[SLE] SLES8 Crash - How to figure out why

From: Richard Mixon (qwest) (rnmixon_at_qwest.net)
Date: 08/31/04

  • Next message: babu walad: "[SLE] where is GLUT in SuSE 9.1 ?" 
    Date: Tue, 31 Aug 2004 12:20:19 -0700
To: suse-linux-e@suse.com

    Our server running SLES8 for AMD64 crashed this morning. It restarted
    fine, but looking at the logs I cannot find anything that suggests a
    problem. The server runs Apache and Tomcat. I have looked at:
      /var/log/messages
      /var/log/apache2/*
      My Tomcat logs

    Where else should I be looking? I included my /var/log/messags below
    from just before the crash and the startup afterwareds - maybe I'm
    missing something. It crashed at 11:29, finally got the coloc operator
    to restart it at 11:54.

    Thanks in advance - Richard

    Aug 31 11:29:28 kingfish pop3d[6519]: mystore: starting txn 2147513745
    Aug 31 11:29:28 kingfish pop3d[6519]: mystore: committing txn 2147513745
    Aug 31 11:29:28 kingfish pop3d[6519]: starttls: TLSv1 with cipher
    RC4-MD5 (128/128 bits new) no authentication
    Aug 31 11:29:28 kingfish saslauthd[704]: pam_sm_authenticate called.
    ... (removed lines reporting on apm authentication with usernames) ...
    Aug 31 11:29:28 kingfish saslauthd[704]: returning 0.
    Aug 31 11:29:28 kingfish saslauthd[704]: pam_mysql: acct_mgmt called but
    not implemented. Dont panic though :)
    Aug 31 11:29:28 kingfish pop3d[6519]: login: [130.13.89.148]
    rnmixon.fromltoj.com plaintext+TLS
    Aug 31 11:29:28 kingfish pop3d[6519]: accepted connection
    Aug 31 11:29:29 kingfish pop3d[6519]: starttls: TLSv1 with cipher
    RC4-MD5 (128/128 bits reused) no authentication
    Aug 31 11:54:09 kingfish syslogd 1.4.1: restart.
    Aug 31 11:54:10 kingfish /etc/hotplug/usb.rc[463]: loaded HCD: usb-ohci
    Aug 31 11:54:14 kingfish kernel: klogd 1.4.1, log source = /proc/kmsg
    started.
    Aug 31 11:54:14 kingfish kernel: Inspecting
    /boot/System.map-2.4.21-241-smp
    Aug 31 11:54:14 kingfish kernel: Loaded 14971 symbols from
    /boot/System.map-2.4.21-241-smp.
    Aug 31 11:54:14 kingfish kernel: Symbols match kernel version 2.4.21.
    Aug 31 11:54:14 kingfish kernel: Error seeking in /dev/kmem
    Aug 31 11:54:14 kingfish kernel: Symbol #st, value a011f000
    Aug 31 11:54:14 kingfish kernel: Error adding kernel module table entry.
    Aug 31 11:54:14 kingfish kernel: ip_tables: (C) 2000-2002 Netfilter core
    team
    Aug 31 11:54:14 kingfish kernel: ip_conntrack version 2.1 (8192 buckets,
    65536 max) - 448 bytes per conntrack
    Aug 31 11:54:14 kingfish kernel: Broadcom Gigabit Ethernet Driver
    bcm5700 with Broadcom NIC Extension (NICE) ver. 6.2.17 (07/14/03)
    Aug 31 11:54:14 kingfish kernel: eth0: Broadcom BCM5704 1000Base-T found
    at mem fc9c0000, IRQ 24, node addr 00e081527800
    Aug 31 11:54:14 kingfish kernel: eth0: Broadcom BCM5704 Integrated
    Copper transceiver found
    Aug 31 11:54:14 kingfish kernel: eth0: Scatter-gather ON, 64-bit DMA ON,
    Tx Checksum ON, Rx Checksum ON, 802.1Q VLAN ON
    Aug 31 11:54:14 kingfish kernel: eth1: Broadcom BCM5704 1000Base-T found
    at mem fc9f0000, IRQ 25, node addr 00e081527801
    Aug 31 11:54:14 kingfish kernel: eth1: Broadcom BCM5704 Integrated
    Copper transceiver found
    Aug 31 11:54:14 kingfish kernel: eth1: Scatter-gather ON, 64-bit DMA ON,
    Tx Checksum ON, Rx Checksum ON, 802.1Q VLAN ON
    Aug 31 11:54:14 kingfish kernel: usb.c: registered new driver usbdevfs
    Aug 31 11:54:14 kingfish kernel: usb.c: registered new driver hub
    Aug 31 11:54:14 kingfish kernel: usb-uhci.c: $Revision: 1.275 $ time
    14:46:30 Aug 10 2004
    Aug 31 11:54:14 kingfish kernel: usb-uhci.c: High bandwidth mode enabled
    Aug 31 11:54:14 kingfish kernel: usb-uhci.c: v1.275:USB Universal Host
    Controller Interface driver
    Aug 31 11:54:14 kingfish kernel: uhci.c: USB Universal Host Controller
    Interface driver v1.1
    Aug 31 11:54:14 kingfish kernel: usb-ohci.c: USB OHCI at membase
    0xffffff0000eaf000, IRQ 19
    Aug 31 11:54:14 kingfish kernel: usb-ohci.c: usb-03:00.0, PCI device
    1022:7464
    Aug 31 11:54:14 kingfish kernel: usb.c: new USB bus registered, assigned
    bus number 1
    Aug 31 11:54:14 kingfish kernel: hub.c: USB hub found
    Aug 31 11:54:14 kingfish kernel: hub.c: 3 ports detected
    Aug 31 11:54:14 kingfish kernel: usb-ohci.c: USB OHCI at membase
    0xffffff0000ed6000, IRQ 19
    Aug 31 11:54:14 kingfish kernel: usb-ohci.c: usb-03:00.1, PCI device
    1022:7464
    Aug 31 11:54:14 kingfish kernel: usb.c: new USB bus registered, assigned
    bus number 2
    Aug 31 11:54:14 kingfish kernel: hub.c: USB hub found
    Aug 31 11:54:14 kingfish kernel: hub.c: 3 ports detected
    Aug 31 11:54:14 kingfish kernel: bcm5700: eth1 NIC Link is UP, 100 Mbps
    half duplex
    Aug 31 11:54:15 kingfish master[682]: setrlimit: Unable to set file
    descriptors limit to -1: Operation not permitted
    Aug 31 11:54:15 kingfish master[682]: retrying with 1024 (current max)
    Aug 31 11:54:15 kingfish master[682]: process started
    Aug 31 11:54:15 kingfish master[683]: about to exec
    /usr/lib/cyrus/bin/ctl_cyrusdb
    Aug 31 11:54:15 kingfish master[682]: process started
    Aug 31 11:54:15 kingfish master[683]: about to exec
    /usr/lib/cyrus/bin/ctl_cyrusdb
    Aug 31 11:54:15 kingfish ctl_cyrusdb[683]: recovering cyrus databases
    Aug 31 11:54:15 kingfish saslauthd[702]: START: saslauthd 2.1.12
    Aug 31 11:54:15 kingfish saslauthd[710]: master PID is: 710
    Aug 31 11:54:15 kingfish saslauthd[710]: daemon started, listening on
    /var/run/sasl2//mux
    Aug 31 11:54:16 kingfish kernel: IPsec Security Association Database
    (SADB): initialized.
    Aug 31 11:54:16 kingfish kernel: IPsec Security Policy Database (SPD):
    initialized.
    Aug 31 11:54:16 kingfish kernel: IPsec PF_KEY V2: initialized
    Aug 31 11:54:16 kingfish kernel: IPv6 v0.8 (usagi-cvs/IPsec6 based StS)
    for NET4.0
    Aug 31 11:54:16 kingfish kernel: IPv6 over IPv4 tunneling driver
    Aug 31 11:54:16 kingfish sshd[730]: Server listening on :: port 22.
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/in.rshd is not
    executable [line=100]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=100]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/in.rlogind is not
    executable [line=111]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=111]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/in.fingerd is not
    executable [line=123]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=123]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/imapd is not
    executable [line=135]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=135]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/imapd is not
    executable [line=148]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=148]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/ipop3d is not
    executable [line=161]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=161]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/ipop3d is not
    executable [line=173]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=173]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/in.talkd is not
    executable [line=195]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=195]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/in.talkd is not
    executable [line=205]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=205]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/in.identd is not
    executable [line=340]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=340]
    Aug 31 11:54:19 kingfish xinetd[842]: Server /usr/sbin/leafnode is not
    executable [line=349]
    Aug 31 11:54:19 kingfish xinetd[842]: Error parsing attribute server -
    DISABLING SERVICE [line=349]
    Aug 31 11:54:19 kingfish xinetd[842]: removing shell
    Aug 31 11:54:19 kingfish xinetd[842]: removing login
    Aug 31 11:54:19 kingfish xinetd[842]: removing finger
    Aug 31 11:54:19 kingfish xinetd[842]: removing imap
    Aug 31 11:54:19 kingfish xinetd[842]: removing imaps
    Aug 31 11:54:19 kingfish xinetd[842]: removing pop3
    Aug 31 11:54:19 kingfish xinetd[842]: removing pop3s
    Aug 31 11:54:19 kingfish xinetd[842]: removing ntalk
    Aug 31 11:54:19 kingfish xinetd[842]: removing talk
    Aug 31 11:54:19 kingfish xinetd[842]: removing ident
    Aug 31 11:54:19 kingfish xinetd[842]: removing nntp
    Aug 31 11:54:19 kingfish xinetd[842]: xinetd Version 20030122 started
    with libwrap loadavg options compiled in.
    Aug 31 11:54:19 kingfish xinetd[842]: Started working: 2 available
    services
    Aug 31 11:54:22 kingfish kernel: keyboard: Timeout - AT keyboard not
    present?(f4)
    Aug 31 11:54:25 kingfish last message repeated 2 times
    Aug 31 11:54:25 kingfish /etc/hotplug/net.agent[757]: No HW description
    found ... exiting
    Aug 31 11:54:26 kingfish kernel: eth1: no IPv6 routers present
    Aug 31 11:54:26 kingfish kernel: keyboard: Timeout - AT keyboard not
    present?(f4)
    Aug 31 11:54:28 kingfish ctl_cyrusdb[683]: skiplist: recovered
    /var/lib/imap/mailboxes.db (43 records, 16796 bytes) in 0 seconds
    Aug 31 11:54:28 kingfish ctl_cyrusdb[683]: done recovering cyrus
    databases
    Aug 31 11:54:28 kingfish master[682]: ready for work
    Aug 31 11:54:28 kingfish master[995]: about to exec
    /usr/lib/cyrus/bin/tls_prune
    Aug 31 11:54:28 kingfish master[996]: about to exec
    /usr/lib/cyrus/bin/ctl_deliver
    Aug 31 11:54:28 kingfish master[997]: about to exec
    /usr/lib/cyrus/bin/ctl_cyrusdb
    Aug 31 11:54:28 kingfish ctl_cyrusdb[997]: checkpointing cyrus databases
    Aug 31 11:54:28 kingfish ctl_cyrusdb[997]: archiving database file:
    /var/lib/imap/mailboxes.db
    Aug 31 11:54:28 kingfish ctl_deliver[996]: duplicate_prune: pruning back
    3 days
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483651
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483651
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483654
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483654
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483655
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483655
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483656
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483656
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483657
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483657
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483658
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483657
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483658
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483658
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483659
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483659
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483660
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483660
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483662
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483662
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483661
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483663
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483661
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483663
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483664
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483665
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483664
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483665
    Aug 31 11:54:28 kingfish ctl_cyrusdb[997]: archiving log file:
    /var/lib/imap/db/log.0000000005
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483666
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483667
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483666
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483667
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: starting txn
    2147483668
    Aug 31 11:54:28 kingfish ctl_cyrusdb[997]: done checkpointing cyrus
    databases
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483669
    Aug 31 11:54:28 kingfish ctl_deliver[996]: mydelete: committing txn
    2147483668
    Aug 31 11:54:28 kingfish master[682]: process 997 exited, status 0
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483669
    Aug 31 11:54:28 kingfish ctl_deliver[996]: duplicate_prune: purged 12
    out of 165 entries
    Aug 31 11:54:28 kingfish master[682]: process 996 exited, status 0
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483670
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483670
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483671
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483671
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483672
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483672
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483673
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483673
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483674
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483674
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483675
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483675
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483676
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483676
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483677
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483677
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: starting txn
    2147483678
    Aug 31 11:54:28 kingfish tls_prune[995]: mydelete: committing txn
    2147483678
    Aug 31 11:54:28 kingfish tls_prune[995]: tls_prune: purged 14 out of 90
    entries
    Aug 31 11:54:28 kingfish master[682]: process 995 exited, status 0
    Aug 31 11:54:29 kingfish kernel: keyboard: Timeout - AT keyboard not
    present?(f4)
    Aug 31 11:54:32 kingfish last message repeated 2 times
    Aug 31 11:54:33 kingfish kernel: ide-floppy driver 0.99.newide
    Aug 31 11:54:33 kingfish kernel: hda: attached ide-cdrom driver.
    Aug 31 11:54:33 kingfish kernel: hda: ATAPI 52X CD-ROM drive, 120kB
    Cache
    Aug 31 11:54:33 kingfish kernel: Uniform CD-ROM driver Revision: 3.12
    Aug 31 11:54:39 kingfish kernel: cdrom: open failed.
    Aug 31 11:54:52 kingfish /usr/sbin/cron[1251]: (CRON) STARTUP (fork ok)
    Aug 31 11:55:36 kingfish sshd[1341]: Could not reverse map address
    130.13.89.148.
    Aug 31 11:56:10 kingfish master[1346]: about to exec
    /usr/lib/cyrus/bin/pop3d
    Aug 31 11:56:10 kingfish pop3s[1346]: executed
    Aug 31 11:56:10 kingfish pop3d[1346]: accepted connection
    Aug 31 11:56:12 kingfish pop3d[1346]: starttls: TLSv1 with cipher
    RC4-MD5 (128/128 bits reused) no authentication
    Aug 31 11:56:12 kingfish saslauthd[711]: pam_sm_authenticate called. 

    -- 
Check the headers for your unsubscription address
For additional commands send e-mail to suse-linux-e-help@suse.com
Also check the archives at http://lists.suse.com
Please read the FAQs: suse-linux-e-faq@suse.com
  • Next message: babu walad: "[SLE] where is GLUT in SuSE 9.1 ?"