aMule Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

We're back! (IN POG FORM)

Author Topic: aMule r10525 hangs  (Read 3842 times)

staxx

  • Approved Newbie
  • *
  • Karma: 8
  • Offline Offline
  • Posts: 29
aMule r10525 hangs
« on: April 26, 2011, 02:27:15 AM »
Amule  (r10525, x86-64, with wx 2.8.12 on RHEL 6 system) hangs from time to time, with 100% cpu usage.  Before that, it eats out lot of ram (like around 500 mb resident ram, when usually it consumes 100-150).  He can be hanging like that for hours until killed. First SIGTERM sent to process does nothing, second instantly terminates it.

When attaching with strace after first SIGTERM I see very fast spamming lines like
Code: [Select]
read(5, 0x1732ac4, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=8, events=POLLOUT}, {fd=7, events=POLLIN}, {fd=69, events=POLLOUT}], 5, 0) = 1 ([{fd=69, revents=POLLOUT}])
read(5, 0x1732ac4, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=8, events=POLLOUT}, {fd=7, events=POLLIN}, {fd=69, events=POLLOUT}], 5, 0) = 1 ([{fd=69, revents=POLLOUT}])
read(5, 0x1732ac4, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=8, events=POLLOUT}, {fd=7, events=POLLIN}, {fd=69, events=POLLOUT}], 5, 0) = 1 ([{fd=69, revents=POLLOUT}])
tons of these lines spamming hundreds per second and nothing else.

fd 5, according to /proc/pid/fd is just some random socket.

I know that this information isn't enough, but I just don't know what else to do to get more. If you can give me some advice (like, maybe some way to attach with gdb or something) I could probably do it next time it happens.
Logged

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: aMule r10525 hangs
« Reply #1 on: April 26, 2011, 03:50:53 AM »
Attaching with gdb is really simple.

http://ftp.gnu.org/old-gnu/Manuals/gdb-5.1.1/html_node/gdb_22.html

Do that, then give us a backtrace.
Logged

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: aMule r10525 hangs
« Reply #2 on: April 26, 2011, 05:12:05 PM »
Also be sure to install debug packages first.
Logged

staxx

  • Approved Newbie
  • *
  • Karma: 8
  • Offline Offline
  • Posts: 29
Re: aMule r10525 hangs
« Reply #3 on: May 09, 2011, 03:58:32 AM »
It happened again, amule was hanging for about 7 hours with 100% cpu usage and no interface redraw.  Looks like during that it was slowly eating up more and more memory,  too.
When I noticed, I attached gdb and created some backtraces:

Code: [Select]
(gdb) attach 1558
Attaching to process 1558
Reading symbols from /usr/bin/amule...Reading symbols from /usr/lib/debug/usr/bin/amule.debug...done.
done.
Reading symbols from /lib64/libpthread.so.0...Reading symbols from /usr/lib/debug/lib64/libpthread-2.12.so.debug...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x7f4a2cca3700 (LWP 1574)]
[New Thread 0x7f4a2e41e700 (LWP 1565)]
[..skipped..]

__pthread_mutex_lock (mutex=0x2b6ef38) at pthread_mutex_lock.c:125
125   mutex->__data.__owner = id;

(gdb) bt
#0  __pthread_mutex_lock (mutex=0x2b6ef38) at pthread_mutex_lock.c:125
#1  0x00000030eca3be41 in IA__g_main_context_check (context=0x2b6ef30,
    max_priority=110, fds=<value optimized out>, n_fds=<value optimized out>)
    at gmain.c:2471
#2  0x00000030eca3c66f in g_main_context_iterate (context=0x2b6ef30, block=1,
    dispatch=1, self=<value optimized out>) at gmain.c:2588
#3  0x00000030eca3cd55 in IA__g_main_loop_run (loop=0x2c5d220) at gmain.c:2799
#4  0x000000389694c2c7 in IA__gtk_main () at gtkmain.c:1218
#5  0x0000003c543e78f8 in wxEventLoop::Run (this=0x59af2a0)
    at src/gtk/evtloop.cpp:76
#6  0x0000003c5446216b in wxAppBase::MainLoop (this=0x2b6e960)
    at src/common/appcmn.cpp:312
#7  0x00000036bb6961c5 in wxEntry (argc=<value optimized out>,
    argv=<value optimized out>) at src/common/init.cpp:448
#8  0x00000000005c98e2 in main (argc=1, argv=<value optimized out>)
    at amule-gui.cpp:93
(gdb) bt full
#0  __pthread_mutex_lock (mutex=0x2b6ef38) at pthread_mutex_lock.c:125
        type = <value optimized out>
        id = 1558
#1  0x00000030eca3be41 in IA__g_main_context_check (context=0x2b6ef30,
    max_priority=110, fds=<value optimized out>, n_fds=<value optimized out>)
    at gmain.c:2471
        result = <value optimized out>
        check = <value optimized out>
        source = 0x2ba2240
        pollrec = <value optimized out>
        n_ready = <value optimized out>
        i = <value optimized out>
#2  0x00000030eca3c66f in g_main_context_iterate (context=0x2b6ef30, block=1,
    dispatch=1, self=<value optimized out>) at gmain.c:2588
        max_priority = 110
        timeout = 0
        some_ready = <value optimized out>
        nfds = 13
        allocated_nfds = <value optimized out>
        fds = 0x8de47a0
        __PRETTY_FUNCTION__ = "g_main_context_iterate"
#3  0x00000030eca3cd55 in IA__g_main_loop_run (loop=0x2c5d220) at gmain.c:2799
        self = 0x2b6fd80
        __PRETTY_FUNCTION__ = "IA__g_main_loop_run"
#4  0x000000389694c2c7 in IA__gtk_main () at gtkmain.c:1218
        tmp_list = <value optimized out>
        functions = 0x0
        init = <value optimized out>
        loop = 0x2c5d220
#5  0x0000003c543e78f8 in wxEventLoop::Run (this=0x59af2a0)
    at src/gtk/evtloop.cpp:76
        activate = {m_evtLoopOld = 0x0}
        exitcode = <value optimized out>
#6  0x0000003c5446216b in wxAppBase::MainLoop (this=0x2b6e960)
    at src/common/appcmn.cpp:312
        mainLoop = {<wxEventLoopPtr> = {m_ptr = 0x59af2a0}, m_pp = 0x2b6e9d8,
          m_pOld = 0x0}
#7  0x00000036bb6961c5 in wxEntry (argc=<value optimized out>,
---Type <return> to continue, or q <return> to quit---
    argv=<value optimized out>) at src/common/init.cpp:448
        initializer = {m_ok = true}
#8  0x00000000005c98e2 in main (argc=1, argv=<value optimized out>)
    at amule-gui.cpp:93
No locals.
(gdb) thread apply all bt

Thread 3 (Thread 0x7f4a2e41e700 (LWP 1565)):
#0  0x00000030ed60ed4d in nanosleep () at ../sysdeps/unix/syscall-template.S:82
#1  0x00000036bb6ebefc in wxMicroSleep (microseconds=<value optimized out>)
    at src/unix/utilsunx.cpp:191
#2  0x000000000057208b in UploadBandwidthThrottler::Entry (this=0x59fe5a0)
    at UploadBandwidthThrottler.cpp:323
#3  0x00000036bb6e74d1 in wxThreadInternal::PthreadStart (thread=0x59fe5a0)
    at src/unix/threadpsx.cpp:766
#4  0x00000030ed6077e1 in start_thread (arg=0x7f4a2e41e700)
    at pthread_create.c:301
#5  0x00000030ecee5dcd in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 2 (Thread 0x7f4a2cca3700 (LWP 1574)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:216
#1  0x00000036bb6e5be6 in wxConditionInternal::WaitTimeout (this=0x2c15450,
    milliseconds=100) at src/unix/threadpsx.cpp:405
#2  0x00000036bb6e7063 in wxSemaphoreInternal::WaitTimeout (this=0x59af390,
    milliseconds=100) at src/unix/threadpsx.cpp:552
#3  0x00000000006987bd in CTimerThread::Entry (this=0x59af340) at Timer.cpp:66
#4  0x00000036bb6e74d1 in wxThreadInternal::PthreadStart (thread=0x59af340)
    at src/unix/threadpsx.cpp:766
#5  0x00000030ed6077e1 in start_thread (arg=0x7f4a2cca3700)
    at pthread_create.c:301
#6  0x00000030ecee5dcd in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 1 (Thread 0x7f4a36d6c940 (LWP 1558)):
#0  __pthread_mutex_lock (mutex=0x2b6ef38) at pthread_mutex_lock.c:125
#1  0x00000030eca3be41 in IA__g_main_context_check (context=0x2b6ef30,
    max_priority=110, fds=<value optimized out>, n_fds=<value optimized out>)
    at gmain.c:2471
#2  0x00000030eca3c66f in g_main_context_iterate (context=0x2b6ef30, block=1,
    dispatch=1, self=<value optimized out>) at gmain.c:2588
#3  0x00000030eca3cd55 in IA__g_main_loop_run (loop=0x2c5d220) at gmain.c:2799
#4  0x000000389694c2c7 in IA__gtk_main () at gtkmain.c:1218
---Type <return> to continue, or q <return> to quit---
#5  0x0000003c543e78f8 in wxEventLoop::Run (this=0x59af2a0)
    at src/gtk/evtloop.cpp:76
#6  0x0000003c5446216b in wxAppBase::MainLoop (this=0x2b6e960)
    at src/common/appcmn.cpp:312
#7  0x00000036bb6961c5 in wxEntry (argc=<value optimized out>,
    argv=<value optimized out>) at src/common/init.cpp:448
#8  0x00000000005c98e2 in main (argc=1, argv=<value optimized out>)
    at amule-gui.cpp:93

If attaching with strace -p, there is very fast non-stop spam (it fills 5000 lines log in terminal in a second) of
Code: [Select]
poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=8, events=POLLOUT}, {fd=6, events=POLLIN}, {fd=506, events=POLLIN}, {fd=458, events=POLLIN}, {fd=525, events=POLLIN}, {fd=145, events=POLLIN}, {fd=234, events=POLLIN}, {fd=397, events=POLLIN}, {fd=29, events=POLLIN}, {fd=122, events=POLLIN}, {fd=303, events=POLLOUT}], 13, 0) = 1 ([{fd=303, revents=POLLOUT}])
read(5, 0x2b8aac4, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=8, events=POLLOUT}, {fd=6, events=POLLIN}, {fd=506, events=POLLIN}, {fd=458, events=POLLIN}, {fd=525, events=POLLIN}, {fd=145, events=POLLIN}, {fd=234, events=POLLIN}, {fd=397, events=POLLIN}, {fd=29, events=POLLIN}, {fd=122, events=POLLIN}, {fd=303, events=POLLOUT}], 13, 0) = 1 ([{fd=303, revents=POLLOUT}])
read(5, 0x2b8aac4, 4096)                = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=8, events=POLLOUT}, {fd=6, events=POLLIN}, {fd=506, events=POLLIN}, {fd=458, events=POLLIN}, {fd=525, events=POLLIN}, {fd=145, events=POLLIN}, {fd=234, events=POLLIN}, {fd=397, events=POLLIN}, {fd=29, events=POLLIN}, {fd=122, events=POLLIN}, {fd=303, events=POLLOUT}], 13, 0) = 1 ([{fd=303, revents=POLLOUT}])
read(5, 0x2b8aac4, 4096)                = -1 EAGAIN (Resource temporarily unavailable)

Just like before, first SIGTERM changes nothing, second instantly kills amule.
Logged