Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[RRM]

Thank you Bill, you are very supportive.
This forum is amazing (the knowledge, the support).

amule VSZ is more than 3Gb!

So, should the virtual memory be lower, or should i buy more memory?
6GB would be enough?

some other of your crashes doesn't seem related

Couldnt it be that when it lacks memory, anything might go wrong?

[btkaos]

So, should the virtual memory be lower, or should i buy more memory?
6GB would be enough?

Don't buy more memory. It seems amule has a bug and it eating more memory than it should.

Unless you are in the realms of 10.000-100.000 simultaneous uploads?

[RRM]

Thank you guys.

Unless you are in the realms of 10.000-100.000 simultaneous uploads?

he he, noooo 
It happens beyond 200 simultaneous uploads,
particularly when combined with multiple downloads.

Probably its irrelevant to the problem you are referring to,
or you already know all this,
but i found this: 

"Here is a checklist for possible causes for increased memory usage:

    * Check which MDAC version is being used. In many of the reported cases, the MDAC version 2.1.3711.11 contained a documented memory leak.
    * Examine ColdFusion query caching to make sure no differences appear.
    * Make sure that growing files, logs, or record sets are not being brought into memory, even if only for the duration of the request.
    * Check accumulating states in persistent scopes, including server, application, and session.
    * Operations on collections (e.g., CFSearch and CFIndex) can cause growth over time.
    * Keep in mind that operations needing large contiguous blocks of memory like CFFile or CFHTTP can take longer to plateau.
    * Homegrown CFX tags may be allocating memory and not properly freeing it up.
    * When running tests for memory usage, check for the existence of any application.cfm files located in the directory path that enable session, application, or server variables. These variables may add processing cycles or additional demand on memory, which may not be immediately apparent.
    * One of the most common causes are improperly locked access to session, application, and server variables. These variables are stored in memory and need to be enclosed in a CFLOCK tag with the appropriate SCOPE attribute specified (i.e. session, application, or server) when referenced.
    * When accessing client variables stored in an Oracle DB with the CFAS native driver for Oracle, see KB article 15885."

[RRM]

Amule has not crashed today, because before going to work i had set maximum
connections to 100 (though even with so few connections amule sometimes
uploads 6 to 10 MB/s), so i get this:

$ ps axl | grep amule

Code: [Select]

F  UID   PID   PPID PRI  NI VSZ     RSS     WCHAN  STAT TTY    TIME COMMAND
0  1000 23494 23472  20   0 162652 90208 -      Rl+  pts/0    621:25 /usr/bin/am

and yet:

$ free

Code: [Select]

             total       used       free 
Mem:       3111840    3042400      69440
So, maybe there is an accumulation?
Or might it be that this is the result of using both
amule and torrents, because since i have to restrict amule activity lately,
i also up/download (about 20) torrents, 283 connections in total,
to utilize the bandwith amule doesnt use.

[RRM]

And another crash:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb6693700 (LWP 23494)]
0x0ad5f030 in ?? ()
(gdb) bt

Code: [Select]

#0  0x0ad5f030 in ?? ()
#1  0xb6cb74af in gdk_io_invoke (source=0xb24a0310,
    condition=<value optimized out>, data=0x3)
    at /build/buildd/gtk+2.0-2.14.4/gdk/gdkevents.c:1013
#2  0xb6a5571d in g_io_unix_dispatch (source=0xb20eee48,
    callback=0xb6cb7450 <gdk_io_invoke>, user_data=0xb247d8e8)
    at /build/buildd/glib2.0-2.18.2/glib/giounix.c:162
#3  0xb6a1e718 in IA__g_main_context_dispatch (context=0x9a88918)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2144
#4  0xb6a21dc3 in g_main_context_iterate (context=0x9a88918, block=1,
    dispatch=1, self=0x9a89ec0)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2778
#5  0xb6a222e2 in IA__g_main_loop_run (loop=0xa27c748)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2986
#6  0xb6e503a9 in IA__gtk_main ()
    at /build/buildd/gtk+2.0-2.14.4/gtk/gtkmain.c:1200
#7  0xb766d10b in wxEventLoop::Run (this=0xa27d6c8)
    at ../src/gtk/evtloop.cpp:76
#8  0xb77113dc in wxAppBase::MainLoop (this=0x9a887d0)
    at ../src/common/appcmn.cpp:312
#9  0xb7711131 in wxAppBase::OnRun (this=0x0) at ../src/common/appcmn.cpp:367
#10 0xb74063fa in wxEntry (argc=@0xb74fe6cc, argv=0x9a76e38)
    at ../src/common/init.cpp:460
---Type <return> to continue, or q <return> to quit---
#11 0xb74064b7 in wxEntry (argc=@0xbfa187b0, argv=0xbfa18834)
    at ../src/common/init.cpp:472
#12 0x082ac21d in main (argc=1, argv=0xb6a9b460) at ../../src/amule-gui.cpp:95(gdb) bt full

Code: [Select]

#0  0x0ad5f030 in ?? ()
No symbol table info available.
#1  0xb6cb74af in gdk_io_invoke (source=0xb24a0310,
    condition=<value optimized out>, data=0x3)
    at /build/buildd/gtk+2.0-2.14.4/gdk/gdkevents.c:1013
gdk_cond = 184218312
#2  0xb6a5571d in g_io_unix_dispatch (source=0xb20eee48,
    callback=0xb6cb7450 <gdk_io_invoke>, user_data=0xb247d8e8)
    at /build/buildd/glib2.0-2.18.2/glib/giounix.c:162
buffer_condition = 0
#3  0xb6a1e718 in IA__g_main_context_dispatch (context=0x9a88918)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2144
No locals.
#4  0xb6a21dc3 in g_main_context_iterate (context=0x9a88918, block=1,
    dispatch=1, self=0x9a89ec0)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2778
max_priority = 2147483647
timeout = 15
some_ready = 1
nfds = 103
allocated_nfds = <value optimized out>
fds = (GPollFD *) 0xa86e6e0
__PRETTY_FUNCTION__ = "g_main_context_iterate"
---Type <return> to continue, or q <return> to quit---
#5  0xb6a222e2 in IA__g_main_loop_run (loop=0xa27c748)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2986
self = (GThread *) 0x9a89ec0
__PRETTY_FUNCTION__ = "IA__g_main_loop_run"
#6  0xb6e503a9 in IA__gtk_main ()
    at /build/buildd/gtk+2.0-2.14.4/gtk/gtkmain.c:1200
tmp_list = (GList *) 0x0
functions = (GList *) 0x0
init = (GtkInitFunction *) 0xa27d6c8
loop = (GMainLoop *) 0xa27c748
#7  0xb766d10b in wxEventLoop::Run (this=0xa27d6c8)
    at ../src/gtk/evtloop.cpp:76
exitcode = <value optimized out>
__FUNCTION__ = "Run"
#8  0xb77113dc in wxAppBase::MainLoop (this=0x9a887d0)
    at ../src/common/appcmn.cpp:312
mainLoop = {<wxEventLoopPtr> = {m_ptr = 0xa27d6c8}, m_pp = 0x9a88810,
  m_pOld = 0x0}
#9  0xb7711131 in wxAppBase::OnRun (this=0x0) at ../src/common/appcmn.cpp:367
No locals.
#10 0xb74063fa in wxEntry (argc=@0xb74fe6cc, argv=0x9a76e38)
    at ../src/common/init.cpp:460
No locals.
---Type <return> to continue, or q <return> to quit---
#11 0xb74064b7 in wxEntry (argc=@0xbfa187b0, argv=0xbfa18834)
    at ../src/common/init.cpp:472
No locals.
#12 0x082ac21d in main (argc=1, argv=0xb6a9b460) at ../../src/amule-gui.cpp:95
No locals.

[RRM]

thread apply all bt full

Code: [Select]

Thread 5 (Thread 0xb450eb90 (LWP 23536)):
#0  0xb7f17430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7edb3a2 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#2  0xb74776e1 in wxConditionInternal::WaitTimeout (this=0x9acc908,
    milliseconds=100) at ../src/unix/threadpsx.cpp:405
tspec = {tv_sec = 1238242285, tv_nsec = 404000000}
err = <value optimized out>
#3  0xb747776a in wxCondition::WaitTimeout (this=0x9ae827c, milliseconds=100)
    at ../include/wx/thrimpl.cpp:256
__FUNCTION__ = "WaitTimeout"
#4  0xb7479524 in wxSemaphoreInternal::WaitTimeout (this=0x9ae8278,
    milliseconds=100) at ../src/unix/threadpsx.cpp:552
remainingTime = -516
locker = {m_isOk = true, m_mutex = @0x9ae8278}
#5  0xb74795aa in wxSemaphore::WaitTimeout (this=0xa286860, milliseconds=100)
    at ../include/wx/thrimpl.cpp:320
__FUNCTION__ = "WaitTimeout"
#6  0x083e39d4 in CTimerThread::Entry (this=0xa286840)
    at ../../src/Timer.cpp:64
---Type <return> to continue, or q <return> to quit---
now = 1291704056
sinceLast = 0
timeout = 100
evt = {<wxEvent> = {<wxObject> = {_vptr.wxObject = 0x85c7608,
      static ms_classInfo = {m_className = 0xb74ab1a4, m_objectSize = 8,
        m_objectConstructor = 0, m_baseInfo1 = 0x0, m_baseInfo2 = 0x0,
        static sm_first = 0xb79a88cc, m_next = 0xb74ff8c4,
        static sm_classTable = 0x9a5b3e8}, m_refData = 0x0},
    m_eventObject = 0x0, m_eventType = 10244, m_timeStamp = 0, m_id = 6128,
    m_callbackUserData = 0x0, m_propagationLevel = 0, m_skipped = false,
    m_isCommandEvent = false, static ms_classInfo = {m_className = 0xb74bc708,
      m_objectSize = 36, m_objectConstructor = 0, m_baseInfo1 = 0xb74ff858,
      m_baseInfo2 = 0x0, static sm_first = 0xb79a88cc, m_next = 0xb7500908,
      static sm_classTable = 0x9a5b3e8}}, <No data fields>}
lastEvent = 1291704056
#7  0xb747a036 in wxThreadInternal::PthreadStart (thread=0xa286840)
    at ../src/unix/threadpsx.cpp:766
pthread = (wxThreadInternal *) 0x9ae8210
rc = <value optimized out>
dontRunAtAll = false
__FUNCTION__ = "PthreadStart"
#8  0xb747a13d in wxPthreadStart (ptr=0xa286840)
    at ../src/unix/threadpsx.cpp:718
---Type <return> to continue, or q <return> to quit---
No locals.
#9  0xb7ed750f in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#10 0xb71aca0e in clone () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.

Thread 3 (Thread 0xb5510b90 (LWP 23532)):
#0  0xb7f17430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7ede906 in nanosleep () from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#2  0xb7481168 in wxMicroSleep (microseconds=0) at ../src/unix/utilsunx.cpp:191
tmReq = {tv_sec = 0, tv_nsec = 4000000}
#3  0xb7481191 in wxMilliSleep (milliseconds=4) at ../src/unix/utilsunx.cpp:212
No locals.
#4  0xb747741d in wxThread::Sleep (milliseconds=4)
    at ../src/unix/threadpsx.cpp:986
No locals.
#5  0x081fd133 in UploadBandwidthThrottler::Entry (this=0xb591bd90)
    at ../../src/UploadBandwidthThrottler.cpp:324
timeSinceLastLoop = 1
minFragSize = 1300
doubleSendSize = 2600
---Type <return> to continue, or q <return> to quit---
bytesToSpend = 29680
sleepTime = 5
thisLoopTick = 1291704104
TIME_BETWEEN_UPLOAD_LOOPS = 1
lastLoopTick = 1291704104
realBytesToSpend = 23040999
allowedDataRate = 2048000
rememberedSlotCounter = 40
extraSleepTime = 5
sendLock = {m_isOk = false, m_mutex = @0xb7ee6ff4}
#6  0xb747a036 in wxThreadInternal::PthreadStart (thread=0xb591bd90)
    at ../src/unix/threadpsx.cpp:766
pthread = (wxThreadInternal *) 0xb591bee0
rc = <value optimized out>
dontRunAtAll = false
__FUNCTION__ = "PthreadStart"
#7  0xb747a13d in wxPthreadStart (ptr=0xb591bd90)
    at ../src/unix/threadpsx.cpp:718
No locals.
#8  0xb7ed750f in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#9  0xb71aca0e in clone () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---

Thread 1 (Thread 0xb6693700 (LWP 23494)):
#0  0x0ad5f030 in ?? ()
No symbol table info available.
#1  0xb6cb74af in gdk_io_invoke (source=0xb24a0310,
    condition=<value optimized out>, data=0x3)
    at /build/buildd/gtk+2.0-2.14.4/gdk/gdkevents.c:1013
gdk_cond = 184218312
#2  0xb6a5571d in g_io_unix_dispatch (source=0xb20eee48,
    callback=0xb6cb7450 <gdk_io_invoke>, user_data=0xb247d8e8)
    at /build/buildd/glib2.0-2.18.2/glib/giounix.c:162
buffer_condition = 0
#3  0xb6a1e718 in IA__g_main_context_dispatch (context=0x9a88918)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2144
No locals.
#4  0xb6a21dc3 in g_main_context_iterate (context=0x9a88918, block=1,
    dispatch=1, self=0x9a89ec0)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2778
max_priority = 2147483647
timeout = 15
some_ready = 1
nfds = 103
allocated_nfds = <value optimized out>
---Type <return> to continue, or q <return> to quit---
fds = (GPollFD *) 0xa86e6e0
__PRETTY_FUNCTION__ = "g_main_context_iterate"
#5  0xb6a222e2 in IA__g_main_loop_run (loop=0xa27c748)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2986
self = (GThread *) 0x9a89ec0
__PRETTY_FUNCTION__ = "IA__g_main_loop_run"
#6  0xb6e503a9 in IA__gtk_main ()
    at /build/buildd/gtk+2.0-2.14.4/gtk/gtkmain.c:1200
tmp_list = (GList *) 0x0
functions = (GList *) 0x0
init = (GtkInitFunction *) 0xa27d6c8
loop = (GMainLoop *) 0xa27c748
#7  0xb766d10b in wxEventLoop::Run (this=0xa27d6c8)
    at ../src/gtk/evtloop.cpp:76
exitcode = <value optimized out>
__FUNCTION__ = "Run"
#8  0xb77113dc in wxAppBase::MainLoop (this=0x9a887d0)
    at ../src/common/appcmn.cpp:312
mainLoop = {<wxEventLoopPtr> = {m_ptr = 0xa27d6c8}, m_pp = 0x9a88810,
  m_pOld = 0x0}
#9  0xb7711131 in wxAppBase::OnRun (this=0x0) at ../src/common/appcmn.cpp:367
No locals.
#10 0xb74063fa in wxEntry (argc=@0xb74fe6cc, argv=0x9a76e38)
---Type <return> to continue, or q <return> to quit---
    at ../src/common/init.cpp:460
No locals.
#11 0xb74064b7 in wxEntry (argc=@0xbfa187b0, argv=0xbfa18834)
    at ../src/common/init.cpp:472
No locals.
#12 0x082ac21d in main (argc=1, argv=0xb6a9b460) at ../../src/amule-gui.cpp:95
No locals.

$ free

Code: [Select]

             total       used       free     shared    buffers     cached
Mem:       3111840    3002768     109072          0      36264    2085076
-/+ buffers/cache:     881428    2230412
Swap:      9100780     429376    8671404
$ ps axl | grep amule

Code: [Select]

0  1000 23472 23445  20   0 203176 198464 poll  S+   pts/0      0:03 gdb amule
F  UID   PID   PPID PRI  NI VSZ     RSS     WCHAN  STAT TTY    TIME COMMAND
0  1000 23494 23472  20   0 187964 113600 ptrace Tl  pts/0    1383:58 /usr/bin/amule
So, even in the 'safe mode ' (only 100 connections), and with a relatively low
VSZ, amule crashes.
I guess its safe to say that with more connections, the memory leak is bigger as well.

[RRM]

Im doing a little monitoring.
I restarted the computer and then checked available memory and VSZ
with time (Amule: 250 connections; Torrents: 16= 301 connections):

$ free

Code: [Select]

           total       used       free     
Mem:       3111840     496256    2615584   = right before starting up Amule
Mem:       3111840     859656    2252184   = 6 seconds after starting up Amule
Mem:       3111840     916388    2195452   = 8 seconds ,,,
Mem:       3111840     921388    2190452   = 10 secs ,,,
Mem:       3111840     932488    2179352   = 12 secs ,,,
Mem:       3111840    1296008    1815832   = 18 secs
Mem:       3111840    3022224      89616   = 29 secs
Mem:       3111840    3020048      91792   = 40 secs
Mem:       3111840    3019724      92116   = 11 minutes
Mem:       3111840    3025576      86264   = 47 min.
Mem:       3111840    3023116      88724   = 68 min.
Mem:       3111840    1926184    1185656   = 1 sec. after closing gdb amule
$ ps axl | grep amule

Code: [Select]

VSZ     RSS     WCHAN  STAT TTY    TIME COMMAND
110464 40604 poll   Sl+  pts/1      0:07 /usr/bin/amule
110772 40920 -      Rl+  pts/1      0:09 /usr/bin/amule
111420 41568 poll   Sl+  pts/1      0:11 /usr/bin/amule
112000 42688 poll   Sl+  pts/1      0:13 /usr/bin/amule
112912 43520 poll   Sl+  pts/1      0:19 /usr/bin/amule
114760 45216 poll   Sl+  pts/1      0:30 /usr/bin/amule
115980 46800 poll   Sl+  pts/1      0:41 /usr/bin/amule
118144 48960 poll   Sl+  pts/1      1:16 /usr/bin/amule
123260 53380 -      Sl+  pts/1      2:36 /usr/bin/amule
140240 69312 -      Rl+  pts/1     11:01 /usr/bin/amule
164044 92316 -      Rl+  pts/1     24:59 /usr/bin/amule
178380 107064 futex_ Sl+ pts/1     48:17 /usr/bin/amule
739292 668084 -     Sl+  pts/1     63:34 /usr/bin/amule
3138656 2384740 sync_p Dl+ pts/1   67:37 /usr/bin/amule
3144228 2220900 ptrace Tl pts/1    67:38 /usr/bin/amule


• Available memory rapidly declines after about 13 secs,
  which plateaus after 30 secs.
• VSZ is over 1GB right from the start...
  and gradually continues to grow.
  After the sudden increase in VSZ (about 55 - 60 min), Amule
  is frozen, but has not crashed yet.
  At 67 min. and 37 secs. Amule crashes.
• Amule apparently keeps my system so busy that the bittorrent transmission client
  is still waiting to verify local data of some torrents 50 mins. after startup.

[RRM]

Program received signal SIGABRT, Aborted.
[Switching to Thread 0xb67f5700 (LWP 6635)]
0xb8079430 in __kernel_vsyscall ()

bt

Code: [Select]

#0  0xb8079430 in __kernel_vsyscall ()
#1  0xb72588a0 in raise () from /lib/tls/i686/cmov/libc.so.6
#2  0xb725a268 in abort () from /lib/tls/i686/cmov/libc.so.6
#3  0xb747b655 in __cxxabiv1::__terminate (handler=0x808a378 <abort@plt>)
    at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:43
#4  0xb747b692 in std::terminate ()
    at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:53
#5  0xb747b7ca in __cxa_throw (obj=0xb74aaaf0, tinfo=0x86e5a6c,
    dest=0xb747bd00 <~bad_alloc>)
    at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:76
#6  0xb747be23 in operator new (sz=24)
    at ../../../../src/libstdc++-v3/libsupc++/new_op.cc:63
#7  0xb7560556 in wxObjectList::CreateNode (this=0x8ef76f8, prev=0x9b222b0,
    next=0x0, data=0x86e80c0, key=@0xb7660750) at ../include/wx/list.h:1178
#8  0xb7575759 in wxListBase::Append (this=0x8ef76f8, object=0x86e80c0)
    at ../src/common/list.cpp:244
#9  0xb75ddc26 in wxEvtHandler::AddPendingEvent (this=0x86e80c0,
    event=@0xbfa784a0) at ../include/wx/list.h:1178
#10 0xb7681d58 in wxSocketBase::OnRequest (this=0x9b8fa90,
    notification=wxSOCKET_OUTPUT) at ../src/common/socket.cpp:1006
#11 0xb7681e54 in wx_socket_callback (notification=GSOCK_OUTPUT,
    cdata=0x9b8fa90 "\b-M\b") at ../src/common/socket.cpp:942
#12 0xb76866bb in GSocket::Detected_Write (this=0x9a49538)
---Type <return> to continue, or q <return> to quit---
    at ../src/unix/gsocket.cpp:1836
#13 0xb77d0be7 in _GSocket_GDK_Input (data=0x9a49538, source=73,
    condition=GDK_INPUT_WRITE) at ../src/gtk/gsockgtk.cpp:36
#14 0xb6e194af in gdk_io_invoke (source=0xb57303c0,
    condition=<value optimized out>, data=0xb5731150)
    at /build/buildd/gtk+2.0-2.14.4/gdk/gdkevents.c:1013
#15 0xb6bb771d in g_io_unix_dispatch (source=0xb572faa0,
    callback=0xb6e19450 <gdk_io_invoke>, user_data=0xb5731150)
    at /build/buildd/glib2.0-2.18.2/glib/giounix.c:162
#16 0xb6b80718 in IA__g_main_context_dispatch (context=0x8ecf918)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2144
#17 0xb6b83dc3 in g_main_context_iterate (context=0x8ecf918, block=1,
    dispatch=1, self=0x8ed0ec0)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2778
#18 0xb6b842e2 in IA__g_main_loop_run (loop=0x919d610)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2986
#19 0xb6fb23a9 in IA__gtk_main ()
    at /build/buildd/gtk+2.0-2.14.4/gtk/gtkmain.c:1200
#20 0xb77cf10b in wxEventLoop::Run (this=0x96fbc68)
    at ../src/gtk/evtloop.cpp:76
#21 0xb78733dc in wxAppBase::MainLoop (this=0x8ecf7d0)
    at ../src/common/appcmn.cpp:312
#22 0xb7873131 in wxAppBase::OnRun (this=0x6) at ../src/common/appcmn.cpp:367
---Type <return> to continue, or q <return> to quit---
#23 0xb75683fa in wxEntry (argc=@0xb76606cc, argv=0x8ebde38)
    at ../src/common/init.cpp:460
#24 0xb75684b7 in wxEntry (argc=@0xbfa78810, argv=0xbfa78894)
    at ../src/common/init.cpp:472
#25 0x082ac21d in main (argc=Cannot access memory at address 0x19eb
) at ../../src/amule-gui.cpp:95

bt full

Code: [Select]

#0  0xb8079430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb72588a0 in raise () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#2  0xb725a268 in abort () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#3  0xb747b655 in __cxxabiv1::__terminate (handler=0x808a378 <abort@plt>)
    at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:43
No locals.
#4  0xb747b692 in std::terminate ()
    at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:53
No locals.
#5  0xb747b7ca in __cxa_throw (obj=0xb74aaaf0, tinfo=0x86e5a6c,
    dest=0xb747bd00 <~bad_alloc>)
    at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:76
header = <value optimized out>
#6  0xb747be23 in operator new (sz=24)
    at ../../../../src/libstdc++-v3/libsupc++/new_op.cc:63
handler = <value optimized out>
p = <value optimized out>
#7  0xb7560556 in wxObjectList::CreateNode (this=0x8ef76f8, prev=0x9b222b0,
    next=0x0, data=0x86e80c0, key=@0xb7660750) at ../include/wx/list.h:1178
No locals.
---Type <return> to continue, or q <return> to quit---
#8  0xb7575759 in wxListBase::Append (this=0x8ef76f8, object=0x86e80c0)
    at ../src/common/list.cpp:244
node = <value optimized out>
__FUNCTION__ = "Append"
#9  0xb75ddc26 in wxEvtHandler::AddPendingEvent (this=0x86e80c0,
    event=@0xbfa784a0) at ../include/wx/list.h:1178
eventCopy = <value optimized out>
__FUNCTION__ = "AddPendingEvent"
#10 0xb7681d58 in wxSocketBase::OnRequest (this=0x9b8fa90,
    notification=wxSOCKET_OUTPUT) at ../src/common/socket.cpp:1006
event = {<wxEvent> = {<wxObject> = {_vptr.wxObject = 0x86e1488,
      static ms_classInfo = {m_className = 0xb760d1a4, m_objectSize = 8,
        m_objectConstructor = 0, m_baseInfo1 = 0x0, m_baseInfo2 = 0x0,
        static sm_first = 0xb7b0a8cc, m_next = 0xb76618c4,
        static sm_classTable = 0x8ea23e8}, m_refData = 0x0},
    m_eventObject = 0x9b8fa90, m_eventType = 10002, m_timeStamp = 0,
    m_id = 6123, m_callbackUserData = 0x0, m_propagationLevel = 0,
    m_skipped = false, m_isCommandEvent = false, static ms_classInfo = {
      m_className = 0xb761e708, m_objectSize = 36, m_objectConstructor = 0,
      m_baseInfo1 = 0xb7661858, m_baseInfo2 = 0x0,
      static sm_first = 0xb7b0a8cc, m_next = 0xb7662908,
      static sm_classTable = 0x8ea23e8}}, m_event = wxSOCKET_OUTPUT,
  m_clientData = 0x0, static ms_classInfo = {m_className = 0xb7689008,
---Type <return> to continue, or q <return> to quit---
    m_objectSize = 44,
    m_objectConstructor = 0xb767fe00 <wxSocketEvent::wxCreateObject()>,
    m_baseInfo1 = 0xb7662920, m_baseInfo2 = 0x0, static sm_first = 0xb7b0a8cc,
    m_next = 0xb769254c, static sm_classTable = 0x8ea23e8}}
flag = <value optimized out>
#11 0xb7681e54 in wx_socket_callback (notification=GSOCK_OUTPUT,
    cdata=0x9b8fa90 "\b-M\b") at ../src/common/socket.cpp:942
No locals.
#12 0xb76866bb in GSocket::Detected_Write (this=0x9a49538)
    at ../src/unix/gsocket.cpp:1836
No locals.
#13 0xb77d0be7 in _GSocket_GDK_Input (data=0x9a49538, source=73,
    condition=GDK_INPUT_WRITE) at ../src/gtk/gsockgtk.cpp:36
No locals.
#14 0xb6e194af in gdk_io_invoke (source=0xb57303c0,
    condition=<value optimized out>, data=0xb5731150)
    at /build/buildd/gtk+2.0-2.14.4/gdk/gdkevents.c:1013
gdk_cond = GDK_INPUT_WRITE
#15 0xb6bb771d in g_io_unix_dispatch (source=0xb572faa0,
    callback=0xb6e19450 <gdk_io_invoke>, user_data=0xb5731150)
    at /build/buildd/glib2.0-2.18.2/glib/giounix.c:162
buffer_condition = 6
#16 0xb6b80718 in IA__g_main_context_dispatch (context=0x8ecf918)
---Type <return> to continue, or q <return> to quit---
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2144
No locals.
#17 0xb6b83dc3 in g_main_context_iterate (context=0x8ecf918, block=1,
    dispatch=1, self=0x8ed0ec0)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2778
max_priority = 110
timeout = 0
some_ready = 1
nfds = 11
allocated_nfds = <value optimized out>
fds = (GPollFD *) 0xb1f6b378
__PRETTY_FUNCTION__ = "g_main_context_iterate"
#18 0xb6b842e2 in IA__g_main_loop_run (loop=0x919d610)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2986
self = (GThread *) 0x8ed0ec0
__PRETTY_FUNCTION__ = "IA__g_main_loop_run"
#19 0xb6fb23a9 in IA__gtk_main ()
    at /build/buildd/gtk+2.0-2.14.4/gtk/gtkmain.c:1200
tmp_list = (GList *) 0x0
functions = (GList *) 0x0
init = (GtkInitFunction *) 0x96fbc68
loop = (GMainLoop *) 0x919d610
#20 0xb77cf10b in wxEventLoop::Run (this=0x96fbc68)
---Type <return> to continue, or q <return> to quit---
    at ../src/gtk/evtloop.cpp:76
exitcode = <value optimized out>
__FUNCTION__ = "Run"
#21 0xb78733dc in wxAppBase::MainLoop (this=0x8ecf7d0)
    at ../src/common/appcmn.cpp:312
mainLoop = {<wxEventLoopPtr> = {m_ptr = 0x96fbc68}, m_pp = 0x8ecf810,
  m_pOld = 0x0}
#22 0xb7873131 in wxAppBase::OnRun (this=0x6) at ../src/common/appcmn.cpp:367
No locals.
#23 0xb75683fa in wxEntry (argc=@0xb76606cc, argv=0x8ebde38)
    at ../src/common/init.cpp:460
No locals.
#24 0xb75684b7 in wxEntry (argc=@0xbfa78810, argv=0xbfa78894)
    at ../src/common/init.cpp:472
No locals.
#25 0x082ac21d in main (argc=Cannot access memory at address 0x19eb
) at ../../src/amule-gui.cpp:95
No locals.


[RRM]

thread apply all bt full

Code: [Select]

Thread 5 (Thread 0xb4684b90 (LWP 6653)):
#0  0xb8079430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb72588a0 in raise () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#2  0xb725a268 in abort () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#3  0x083e9e51 in OnUnhandledException ()
    at ../../../../src/libs/common/MuleDebug.cpp:103
t = (class std::type_info *) 0x86e5a6c
output = (FILE *) 0xb7387560
#4  0xb747b655 in __cxxabiv1::__terminate (
    handler=0x83e9ae0 <OnUnhandledException()>)
    at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:43
No locals.
#5  0xb747b692 in std::terminate ()
    at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:53
No locals.
#6  0xb747b7ca in __cxa_throw (obj=0xb74aa8f0, tinfo=0x86e5a6c,
    dest=0xb747bd00 <~bad_alloc>)
    at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:76
header = <value optimized out>
---Type <return> to continue, or q <return> to quit---
#7  0xb747be23 in operator new (sz=36)
    at ../../../../src/libstdc++-v3/libsupc++/new_op.cc:63
handler = <value optimized out>
p = <value optimized out>
#8  0x083e3555 in CTimerEvent::Clone (this=0xb46842ec)
    at ../../src/Timer.cpp:158
No locals.
#9  0xb75ddbb5 in wxEvtHandler::AddPendingEvent (this=0x8ecf7d0,
    event=@0xb46842ec) at ../src/common/event.cpp:1130
eventCopy = <value optimized out>
__FUNCTION__ = "AddPendingEvent"
#10 0x08190990 in wxPostEvent (dest=0x8ecf7d0, event=@0xb46842ec)
    at /usr/include/wx-2.8/wx/event.h:2565
__FUNCTION__ = "wxPostEvent"
#11 0x083e3a04 in CTimerThread::Entry (this=0x919d5c0)
    at ../../src/Timer.cpp:68
now = 1320027760
sinceLast = 0
timeout = 100
evt = {<wxEvent> = {<wxObject> = {_vptr.wxObject = 0x85c7608,
      static ms_classInfo = {m_className = 0xb760d1a4, m_objectSize = 8,
        m_objectConstructor = 0, m_baseInfo1 = 0x0, m_baseInfo2 = 0x0,
        static sm_first = 0xb7b0a8cc, m_next = 0xb76618c4,
---Type <return> to continue, or q <return> to quit---
        static sm_classTable = 0x8ea23e8}, m_refData = 0x0},
    m_eventObject = 0x0, m_eventType = 10244, m_timeStamp = 0, m_id = 6128,
    m_callbackUserData = 0x0, m_propagationLevel = 0, m_skipped = false,
    m_isCommandEvent = false, static ms_classInfo = {m_className = 0xb761e708,
      m_objectSize = 36, m_objectConstructor = 0, m_baseInfo1 = 0xb7661858,
      m_baseInfo2 = 0x0, static sm_first = 0xb7b0a8cc, m_next = 0xb7662908,
      static sm_classTable = 0x8ea23e8}}, <No data fields>}
lastEvent = 1320027860
#12 0xb75dc036 in wxThreadInternal::PthreadStart (thread=0x919d5c0)
    at ../src/unix/threadpsx.cpp:766
pthread = (wxThreadInternal *) 0x919d5e8
rc = <value optimized out>
dontRunAtAll = false
__FUNCTION__ = "PthreadStart"
#13 0xb75dc13d in wxPthreadStart (ptr=0x919d5c0)
    at ../src/unix/threadpsx.cpp:718
No locals.
#14 0xb803950f in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#15 0xb730ea0e in clone () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.

Thread 3 (Thread 0xb5686b90 (LWP 6651)):
---Type <return> to continue, or q <return> to quit---
#0  0xb8079430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb8040906 in nanosleep () from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#2  0xb75e3168 in wxMicroSleep (microseconds=0) at ../src/unix/utilsunx.cpp:191
tmReq = {tv_sec = 1, tv_nsec = 0}
#3  0xb75e3191 in wxMilliSleep (milliseconds=1000)
    at ../src/unix/utilsunx.cpp:212
No locals.
#4  0xb75d941d in wxThread::Sleep (milliseconds=1000)
    at ../src/unix/threadpsx.cpp:986
No locals.
#5  0x081fd133 in UploadBandwidthThrottler::Entry (this=0x9566bb8)
    at ../../src/UploadBandwidthThrottler.cpp:324
timeSinceLastLoop = 0
minFragSize = 1300
doubleSendSize = 2600
bytesToSpend = 2130432
sleepTime = 1000
thisLoopTick = 1320030875
TIME_BETWEEN_UPLOAD_LOOPS = 1
lastLoopTick = 1320030875
realBytesToSpend = 82432999
---Type <return> to continue, or q <return> to quit---
allowedDataRate = 2048000
rememberedSlotCounter = 84
extraSleepTime = 1000
sendLock = {m_isOk = false, m_mutex = @0x0}
#6  0xb75dc036 in wxThreadInternal::PthreadStart (thread=0x9566bb8)
    at ../src/unix/threadpsx.cpp:766
pthread = (wxThreadInternal *) 0x957aa28
rc = <value optimized out>
dontRunAtAll = false
__FUNCTION__ = "PthreadStart"
#7  0xb75dc13d in wxPthreadStart (ptr=0x9566bb8)
    at ../src/unix/threadpsx.cpp:718
No locals.
#8  0xb803950f in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#9  0xb730ea0e in clone () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.

Thread 1 (Thread 0xb67f5700 (LWP 6635)):
#0  0xb8079430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb72588a0 in raise () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#2  0xb725a268 in abort () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#3  0xb747b655 in __cxxabiv1::__terminate (handler=0x808a378 <abort@plt>)
    at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:43
No locals.
#4  0xb747b692 in std::terminate ()
    at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:53
No locals.
#5  0xb747b7ca in __cxa_throw (obj=0xb74aaaf0, tinfo=0x86e5a6c,
    dest=0xb747bd00 <~bad_alloc>)
    at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:76
header = <value optimized out>
#6  0xb747be23 in operator new (sz=24)
    at ../../../../src/libstdc++-v3/libsupc++/new_op.cc:63
handler = <value optimized out>
p = <value optimized out>
#7  0xb7560556 in wxObjectList::CreateNode (this=0x8ef76f8, prev=0x9b222b0,
    next=0x0, data=0x86e80c0, key=@0xb7660750) at ../include/wx/list.h:1178
No locals.
#8  0xb7575759 in wxListBase::Append (this=0x8ef76f8, object=0x86e80c0)
    at ../src/common/list.cpp:244
node = <value optimized out>
__FUNCTION__ = "Append"
---Type <return> to continue, or q <return> to quit---
#9  0xb75ddc26 in wxEvtHandler::AddPendingEvent (this=0x86e80c0,
    event=@0xbfa784a0) at ../include/wx/list.h:1178
eventCopy = <value optimized out>
__FUNCTION__ = "AddPendingEvent"
#10 0xb7681d58 in wxSocketBase::OnRequest (this=0x9b8fa90,
    notification=wxSOCKET_OUTPUT) at ../src/common/socket.cpp:1006
event = {<wxEvent> = {<wxObject> = {_vptr.wxObject = 0x86e1488,
      static ms_classInfo = {m_className = 0xb760d1a4, m_objectSize = 8,
        m_objectConstructor = 0, m_baseInfo1 = 0x0, m_baseInfo2 = 0x0,
        static sm_first = 0xb7b0a8cc, m_next = 0xb76618c4,
        static sm_classTable = 0x8ea23e8}, m_refData = 0x0},
    m_eventObject = 0x9b8fa90, m_eventType = 10002, m_timeStamp = 0,
    m_id = 6123, m_callbackUserData = 0x0, m_propagationLevel = 0,
    m_skipped = false, m_isCommandEvent = false, static ms_classInfo = {
      m_className = 0xb761e708, m_objectSize = 36, m_objectConstructor = 0,
      m_baseInfo1 = 0xb7661858, m_baseInfo2 = 0x0,
      static sm_first = 0xb7b0a8cc, m_next = 0xb7662908,
      static sm_classTable = 0x8ea23e8}}, m_event = wxSOCKET_OUTPUT,
  m_clientData = 0x0, static ms_classInfo = {m_className = 0xb7689008,
    m_objectSize = 44,
    m_objectConstructor = 0xb767fe00 <wxSocketEvent::wxCreateObject()>,
    m_baseInfo1 = 0xb7662920, m_baseInfo2 = 0x0, static sm_first = 0xb7b0a8cc,
    m_next = 0xb769254c, static sm_classTable = 0x8ea23e8}}
---Type <return> to continue, or q <return> to quit---
flag = <value optimized out>
#11 0xb7681e54 in wx_socket_callback (notification=GSOCK_OUTPUT,
    cdata=0x9b8fa90 "\b-M\b") at ../src/common/socket.cpp:942
No locals.
#12 0xb76866bb in GSocket::Detected_Write (this=0x9a49538)
    at ../src/unix/gsocket.cpp:1836
No locals.
#13 0xb77d0be7 in _GSocket_GDK_Input (data=0x9a49538, source=73,
    condition=GDK_INPUT_WRITE) at ../src/gtk/gsockgtk.cpp:36
No locals.
#14 0xb6e194af in gdk_io_invoke (source=0xb57303c0,
    condition=<value optimized out>, data=0xb5731150)
    at /build/buildd/gtk+2.0-2.14.4/gdk/gdkevents.c:1013
gdk_cond = GDK_INPUT_WRITE
#15 0xb6bb771d in g_io_unix_dispatch (source=0xb572faa0,
    callback=0xb6e19450 <gdk_io_invoke>, user_data=0xb5731150)
    at /build/buildd/glib2.0-2.18.2/glib/giounix.c:162
buffer_condition = 6
#16 0xb6b80718 in IA__g_main_context_dispatch (context=0x8ecf918)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2144
No locals.
#17 0xb6b83dc3 in g_main_context_iterate (context=0x8ecf918, block=1,
    dispatch=1, self=0x8ed0ec0)
---Type <return> to continue, or q <return> to quit---
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2778
max_priority = 110
timeout = 0
some_ready = 1
nfds = 11
allocated_nfds = <value optimized out>
fds = (GPollFD *) 0xb1f6b378
__PRETTY_FUNCTION__ = "g_main_context_iterate"
#18 0xb6b842e2 in IA__g_main_loop_run (loop=0x919d610)
    at /build/buildd/glib2.0-2.18.2/glib/gmain.c:2986
self = (GThread *) 0x8ed0ec0
__PRETTY_FUNCTION__ = "IA__g_main_loop_run"
#19 0xb6fb23a9 in IA__gtk_main ()
    at /build/buildd/gtk+2.0-2.14.4/gtk/gtkmain.c:1200
tmp_list = (GList *) 0x0
functions = (GList *) 0x0
init = (GtkInitFunction *) 0x96fbc68
loop = (GMainLoop *) 0x919d610
#20 0xb77cf10b in wxEventLoop::Run (this=0x96fbc68)
    at ../src/gtk/evtloop.cpp:76
exitcode = <value optimized out>
__FUNCTION__ = "Run"
#21 0xb78733dc in wxAppBase::MainLoop (this=0x8ecf7d0)
---Type <return> to continue, or q <return> to quit---
    at ../src/common/appcmn.cpp:312
mainLoop = {<wxEventLoopPtr> = {m_ptr = 0x96fbc68}, m_pp = 0x8ecf810,
  m_pOld = 0x0}
#22 0xb7873131 in wxAppBase::OnRun (this=0x6) at ../src/common/appcmn.cpp:367
No locals.
#23 0xb75683fa in wxEntry (argc=@0xb76606cc, argv=0x8ebde38)
    at ../src/common/init.cpp:460
No locals.
#24 0xb75684b7 in wxEntry (argc=@0xbfa78810, argv=0xbfa78894)
    at ../src/common/init.cpp:472
No locals.
#25 0x082ac21d in main (argc=Cannot access memory at address 0x19eb
) at ../../src/amule-gui.cpp:95
No locals.


[RRM]

No crash since last night (allowing only 80 Amule connections; torrents: 272 connections)
However, even now VSZ slowly but steadily grows:
ps axl | grep amule, with time

Code: [Select]

VSZ
133468 79096 sync_p Dl+  pts/0    497:10 /usr/bin/amule
138824 84292 poll   Sl+  pts/0    575:57 /usr/bin/amule
139072 84520 poll   Sl+  pts/0    589:16 /usr/bin/amule
140408 85756 poll   Sl+  pts/0    622:53 /usr/bin/amule
141416 86764 poll   Sl+  pts/0    648:21 /usr/bin/amule
149164 94596 -      Rl+  pts/0    857:55 /usr/bin/amule
157612 103428 poll  Sl+  pts/0    1165:56 /usr/bin/amule

So that eventually, amule will crash again.

[btkaos]

Im doing a little monitoring.

Good job RRM, thanks to your work it is clear what is going on. I'm working on a patch so we can check what internal structure of amule is wasting so much memory

I restarted the computer and then checked available memory and VSZ
with time (Amule: 250 connections; Torrents: 16= 301 connections):

$ free

Code: [Select]

           total       used       free     
Mem:       3111840     496256    2615584   = right before starting up Amule
....


Be careful, you have to check free output under +/- buffers sections (it is, not counting cache)

$ ps axl | grep amule

Code: [Select]

VSZ     RSS     WCHAN  STAT TTY    TIME COMMAND
....
164044 92316 -      Rl+  pts/1     24:59 /usr/bin/amule
178380 107064 futex_ Sl+ pts/1     48:17 /usr/bin/amule
739292 668084 -     Sl+  pts/1     63:34 /usr/bin/amule
3138656 2384740 sync_p Dl+ pts/1   67:37 /usr/bin/amule
3144228 2220900 ptrace Tl pts/1    67:38 /usr/bin/amule


• VSZ is over 1GB right from the start...
  and gradually continues to grow.
  After the sudden increase in VSZ (about 55 - 60 min), Amule
  is frozen, but has not crashed yet.
  At 67 min. and 37 secs. Amule crashes.
• Amule apparently keeps my system so busy that the bittorrent transmission client
  is still waiting to verify local data of some torrents 50 mins. after startup.

Note that you start with 100MiB of VSZ memory. This is a normal value. Then in min 60 amule start eating memory like crazy. Here is the bug. Now we should know where is amule throwing all the memory away.

What is your max clients in queue setting?

A side note about performance, Festor packages are compiled with debug in mind so they are lacking some important optimizations.

[RRM]

Thank You,

I'm working on a patch so we can check what internal structure of amule is wasting so much memory

Wow, thats interesting!
Im looking forward to having it checking my amule...

Be careful, you have to check free output under +/- buffers sections

Ah, ok, thanks. 

Note that you start with 100MiB of VSZ memory.

Ooops, yes, you are right; i misread that.
Right after starting up after another crash,
VSZ was even as low as 87280.

What is your max clients in queue setting?

5000. Shall i play with it to check whether it makes a difference?

A side note about performance, Festor packages are compiled with debug in mind so they are lacking some important optimizations.

Yes, I understood that our first priority is to be able to get to know whats going on.
If you cannot pinpoint the bug, you cannot fix it...

[RRM]

BTW, i guess its not always the oversized VSZ that makes my system crash,
because many times it crashes, the VSZ is just 170MB or so.

[btkaos]

BTW, i guess its not always the oversized VSZ that makes my system crash,
because many times it crashes, the VSZ is just 170MB or so.

Interesting. Could you post a backtrace of that situation?

The good backtraces we have from you show an out of memory condition. Make sure to check vsz after amule crashed and before exiting gdb.

Regards!

[RRM]

Interesting. Could you post a backtrace of that situation?

I already did: reply 90 + 91 and reply 100 + 101 on page 7 (I initially misread that the VSZ was 1.9GB).
But i will also post the backtrace of the next crash with such a low VSZ

Make sure to check vsz after amule crashed and before exiting gdb.

Yes, i did that. Thank you.

Regards!