Topic: RRM's epic struggle for a better aMule on high-speed connections (Read 165936 times)

RRM · « **Reply #195 on:** May 16, 2009, 08:02:25 AM »

Quote from: Stu Redman on May 14, 2009, 09:39:40 PM

So trying each new version in hope of improvement will not be usefull, sorry.

Ok, thank you for letting me know.

Quote

100 Megabyte/s UL bandwidth ?!? Are you kidding ?

Officially, my DL bandwidth is 'only' 120 MB/s and UL 10 MB/s.
However, this is a fairly new service with not too many customers, so the first users
have much more bandwidth in practise. (it will end soon, i guess

)
Trying to find out about when aMule starts crashing, i closely watched it for a few days
and witnessed uploads of well over 10 MB/s.
And, many times before it actually crashed, i witnessed uploads of 40 to 90 MB/s.

Stu Redman · « **Reply #196 on:** May 16, 2009, 12:54:15 PM »

I'm not sure the rate measurement works reliable at such speeds. 90MB/s, that would be close to the limits of harddisc and gigabit LAN.
How many active uploads in such moments?

RRM · « **Reply #197 on:** May 16, 2009, 07:14:53 PM »

Quote from: Stu Redman on May 16, 2009, 12:54:15 PM

I'm not sure the rate measurement works reliable at such speeds.

Yes, you might be right; amule is also 'in the freezing process' whenever the numbers are
(over 50 MB/s) that high; as if the results are added up, whenever it can still report 'something'.
(between frozen moments)

Quote

How many active uploads in such moments?

Something like 400 to 500.
The downloads that i can still see at that point (i cant scroll or change pages in that
condition) are often in the range of 100 - 300 kb/s (some much lower, some higher)

Most of the time (when im not home), i have the max. connections set at 50, to prevent crashes.
When the max. connections are set to 200, i dont get above 20 MB/s UL, but that still
(usually) results in high VSZ crashes.

RRM · « **Reply #198 on:** May 16, 2009, 07:18:59 PM »

Quote from: btkaos on May 14, 2009, 04:24:58 PM

Dear RRM, your assumptions are correct. However, I guess we have enough backtraces for now.

Perfect.
However, i just had a new type of crash that i never experienced before
and that i like to share with you (see next post).
It will be the last one.

Quote

I was going to have a look into this problem last week, but I stepped into

http://seagate.custkb.com/seagate/crm/selfservice/search.jsp?DocId=207931

Not funny

Ouch!!

Not funny at all, indeed...

Regards!

RRM · « **Reply #199 on:** May 16, 2009, 07:19:57 PM »

I did not see this (also high VSZ) crash before,
so forgive me for posting it...

Program received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 0xb65f6750 (LWP 5288)]
0xb7f22430 in __kernel_vsyscall ()

$ ps axl | grep amule

Code: [Select]

0  1000  4175  4158  20   0   3336   800 pipe_w S+   pts/1      0:00 grep amule
0  1000  5287  5270  20   0 245668 84912 poll   S+   pts/0      0:04 gdb amule
0  1000  5288  5287  20   0 3145724 2313580 ptrace Tl pts/0   495:55 /usr/bin/amule

(gdb) bt

Code: [Select]

#0  0xb7f22430 in __kernel_vsyscall ()
#1  0xb7f034b0 in raise (sig=5)
    at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
#2  0xb73dfb8e in wxTrap () at ../src/common/appbase.cpp:674
#3  0xb73e0a13 in wxOnAssert (szFile=0xb74d0e4c, nLine=1097, 
    szFunc=0xb74d2677 "UngetWriteBuf", szCond=0xb74d0de0, szMsg=0xb74d0da4)
    at ../src/common/appbase.cpp:694
#4  0xb744df63 in wxString::UngetWriteBuf (this=0xbff3d628, nLen=0)
    at ../src/common/string.cpp:1097
#5  0xb744f146 in wxString::UngetWriteBuf (this=0xbff3d628)
    at ../src/common/string.cpp:1090
#6  0xb7453423 in wxString::PrintfV (this=0xbff3d628, pszFormat=0xb74b3be0, 
    argptr=0xbff3d608 "L\016M�I\004") at ../include/wx/string.h:1510
#7  0xb74534db in wxString::Printf (this=0xbff3d628, pszFormat=0xb74b3be0)
    at ../src/common/string.cpp:1826
#8  0xb73e0092 in ShowAssertDialog (szFile=0xb74d0e4c, nLine=1097, 
    szFunc=0xcdc943c, szCond=0xb74d0de0, szMsg=0xb74d0da4, traits=0x992cc50)
    at ../src/common/appbase.cpp:791
#9  0xb7672409 in wxApp::OnAssertFailure (this=0x98dbe40, file=0xb74d0e4c, 
    line=1097, func=0xcdc943c, cond=0xb74d0de0, msg=0xb74d0da4)
    at ../src/gtk/app.cpp:556
#10 0x08096bb2 in CamuleApp::OnAssertFailure (this=0x98dbe40, file=0xb74d0e4c, 
    line=1097, func=0xcdc943c, cond=0xb74d0de0, msg=0xb74d0da4)
---Type <return> to continue, or q <return> to quit---
    at ../../src/amule.cpp:1385
#11 0xb73e0a8b in wxOnAssert (szFile=0xb74d0e4c, nLine=1097, 
    szFunc=0xb74d2677 "UngetWriteBuf", szCond=0xb74d0de0, szMsg=0xb74d0da4)
    at ../src/common/appbase.cpp:711
#12 0xb744df63 in wxString::UngetWriteBuf (this=0xbff3d834, nLen=0)
    at ../src/common/string.cpp:1097
#13 0xb744f146 in wxString::UngetWriteBuf (this=0xbff3d834)
    at ../src/common/string.cpp:1090
#14 0xb7453423 in wxString::PrintfV (this=0xbff3d834, pszFormat=0x9a34f64, 
    argptr=0xbff3d7c8 "|\003") at ../include/wx/string.h:1510
#15 0xb745358d in wxString::Format (pszFormat=0x9a34f64)
    at ../src/common/string.cpp:1806
#16 0x082eb5d5 in CKadDlg::UpdateGraph (this=0xa11c1d8, update=@0xbff3d890)
    at ../../src/KadDlg.cpp:132
#17 0x082d5dfd in CamuleDlg::OnGUITimer (this=0x9eac460)
    at ../../src/amuleDlg.cpp:1087
#18 0xb73dfa81 in wxAppConsole::HandleEvent (this=0x98dbe40, 
    handler=0x9eac460, func=0x14a8, event=@0xbff3d9b4)
    at ../src/common/appbase.cpp:322
#19 0xb749da6a in wxEvtHandler::ProcessEventIfMatches (entry=@0x890f378, 
    handler=0x9eac460, event=@0xbff3d9b4) at ../src/common/event.cpp:1231
#20 0xb749f1f4 in wxEventHashTable::HandleEvent (this=0x890f220, 
    event=@0xbff3d9b4, self=0x9eac460) at ../src/common/event.cpp:906
---Type <return> to continue, or q <return> to quit---
#21 0xb749f346 in wxEvtHandler::ProcessEvent (this=0x9eac460, 
    event=@0xbff3d9b4) at ../src/common/event.cpp:1293
#22 0xb77c25a7 in wxTimerBase::Notify (this=0xa2238a8)
    at ../src/common/timercmn.cpp:57
#23 0xb769950d in timeout_callback (data=0xa2238a8) at ../src/gtk/timer.cpp:43
#24 0xb6a1e2b6 in g_timeout_dispatch (source=0x992bc00, callback=0, 
    user_data=0xa2238a8) at /build/buildd/glib2.0-2.20.1/glib/gmain.c:3253
#25 0xb6a1db88 in IA__g_main_context_dispatch (context=0x98dbff8)
    at /build/buildd/glib2.0-2.20.1/glib/gmain.c:1814
#26 0xb6a210eb in g_main_context_iterate (context=0x98dbff8, block=1, 
    dispatch=1, self=0x98dd570)
    at /build/buildd/glib2.0-2.20.1/glib/gmain.c:2448
#27 0xb6a215ba in IA__g_main_loop_run (loop=0xb404268)
    at /build/buildd/glib2.0-2.20.1/glib/gmain.c:2656
#28 0xb6e617d9 in IA__gtk_main ()
    at /build/buildd/gtk+2.0-2.16.1/gtk/gtkmain.c:1205
#29 0xb768f41b in wxEventLoop::Run (this=0xb3be0f0)
    at ../src/gtk/evtloop.cpp:76
#30 0xb7733b3c in wxAppBase::MainLoop (this=0x98dbe40)
    at ../src/common/appcmn.cpp:312
#31 0xb7733891 in wxAppBase::OnRun (this=0x5) at ../src/common/appcmn.cpp:367
#32 0xb74277ca in wxEntry (argc=@0xb75206cc, argv=0x98ca480)
    at ../src/common/init.cpp:460
---Type <return> to continue, or q <return> to quit---
#33 0xb7427887 in wxEntry (argc=@0xbff3dca0, argv=0xbff3dd24)
    at ../src/common/init.cpp:472
#34 0x082cba50 in main (argc=0, argv=0x0) at ../../src/amule-gui.cpp:94

(gdb) bt full

Code: [Select]

#0  0xb7f22430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7f034b0 in raise (sig=5)
    at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
	resultvar = 5
#2  0xb73dfb8e in wxTrap () at ../src/common/appbase.cpp:674
No locals.
#3  0xb73e0a13 in wxOnAssert (szFile=0xb74d0e4c, nLine=1097, 
    szFunc=0xb74d2677 "UngetWriteBuf", szCond=0xb74d0de0, szMsg=0xb74d0da4)
    at ../src/common/appbase.cpp:694
	guard = {m_flag = @0xb751a1f0, m_isInside = true}
	strFunc = {<wxStringBase> = {static npos = 4294967295, 
    m_pchData = 0xbff3d578}, <No data fields>}
	s_bInAssert = 2
#4  0xb744df63 in wxString::UngetWriteBuf (this=0xbff3d628, nLen=0)
    at ../src/common/string.cpp:1097
	pData = (wxStringData * const) 0xb74d27a8
	__FUNCTION__ = "UngetWriteBuf"
#5  0xb744f146 in wxString::UngetWriteBuf (this=0xbff3d628)
    at ../src/common/string.cpp:1090
No locals.
#6  0xb7453423 in wxString::PrintfV (this=0xbff3d628, pszFormat=0xb74b3be0, 
    argptr=0xbff3d608 "L\016M�I\004") at ../include/wx/string.h:1510
---Type <return> to continue, or q <return> to quit---
	len = <value optimized out>
	size = 1024
#7  0xb74534db in wxString::Printf (this=0xbff3d628, pszFormat=0xb74b3be0)
    at ../src/common/string.cpp:1826
	iLen = 0
#8  0xb73e0092 in ShowAssertDialog (szFile=0xb74d0e4c, nLine=1097, 
    szFunc=0xcdc943c, szCond=0xb74d0de0, szMsg=0xb74d0da4, traits=0x992cc50)
    at ../src/common/appbase.cpp:791
	msg = {<wxStringBase> = {static npos = 4294967295, 
    m_pchData = 0xb74d27b4}, <No data fields>}
	s_bNoAsserts = false
#9  0xb7672409 in wxApp::OnAssertFailure (this=0x98dbe40, file=0xb74d0e4c, 
    line=1097, func=0xcdc943c, cond=0xb74d0de0, msg=0xb74d0da4)
    at ../src/gtk/app.cpp:556
No locals.
#10 0x08096bb2 in CamuleApp::OnAssertFailure (this=0x98dbe40, file=0xb74d0e4c, 
    line=1097, func=0xcdc943c, cond=0xb74d0de0, msg=0xb74d0da4)
    at ../../src/amule.cpp:1385
No locals.
#11 0xb73e0a8b in wxOnAssert (szFile=0xb74d0e4c, nLine=1097, 
    szFunc=0xb74d2677 "UngetWriteBuf", szCond=0xb74d0de0, szMsg=0xb74d0da4)
    at ../src/common/appbase.cpp:711
	guard = {m_flag = @0xb751a1f0, m_isInside = false}
---Type <return> to continue, or q <return> to quit---
	strFunc = {<wxStringBase> = {static npos = 4294967295, 
    m_pchData = 0xcdc943c}, <No data fields>}
	s_bInAssert = 2
#12 0xb744df63 in wxString::UngetWriteBuf (this=0xbff3d834, nLen=0)
    at ../src/common/string.cpp:1097
	pData = (wxStringData * const) 0xb74d27a8
	__FUNCTION__ = "UngetWriteBuf"
#13 0xb744f146 in wxString::UngetWriteBuf (this=0xbff3d834)
    at ../src/common/string.cpp:1090
No locals.
#14 0xb7453423 in wxString::PrintfV (this=0xbff3d834, pszFormat=0x9a34f64, 
    argptr=0xbff3d7c8 "|\003") at ../include/wx/string.h:1510
	len = <value optimized out>
	size = 1024
#15 0xb745358d in wxString::Format (pszFormat=0x9a34f64)
    at ../src/common/string.cpp:1806
	s = {<wxStringBase> = {static npos = 4294967295, 
    m_pchData = 0xbff3d834}, <No data fields>}
#16 0x082eb5d5 in CKadDlg::UpdateGraph (this=0xa11c1d8, update=@0xbff3d890)
    at ../../src/KadDlg.cpp:132
	v = {<std::__norm::vector<float*, std::allocator<float*> >> = {<std::__norm::_Vector_base<float*, std::allocator<float*> >> = {
      _M_impl = {<std::allocator<float*>> = {<__gnu_cxx::new_allocator<float*>> ---Type <return> to continue, or q <return> to quit---
= {<No data fields>}, <No data fields>}, _M_start = 0xb3650648, 
        _M_finish = 0xb3650654, 
        _M_end_of_storage = 0xb3650654}}, <No data fields>}, <__gnu_debug::_Safe_sequence<std::__debug::vector<float*, std::allocator<float*> > >> = {<__gnu_debug::_Safe_sequence_base> = {_M_iterators = 0x0, _M_const_iterators = 0x0, 
      _M_version = 1}, <No data fields>}, _M_guaranteed_capacity = 3}
	apfKad = (
    const class std::__debug::vector<float*, std::allocator<float*> >
     &) @0xbff3d818: {<std::__norm::vector<float*, std::allocator<float*> >> = {<std::__norm::_Vector_base<float*, std::allocator<float*> >> = {
      _M_impl = {<std::allocator<float*>> = {<__gnu_cxx::new_allocator<float*>> = {<No data fields>}, <No data fields>}, _M_start = 0xb3650648, 
        _M_finish = 0xb3650654, 
        _M_end_of_storage = 0xb3650654}}, <No data fields>}, <__gnu_debug::_Safe_sequence<std::__debug::vector<float*, std::allocator<float*> > >> = {<__gnu_debug::_Safe_sequence_base> = {_M_iterators = 0x0, _M_const_iterators = 0x0, 
      _M_version = 1}, <No data fields>}, _M_guaranteed_capacity = 3}
	nodeCount = 892
	label = (class wxStaticText *) 0xa11e660
	__FUNCTION__ = "UpdateGraph"
#17 0x082d5dfd in CamuleDlg::OnGUITimer (this=0x9eac460)
    at ../../src/amuleDlg.cpp:1087
	update = {timestamp = 140643.049, downloads = {10.7057018, 4.7475009, 
---Type <return> to continue, or q <return> to quit---
    4.52141333}, uploads = {339.60022, 268.887817, 262.159088}, connections = {
    37, 51, 3}, kadnodes = {846.082214, 886.651245, 892}}
	msCur = 140921301
	msGraphUpdate = 10000
	sStatsUpdate = -1230778926
	msPrev1 = 140919707
	msPrev5 = 140919904
	msPrevStats = 0
	msPrevGraph = 140921301
#18 0xb73dfa81 in wxAppConsole::HandleEvent (this=0x98dbe40, 
    handler=0x9eac460, func=0x14a8, event=@0xbff3d9b4)
    at ../src/common/appbase.cpp:322
No locals.
#19 0xb749da6a in wxEvtHandler::ProcessEventIfMatches (entry=@0x890f378, 
    handler=0x9eac460, event=@0xbff3d9b4) at ../src/common/event.cpp:1231
	tableId1 = <value optimized out>
	tableId2 = <value optimized out>
#20 0xb749f1f4 in wxEventHashTable::HandleEvent (this=0x890f220, 
    event=@0xbff3d9b4, self=0x9eac460) at ../src/common/event.cpp:906
	n = 0
	count = 1
	eventType = 10096
	eTTnode = (wxEventHashTable::EventTypeTable * const) 0xa11acf8
---Type <return> to continue, or q <return> to quit---
#21 0xb749f346 in wxEvtHandler::ProcessEvent (this=0x9eac460, 
    event=@0xbff3d9b4) at ../src/common/event.cpp:1293
	__FUNCTION__ = "ProcessEvent"
#22 0xb77c25a7 in wxTimerBase::Notify (this=0xa2238a8)
    at ../src/common/timercmn.cpp:57
	event = {<wxEvent> = {<wxObject> = {_vptr.wxObject = 0x8725be8, 
      static ms_classInfo = {m_className = 0xb74cd6c4, m_objectSize = 8, 
        m_objectConstructor = 0, m_baseInfo1 = 0x0, m_baseInfo2 = 0x0, 
        static sm_first = 0xb79cb8cc, m_next = 0xb75218c4, 
        static sm_classTable = 0x98ae3e8}, m_refData = 0x0}, 
    m_eventObject = 0xa2238a8, m_eventType = 10096, m_timeStamp = 0, 
    m_id = 6129, m_callbackUserData = 0x0, m_propagationLevel = 0, 
    m_skipped = false, m_isCommandEvent = false, static ms_classInfo = {
      m_className = 0xb74deef0, m_objectSize = 36, m_objectConstructor = 0, 
      m_baseInfo1 = 0xb7521858, m_baseInfo2 = 0x0, 
      static sm_first = 0xb79cb8cc, m_next = 0xb7522908, 
      static sm_classTable = 0x98ae3e8}}, m_interval = 100, 
  static ms_classInfo = {m_className = 0xb785c89c, m_objectSize = 40, 
    m_objectConstructor = 0xb77c2400 <wxTimerEvent::wxCreateObject()>, 
    m_baseInfo1 = 0xb7522920, m_baseInfo2 = 0x0, static sm_first = 0xb79cb8cc, 
    m_next = 0xb78fee48, static sm_classTable = 0x98ae3e8}}
	__FUNCTION__ = "Notify"
#23 0xb769950d in timeout_callback (data=0xa2238a8) at ../src/gtk/timer.cpp:43
---Type <return> to continue, or q <return> to quit---
No locals.
#24 0xb6a1e2b6 in g_timeout_dispatch (source=0x992bc00, callback=0, 
    user_data=0xa2238a8) at /build/buildd/glib2.0-2.20.1/glib/gmain.c:3253
No locals.
#25 0xb6a1db88 in IA__g_main_context_dispatch (context=0x98dbff8)
    at /build/buildd/glib2.0-2.20.1/glib/gmain.c:1814
No locals.
#26 0xb6a210eb in g_main_context_iterate (context=0x98dbff8, block=1, 
    dispatch=1, self=0x98dd570)
    at /build/buildd/glib2.0-2.20.1/glib/gmain.c:2448
	max_priority = 0
	timeout = 0
	some_ready = 1
	nfds = 11
	allocated_nfds = <value optimized out>
	fds = (GPollFD *) 0xa689f68
	__PRETTY_FUNCTION__ = "g_main_context_iterate"
#27 0xb6a215ba in IA__g_main_loop_run (loop=0xb404268)
    at /build/buildd/glib2.0-2.20.1/glib/gmain.c:2656
	self = (GThread *) 0x98dd570
	__PRETTY_FUNCTION__ = "IA__g_main_loop_run"
#28 0xb6e617d9 in IA__gtk_main ()
    at /build/buildd/gtk+2.0-2.16.1/gtk/gtkmain.c:1205
---Type <return> to continue, or q <return> to quit---
	tmp_list = (GList *) 0x0
	functions = (GList *) 0x0
	init = (GtkInitFunction *) 0xb3be0f0
	loop = (GMainLoop *) 0xb404268
#29 0xb768f41b in wxEventLoop::Run (this=0xb3be0f0)
    at ../src/gtk/evtloop.cpp:76
	exitcode = <value optimized out>
	__FUNCTION__ = "Run"
#30 0xb7733b3c in wxAppBase::MainLoop (this=0x98dbe40)
    at ../src/common/appcmn.cpp:312
	mainLoop = {<wxEventLoopPtr> = {m_ptr = 0xb3be0f0}, m_pp = 0x98dbe80, 
  m_pOld = 0x0}
#31 0xb7733891 in wxAppBase::OnRun (this=0x5) at ../src/common/appcmn.cpp:367
No locals.
#32 0xb74277ca in wxEntry (argc=@0xb75206cc, argv=0x98ca480)
    at ../src/common/init.cpp:460
No locals.
#33 0xb7427887 in wxEntry (argc=@0xbff3dca0, argv=0xbff3dd24)
    at ../src/common/init.cpp:472
No locals.
#34 0x082cba50 in main (argc=0, argv=0x0) at ../../src/amule-gui.cpp:94
No locals.

RRM · « **Reply #200 on:** May 16, 2009, 07:20:39 PM »

(gdb) thread apply all bt full

Code: [Select]

Thread 5 (Thread 0xb44f0b90 (LWP 5550)):
#0  0xb7f22430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7eff412 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_timedwait.S:179
No locals.
#2  0xb7499b51 in wxConditionInternal::WaitTimeout (this=0x9967ed8, 
    milliseconds=100) at ../src/unix/threadpsx.cpp:405
	tspec = {tv_sec = 1242473334, tv_nsec = 911000000}
	err = <value optimized out>
#3  0xb7499bda in wxCondition::WaitTimeout (this=0xb3c05ac, milliseconds=100)
    at ../include/wx/thrimpl.cpp:256
	__FUNCTION__ = "WaitTimeout"
#4  0xb749b994 in wxSemaphoreInternal::WaitTimeout (this=0xb3c05a8, 
    milliseconds=100) at ../src/unix/threadpsx.cpp:552
	remainingTime = -516
	locker = {m_isOk = true, m_mutex = @0xb3c05a8}
#5  0xb749ba1a in wxSemaphore::WaitTimeout (this=0xb3fb780, milliseconds=100)
    at ../include/wx/thrimpl.cpp:320
	__FUNCTION__ = "WaitTimeout"
#6  0x0841692c in CTimerThread::Entry (this=0xb3fb760)
---Type <return> to continue, or q <return> to quit---
    at ../../src/Timer.cpp:64
	now = 1227786267
	sinceLast = 0
	timeout = 100
	evt = {<wxEvent> = {<wxObject> = {_vptr.wxObject = 0x85fd8e8, 
      static ms_classInfo = {m_className = 0xb74cd6c4, m_objectSize = 8, 
        m_objectConstructor = 0, m_baseInfo1 = 0x0, m_baseInfo2 = 0x0, 
        static sm_first = 0xb79cb8cc, m_next = 0xb75218c4, 
        static sm_classTable = 0x98ae3e8}, m_refData = 0x0}, 
    m_eventObject = 0x0, m_eventType = 10244, m_timeStamp = 0, m_id = 6128, 
    m_callbackUserData = 0x0, m_propagationLevel = 0, m_skipped = false, 
    m_isCommandEvent = false, static ms_classInfo = {m_className = 0xb74deef0, 
      m_objectSize = 36, m_objectConstructor = 0, m_baseInfo1 = 0xb7521858, 
      m_baseInfo2 = 0x0, static sm_first = 0xb79cb8cc, m_next = 0xb7522908, 
      static sm_classTable = 0x98ae3e8}}, <No data fields>}
	lastEvent = 1227786267
#7  0xb749c4a6 in wxThreadInternal::PthreadStart (thread=0xb3fb760)
    at ../src/unix/threadpsx.cpp:766
	pthread = (wxThreadInternal *) 0x99767c0
	rc = <value optimized out>
	dontRunAtAll = false
	__FUNCTION__ = "PthreadStart"
#8  0xb749c5ad in wxPthreadStart (ptr=0xb3fb760)
---Type <return> to continue, or q <return> to quit---
    at ../src/unix/threadpsx.cpp:718
No locals.
#9  0xb7efb4ff in start_thread (arg=0xb44f0b90) at pthread_create.c:297
	__res = <value optimized out>
	__ignore1 = <value optimized out>
	__ignore2 = <value optimized out>
	pd = (struct pthread *) 0xb44f0b90
	now = <value optimized out>
	unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1208963084, 0, 0, 
        -1269890008, 596454441, 1658991150}, mask_was_saved = 0}}, priv = {
    pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, 
      canceltype = 0}}}
	not_first_call = <value optimized out>
	robust = <value optimized out>
#10 0xb71ca49e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
No locals.

Thread 3 (Thread 0xb54f2b90 (LWP 5429)):
#0  0xb7f22430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7f028f6 in nanosleep () from /lib/tls/i686/cmov/libpthread.so.0
No locals.
#2  0xb74a35d8 in wxMicroSleep (microseconds=0) at ../src/unix/utilsunx.cpp:191
---Type <return> to continue, or q <return> to quit---
	tmReq = {tv_sec = 1, tv_nsec = 0}
#3  0xb74a3601 in wxMilliSleep (milliseconds=1000)
    at ../src/unix/utilsunx.cpp:212
No locals.
#4  0xb749988d in wxThread::Sleep (milliseconds=1000)
    at ../src/unix/threadpsx.cpp:986
No locals.
#5  0x0821dcce in UploadBandwidthThrottler::Entry (this=0xa11ea28)
    at ../../src/UploadBandwidthThrottler.cpp:320
	timeSinceLastLoop = 0
	minFragSize = 1300
	doubleSendSize = 2600
	bytesToSpend = 1672082
	sleepTime = 1000
	thisLoopTick = 1227786318
	TIME_BETWEEN_UPLOAD_LOOPS = 1
	lastLoopTick = 1227786318
	realBytesToSpend = 22016999
	allowedDataRate = 10240000
	rememberedSlotCounter = 28
	extraSleepTime = 1000
	sendLock = {m_isOk = false, m_mutex = @0xb7f0aff4}
#6  0xb749c4a6 in wxThreadInternal::PthreadStart (thread=0xa11ea28)
---Type <return> to continue, or q <return> to quit---
    at ../src/unix/threadpsx.cpp:766
	pthread = (wxThreadInternal *) 0xa123a48
	rc = <value optimized out>
	dontRunAtAll = false
	__FUNCTION__ = "PthreadStart"
#7  0xb749c5ad in wxPthreadStart (ptr=0xa11ea28)
    at ../src/unix/threadpsx.cpp:718
No locals.
#8  0xb7efb4ff in start_thread (arg=0xb54f2b90) at pthread_create.c:297
	__res = <value optimized out>
	__ignore1 = <value optimized out>
	__ignore2 = <value optimized out>
	pd = (struct pthread *) 0xb54f2b90
	now = <value optimized out>
	unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1208963084, 0, 0, 
        -1253104600, 600648747, 1658991150}, mask_was_saved = 0}}, priv = {
    pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, 
      canceltype = 0}}}
	not_first_call = <value optimized out>
	robust = <value optimized out>
#9  0xb71ca49e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
No locals.

---Type <return> to continue, or q <return> to quit---
Thread 1 (Thread 0xb65f6750 (LWP 5288)):
#0  0xb7f22430 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7f034b0 in raise (sig=5)
    at ../nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
	resultvar = 5
#2  0xb73dfb8e in wxTrap () at ../src/common/appbase.cpp:674
No locals.
#3  0xb73e0a13 in wxOnAssert (szFile=0xb74d0e4c, nLine=1097, 
    szFunc=0xb74d2677 "UngetWriteBuf", szCond=0xb74d0de0, szMsg=0xb74d0da4)
    at ../src/common/appbase.cpp:694
	guard = {m_flag = @0xb751a1f0, m_isInside = true}
	strFunc = {<wxStringBase> = {static npos = 4294967295, 
    m_pchData = 0xbff3d578}, <No data fields>}
	s_bInAssert = 2
#4  0xb744df63 in wxString::UngetWriteBuf (this=0xbff3d628, nLen=0)
    at ../src/common/string.cpp:1097
	pData = (wxStringData * const) 0xb74d27a8
	__FUNCTION__ = "UngetWriteBuf"
#5  0xb744f146 in wxString::UngetWriteBuf (this=0xbff3d628)
    at ../src/common/string.cpp:1090
No locals.
#6  0xb7453423 in wxString::PrintfV (this=0xbff3d628, pszFormat=0xb74b3be0, 
---Type <return> to continue, or q <return> to quit---
    argptr=0xbff3d608 "L\016M�I\004") at ../include/wx/string.h:1510
	len = <value optimized out>
	size = 1024
#7  0xb74534db in wxString::Printf (this=0xbff3d628, pszFormat=0xb74b3be0)
    at ../src/common/string.cpp:1826
	iLen = 0
#8  0xb73e0092 in ShowAssertDialog (szFile=0xb74d0e4c, nLine=1097, 
    szFunc=0xcdc943c, szCond=0xb74d0de0, szMsg=0xb74d0da4, traits=0x992cc50)
    at ../src/common/appbase.cpp:791
	msg = {<wxStringBase> = {static npos = 4294967295, 
    m_pchData = 0xb74d27b4}, <No data fields>}
	s_bNoAsserts = false
#9  0xb7672409 in wxApp::OnAssertFailure (this=0x98dbe40, file=0xb74d0e4c, 
    line=1097, func=0xcdc943c, cond=0xb74d0de0, msg=0xb74d0da4)
    at ../src/gtk/app.cpp:556
No locals.
#10 0x08096bb2 in CamuleApp::OnAssertFailure (this=0x98dbe40, file=0xb74d0e4c, 
    line=1097, func=0xcdc943c, cond=0xb74d0de0, msg=0xb74d0da4)
    at ../../src/amule.cpp:1385
No locals.
#11 0xb73e0a8b in wxOnAssert (szFile=0xb74d0e4c, nLine=1097, 
    szFunc=0xb74d2677 "UngetWriteBuf", szCond=0xb74d0de0, szMsg=0xb74d0da4)
    at ../src/common/appbase.cpp:711
---Type <return> to continue, or q <return> to quit---
	guard = {m_flag = @0xb751a1f0, m_isInside = false}
	strFunc = {<wxStringBase> = {static npos = 4294967295, 
    m_pchData = 0xcdc943c}, <No data fields>}
	s_bInAssert = 2
#12 0xb744df63 in wxString::UngetWriteBuf (this=0xbff3d834, nLen=0)
    at ../src/common/string.cpp:1097
	pData = (wxStringData * const) 0xb74d27a8
	__FUNCTION__ = "UngetWriteBuf"
#13 0xb744f146 in wxString::UngetWriteBuf (this=0xbff3d834)
    at ../src/common/string.cpp:1090
No locals.
#14 0xb7453423 in wxString::PrintfV (this=0xbff3d834, pszFormat=0x9a34f64, 
    argptr=0xbff3d7c8 "|\003") at ../include/wx/string.h:1510
	len = <value optimized out>
	size = 1024
#15 0xb745358d in wxString::Format (pszFormat=0x9a34f64)
    at ../src/common/string.cpp:1806
	s = {<wxStringBase> = {static npos = 4294967295, 
    m_pchData = 0xbff3d834}, <No data fields>}
#16 0x082eb5d5 in CKadDlg::UpdateGraph (this=0xa11c1d8, update=@0xbff3d890)
    at ../../src/KadDlg.cpp:132
	v = {<std::__norm::vector<float*, std::allocator<float*> >> = {<std::__norm::_Vector_base<float*, std::allocator<float*> >> = {
---Type <return> to continue, or q <return> to quit---
      _M_impl = {<std::allocator<float*>> = {<__gnu_cxx::new_allocator<float*>> = {<No data fields>}, <No data fields>}, _M_start = 0xb3650648, 
        _M_finish = 0xb3650654, 
        _M_end_of_storage = 0xb3650654}}, <No data fields>}, <__gnu_debug::_Safe_sequence<std::__debug::vector<float*, std::allocator<float*> > >> = {<__gnu_debug::_Safe_sequence_base> = {_M_iterators = 0x0, _M_const_iterators = 0x0, 
      _M_version = 1}, <No data fields>}, _M_guaranteed_capacity = 3}
	apfKad = (
    const std::__debug::vector<float*, std::allocator<float*> > &) @0xbff3d818: {<std::__norm::vector<float*, std::allocator<float*> >> = {<std::__norm::_Vector_base<float*, std::allocator<float*> >> = {
      _M_impl = {<std::allocator<float*>> = {<__gnu_cxx::new_allocator<float*>> = {<No data fields>}, <No data fields>}, _M_start = 0xb3650648, 
        _M_finish = 0xb3650654, 
        _M_end_of_storage = 0xb3650654}}, <No data fields>}, <__gnu_debug::_Safe_sequence<std::__debug::vector<float*, std::allocator<float*> > >> = {<__gnu_debug::_Safe_sequence_base> = {_M_iterators = 0x0, _M_const_iterators = 0x0, 
      _M_version = 1}, <No data fields>}, _M_guaranteed_capacity = 3}
	nodeCount = 892
	label = (class wxStaticText *) 0xa11e660
	__FUNCTION__ = "UpdateGraph"
#17 0x082d5dfd in CamuleDlg::OnGUITimer (this=0x9eac460)
    at ../../src/amuleDlg.cpp:1087
---Type <return> to continue, or q <return> to quit---
	update = {timestamp = 140643.049, downloads = {10.7057018, 4.7475009, 
    4.52141333}, uploads = {339.60022, 268.887817, 262.159088}, connections = {
    37, 51, 3}, kadnodes = {846.082214, 886.651245, 892}}
	msCur = 140921301
	msGraphUpdate = 10000
	sStatsUpdate = -1230778926
	msPrev1 = 140919707
	msPrev5 = 140919904
	msPrevStats = 0
	msPrevGraph = 140921301
#18 0xb73dfa81 in wxAppConsole::HandleEvent (this=0x98dbe40, 
    handler=0x9eac460, func=0x14a8, event=@0xbff3d9b4)
    at ../src/common/appbase.cpp:322
No locals.
#19 0xb749da6a in wxEvtHandler::ProcessEventIfMatches (entry=@0x890f378, 
    handler=0x9eac460, event=@0xbff3d9b4) at ../src/common/event.cpp:1231
	tableId1 = <value optimized out>
	tableId2 = <value optimized out>
#20 0xb749f1f4 in wxEventHashTable::HandleEvent (this=0x890f220, 
    event=@0xbff3d9b4, self=0x9eac460) at ../src/common/event.cpp:906
	n = 0
	count = 1
	eventType = 10096
---Type <return> to continue, or q <return> to quit---
	eTTnode = (wxEventHashTable::EventTypeTable * const) 0xa11acf8
#21 0xb749f346 in wxEvtHandler::ProcessEvent (this=0x9eac460, 
    event=@0xbff3d9b4) at ../src/common/event.cpp:1293
	__FUNCTION__ = "ProcessEvent"
#22 0xb77c25a7 in wxTimerBase::Notify (this=0xa2238a8)
    at ../src/common/timercmn.cpp:57
	event = {<wxEvent> = {<wxObject> = {_vptr.wxObject = 0x8725be8, 
      static ms_classInfo = {m_className = 0xb74cd6c4, m_objectSize = 8, 
        m_objectConstructor = 0, m_baseInfo1 = 0x0, m_baseInfo2 = 0x0, 
        static sm_first = 0xb79cb8cc, m_next = 0xb75218c4, 
        static sm_classTable = 0x98ae3e8}, m_refData = 0x0}, 
    m_eventObject = 0xa2238a8, m_eventType = 10096, m_timeStamp = 0, 
    m_id = 6129, m_callbackUserData = 0x0, m_propagationLevel = 0, 
    m_skipped = false, m_isCommandEvent = false, static ms_classInfo = {
      m_className = 0xb74deef0, m_objectSize = 36, m_objectConstructor = 0, 
      m_baseInfo1 = 0xb7521858, m_baseInfo2 = 0x0, 
      static sm_first = 0xb79cb8cc, m_next = 0xb7522908, 
      static sm_classTable = 0x98ae3e8}}, m_interval = 100, 
  static ms_classInfo = {m_className = 0xb785c89c, m_objectSize = 40, 
    m_objectConstructor = 0xb77c2400 <wxTimerEvent::wxCreateObject()>, 
    m_baseInfo1 = 0xb7522920, m_baseInfo2 = 0x0, static sm_first = 0xb79cb8cc, 
    m_next = 0xb78fee48, static sm_classTable = 0x98ae3e8}}
	__FUNCTION__ = "Notify"
---Type <return> to continue, or q <return> to quit---
#23 0xb769950d in timeout_callback (data=0xa2238a8) at ../src/gtk/timer.cpp:43
No locals.
#24 0xb6a1e2b6 in g_timeout_dispatch (source=0x992bc00, callback=0, 
    user_data=0xa2238a8) at /build/buildd/glib2.0-2.20.1/glib/gmain.c:3253
No locals.
#25 0xb6a1db88 in IA__g_main_context_dispatch (context=0x98dbff8)
    at /build/buildd/glib2.0-2.20.1/glib/gmain.c:1814
No locals.
#26 0xb6a210eb in g_main_context_iterate (context=0x98dbff8, block=1, 
    dispatch=1, self=0x98dd570)
    at /build/buildd/glib2.0-2.20.1/glib/gmain.c:2448
	max_priority = 0
	timeout = 0
	some_ready = 1
	nfds = 11
	allocated_nfds = <value optimized out>
	fds = (GPollFD *) 0xa689f68
	__PRETTY_FUNCTION__ = "g_main_context_iterate"
#27 0xb6a215ba in IA__g_main_loop_run (loop=0xb404268)
    at /build/buildd/glib2.0-2.20.1/glib/gmain.c:2656
	self = (GThread *) 0x98dd570
	__PRETTY_FUNCTION__ = "IA__g_main_loop_run"
#28 0xb6e617d9 in IA__gtk_main ()
---Type <return> to continue, or q <return> to quit---
    at /build/buildd/gtk+2.0-2.16.1/gtk/gtkmain.c:1205
	tmp_list = (GList *) 0x0
	functions = (GList *) 0x0
	init = (GtkInitFunction *) 0xb3be0f0
	loop = (GMainLoop *) 0xb404268
#29 0xb768f41b in wxEventLoop::Run (this=0xb3be0f0)
    at ../src/gtk/evtloop.cpp:76
	exitcode = <value optimized out>
	__FUNCTION__ = "Run"
#30 0xb7733b3c in wxAppBase::MainLoop (this=0x98dbe40)
    at ../src/common/appcmn.cpp:312
	mainLoop = {<wxEventLoopPtr> = {m_ptr = 0xb3be0f0}, m_pp = 0x98dbe80, 
  m_pOld = 0x0}
#31 0xb7733891 in wxAppBase::OnRun (this=0x5) at ../src/common/appcmn.cpp:367
No locals.
#32 0xb74277ca in wxEntry (argc=@0xb75206cc, argv=0x98ca480)
    at ../src/common/init.cpp:460
No locals.
#33 0xb7427887 in wxEntry (argc=@0xbff3dca0, argv=0xbff3dd24)
    at ../src/common/init.cpp:472
No locals.
#34 0x082cba50 in main (argc=0, argv=0x0) at ../../src/amule-gui.cpp:94
No locals.

Stu Redman · « **Reply #201 on:** May 16, 2009, 08:32:36 PM »

You mean out of memory again?
Well, imagine your running at 100km/h off the road. You might hit a tree, you might hit a rock. It's neither the rock's fault nor the tree's - problem is you were running off the road in the first place. And if you hit a barn for a change it's the same, and examining the barn won't get you to the root of the problem either.

To be frank: I doubt aMule has ever been run anywhere at conditions similiar to yours. In theory it should work, but obviously it doesn't. I would have expected it to draw 100% CPU at such
conditions and not reach the possible upload.
Problem is, nobody has an environment to reproduce the problem. The networking code is really complex (Kry will probably laugh at me again), with packets being queued, posted through events, queued again, etc, and in some place something is breaking at the speed you are running it. I doubt it's a memory leak at all - I'd rather guess one of the queues is getting filled until memory runs out.

Quote from: RRM on May 16, 2009, 07:14:53 PM

When the max. connections are set to 200, i dont get above 20 MB/s UL, but that still
(usually) results in high VSZ crashes.

What IS the CPU load in that condition (stable upload of 20MB/s) ? (And what kind of CPU do you have?)

RRM · « **Reply #202 on:** May 17, 2009, 07:06:19 PM »

Quote from: Stu Redman on May 16, 2009, 08:32:36 PM

You mean out of memory again?

Yes, the VSZ was over 3GB again.

Quote

Well, imagine your running at 100km/h off the road....

Yes, i understand.
However, aMule also keeps crashing while running much, much slower.
Even at an average UL of 2MB, with 50 connections max,
Amule still crashes once in 2 or 3 days.
Also, i guess in the near future everybody will have such fast internet
connections; doesnt it get faster all the time?
And maybe aMule needs to get prepared for that future?

Quote

I'd rather guess one of the queues is getting filled until memory runs out.

That seems logical to me.
And then maybe the sharp increase in VSZ is due to the repeated requests of the queued?
An accumulating effect maybe?
So, maybe the solution might be to slow down / stall the requests of the queued?
So, that there is not such a sharp accumulative effect?
Then aMule can slow down instead of crash, no?

Quote

What IS the CPU load in that condition (stable upload of 20MB/s) ?

There is no stable upload above 1MB/1.5 MB;
The greater the speed, the greater the fluctuations; it bounces up and down,
from max to zero to half-max etc..
But, before aMule crashes, its over 90% (correlating with VSZ, i guess)

Quote

And what kind of CPU do you have?

3GB AMD Phenom QuadCore 9500

RRM · « **Reply #203 on:** May 17, 2009, 07:11:42 PM »

A while ago, i monitored the VSZ build up right before a crash:

Quote from: RRM on April 07, 2009, 01:41:07 PM

...
this is what i recorded (within 3 minutes!):

$ ps axl | grep amule
Code: [Select]
0 1000 6576 6556 20 0 1516768 1454528 - Rl+ pts/0 217:28 /usr/bin/amule 0 1000 6576 6556 20 0 1852180 1790016 - Rl+ pts/0 218:02 /usr/bin/amule 0 1000 6576 6556 20 0 1953028 1890828 - Rl+ pts/0 218:12 /usr/bin/amule 0 1000 6576 6556 20 0 2005828 1943644 - Rl+ pts/0 218:17 /usr/bin/amule 0 1000 6576 6556 20 0 2074600 2012388 - Sl+ pts/0 218:24 /usr/bin/amule 0 1000 6576 6556 20 0 2181124 2118920 - Rl+ pts/0 218:35 /usr/bin/amule 0 1000 6576 6556 20 0 2237224 2175068 - Rl+ pts/0 218:41 /usr/bin/amule 0 1000 6576 6556 20 0 2291872 2229732 - Rl+ pts/0 218:46 /usr/bin/amule 0 1000 6576 6556 20 0 2362096 2295424 - Rl+ pts/0 218:53 /usr/bin/amule 0 1000 6576 6556 20 0 2471392 2341024 - Sl+ pts/0 219:04 /usr/bin/amule 0 1000 6576 6556 20 0 2540692 2356620 - Rl+ pts/0 219:11 /usr/bin/amule 0 1000 6576 6556 20 0 2663716 2416916 - Rl+ pts/0 219:22 /usr/bin/amule 0 1000 6576 6556 20 0 2781196 2420808 - Rl+ pts/0 219:33 /usr/bin/amule 0 1000 6576 6556 20 0 2859140 2444392 - Rl+ pts/0 219:41 /usr/bin/amule 0 1000 6576 6556 20 0 2917508 2468548 - Rl+ pts/0 219:46 /usr/bin/amule 0 1000 6576 6556 20 0 2965636 2448640 - Rl+ pts/0 219:51 /usr/bin/amule 0 1000 6576 6556 20 0 3007620 2431868 - Rl+ pts/0 219:55 /usr/bin/amule 0 1000 6576 6556 20 0 3052676 2477072 - Rl+ pts/0 219:59 /usr/bin/amule 0 1000 6576 6556 20 0 3108996 2475080 - Rl+ pts/0 220:05 /usr/bin/amule 0 1000 6576 6556 20 0 3145724 2501608 sync_p Dl+ pts/0 220:08 /usr/bin/amule 0 1000 6576 6556 20 0 3145724 2499884 ptrace Tl pts/0 220:08 /usr/bin/amule

Stu Redman · « **Reply #204 on:** May 17, 2009, 07:19:25 PM »

Question was - what is the CPU-Load at high upload rates?
And please consider your quad-core - is one of the cores running aMule at 100% ? Does Linux use multiple cores per app (one per thread) ?

I could try to build a limit into some of the queues and see if that takes care of your memory situation.

RRM · « **Reply #205 on:** May 17, 2009, 07:54:33 PM »

Quote from: Stu Redman on May 17, 2009, 07:19:25 PM

...is one of the cores running aMule at 100% ? Does Linux use multiple cores per app (one per thread) ?

I have no idea...
How can i find out?

Quote

I could try to build a limit into some of the queues and see if that takes care of your memory situation.

Wow, if you could actually do that, it would certainly be worth a try!!!

myth · « **Reply #206 on:** August 07, 2009, 03:01:44 AM »

Where do you live to have such a fucking rocking connection?!? I want this, too!

RRM · « **Reply #207 on:** September 03, 2009, 07:37:52 PM »

Quote from: myth on August 07, 2009, 03:01:44 AM

Where do you live to have such a fucking rocking connection?!? I want this, too!

Amsterdam, The Netherlands. (UPC " Fiber Power")
Unfortunately, (for obvious reasons) this service seems to have become more popular;
according to my amule stats, my upload capacity has shrunk to 3 MB/s...

myth · « **Reply #208 on:** September 03, 2009, 09:08:27 PM »

Well, it's still crazy fast!

I pay about 40 Euros a month to have a 7000/384 Kbit/s line which goes at 2100/330 Kbit/s

Stu Redman · « **Reply #209 on:** September 04, 2009, 09:31:00 PM »

Try for cable networking, that's what I did recently. (If you have cable, that is.)

aMule Forum

News:

Author Topic: RRM's epic struggle for a better aMule on high-speed connections (Read 165936 times)

RRM

RRM's epic struggle for a better aMule on high-speed connections

Stu Redman

RRM's epic struggle for a better aMule on high-speed connections

RRM

RRM's epic struggle for a better aMule on high-speed connections

RRM

RRM's epic struggle for a better aMule on high-speed connections

RRM

RRM's epic struggle for a better aMule on high-speed connections

RRM

RRM's epic struggle for a better aMule on high-speed connections

Stu Redman

RRM's epic struggle for a better aMule on high-speed connections

RRM

RRM's epic struggle for a better aMule on high-speed connections

RRM

RRM's epic struggle for a better aMule on high-speed connections

Stu Redman

RRM's epic struggle for a better aMule on high-speed connections

RRM

RRM's epic struggle for a better aMule on high-speed connections

myth

RRM's epic struggle for a better aMule on high-speed connections

RRM

RRM's epic struggle for a better aMule on high-speed connections

myth

RRM's epic struggle for a better aMule on high-speed connections

Stu Redman

RRM's epic struggle for a better aMule on high-speed connections