aMule Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

We're back! (IN POG FORM)

Pages: 1 ... 30 31 [32] 33 34 ... 37

Author Topic: RRM's epic struggle for a better aMule on high-speed connections  (Read 165787 times)

RRM

  • Sr. Member
  • ****
  • Karma: 40
  • Offline Offline
  • Posts: 444
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #465 on: February 15, 2011, 05:08:11 PM »

But, wait a second, Stu already supplied me with the fix on Dec 27, 2009
here: http://forum.amule.org/index.php?topic=17507.405
I had to replace the entire EMSocket.cpp file (Stu enabled the busy-check for all packets),
and that was what fixed the problem.
It was commited in 9913.
Later (Jan 03, 2010) Stu disabled the busy logic for control packets instead of data packets,
Code: [Select]
if(byConnected == ES_CONNECTED && IsEncryptionLayerReady() && (!m_bBusy || onlyAllowedToSendControlPacket)) {
instead of
Code: [Select]
if(byConnected == ES_CONNECTED && IsEncryptionLayerReady() && !m_bBusy) {
which was commited on Jan 05, 2010.

So, dont i need a pre-9913 EMSocket.cpp ?
Logged

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #466 on: February 15, 2011, 05:10:56 PM »

Oh sorry RRM I had a brainfade, of course you have to disable STU fix and see if it crashes.
Logged

RRM

  • Sr. Member
  • ****
  • Karma: 40
  • Offline Offline
  • Posts: 444
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #467 on: February 15, 2011, 05:49:24 PM »

Ok, but how do i edit the EMSocket.cpp back to its pre-9913 version?
Anyone here with 2.2.6 that can provide me with its EMsocket.cpp?
Logged

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #468 on: February 15, 2011, 06:08:11 PM »

RRM I don't understand your question fully, the change should be:

Code: [Select]
diff --git a/src/EMSocket.cpp b/src/EMSocket.cpp
index 3469165..e556f7a 100644
--- a/src/EMSocket.cpp
+++ b/src/EMSocket.cpp
@@ -476,7 +476,7 @@ SocketSentBytes CEMSocket::Send(uint32 maxNumberOfBytesToSend, uint32 minFragSiz
     uint32 sentStandardPacketBytesThisCall = 0;
     uint32 sentControlPacketBytesThisCall = 0;
        
-    if (byConnected == ES_CONNECTED && IsEncryptionLayerReady() && (!m_bBusy || onlyAllowedToSendControlPacket)) {
+    if (byConnected == ES_CONNECTED && IsEncryptionLayerReady() && !(m_bBusy || onlyAllowedToSendControlPacket)) {
 
                //printf("* Internal attemptto send on %p\n", this);
EDIT: I'm stupid, the correct change is the following:
Code: [Select]
diff --git a/src/EMSocket.cpp b/src/EMSocket.cpp
index 3469165..e556f7a 100644
--- a/src/EMSocket.cpp
+++ b/src/EMSocket.cpp
@@ -476,7 +476,7 @@ SocketSentBytes CEMSocket::Send(uint32 maxNumberOfBytesToSend, uint32 minFragSiz
     uint32 sentStandardPacketBytesThisCall = 0;
     uint32 sentControlPacketBytesThisCall = 0;
       
-    if (byConnected == ES_CONNECTED && IsEncryptionLayerReady() && (!m_bBusy || onlyAllowedToSendControlPacket)) {
+    if (byConnected == ES_CONNECTED && IsEncryptionLayerReady() && !(m_bBusy && onlyAllowedToSendControlPacket)) {
 
                //printf("* Internal attemptto send on %p\n", this);

The - indicates that line should be removed, the + the line should be added. You can save the above snippet to a em.patch file and type
Code: [Select]
$ patch -p1 em.patch
to apply the change automatically.

Note the end of the discussion http://forum.emule-project.net/index.php?showtopic=148119here, maybe we should revert the patch? If Enig123 is right (a wx bug) I don't know if this makes sense.
« Last Edit: February 22, 2011, 08:11:01 PM by btkaos »
Logged

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #469 on: February 15, 2011, 06:11:57 PM »

Some points about the patch that is in amule but not in emule:

Quote
After testing of this patch for some time, I realized the original code is logically right. When m_bBusy is true, which generally means a data packet has sent part and there's other splitted parts need to be sent. If there's an only control packet to send, it should not be in the middle of it. So in this condition, the program just skip sending part and put the control packet directly to the control packet queue.

With the patch SS offered, my tester encountered several crashes randomly, which never happened before.

Quote
However, recent high upload experiments show that this patch compromises upload capacity of eMule. My test team with 100Mbit/s upload no longer get a near straight line at upload limit of 4MB/s with the patch applied, as it did before.

My guess is that this is not really a eMule bug, more like a wxWidgets bug.
So indeed the patch may have modified upload routine in a manner the wx bug is not triggered anymore at this particular load.
Logged

RRM

  • Sr. Member
  • ****
  • Karma: 40
  • Offline Offline
  • Posts: 444
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #470 on: February 15, 2011, 07:11:53 PM »

Thank you, Bill.
Im now running SVN 10466, with the modified EMSocket.cpp,
though its compiled with wxGTK2 v2.8.10
....
ehrrr, im not, the mule crashed crashed already...

Program received signal SIGINT, Interrupt.
0x0012d422 in __kernel_vsyscall ()

(gdb) bt full
Code: [Select]
bt full
#0  0x0012d422 in __kernel_vsyscall ()
No symbol table info available.
#1  0x00ebbd96 in poll () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#2  0x0161754b in g_poll () from /lib/libglib-2.0.so.0
No symbol table info available.
#3  0x0083eef7 in wxapp_poll_func (ufds=0x9434f48, nfds=17, timeout=7)
    at ../src/gtk/app.cpp:259
        res = <value optimized out>
#4  0x0160a56b in ?? () from /lib/libglib-2.0.so.0
No symbol table info available.
#5  0x0160ab9f in g_main_loop_run () from /lib/libglib-2.0.so.0
No symbol table info available.
#6  0x0107d419 in gtk_main () from /usr/lib/libgtk-x11-2.0.so.0
No symbol table info available.
#7  0x0085bec8 in wxEventLoop::Run (this=0xb751b3f8)
    at ../src/gtk/evtloop.cpp:76
        __FUNCTION__ = "Run"
        exitcode = <value optimized out>
#8  0x008fd2ac in wxAppBase::MainLoop (this=0x86d5620)
    at ../src/common/appcmn.cpp:312
        mainLoop = {<wxEventLoopPtr> = {m_ptr = 0xb751b3f8}, m_pp = 0x86d5660,
          m_pOld = 0x0}
---Type <return> to continue, or q <return> to quit---
#9  0x008fd001 in wxAppBase::OnRun (this=0xfffffdfc)
    at ../src/common/appcmn.cpp:367
No locals.
#10 0x00b953ea in wxEntry (argc=@0xc8f6ac, argv=0x86c4e80)
    at ../src/common/init.cpp:460
No locals.
#11 0x00b95497 in wxEntry (argc=@0xbffff460, argv=0xbffff504)
    at ../src/common/init.cpp:472
No locals.
#12 0x0823ccfb in main (argc=1, argv=0xbffff504) at amule-gui.cpp:93
No locals.

(gdb) thread apply all bt full
Code: [Select]
Thread 6 (Thread 0xb6405b70 (LWP 29045)):
#0  0x0012d422 in __kernel_vsyscall ()
No symbol table info available.
#1  0x00138142 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
   from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#2  0x00c07a3e in wxConditionInternal::WaitTimeout (this=0xb751adc8,
    milliseconds=100) at ../src/unix/threadpsx.cpp:405
        tspec = {tv_sec = 1297793200, tv_nsec = 960000000}
        err = <value optimized out>
#3  0x00c07aca in wxCondition::WaitTimeout (this=0xb751b804, milliseconds=100)
    at ../include/wx/thrimpl.cpp:256
        __FUNCTION__ = "WaitTimeout"
#4  0x00c097a4 in wxSemaphoreInternal::WaitTimeout (this=0xb751b800,
    milliseconds=100) at ../src/unix/threadpsx.cpp:552
        remainingTime = <value optimized out>
        locker = {m_isOk = true, m_mutex = @0xb751b800}
#5  0x00c0982a in wxSemaphore::WaitTimeout (this=0xb751b438, milliseconds=100)
    at ../include/wx/thrimpl.cpp:320
        __FUNCTION__ = "WaitTimeout"
#6  0x0833d882 in CTimerThread::Entry (this=0xb751b418) at Timer.cpp:66
        now = <value optimized out>
---Type <return> to continue, or q <return> to quit---
        delta = 0
        evt = {<wxEvent> = {<wxObject> = {_vptr.wxObject = 0x847e2c8,
              static ms_classInfo = {m_className = 0xc3ad98 L"wxObject",
                m_objectSize = 8, m_objectConstructor = 0, m_baseInfo1 = 0x0,
                m_baseInfo2 = 0x0, static sm_first = 0x71d8cc,
                m_next = 0xc908a4, static sm_classTable = 0x86a5008},
              m_refData = 0x0}, m_eventObject = 0x0, m_eventType = 10244,
            m_timeStamp = 0, m_id = 6128, m_callbackUserData = 0x0,
            m_propagationLevel = 0, m_skipped = false,
            m_isCommandEvent = false, static ms_classInfo = {
              m_className = 0xc4c590 L"wxEvent", m_objectSize = 36,
              m_objectConstructor = 0, m_baseInfo1 = 0xc90838,
              m_baseInfo2 = 0x0, static sm_first = 0x71d8cc,
              m_next = 0xc918e8,
              static sm_classTable = 0x86a5008}}, <No data fields>}
        lastEvent = 713077468
#7  0x00c0a9e3 in wxThreadInternal::PthreadStart (thread=0xb751b418)
    at ../src/unix/threadpsx.cpp:766
        pthread = 0xb751aca8
        rc = <value optimized out>
        dontRunAtAll = false
        __FUNCTION__ = "PthreadStart"
#8  0x00c0aaed in wxPthreadStart (ptr=0xb751b418)
---Type <return> to continue, or q <return> to quit---
    at ../src/unix/threadpsx.cpp:718
No locals.
#9  0x0013380e in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#10 0x00ec9a0e in clone () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.

Thread 3 (Thread 0xb7407b70 (LWP 29042)):
#0  0x0012d422 in __kernel_vsyscall ()
No symbol table info available.
#1  0x0013b466 in nanosleep () from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#2  0x00c11468 in wxMicroSleep (microseconds=625000)
    at ../src/unix/utilsunx.cpp:191
        tmReq = {tv_sec = 0, tv_nsec = 625000000}
#3  0x00c11491 in wxMilliSleep (milliseconds=625)
    at ../src/unix/utilsunx.cpp:212
No locals.
#4  0x00c0776d in wxThread::Sleep (milliseconds=625)
    at ../src/unix/threadpsx.cpp:986
No locals.
#5  0x081d7830 in UploadBandwidthThrottler::Entry (this=0x8d10968)
    at UploadBandwidthThrottler.cpp:323
---Type <return> to continue, or q <return> to quit---
        timeSinceLastLoop = <value optimized out>
        minFragSize = 1300
        doubleSendSize = 2600
        sleepTime = 625
        thisLoopTick = 713077131
        bytesToSpend = <value optimized out>
        extraSleepTime = 625
        lastLoopTick = 713077131
        allowedDataRate = 3072000
        rememberedSlotCounter = 0
        sendLock = {m_isOk = 176, m_mutex = @0x0}
#6  0x00c0a9e3 in wxThreadInternal::PthreadStart (thread=0x8d10968)
    at ../src/unix/threadpsx.cpp:766
        pthread = 0x8d22c48
        rc = <value optimized out>
        dontRunAtAll = false
        __FUNCTION__ = "PthreadStart"
#7  0x00c0aaed in wxPthreadStart (ptr=0x8d10968)
    at ../src/unix/threadpsx.cpp:718
No locals.
#8  0x0013380e in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
No symbol table info available.
#9  0x00ec9a0e in clone () from /lib/tls/i686/cmov/libc.so.6
---Type <return> to continue, or q <return> to quit---
No symbol table info available.

Thread 1 (Thread 0xb7fdc760 (LWP 29026)):
#0  0x0012d422 in __kernel_vsyscall ()
No symbol table info available.
#1  0x00ebbd96 in poll () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#2  0x0161754b in g_poll () from /lib/libglib-2.0.so.0
No symbol table info available.
#3  0x0083eef7 in wxapp_poll_func (ufds=0x9434f48, nfds=17, timeout=7)
    at ../src/gtk/app.cpp:259
        res = <value optimized out>
#4  0x0160a56b in ?? () from /lib/libglib-2.0.so.0
No symbol table info available.
#5  0x0160ab9f in g_main_loop_run () from /lib/libglib-2.0.so.0
No symbol table info available.
#6  0x0107d419 in gtk_main () from /usr/lib/libgtk-x11-2.0.so.0
No symbol table info available.
#7  0x0085bec8 in wxEventLoop::Run (this=0xb751b3f8)
    at ../src/gtk/evtloop.cpp:76
        __FUNCTION__ = "Run"
        exitcode = <value optimized out>
#8  0x008fd2ac in wxAppBase::MainLoop (this=0x86d5620)
---Type <return> to continue, or q <return> to quit---
    at ../src/common/appcmn.cpp:312
        mainLoop = {<wxEventLoopPtr> = {m_ptr = 0xb751b3f8}, m_pp = 0x86d5660,
          m_pOld = 0x0}
#9  0x008fd001 in wxAppBase::OnRun (this=0xfffffdfc)
    at ../src/common/appcmn.cpp:367
No locals.
#10 0x00b953ea in wxEntry (argc=@0xc8f6ac, argv=0x86c4e80)
    at ../src/common/init.cpp:460
No locals.
#11 0x00b95497 in wxEntry (argc=@0xbffff460, argv=0xbffff504)
    at ../src/common/init.cpp:472
No locals.
#12 0x0823ccfb in main (argc=1, argv=0xbffff504) at amule-gui.cpp:93
No locals.
Logged

RRM

  • Sr. Member
  • ****
  • Karma: 40
  • Offline Offline
  • Posts: 444
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #471 on: February 15, 2011, 07:16:48 PM »

$ ps axl | grep amule
Code: [Select]
0  1000  1985  1953  20   0  37796 30192 poll_s S+   pts/0      0:00 gdb amule
0  1000 28977 19921  20   0 134652 123776 poll_s S+  pts/1      0:03 gdb amule
0  1000 29026 28977  20   0 110992 58764 ptrace Tl   pts/1      0:07 /usr/local/stow/amule-pre-stu/bin/amule
0  1000 29129 29098  20   0   3048   800 pipe_w S+   pts/2      0:00 grep amule
Logged

RRM

  • Sr. Member
  • ****
  • Karma: 40
  • Offline Offline
  • Posts: 444
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #472 on: February 15, 2011, 08:15:19 PM »

Just to make sure, the only modification in the EMSocket.cpp file was
(!m_bBusy 
replaced by:
!(m_bBusy
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #473 on: February 15, 2011, 09:24:06 PM »

Yep. Small change, big effect.  ;)
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #474 on: February 15, 2011, 09:42:13 PM »

Umm strange backtrace. Are you using the patched wx?
Logged

RRM

  • Sr. Member
  • ****
  • Karma: 40
  • Offline Offline
  • Posts: 444
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #475 on: February 15, 2011, 10:16:46 PM »

Are you using the patched wx?

No, not yet.
The version that crashed was stock SVN with the modified EMSocket.cpp
Now im back at using the stock SVN (with Stu's patch included, of course).
Naturally, this one runs perfectly, so how can we notice any effect of the patched wx?
Logged

RRM

  • Sr. Member
  • ****
  • Karma: 40
  • Offline Offline
  • Posts: 444
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #476 on: February 15, 2011, 10:23:12 PM »

Anything i should try?
Logged

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #477 on: February 15, 2011, 10:34:47 PM »

Dear RRM,

you should try the stock SVN with the modified EMSocket but using a modified wx.

In order to do this, build wx using the instructions I sent to you before.

Then, run the amule with the modified EMSocket (let's call it bad-amule) but using the new wx. In order to tell Linux that we want to run with a different lib we must use a variable.

Once you have built wxwindows do the following command:
Code: [Select]
$ LD_LIBRARY_PATH=~/temp-amule/build/lib ldd amule

Note that the path must be the place where the new wx is built and amule should be the "bad amule"

This should output the libraries amule is going to use, see if libwx is the new one. You can safely post the output here if you are not sure. If that checks then run the bad amule as follows
Code: [Select]
$ LD_LIBRARY_PATH=~/temp-amule/build/lib amule

And report back.

We believe the bug is in the library, not in amule, so if you don't build wx no testing is possible.
Logged

RRM

  • Sr. Member
  • ****
  • Karma: 40
  • Offline Offline
  • Posts: 444
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #478 on: February 15, 2011, 11:16:41 PM »

you should try the stock SVN with the modified EMSocket but using a modified wx.

Yes, of course; my mistake.
Gotta go to sleep now, i will try it tomorrow.

BTW, i used to have both high VSZ and low VSZ crashes, so that there might be 2 problems:
one in the library and one in aMule.
This last (pre-Stu) crash was a low VSZ crash.
As far as i know, there were only high VSZ crashes that were fixed by the wx patch.
Maybe Stu's patch fixes the low VSZ crashes, and somehow also prevents high VSZ crashes,
but only in certain conditions (mine).

Logged

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: RRM's epic struggle for a better aMule on high-speed connections
« Reply #479 on: February 15, 2011, 11:30:21 PM »

BTW, i used to have both high VSZ and low VSZ crashes, so that there might be 2 problems:
one in the library and one in aMule.
This last (pre-Stu) crash was a low VSZ crash.
As far as i know, there were only high VSZ crashes that were fixed by the wx patch.
Maybe Stu's patch fixes the low VSZ crashes, and somehow also prevents high VSZ crashes,
but only in certain conditions (mine).
We had both kind of crashes, glib context was corrupted in some weird way and that lead to either glib entering a loop and posting infinite events (OOM crash) or segfault (trying to access mutex of the context, invalid address)

RRM, the only way we have to know is testing :)
Logged
Pages: 1 ... 30 31 [32] 33 34 ... 37