aMule Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

We're back! (IN POG FORM)

Author Topic: When a file completes, all sessions are interrupted.  (Read 5083 times)

stoatwblr

  • Sr. Member
  • ****
  • Karma: 12
  • Offline Offline
  • Posts: 318
When a file completes, all sessions are interrupted.
« on: September 11, 2009, 03:29:52 AM »

This has has been happening for a long time.

It seems to be dependent on the number of downloads active+waiting, and that in turn seems to be dependent on system memory.

Basically when the behaviour is triggered: Every time a file is completed, ALL active sessions (down and uploads) are dumped (and of course, when resuming downloads, you end up at the end of the queue again - as does anyone downloading from you)

Any ideas?

Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: When a file completes, all sessions are interrupted.
« Reply #1 on: September 11, 2009, 08:22:32 PM »
What platform? What's your CPU and memory load? Ordinary and during hashing?
Looks like it gets so loaded up that the events aren't processed fast enough anymore.
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

stoatwblr

  • Sr. Member
  • ****
  • Karma: 12
  • Offline Offline
  • Posts: 318
Re: When a file completes, all sessions are interrupted.
« Reply #2 on: September 12, 2009, 02:49:03 AM »
Linux, x86, P4 3Ghz hyperthreading, DDR400, 3Gb ram

Not cutting into swap at all and yes it does happen when hashing newly added shares as well as after moving shared files around.

Top says (this is while downloading)

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
27423 xyz      20   0  248m 142m 8872 R   16  4.7  99:11.30 amuled            

After a dl completes:

27423 xyz      20   0  241m 142m 5872 S   108  4.7  99:49.42 amuled            

Then it drops back to about 65-68%


Could well be what you're surmising. Perhaps hashing should be a low priority thread.


(EDIT: I had 67% while dling, that was with dynamic cpu frequency control and on a 700Mhz clock.
 Locking to 3Ghz gave 16% and 108%)
« Last Edit: September 12, 2009, 04:05:38 AM by stoatwblr »
Logged

GonoszTopi

  • The current man in charge of most things.
  • Administrator
  • Hero Member
  • *****
  • Karma: 169
  • Offline Offline
  • Posts: 2685
Re: When a file completes, all sessions are interrupted.
« Reply #3 on: September 12, 2009, 11:52:00 AM »
Quote from: stoatwblr on September 12, 2009, 02:49:03 AM
Perhaps hashing should be a low priority thread.

It is.
Code: [Select]
// Try to avoid reducing the latency of the main thread
m_thread->SetPriority(WXTHREAD_MIN_PRIORITY);
Logged
concordia cum veritate

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: When a file completes, all sessions are interrupted.
« Reply #4 on: September 12, 2009, 07:49:23 PM »
Maybe your hard drive access is slow and therefore the process spends too much time waiting for I/O? That could block network polling.
Logged

stoatwblr

  • Sr. Member
  • ****
  • Karma: 12
  • Offline Offline
  • Posts: 318
Re: When a file completes, all sessions are interrupted.
« Reply #5 on: September 13, 2009, 06:56:23 PM »

Top says only 2-3% of the time is spent in IOwait. Load average bears that out, as does iotop.

~/.aMule is a dedicated 320Gb 7200rpm SATA2 drive w/32Mb cache (Seagate barracuda)

Shared files are on 3 similar dedicated disks (Maxtor Diamondmax)

Everything else is on its own hardware RAID array.

These have 7ms seek time and all sustain at least ~33-35Mb/s disk-to-disk.

Can't get much faster short of moving to $$$expensive 15,000rpm SAS drives or SSD - and SAS doesn't get
much improvement as seek times aren't any better than SATA.
« Last Edit: September 13, 2009, 07:01:45 PM by stoatwblr »
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: When a file completes, all sessions are interrupted.
« Reply #6 on: September 13, 2009, 07:54:53 PM »
Quote from: Kry on September 12, 2009, 07:49:23 PM
Maybe your hard drive access is slow and therefore the process spends too much time waiting for I/O? That could block network polling.
Hashing is an extra thread so I can't see how a slow access there could block anything.

stoatwblr, how many files are you downloading? amuled shouldn't take that much CPU on a 3GHz platform.
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: When a file completes, all sessions are interrupted.
« Reply #7 on: September 13, 2009, 08:16:28 PM »
Quote from: Stu Redman on September 13, 2009, 07:54:53 PM
Quote from: Kry on September 12, 2009, 07:49:23 PM
Maybe your hard drive access is slow and therefore the process spends too much time waiting for I/O? That could block network polling.
Hashing is an extra thread so I can't see how a slow access there could block anything.

Because a single thread blocking I/O had never caused problems for other threads, right?

Anyway that has been discarded, so it has to be something else (probably CPU usage, which is way too high in this case).
Logged

stoatwblr

  • Sr. Member
  • ****
  • Karma: 12
  • Offline Offline
  • Posts: 318
Re: When a file completes, all sessions are interrupted.
« Reply #8 on: September 14, 2009, 04:49:41 PM »
Quote from: Stu Redman on September 13, 2009, 07:54:53 PM
stoatwblr, how many files are you downloading? amuled shouldn't take that much CPU on a 3GHz platform.

A lot (mostly rare old stuff, described before. Some have taken nearly a year to complete)

Problems start manifesting about 340 active/waiting sessions.

It does depend what else is running. With a GUI intensive desktop it can happen with as few as 100 sessions.

These figures might seem extreme, but given that someone pulling a complete TV series from one of the indexing sites might add 400 links in one go, they're unlikely to be that uncommon.

Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: When a file completes, all sessions are interrupted.
« Reply #9 on: September 14, 2009, 06:32:37 PM »
Quote from: stoatwblr on September 12, 2009, 02:49:03 AM
that was with dynamic cpu frequency control and on a 700Mhz clock.
Maybe something is screwed with "dynamic cpu frequency control" and thread priority on your system.

And yes, with that many downloads be prepared for problems. Are you using the SVN version?
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

stoatwblr

  • Sr. Member
  • ****
  • Karma: 12
  • Offline Offline
  • Posts: 318
Re: When a file completes, all sessions are interrupted.
« Reply #10 on: September 14, 2009, 07:58:10 PM »
Quote from: Stu Redman on September 14, 2009, 06:32:37 PM
Quote from: stoatwblr on September 12, 2009, 02:49:03 AM
that was with dynamic cpu frequency control and on a 700Mhz clock.
Maybe something is screwed with "dynamic cpu frequency control" and thread priority on your system.

It just takes a few hundred ms to react. That's normal. In any case there's no power consumption difference on full speed vs dynamic (on demand) clocking, so I've left it on performance settings

Quote
And yes, with that many downloads be prepared for problems.

If it isn't stressed, you don't find the edge cases...

Quote
Are you using the SVN version?

Yes - Build 9794

Logged