aMule Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

We're back! (IN POG FORM)

Author Topic: amuled high CPU load with active downloads  (Read 9869 times)

Malden

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
amuled high CPU load with active downloads
« on: April 23, 2010, 04:55:30 PM »

Hello,

I have read almost everything with a similar topic (and there is a lot around), specifically this topic and my case seems really similar (however the solution didn't work for me).

I am using amuled 2.2.6 on Ubuntu 10.04. First I tried the package from the repos and then I recompiled with optimization turned on, but the result is about the same. When I start amuled without connecting, things look normal and CPU usage is below 5%. But when I connect to a server the CPU load starts to go high for a few seconds, then goes back to normal. This happens a few times and after about 10mins amuled  uses up all my CPU and even after more than an hour doesn't return to normal.

I have just one download in the list and max 10 clients connected. So what the hell is it doing ?  :o
I don't think that it's the hashing issue because all my shared file have been hashed once already.

Any idea what could be causing this?

Thanks, Malden
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: amuled high CPU load with active downloads
« Reply #1 on: April 23, 2010, 09:41:21 PM »

First I tried the package from the repos and then I recompiled with optimization turned on, but the result is about the same.
Sure. It would be quite stupid if they provided a package without optimization.
The issue in the thread you mentioned was not about hashing but about gap list implementation. When a client connects to you you tell it which parts of the file you have and which you don't. aMule 2.2 does this in way that is very inefficient if you have a large file with many gaps. The 10 clients don't matter, because everyone queued up asks you periodically for your status.
I don't think you can run in problems with one download and optimization on though, but maybe you could try aMule SVN anyway. With optimization on of course.  :)
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

Malden

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
Re: amuled high CPU load with active downloads
« Reply #2 on: April 24, 2010, 03:40:49 PM »

Thanks for your anser, Stu Redman.

Haven't tried the SVN version yet, but here's something else I noticed. The one download I have in the queue is paused and I am not connected to any server. However after some time I get about 10, sometimes 11 clients connected (i allowed only 10 connections) and amuled memory consumption gets really big. It starts at about 1% and after 3h it gets to 50% starting to swap memory to disk.

Could there be some memory leak?

Malden
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: amuled high CPU load with active downloads
« Reply #3 on: April 25, 2010, 11:35:19 AM »

Are you connected to Kad?
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

Malden

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
Re: amuled high CPU load with active downloads
« Reply #4 on: April 26, 2010, 11:08:50 AM »

I have disabled "auto connect on startup". During the last test I never pushed the connect button, but I got client connections (maybe these clients still knew my address from former connections?). This morning I started amuled again without being connected and after 1 hour there were still no client connections. CPU load and memory consumptions are low. Now as soon as I hit the connect button I get connected to eD2k and Kad (initially it says firewalled and then it becomes OK). Memory consumptions jumps to 30% (I have only 256MB of RAM).
Initially everything looks OK but during the previous tests, after some time, either CPU load or memory consumption increased dramatically.

Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: amuled high CPU load with active downloads
« Reply #5 on: April 26, 2010, 08:03:59 PM »

256MB is not much, especially with Kad. Kad takes more memory over time to store information received from the network, that's not a leak. Do you have an X server running too? Are you using an IP filter?

I suggest you try the SVN version, it has some optimizations for low memory systems.

Clients keep connecting for a while if you were connected before, that's perfectly normal.
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

Malden

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
Re: amuled high CPU load with active downloads
« Reply #6 on: April 27, 2010, 10:26:04 AM »

I tried disabling KAD and memory consumption is indeed much less. I deinstalled X so it's a command line server only now.   The  IP filtering settings in the  security preferences are the default settings (all option selected, except "Use system-wide ipfilter.dat").

Without KAD, my download remains in waiting state because it doesn't find any sources. Nevertheless amuled starts to consume all CPU after about 1hour. What do you think it is doing? With the hashing already done and the download waiting, what could consume all the CPU cycles? The only download in the queue is a 70MB file with 9 file parts.

I don't think I am doing anything strange here and I wonder why nobody else is reporting this issue.
OK, I will try the SVN version and report later.  Are there differences with respect to 2.2.6 that would address this issue?
Logged

Malden

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
Re: amuled high CPU load with active downloads
« Reply #7 on: April 27, 2010, 04:52:48 PM »

I tried with the latest SVN version but the behaviour is about the same. Memory consumption stays constant at about 35% during the first hour (ed2k and Kad enabled). CPU load is below 10% initially but goes to 100% after 30min.

I noticed that the high CPU load happens at the time when the number of  connected clients is equal to the allowed connections. When some client disconnects, CPU load goes back to normal. Just a coincidence ???  ::)
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: amuled high CPU load with active downloads
« Reply #8 on: April 27, 2010, 07:45:42 PM »

I noticed that the high CPU load happens at the time when the number of  connected clients is equal to the allowed connections.
Tell me your
  • physical max upload speed
  • upload bandwidth limit
  • slot allocation
  • max simultaneous connections
  • "number of connected clients" you mentioned
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

Malden

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
Re: amuled high CPU load with active downloads
« Reply #9 on: April 28, 2010, 09:39:33 AM »

Here's the connection settings from my amule.conf
Code: [Select]
QueueSizePref=11                                                                                                               
MaxUpload=100                                                                                                                 
MaxDownload=1000                                                                                                               
SlotAllocation=4                                                                                                               
Port=4662                                                                                                                     
UDPPort=4672
Address=
Autoconnect=1
MaxSourcesPerFile=50
MaxConnections=40
MaxConnectionsPerFiveSeconds=20

My ADLS connections has 4096kbps(down)/512kbps(up).

I can confirm my assumption above. Setting MaxConnections to 10 or 20 I always get 100% CPU load when reaching the corresponding number of clients (+/- 1). When I tried with 40 everything worked fine for hours because I never got 40 clients connected.

Any suspicion where the problem is located?
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: amuled high CPU load with active downloads
« Reply #10 on: April 29, 2010, 06:57:51 PM »

Your settings are wrong:
- 512 kbps upload, that's 64 kBps. You have set 100 which is way more than your connection can handle. Try 50.
- 100 upload, 4 SlotAlloc -> 25 upload slots. That's also a bit much. Try 10 kB/s Slot allocation to reduce the number of concurrent uploads.
- MaxConnections 40 is very low. That's not the number of uploads, it's the number of total TCP connections. Does your router crash with more? Try 200 or something.

There is obviously a bug in aMule when the number of uploads extends the number of connections, but with proper settings this should never happen.
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

Malden

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
Re: amuled high CPU load with active downloads
« Reply #11 on: April 30, 2010, 09:46:36 AM »

Hi Stu Redman, you made my day  8)

 You were right about the settings. I just hadn't noticed the subtle difference between b and B ! So aMule wants the bandwidth settings in kByte/s. Then of course I completely choked my internet connection with the connection settings I used. I changed them according to your suggestions and now they make much more sense. I get much less clients connected and I see download speeds I never had before.

So actually the connection settings can mess up the whole thing. But it seems that these settings could be easily determined by aMule itself just telling it your physical connection up/down speed limits. The rest of the settings should be hidden in the Advanced settings tab, so you usually don't need to touch them.

Default settings that should work fine with an ADLS connection could be calculated from the connection speed limits MAXUP and MAXDOWN as follows:
Code: [Select]
MaxUpload=MAXUP/2    /* assign half of your upload capacity to amule */
MaxDownload=MAXDOWN/4 /* ADLS download speed is usually higher than upload so we divide that further */
SlotAllocation=MaxUpload/5 /* allow 5 clients to connect simultaneously */
MaxConnections=200 /* constant default value */

This could probably be made more sophisticated, connection bandwidth could even be determined by aMule itself during initial setup so the user doesn't have to mess with the settings at all. Just some suggestion for improvements.

Thanks.
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3739
  • Engines screaming
Re: amuled high CPU load with active downloads
« Reply #12 on: May 01, 2010, 10:20:49 PM »

Well, it is not that simple.  :)
- If you want to share you can use way more than half your capacity
- No need to cut your download that much. No need to cut it at all actually.
- High upload speed can support much more than just 5 clients (which might be too slow to download at that speed)
- MaxConnect depends on OS and router. Some cheap routers can crash on that many connections.

eMule has a nice setup wizard for these parameters, but we are either too lazy to import it, or we believe that Linux users know so much more than Windows users and don't need that.  ;)
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

Malden

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
Re: amuled high CPU load with active downloads
« Reply #13 on: May 03, 2010, 04:08:51 PM »

eMule has a nice setup wizard for these parameters

Oooh, so that's why all the housewives in my neighbourhood are downloading 24hours a day and I just couldn't get this to work even with an engineering degree.  ;D
Logged