aMule Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

We're back! (IN POG FORM)

Pages: [1] 2

Author Topic: upredictable memory leak  (Read 7402 times)

beroal

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 7
upredictable memory leak
« on: September 28, 2010, 09:12:45 PM »

After startup amule uses ~ 50 MB of memory, resident size. It may work peacefully for hours, but at some moment it increases resident size to > 500 MB (HDD are working like crazy), so almost completely hanging a computer with 1 GB RAM. This bug was triggered several times. I estimate that ~ 10 hours is enough to trigger the bug. I can not actually determine that moment. I just leave the computer, and when I return, amule is gone mad. This is not related to file hashing, I readded shared files (~ 1700 files) and resident size is < 100 MB. I know this info is not useful, how can I obtain some logs to determine what is going on? amule version: 2.2.6 .
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3740
  • Engines screaming
Re: upredictable memory leak
« Reply #1 on: September 28, 2010, 09:42:14 PM »

It's probably not at some moment, but rather creeps up over time. I suspect the Kad poisoning problem.
What's the size of your key_index.dat file ?
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: upredictable memory leak
« Reply #2 on: September 29, 2010, 02:11:25 AM »

Please post more details about your setup. Operating system vendor and version, etc...
Logged

beroal

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 7
Re: upredictable memory leak
« Reply #3 on: September 29, 2010, 01:07:01 PM »

It's probably not at some moment, but rather creeps up over time. I suspect the Kad poisoning problem.
What's the size of your key_index.dat file ?
Quote
$ ls -l ~/.aMule/key_index.dat
-rw------- 1 {{{skipped}}} 1922073 Sep 29 13:59 /home/slut/.aMule/key_index.dat
Do you suggest to disable Kad?
Logged

beroal

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 7
Re: upredictable memory leak
« Reply #4 on: September 29, 2010, 01:18:41 PM »

Please post more details about your setup. Operating system vendor and version, etc...
OS: Arch Linux
kernel: Linux 2.6.33.7 with real time patch 29
amule package
Quote
$ amule --version
aMule 2.2.6 using wxGTK2 v2.8.10 (OS: Linux)
wxgtk: 2.8.11-1
Quote
$ df ~/.aMule
Filesystem           1K-blocks      Used Available Use% Mounted on
{{{skipped}}}              2018576   1729664    288912  86% /home
Logged

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: upredictable memory leak
« Reply #5 on: September 29, 2010, 03:05:47 PM »

Umm, xlib version?

Also, you could try to compile amule with debug and optimization enabled and run it under gdb.

Then, when it dies, post a backtrace.
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3740
  • Engines screaming
Re: upredictable memory leak
« Reply #6 on: September 29, 2010, 07:52:13 PM »

btkaos, we've been through this. Posting backtraces when the app runs out of memory is useless.
I'll see if I can slap together a script that filters fileview output for IP adresses and sorts them so we can identify the bad guy network. Then you can block it.
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3740
  • Engines screaming
Re: upredictable memory leak
« Reply #7 on: September 29, 2010, 08:24:12 PM »

OK, here it is. Run
fileview key_index.dat | perl countIPs.pl

There should be a bunch of addresses sticking out at the top. Block them.
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

beroal

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 7
Re: upredictable memory leak
« Reply #8 on: September 29, 2010, 09:07:56 PM »

Umm, xlib version?
Quote
$ pacman -Q libx11
libx11 1.3.5-1

Also, you could try to compile amule with debug and optimization enabled and run it under gdb.
Then, when it dies, post a backtrace.
amule does not die. When I send the TERM signal, it exits by its own within a few minutes. When I compile amule:
Quote
wxcasframe.cpp: In member function ‘wxImage* WxCasFrame::GetStatImage() const’:
wxcasframe.cpp:292:58: error: cannot call constructor ‘wxFont::wxFont’ directly
wxcasframe.cpp:292:58: error:   for a function-style cast, remove the redundant ‘::wxFont’
source=(http://downloads.sourceforge.net/project/${pkgname}/aMule/${pkgver}/aMule-${pkgver}.tar.bz2)
pkgname=amule
pkgver=2.2.6
Logged

beroal

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 7
Re: upredictable memory leak
« Reply #9 on: September 29, 2010, 09:16:47 PM »

I'll see if I can slap together a script that filters fileview output for IP adresses and sorts them so we can identify the bad guy network. Then you can block it.
So you say that 500 MB of RAM is too low for working with 2 MB file?  ;D It does not make sense, sorry. Nevertheless, what is "fileview" and where can I find it?
« Last Edit: September 29, 2010, 09:48:12 PM by beroal »
Logged

Stu Redman

  • Administrator
  • Hero Member
  • *****
  • Karma: 214
  • Offline Offline
  • Posts: 3740
  • Engines screaming
Re: upredictable memory leak
« Reply #10 on: September 29, 2010, 10:39:40 PM »

src/utils/fileview , configure --with-fileview iirc (at least in the SVN version)

You make it sound silly.  :) But the data isn't stored in memory as compact as in the file, with all the list linking overhead and stuff.
Actually the math was 1MB = ok, 11MB = spammed, so I'm not sure with 2MB if that's your problem here at all. But it's the one big mem leak I'm aware of. (Unless you have 100 Mbit/s upload speed, then there's another one.)

Oh, and you can always try the SVN version to see if that performs better. It fixes the high speed leak, but not the Kad spamming.
Logged
The image of mother goddess, lying dormant in the eyes of the dead, the sheaf of the corn is broken, end the harvest, throw the dead on the pyre -- Iron Maiden, Isle of Avalon

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: upredictable memory leak
« Reply #11 on: September 30, 2010, 12:00:35 AM »

btkaos, we've been through this. Posting backtraces when the app runs out of memory is useless.
I don't think so, I've found literally dozens of bugs causing OOM errors this way. Usually the stack trace reveals the path to blame for excess allocation.
Logged

btkaos

  • Global Moderator
  • Sr. Member
  • *****
  • Karma: 110
  • Offline Offline
  • Posts: 486
  • Kaos is infinite!
Re: upredictable memory leak
« Reply #12 on: September 30, 2010, 12:10:43 AM »

Umm, xlib version?
Quote
$ pacman -Q libx11
libx11 1.3.5-1
Weird, I cannot find that version upstream, it jumps from libx11 1.3.4 to 1.3.99. I'm sorry I'm not familiar with Arch Linux, do you have a package history? Could you see if the problem arose at the same time that a libx11 update?
Quote
amule does not die. When I send the TERM signal, it exits by its own within a few minutes. When I compile amule:
Quote
wxcasframe.cpp: In member function ‘wxImage* WxCasFrame::GetStatImage() const’:
wxcasframe.cpp:292:58: error: cannot call constructor ‘wxFont::wxFont’ directly
wxcasframe.cpp:292:58: error:   for a function-style cast, remove the redundant ‘::wxFont’
source=(http://downloads.sourceforge.net/project/${pkgname}/aMule/${pkgver}/aMule-${pkgver}.tar.bz2)
pkgname=amule
pkgver=2.2.6
Well, I'd recommed several things:
  • Try to compile git amule instead of 2.2.6. Use git clone git://repo.or.cz/amule.git to get the source. If you cannot stand the new GUI like me use rev 10140 :p
  • Start amule under gdb, then when amule starts behaving like crazy press Ctrl-C in gdb and get the backtrace. If possible, leave the program as is, or you can use continue in order to get another backtrace. Having a gdb open with aMule in Ctrl-C state could be very useful for debugging once we see the backtrace.
Logged

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: upredictable memory leak
« Reply #13 on: September 30, 2010, 12:34:35 AM »

btkaos, we've been through this. Posting backtraces when the app runs out of memory is useless.
I don't think so, I've found literally dozens of bugs causing OOM errors this way. Usually the stack trace reveals the path to blame for excess allocation.

That's very much true.
Logged

beroal

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 7
Re: upredictable memory leak
« Reply #14 on: October 03, 2010, 05:07:12 PM »

Sorry for a long delay. It was really hard to spot this bug. I spotted it 3 times in a very short time. Now I understand why I encountered it often — my internet provider has disconnects quite often. Steps to reproduce:
  • Start amule.
  • Disconnect a network cable.
  • Wait for few minutes. In my case 80 or 130 seconds.
Then amule will allocate memory during several minutes until swap is exhausted. Also CPU is used at 90-100% while allocating memory. ~/.aMule/logfile and tty output contains nothing interesting. I run a program which records resident memory usage as reported by «ps». Here is an excerpt from the log:

Quote
Sun Oct  3 17:20:20 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 13296 189824 S pts/2    00:00:00 amule
Sun Oct  3 17:20:30 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 43588 312432 R pts/2    00:00:11 amule
Sun Oct  3 17:20:40 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 47680 332272 S pts/2    00:00:14 amule
Sun Oct  3 17:20:50 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 48096 332580 S pts/2    00:00:15 amule
{{{skipped}}}
Sun Oct  3 17:23:30 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 49420 333844 S pts/2    00:00:29 amule
Sun Oct  3 17:23:40 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 49420 333844 S pts/2    00:00:30 amule
Sun Oct  3 17:23:50 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 178992 399380 R pts/2   00:00:36 amule
Sun Oct  3 17:24:00 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 418116 661524 R pts/2   00:00:46 amule
Sun Oct  3 17:24:10 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 554488 858132 R pts/2   00:00:54 amule
Sun Oct  3 17:24:20 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 597708 1054740 D pts/2  00:01:02 amule
Sun Oct  3 17:24:30 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 595536 1185812 R pts/2  00:01:10 amule
Sun Oct  3 17:24:42 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 583356 1382420 R pts/2  00:01:18 amule
Sun Oct  3 17:24:50 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 648988 1579028 R pts/2  00:01:26 amule
Sun Oct  3 17:25:01 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 608804 1775636 R pts/2  00:01:36 amule
Sun Oct  3 17:25:10 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 629812 1906708 D pts/2  00:01:43 amule
Sun Oct  3 17:25:20 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 638216 2103316 R pts/2  00:01:51 amule
Sun Oct  3 17:25:32 EEST 2010
  PID   RSS    VSZ S TTY          TIME COMMAND
 2292 643860 2299924 T pts/2  00:02:01 amule
Logged
Pages: [1] 2