Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[anp]

i have share consisting 4K+ directories and 15K+ files;
amule start takes about 40 minutes (hashing files took 5 hours) on AMD Athlon64 3000+ linux/i686 binary.

and before this process finishes, amuled do not connect to any servers or KAD.

why amuled does not hash files in background? and what it does with files? (about 10-20 new files created between executions, 1-2 files moved)

[Stu Redman]

What version, what platform ?

[anp]

aMule 2.2.1, linux/i686 self-compiled binary (gcc 4.0.3)

[anp]

in addition, amuled stuck with ~90% CPU usage while new file is downloaded for long time (at least 10 minutes), while time, needed to hash such file is 10-15 seconds (with 10GB share)

[macias]

See also:
http://forum.amule.org/index.php?topic=15329.0

did it happen with previous versions of amule?

[anp]

sorry, i dont know, i started using amule with big shares from version 2.2.1

[wuischke]

I advise to share less files for the network's sake, althought this is surely not the solution to the problem. I can't say anything about the high CPU usage problems, I'm afraid.

[Kry]

15k files is, very much probably, more than the servers are allowing indexing when you publish them.

[anp]

only about 10% of servers not allowing shares with more than 1000 files.
i'm not using it.
and KAD clients can search all my files - it is not problem.

but why do we need rehashing all files at start and every time when file download completed?

[Kry]

It doesn't do that.

[anp]

long time, 90% CPU usage, and hdd activity....
if not hashing, what it does?

[Kry]

I don't know. You tell me.  But it's not hashing, or at least it shouldn't be. Also hashing is done on a separate lower prio thread.

You need to provide us with more information if you want us to find out what's going on in your aMule. Compile it with debug information, enable some relevant debug categories, run it to reproduce the problem and then give us the log with all the debug information. I am not going to create 15k files in my hard drive explicitely to reproduce it, specially when I ahve told you that most servers will never index that amount of files.

[anp]

enable some relevant debug categories

section "Debug" in amule.conf?
what categories i should enable?

Compile it with debug information

  --with-wxdebug          Force building against a debug build of wxWidgets,
                          even if --disable-debug is given (default is auto)

is it enought?

specially when I ahve told you that most servers will never index that amount of files

maybe, but files can be accessed via KAD...

[Archmage]

Honestly you should try to reduce the number of files that you are sharing.

But my guess is, that aMule is publishing the files over KAD, before you can start.

[anp]

log says, that server connection (and after some time KAD connection, "firewalled" and then "ok")
goes some time after the amule ends its HDD and CPU activity

log:

2008-06-24 23:52:51: ClientCreditsList.cpp(168): Creditfile loaded, 1108 clients are known
2008-06-24 23:52:51: IPFilter.cpp(109): Loading IP-filters 'ipfilter.dat' and 'ipfilter_static.dat'.
2008-06-24 23:52:51: IPFilter.cpp(333): Loaded 0 IP-ranges from '/home/p2p/.aMule/ipfilter.dat'. 0 malformed lines were disca
rded.
2008-06-24 23:52:51: IPFilter.cpp(333): Loaded 0 IP-ranges from '/home/p2p/.aMule/ipfilter_static.dat'. 0 malformed lines wer
e discarded.
2008-06-24 23:52:51: ExternalConn.cpp(160): *** TCP socket (ECServer) listening on 0.0.0.0:4712
2008-06-24 23:52:51: MuleUDPSocket.cpp(81): Created Server UDP-Socket at port 41299
2008-06-24 23:52:51: MuleUDPSocket.cpp(81): Created Client UDP-Socket at port 41296
2008-06-24 23:52:51: ServerList.cpp(83): Loading server.met file: /home/p2p/.aMule/server.met
2008-06-24 23:52:51: ServerList.cpp(168): 242 servers in server.met found
2008-06-24 23:52:51: ServerList.cpp(817): No server list address entry in 'addresses.dat' found. Please paste a valid server
list address into this file in order to auto-update your server list
2008-06-24 23:52:51: DownloadQueue.cpp(176): Found 36 part files
2008-06-25 00:18:10: SharedFileList.cpp(358): Found 29266 known shared files, 59 unknown
2008-06-25 00:18:10: amule.cpp(801): Connecting
2008-06-25 00:18:10: ServerSocket.cpp(621): Servers: Trying to connect
2008-06-25 00:18:10: ServerSocket.cpp(754): Connecting to Razorback 3.3 (92.48.193.14 - 92.48.193.14:5000) using protocol obf
uscation.

high CPU and HDD usage happens between this messages:

2008-06-24 23:52:51: DownloadQueue.cpp(176): Found 36 part files
2008-06-25 00:18:10: SharedFileList.cpp(358): Found 29266 known shared files, 59 unknown

in this start files are fully unchanged (shared filesystem locked in read-only mode, except "ed2k-incoming" directory, which resides in another filesystem)

59 unknown files are zero-length files:

2008-06-25 00:18:11: RoutingZone.cpp(188): Read 200 Kad contacts
2008-06-25 00:18:11: ThreadTasks.cpp(111): Hasher: Warning, 0-size file, skipping: /mnt/fs3/games/mixed/linuxcenter-games-dvd
/Installers/Daikatana/daikatana_1.2-english.uk.run
2008-06-25 00:18:11: ThreadScheduler.cpp(308): ThreadScheduler: Completed task 'Hashing - /mnt/fs3/games/mixed/linuxcenter-ga
mes-dvd/Installers/Daikatana/daikatana_1.2-english.uk.run', 58 tasks remaining.

after downloading file it gets loss of network connection, and same CPU and HDD usage for a long time:

2008-06-27 09:29:54: PartFile.cpp(2268): Finished downloading: Peremotka.2008.L2.DVDRip_INTERFILM.avi
2008-06-27 09:29:55: ThreadScheduler.cpp(308): ThreadScheduler: Completed task 'AICH Syncronizing', 0 tasks remaining.
2008-06-27 09:30:58: ClientTCPSocket.cpp(2066): Packet Parsing Errors: Caught EOF exception: SafeIO::EOF: Attempt to read pas
t end of file.
On packet with protocol e3, opcode 4c, size 156 ClientData: Client Unknown on IP:Port 84.109.142.171:4662 using Unknown Unkno
wn
2008-06-27 09:51:52: ClientTCPSocket.cpp(2066): Packet Parsing Errors: Caught EOF exception: SafeIO::EOF: Attempt to read pas
t end of file.
On packet with protocol e3, opcode 4c, size 156 ClientData: Client Unknown on IP:Port 84.109.142.171:4662 using Unknown Unkno
wn
2008-06-27 09:59:16: Connected to Kad (firewalled)
2008-06-27 09:59:17: Connected to Kad (ok)

and shared filesystem is not so slow...

[root@ebp3 /]# time ls -Rrltsh /mnt >/dev/null

real    0m1.466s
user    0m0.730s
sys     0m0.410s