Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[p.r.]

I run both rtorrent and amuled at my home NAS (ppc603e 200Mhz/32MB/uClibc Linux), and I've noticed a strange behavior of Amule.

When rtorrent performs procedure of final file hashing it does NOT slow down itself (speed of downloading/uploading) or other applications, such as amule.

When amule performs procedure of final file hashing it DOES very slow down itself and other applications, such as rtorrent.

I think that this difference may be due to different libraries: openssl 0.9.7m (libcrypto.so.0.9.7) and crypto++ 5.5.2 (built into Amule 2.2.3).

Therefore, I want to know:
1. is it possible to compile amule with openssl?
2. is it possible to make amule (by changing the settings, or source code) NOT to use more than 50% of cpu time?

[GonoszTopi]

1. is it possible to compile amule with openssl?

No.

2. is it possible to make amule (by changing the settings, or source code) NOT to use more than 50% of cpu time?

Most likely, yes.

[p.r.]

2. is it possible to make amule (by changing the settings, or source code) NOT to use more than 50% of cpu time?

Most likely, yes.

But how ?

Could some kind of  "hash strategies switcher" be implemented, so
user could choose, for example, between different options:
1. as fast as possible
2. don't exceed certain data rate (xx MB/s)
3. don't use more than xx% of cpu time ?

Sorry if I've written something stupid, but it looks like amule by default try to hash files as fast as possible.

[GonoszTopi]

Sorry if I've written something stupid, but it looks like amule by default try to hash files as fast as possible.

When the final rehash happens the file is (yet) a partfile, so the hasher thread gets high priority. That's what causing the general slow-down.

[p.r.]

Sorry if I've written something stupid, but it looks like amule by default try to hash files as fast as possible.

When the final rehash happens the file is (yet) a partfile, so the hasher thread gets high priority. That's what causing the general slow-down.

And how to reduce priority of   hasher thread ?

So I found a solution.

Check ThreadTasks.cpp:50

CHashingTask::CHashingTask(const CPath& path, const CPath& filename, const CPartFile* part)
   // GetPrintable is used to improve the readability of the log.
   : CThreadTask(wxT("Hashing"), path.JoinPaths(filename).GetPrintable(), (part ? ETP_High : ETP_Normal)),

You can change ETP_High to ETP_Low if you want, even the ETP_Normal, but it will make hashing way slower in your system. OTOH it should prevent CPU starvation.

Let's see how it will work.

[Archmage]

Will amuled also slow down?

[p.r.]

No luck with reducing priority of  hasher thread:
" : CThreadTask(wxT("Hashing"), path.JoinPaths(filename).GetPrintable(), (part ? ETP_Low : ETP_Low))"
No changes in amuled behaviour,  got the same slow down.

[freddy77]

This priority affect only the order in which tasks are executed in aMule. Probably hashing take too much time.

[GonoszTopi]

Code: (src/ThreadScheduler.cpp:156) [Select]

                // Try to avoid reducing the latency of the main thread
                m_thread->SetPriority(WXTHREAD_MIN_PRIORITY);

There seems to be no easy way to reduce CPU load.

[p.r.]

Final file hashing speed:
rtorrent   ~150 MB/minute  and    NO        system slow down
amuled      ~50 MB/minute   and    FULL   system slow down

What could be a cause of such awful inefficiency?

[Kry]

Are you sure it is the hashing? Are your incoming and temp in the same partition/disk?

[wuischke]

Well, torrent uses sha1 on small subsets (64kB to 4MB according to a large online encyclopaedia). ed2k (i.e. aMule) uses bigger chunks of 9500 kB which are hashed using MD4. aMule uses additionally AICH which uses sha1 to hash small parts of 180 kB, i.e. it creates 52+1 hashes per chunk. (Furthermore aMule copies the whole file after verification whereas torrent usually does not. That's what Kry's question is targetting.)

Therefore it's safe to assume that aMule is doing more work when hashing. This does, however not explain the slowdown. If, however, your computer is low on RAM this might be caused by the higher RAM usage of aMule when hashing. (It does currently load the whole 9.28 MB chunk into RAM, iirc.) This might cause swapping which is always a lot slower. freddy77 is working on patches to lower the RAM usage, so this might be improved in the future.

[p.r.]

Are you sure it is the hashing? Are your incoming and temp in the same partition/disk?

Yes, these directories are in same disk /partition, and the disk is internal ata133, not external usb.
All directories locations are default:

# ls -la1 /mnt/HD_a2/home/p2p/.aMule
.
..
ED2KLinks_lock
Incoming
Queue
Temp
amule.conf
clients.met
clients.met.BAK
cryptkey.dat
emfriends.met
ipfilter.dat
ipfilter_static.dat
key_index.dat
known.met
known2_64.met
last_version_check
lastversion
load_index.dat
logfile
logfile.bak
muleLock
nodes.dat
preferences.dat
preferencesKad.dat
server.met
server_met.old
shareddir.dat
src_index.dat

wuischke
Thank you for comprehensive explanation of difference between torrent and mule hashing.

AICH was disabled by default:

Code: [Select]

ICH=1
AICHTrust=0Should I disable ICH too?


I'll hope that freddy77 will make this patch.

Thank you all for understanding.

[wuischke]

These options do not what you think. AICHTrust means that you trust every hash you receive by someone else. If you disable this (which is the right thing to do™), you only trust a hash if more than 10 clients have sent you the hash and at least 92% of all clients had this hash.
ICH is an old way to recover data, but a lot less efficient than AICH. You can savely disable it, but it will not improve hashing speed.

Please have a look at the available RAM during hashing and see if it's really caused by swapping. Seeing your machine has 32 MB RAM, it's very well possible.

[p.r.]

wuischke
So there is no way to disable AICH?

Swapping...  yes,  it present  but  it'snt so heavy.
When I run both rtorrent and amuled  free MEM  is almost constant:  500...700 KB,
 used SWAP is variable: 16000 ... 29000 KB.