Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[hopelessone]

Hi Ya,

I would like to see:

 to be able to load more than 600 + files in paused mode without aMule crashing

...i have 30 downloads going and every time i get above 600+ downloads waiting on paused mode ...i get the error 24 random cannot write... i been waiting for a feature like this..

Thanks...

[Kry]

Raise your system's ulimits for open files.

[lfroen]

amuled will likely to crash in this case anyway (when fd number will pass 1023)

[Kry]

Then raise it.

[skolnick]

I have had more than 800 files queued on my aMule and it works fine. Using it under fedora 10 with default ulimit settings.

Regards.

BTW: Why is the 1023 limit you talk about lfroen? I have loaded like 1300 torrents on azureus and it works, so if aureus can, i guess amule can do it also. Is it something to do with the C++ structure or something?

Regards.

[Kry]

No, the aMuled code. amuleD

[skolnick]

It seems I didn't read properly the post. However I do not get why amule can but amuleD can't if AFAIK they share tons of code.

Regards.

[Kry]

But not the low-level network one. A simple change would suffice to increase it, of course. Or even make it as big as ulimit's

[Stu Redman]

Raise your system's ulimits for open files.

We could also make a pool of, say, 50 file handles for the downloads instead of permanently assigning one handle to each. And then assign the handles at need.

But not the low-level network one.

Can we change that when we switch to wx 3.0 (when wx-sockets become available for non-GUI apps) ?

[Kry]

If we ever do.

[freddy77]

Mumble mumble... I didn't understand the problem. I though was sufficient a number of file descriptors like

   upload connections + download connections + uploading/hashing files

but I noted that partial files require always an open file descriptor, from PartFile.h

   CFile m_hpartfile;  //permanent opened handle to avoid write conflicts

which conflicts refer this comment??

[Stu Redman]

It's just a lazy implementation imho. It would of course be possible to make a pool of filehandles for the downloads and release the longest unused one to save handles. Well - it had been possible at least until you came up with the mapping code. 

[freddy77]

It's just a lazy implementation imho. It would of course be possible to make a pool of filehandles for the downloads and release the longest unused one to save handles. Well - it had been possible at least until you came up with the mapping code. 

Oh my god Well... believe or not is instead a way to support more than 1024 open files!! Well... I have still to test this idea but open a file with open, mmap it, close file descriptor, now you have mmap-ed area which you can write. Then you have remapXXX (I don't remember XXX) which is Linux extension which allow you to move the area without using file descriptor However I think it would be better (if not already implemented) to have some sort of queuing long processes (like hashing) in order to limit disk seeking (wait to finish an hashing before starting with another file) and file opening.

I still don't understand write conflicts problems... a PartFile represent a partial file and real file is accesses using the PartFile object. Obviously you can't handle two tasks on same PartFile at the same time so you use m_hpartfile serialized so I don't see any problem if I open file before operation and close it just after read/write.

About cache I would suggest a LazyClose/LazyOpen either in PartFile or in CFile that don't close file if there are less than X file "lazy opened". A simple counter and a fixed array suffice.

[Stu Redman]

Before you start digging into it:

- The 1024 is only a amuled issue, not an amule issue. With amule, file handles are unlimited (if they are unlimited in your OS at least). For amuled there is a chance to get rid of the limit and the legacy code when wx 3.0 comes out and sockets/events for non-gui apps become available.
- 600 downloads at a time is overdoing things anyway. A machine capable of that can also run amule instead of amuled and so work around the problem.
- I don't want to know what happens if you open a partfile, memory-map part of it for reading, close it (still mapped), reopen it, and then write to it. Especially on the multiple platforms we support. No, thanks (shiver).
- Full hashing is done on background thread (single part hashing is done in the main thread). Download is finished then anyway, and upload is suspended meanwhile. Otherwise access is single-threaded, so there are no write conflicts. Don't trust any ole comment lying around. 

Bottom line: I would just leave it like it is. The risk outweights the gain imho.

[freddy77]

The 1024 is only a amuled issue, not an amule issue. With amule, file handles are unlimited (if they are unlimited in your OS at least). For amuled there is a chance to get rid of the limit and the legacy code when wx 3.0 comes out and sockets/events for non-gui apps become available.

?? I though 1024 was a OS system limit so it would be a limit even for amule GUI... I think that even wx will use a file descriptor for sockets and files... I'm not digging this stuff that much... just I would like to know some aMule detail...