Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[fluffy]

Hi

I am having a problem with files getting seriously fragmented on my EXT3 fs, so therefore i want to know if it would be possible to add an option to prefs to make aMule allocate an entire chunk as soon as data is for that chunk is received, and perhaps even for allocating the entire file (disabling the use of sparse files completely).

Maybe it could look something like this in prefs:
--> Reduce Fragmentation
----------> (x) Allocate Full Chunks
----------> (x) Allocate Full File

The Full Chunk feature would take up a little more space on disc, but would seriously reduce the fragmentation of the downloads, while the Full File feature would completely eliminate fragmentation but would take up the complete space of all the files you are downloading.

What do you guys think?

[KingFish]

afaik *mule allocates data on hdd as soon as it got the first few bytes. Even if you don't have the file complete the is already a file there filled with zeros. Also, it's the first time I hear ext3 fragments very much. iirc I read in the infos about this fs (before I switched my hdds to this format) that ext3 fragments a lot less than other fss.

Don't blame me if it's wrong though, I'm no dev here or at kernel.org

[fluffy]

No. aMule (and xMule) does NOT allocate the space on hdd for the whole file. Only on file systems which does not support sparse files (fat32?) the entire file is written right away.

And the way to allocate the space, which i forgot to mention would be to fill the chunk with zeros, in case that wasnt obvious.

Why does everybody seem to think that ext3 does not get fragmented? If I didn't have a problem with fragmentation, I wouldn't have made this feature request in the first place...

Just to see the extend of my problem: When I transfer a file downloaded by aMule to another computer, I can only transfer at 1200-2500kb/s, whereas when I transfer a unfragmented file, then I transfer at 7000-8000kb/s!! 8o Also aMule is using 5-6 minutes to complete a file, where if the file was unfragmented or at least less fragmented should be able to complete the file (700mb) in less than a minute. If that's not a performance drop caused by fragmentation, I dont know what it is

[Kry]

This thread was created by my request as I approved the feature request on IRC (btw, the 'preferences idea is mine too heh) so it's gonna be on TODO.

[fluffy]

Just to sound impatient and to nag a bit

How far are you with this feature?

I will donate a big *fluff* on irc to whoever implements this

[Kry]

lol fluffy, I'm just busy now

[m2kio]

hi,

you can already select "get 1st and last chunk first" (or similar) in the settings dialog. this should grow the part file to it's final size as soon as the last chunk is retrieved.

       ... m2kio !

[Kry]

No, it would not. Google for 'sparse' files

[Mr Faber]

Does it only works with ext3? I can't activate the function with xfs and I guess in reiser fs too because I used it in the past.
The fragmentation is really horrible but sparse files have  advantages too. Isn't it possible to use sparse file but to try to write bytes after bytes so if there is no other file blocking aMule writes the file in one stream?

cu
Mr Faber

[GonoszTopi]

Yeah, nice idea, next aMule version will contain a kernel patch, too. :rolleyes:

[Mr Faber]

Why not   . But you didn't answered my first question.

cu
Mr Faber

[lfroen]

The whole story: on ext2/3 filesystem space is not allocated when file size is set. Instead, it's allocated only on actual write. This means, that you can have 10 files each 1Gb on disk 2 Gb. An application (amule) can force allocation of disk space by writing let's say zeros to all the file after setting it's size.
This of cause doesn't apply to windoze's of any kind: neither ntfs nor fat32 supports sparse files, and space is allocated in the moment SetFileSize() is called.

GonoszTopi: yea, right, what about amulefs ?  ;(

[Mr Faber]

One point isn't correct. NTFS support sparse files. At least the new eMule client support it with NTFS. I know what sparse files are, from Windows, eMule .
My question isn't answered. XFS and reiserfs definatly support sparse files because only the needed space is allocated but can aMule disable this function too?
Is there any defragmentation program out there (Please no comment that linux doesn't need one )? I really like sparse files. It saves so much space especially if I download many big rare files. I thought that Linux hasn't problems with fragmentation if the disk isn't nearly full (I know that a little bit fragmentation is normal). Don't get me wrong. It is not your problem if the file system code is to stupid to write sparse files in one stream. Maybe they don't wanted fragmentation so they write each file in a row. That is ok if there are no sparse files. But with sparse files they write 180 KB from the first file, 180 from the second and so on (with ed2k blocks). That didn't make sense in this case but I am no expert.

cu
Mr Faber

[lfroen]

NTFS support sparse files.

Yep, wrong me. Starting from 2000 and XP ntfs supports in.

[Mr Faber]

It seems that someone wanted to kid me . I have switched my home partition to ext3 and still can't disable sparsed files? ?( The checkboxes are always grey.
The advantage is that I can test the fragmentation of ext3. Reiserfs and XFS has both very bad performance on downloaded files. 1/3-1/2 from normal transfer rates.
Another point for ext3 ist that if the computer gets halted XFS is very, very bad. By the way aMule have repaired all my part.met files with the backup automatically at start. That was really impressive.  :baby:
A backup of the configuration file would be usefull too If you don't believe me just use XFS and celebrate a blackout

cu
Mr Faber