Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[oranies]

I felt very useful to be able to load from a external source the parts of file you lack. I used it in eMule 0.49b Morph 11.0.

When you get the file you requested in Mule from a external site (FTP for instance), the better way I found to put it in the eMule and be sure every chuck was right is to "import" it to the downloading file to complete it. Some times is the whole file, sometimes is only part of it because the FTP failed before ending de downloading.

It's no mauch difficult and it's usefull when you work in a team to share the same file.

[skolnick]

There are problems with this: how to make sure the file is the same? where would you get the parthashes from? I don't understand clearly how would this work.

Regards.

[nikio]

Correct me if  I'm wrong, this is the scenario that I can guess:
I find some other people that are trying to download the same file, so we organize a fast ftp exchange of our partial files to merge them?
this would be quite alike assigning slots each other... but maybe I've misunderstood

[oranies]

The scenario is a launch team.

We live spread (some in Barcelona, some in the other counties of Catalonia and some even Catalonnians over the world, working or studing mainly) and use several methods to deliver the file we want to launch (FTP, megaupdoad, DVD, etc).

To check integrity of files we could use MD5 but we use the hash of the ed2k protocol.

The team manager announce the launch of a file (usually a film Spanish shops doesn't want to sell in Catalonian in spite of Spanish laws) but we got the sound track in this language by another mean (usually Catalonnians Broadcasting TV).

Some members of the launch team have already got the file by direct download (FTP, megaupdoad, DVD, etc). Some only can get the ed2k link and download it (they are the snails). But some get some part of the file from one of theese ways or get some part of the file corrupted.

Then (in this special case) we begin to start to download the file from ed2k and with the headers we ask eMule 0.49b Morph 11.0, to import parts from the external incomplete or corrupted received file.

Then eMule checks every chuck integrety and accept the right ones (the beginning of the file partially downloaded but interrupted or DVD file with wrong chuks).

We've checked that a video seen by Windows Media Player change its MD5, but not only because Windows Media Player show the video but also because it makes a mark in it. It's no the same a DVD/CD with the right MD5 that one slighty wrong that a friend has brought to you. So Quality Controlers (referees) musn't use Windows Media Player.

We are making big efforts to deliver the files with the rigt MD5 but Spanish ADSL not allway alow to have the complete file for the day of the announcement.

As much as Catalonnian communnity is not as large as English or Spanish ones we take the ed2k link, import (with Windows eMule Morph) the main part of the file and we have to download by P2P only the bad chucks.

As much as we consider to move to Linux we miss this utility. At present I've got how to do the whole process in Linux except to import external part files to an existing P2P downloading file. I'm still doing it in Windows and after I copy the nnn.part and nnn.part.met to Linux Temp directory.

The process of importing parts doesn't seem to difficult but it's just a propose you could consider a priority to include or not.

In anything I could help ask for it! (But my priority aim is to preserve Catalonnian from Spanish goverment attacks -in spite 1/3 of the European Spanish people are Catalonnian speakers-).

Thank you,

Jaume.

PS. aMule is giving a very good result even in old machines as Pentium II at 450 MHz.

[Kry]

Careful with the politics. Amongst other things, catalonians represent only 16% if the total Spanish population, so please keep your FUD elsewhere. Along those lines, DVDs being released or not in catalonian language has nothing to do with Spanish law, but with film publishers and producers.

Cut the bullshit and the FUD, I'm Spanish so it won't work with me. Concentrate on your technical issues, and leave politics aside. This is the last and only warning.

[wuischke]

You want to import the incomplete file into a new download or you want to merge the external incomplete file with a running incomplete download?

[Archmage]

I think the idea is good and work if you have the AICH. With this you simply check each Chunk if it is already complete. If it is complete, than skip this chunk, if it isn't than accept the chunk from an extern file, but do a rehash of the file later.

I did try it a few times with Neomule and it seems that it works okay. Best case is where you have a broken FTP download, and a not yet started download in eMule, where you can get nearly all the things that you downloaded and worst case is a nearly finish eMule and nearly finished BT download, where you can get nearly nothing, in fact you might destroy some already started Chunks, but AICH might save later a few chunks (this is because of the different chunks system in both systems).

This is a nice features for people that use different systems to get there files and often have incomplete downloads. But I don't know if this is THAT feature that you need in the next aMule-version.

[Stu Redman]

This would work even without AICH (checking only full chunks), but with AICH it's even better. For this scenario it would be best if the releaser publishes a ED2K link with integrated AICH root hash right away.

I find it useful and don't think it's much of a problem. I'd give it a try if no dev is against it. (I don't care about Spanish/Catalonian politics btw.  )

You want to import the incomplete file into a new download or you want to merge the external incomplete file with a running incomplete download?

I'd say, start the download. When the hashes are there the context menu entry "import incomplete file" gets available. It opens a file dialog to select the file to import.

What about the AICH feature -
- wait until the AICH hash tree is there, then start importing ?
- first import only full chunks, then give it a second pass when AICH gets available ?
- just fill up all available gaps in the part file, rehash and then let AICH sort it out ? I think that's the best.

[Kry]

Sure, I'm not against the idea per se.

[GonoszTopi]

I'd give it a try if no dev is against it.

Sure, go ahead.

About context menu entry, I'd suggest something like:
- when no hashsets available: (disabled) 'Import'
- only ed2k hashset: 'Import (full parts only)'
- both ed2k and AICH hashsets available: 'Import (AICH aided)'

- just fill up all available gaps in the part file, rehash and then let AICH sort it out ? I think that's the best.

I wouldn't want to lose a single byte already downloaded data, even if I import from a completely empty/invalid/etc source.

[oranies]

Careful with the politics

I agree, in Spain, in Neederland, in Belgium, anywhere. In Catalunya, Valencia and Balearic Islands the susceptibilities vary very much (35 % of Spanish popullation is from Catalan/Valencian counties -county is smaller than country- and at least understood this language most of them) so I'm not going to enter to the three parts of the art. 3 of the Spanish Constitution of 1978. The French Pyrenees Orientales has a different social reallity nowadays but they also belong to Ramon Llull Institute. So for me it's a closed mater with no relationship with the feature.

I have interest in the feature itself. It will be usefull for us and for any team working in a spread methode.

You want to import the incomplete file into a new download or you want to merge the external incomplete file with a running incomplete download?

Merge. Include the missing parts to a current download. So I think it's needed the AICH included in some part of the "headers" of the file to complete it.

This would work even without AICH (checking only full chunks), but with AICH it's even better. For this scenario it would be best if the releaser publishes a ED2K link with integrated AICH root hash right away.
[...]
What about the AICH feature -
- wait until the AICH hash tree is there, then start importing ?
- first import only full chunks, then give it a second pass when AICH gets available ?
- just fill up all available gaps in the part file, rehash and then let AICH sort it out ? I think that's the best.

I've seen the feature when you have the ed2k link in the download list, in any state of completition, and then you can import the part of the file you have (or the damaged file). So it would be what you say the best option: "just fill up all available gaps in the part file, rehash and then let AICH sort it out".

About context menu entry, I'd suggest something like:
- when no hashsets available: (disabled) 'Import'
- only ed2k hashset: 'Import (full parts only)'
- both ed2k and AICH hashsets available: 'Import (AICH aided)'

Exactly but as much the user is in a downloading context there's at least the ed2k hashset for the downloading files.

I think the idea is good and work if you have the AICH. With this you simply check each Chunk if it is already complete. If it is complete, than skip this chunk, if it isn't than accept the chunk from an extern file, but do a rehash of the file later.
I did try it a few times with Neomule and it seems that it works okay. Best case is where you have a broken FTP download, and a not yet started download in eMule, where you can get nearly all the things that you downloaded and worst case is a nearly finish eMule and nearly finished BT download, where you can get nearly nothing, in fact you might destroy some already started Chunks, but AICH might save later a few chunks (this is because of the different chunks system in both systems).
This is a nice features for people that use different systems to get there files and often have incomplete downloads. But I don't know if this is THAT feature that you need in the next aMule-version.

Otherwise it's the one I've seen earlier than others like to comment multiple selection file, acknowledge about obfuscation in servers, limit of the share file number in servers, etc.

Others features more "important" I've never used as preview or unzip part files because the references I ask already have certain guaranty.

Our field of work is the Science-Fiction, Fantasy and Horror. And everyday more people get the free registration to get the right ed2k references. Others teams as our also give verified references without registration to aviod nerd stuff run on the network. I think it's not necessay to say names of the webs I work with (Catalonian and not Catalonian); some webs ask for registration other not, some with no registration also give a very good quality works.

I wouldn't want to lose a single byte already downloaded data, even if I import from a completely empty/invalid/etc source.

It's exactly my aim. I'm only asking you to put a certain priority (if any) to importing incomplete files. It will help the migration of the property software to GNU software. Thank you.

[GonoszTopi]

as much the user is in a downloading context there's at least the ed2k hashset for the downloading files.

If you download a file from an ed2k link with only the master-hash and no sources have been yet found, there's no way to know the hashset and thus there's no way to import anything.

And re-thinking naming, I'd better suggest "Import file data" instead of  "Import" in my previous post, to make a clear(?) distinction from the toolbar 'Import' button and functionality.

[Vollstrecker]

In this case, the data could be imported. If the file is complete, the masterhash should be enough to know if all went fine, if the masterhash doesn't match, or the file isn't complete, the parts can be checked when sources are avialable.

[Stu Redman]

Let's keep this (minor) feature not overly complicated:
- import only when ED2K hashset is available
- import only full chunks where either
a) chunk is fully ok in imported file
b) chunk is ok when gaps in partfile are filled from imported file

Everything else requires tracking if a gap is a true gap or a gap filled with untrusted data from import, and that's simply not worth it. Usecase is importing a truncated file or a damaged file. In both cases I don't expect lots of fragmentation (parts smaller than a chunk being ok), so going to AICH will probably benefit in close to nothing. Even b) is already an optimization I rarely expect to yield in anything.

[oranies]

Let's keep this (minor) feature not overly complicated:
- import only when ED2K hashset is available
- import only full chunks where either
a) chunk is fully ok in imported file
b) chunk is ok when gaps in partfile are filled from imported file

Everything else requires tracking if a gap is a true gap or a gap filled with untrusted data from import, and that's simply not worth it. Usecase is importing a truncated file or a damaged file. In both cases I don't expect lots of fragmentation (parts smaller than a chunk being ok), so going to AICH will probably benefit in close to nothing. Even b) is already an optimization I rarely expect to yield in anything.

Thank you for the abstract!

I can't hardly have time for this forum because I prefer to get in in Linux world properly. I'm in the phase of migration and meanwhile keeping a aMule and eMule in production.

As much as Linux lets to work in older machines I'm studying the way to take advantge of the teans like mine have and they are going to let appart because Windows says they're not powerful eanough.

Thank you to you all. I'll keep watching what are you doing and even if I feel sure enough I'll pass from stable to developement version of aMule. I've got very good results with version 2.2.2.