aMule Forum

English => Feature requests => Topic started by: Eleksir on October 07, 2006, 10:01:13 AM

Title: Dynamic hashing
Post by: Eleksir on October 07, 2006, 10:01:13 AM
Very simple(to say  :) ) - calculate hash of file while downloading this file, for incomplete files, store incomplete hash in temporary file and time to time update this file. It should be _possible_ to implement.

I'd like to see this function, because during calculating hash of big(700+ Mb), just downloaded file performace of computer is very poor because of very huge disk activity.
Title: Re: Dynamic hashing
Post by: wuischke on October 07, 2006, 10:21:52 AM
Unfortunately this is not possible. You need the whole file in order to create the hash. If you feel like reading some more about this: http://www.ietf.org/rfc/rfc1320.txt (technical specifications) In point 3 all necessary steps are explained and as you'll see this is only possible having the whole file.
Title: Re: Dynamic hashing
Post by: Vollstrecker on October 07, 2006, 11:35:23 AM
Hashing of a noncomplete file, would create a complete different hash that has absolutely nothing to do with the hash of the complete file. Haashing is not an incremental process, it show just the current state.
Title: Re: Dynamic hashing
Post by: Eleksir on October 07, 2006, 12:18:38 PM
Right, it's very hard to implement, it will need firstly space for itermediate data (at least 1/4 - 1/3 of already dowloaded chunk of file) and secondly very inordinary implementation of md4 algorithm, oh, and a huge brain work, but anyway it is possible to implement, theoretically  ;)

http://www.ietf.org/rfc/rfc1320.txt - very interesting stuff  :) , thanks pointing me out.

I asked my friend (he's a big guru in cryptography) he sayd that there is already exist implementation of md5 with "save states" for incomplete files but it requires to save addidtional data which requires (sizeof(incomplete data))/2 of additional space, and it was really hard to implement it,
so this thread may be considered as closed.
Title: Re: Dynamic hashing
Post by: lfroen on October 09, 2006, 11:59:14 AM
Quote
Originally posted by Eleksir
Right, it's very hard to implement

No means No. It's not "hard" it's "theoretically impossible". Read answers again. Go RTFM (link to RFC) and tell your friend to do it too.
Title: Re: Dynamic hashing
Post by: morricone on October 22, 2006, 09:04:36 PM
Well, actually it's impossible to calculate the correct hash value if you don't get the data in a continuous form.

But you can just calculate the hash of the first chunk and safe the context. And as soon as there's new proximate (I had to look that up ;) ) data restart feeding the algorithm. Obviously this only works well, if you don't get the first chunk at last ;)
Title: Re: Dynamic hashing
Post by: kreegee on October 23, 2006, 12:46:15 AM
well, yes, but that only saves you little cpu time, in the end you still have the whole file to know the hash...

and it somehow doesn't make much sense, as we only hash to guarantee the integrity of the file at the time it was finished. if half of the data is weeks old, we could as well just skip it ;)
Title: Re: Dynamic hashing
Post by: amadvance on January 03, 2007, 08:59:35 PM
Why not using the AICH information if available ?

On my understanding it's possibile to validate the file building the AICH tree during the download and compare only the final root hash when the file is complete.

At present the rehashing is very annoying when you are burning a DVD or watching a film :(