aMule Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

We're back! (IN POG FORM)

Author Topic: Dynamic hashing  (Read 4168 times)

Eleksir

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 17
Dynamic hashing
« on: October 07, 2006, 10:01:13 AM »
Very simple(to say  :) ) - calculate hash of file while downloading this file, for incomplete files, store incomplete hash in temporary file and time to time update this file. It should be _possible_ to implement.

I'd like to see this function, because during calculating hash of big(700+ Mb), just downloaded file performace of computer is very poor because of very huge disk activity.
Logged

wuischke

  • Developer
  • Hero Member
  • *****
  • Karma: 183
  • Offline Offline
  • Posts: 4292
Re: Dynamic hashing
« Reply #1 on: October 07, 2006, 10:21:52 AM »
Unfortunately this is not possible. You need the whole file in order to create the hash. If you feel like reading some more about this: http://www.ietf.org/rfc/rfc1320.txt (technical specifications) In point 3 all necessary steps are explained and as you'll see this is only possible having the whole file.
Logged

Vollstrecker

  • Administrator
  • Hero Member
  • *****
  • Karma: 67
  • Offline Offline
  • Posts: 1549
  • Unofficial Debian Packager
    • http://vollstreckernet.de
Re: Dynamic hashing
« Reply #2 on: October 07, 2006, 11:35:23 AM »
Hashing of a noncomplete file, would create a complete different hash that has absolutely nothing to do with the hash of the complete file. Haashing is not an incremental process, it show just the current state.
Logged
Homefucking is killing prostitution

Eleksir

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 17
Re: Dynamic hashing
« Reply #3 on: October 07, 2006, 12:18:38 PM »
Right, it's very hard to implement, it will need firstly space for itermediate data (at least 1/4 - 1/3 of already dowloaded chunk of file) and secondly very inordinary implementation of md4 algorithm, oh, and a huge brain work, but anyway it is possible to implement, theoretically  ;)

http://www.ietf.org/rfc/rfc1320.txt - very interesting stuff  :) , thanks pointing me out.

I asked my friend (he's a big guru in cryptography) he sayd that there is already exist implementation of md5 with "save states" for incomplete files but it requires to save addidtional data which requires (sizeof(incomplete data))/2 of additional space, and it was really hard to implement it,
so this thread may be considered as closed.
« Last Edit: October 07, 2006, 12:47:55 PM by Eleksir »
Logged

lfroen

  • Guest
Re: Dynamic hashing
« Reply #4 on: October 09, 2006, 11:59:14 AM »
Quote
Originally posted by Eleksir
Right, it's very hard to implement

No means No. It's not "hard" it's "theoretically impossible". Read answers again. Go RTFM (link to RFC) and tell your friend to do it too.
Logged

morricone

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 11
Re: Dynamic hashing
« Reply #5 on: October 22, 2006, 09:04:36 PM »
Well, actually it's impossible to calculate the correct hash value if you don't get the data in a continuous form.

But you can just calculate the hash of the first chunk and safe the context. And as soon as there's new proximate (I had to look that up ;) ) data restart feeding the algorithm. Obviously this only works well, if you don't get the first chunk at last ;)
Logged

kreegee

  • Full Member
  • ***
  • Karma: 2
  • Offline Offline
  • Posts: 160
    • http://kreegee.cycovery.com
Re: Dynamic hashing
« Reply #6 on: October 23, 2006, 12:46:15 AM »
well, yes, but that only saves you little cpu time, in the end you still have the whole file to know the hash...

and it somehow doesn't make much sense, as we only hash to guarantee the integrity of the file at the time it was finished. if half of the data is weeks old, we could as well just skip it ;)
Logged

amadvance

  • Newbie
  • Karma: 2
  • Offline Offline
  • Posts: 4
Re: Dynamic hashing
« Reply #7 on: January 03, 2007, 08:59:35 PM »
Why not using the AICH information if available ?

On my understanding it's possibile to validate the file building the AICH tree during the download and compare only the final root hash when the file is complete.

At present the rehashing is very annoying when you are burning a DVD or watching a film :(
Logged