aMule Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

We're back! (IN POG FORM)

Pages: 1 2 3 [4] 5

Author Topic: Download part _X_ is Corrupt -- too often  (Read 29278 times)

whomnet

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 17
Re: Download part _X_ is Corrupt -- too often
« Reply #45 on: January 28, 2004, 05:13:24 PM »

I have ran for 24 hours amule (ext filesystem, could'nt change to FAT32 though). The results are as follows:

What I have seen is that, when downloading a new file (about 500MB size), the bad chunks didn't appear (except for 2 bad after 60 good). Then, when the file is almost downloaded (1 or 2 chunks last), these last chunks are the erroneus ones, and aMule keeps saying they are bad although it downloads them many times (5 as my patience lasted).
Finally, I made a couple of experiments: the first one was to force a sync before test were done, in order to have the same data in the HD than in the cache... it seems that didn't work.
The second experiment was to pause the file (1 chunk was left to finish, but always being wrongly downloaded).  As more files were being downloaded, the file begun downloading after the first (other) download finished and... ¡Oooh the chunk received was OK and the download finished!

My theory is that there are some desynchronized content, and it gets synchronized after a while. I will keep on testing possibilities to see if my theory is correct.
Logged

werner

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 47
Re: Download part _X_ is Corrupt -- too often
« Reply #46 on: January 28, 2004, 05:56:17 PM »

Quote
Originally posted by whomnet
Finally, I made a couple of experiments: the first one was to force a sync before test were done

As in typing "sync", or adding an fsync() call ? Unless you treat your
system extremely roughly (power off or hard reset without shutdown,
disconnect drive during operation, etc.), this won't make a difference.

BTW, does anyone know if aMule shares data from chunks that have
not been checksummed yet ? If it does, this would be a good thing to
make at least optional, to limit the spread of corrupt data and (more
importantly) to avoid downloading of good data into chunks already
containing bad data.

- Werner
Logged

werner

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 47
simulation says: bad may be good
« Reply #47 on: January 29, 2004, 02:24:50 AM »

Quote
Originally posted by werner
this would be a good thing to make at least optional

I'm no longer so sure about this. A little simulation shows that it
may actually be faster to also download bad data. The traffic
increases, but good data propagates considerably faster than
when waiting for verification.

I've attached the simulation program. See README for details
The .zip is of course .gz - silly forum won't let me pick the
extension I want. Warning: extracts into the current directory.

- Werner
Logged

werner

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 47
37h run
« Reply #48 on: January 29, 2004, 05:54:31 AM »

For the data collectors, here is a ~37h run (Tuesday UTC 15 to
Thursday UTC 4). I don't have the exact average download rate,
but most of the time, it was probably not more than 5-15 kB/s.

bad 0 good 1 ich 0 other 110
bad 0 good 2 ich 0 other 391
bad 0 good 3 ich 0 other 398
bad 0 good 4 ich 0 other 539
bad 0 good 5 ich 0 other 653
bad 0 good 6 ich 0 other 728
bad 0 good 7 ich 0 other 829
bad 0 good 8 ich 0 other 878
bad 0 good 9 ich 0 other 901
bad 0 good 10 ich 0 other 1132
bad 0 good 11 ich 0 other 1149
bad 0 good 12 ich 0 other 1152
bad 0 good 13 ich 0 other 1168
bad 0 good 14 ich 0 other 1201
bad 0 good 15 ich 0 other 1223
bad 0 good 16 ich 0 other 1229
bad 0 good 17 ich 0 other 1257
bad 0 good 18 ich 0 other 1393
bad 1 good 18 ich 0 other 1423
bad 1 good 19 ich 0 other 1437
bad 1 good 20 ich 0 other 1454
bad 1 good 21 ich 0 other 1492
bad 1 good 22 ich 0 other 1662
bad 1 good 23 ich 0 other 1666
bad 2 good 23 ich 0 other 1785
bad 2 good 24 ich 0 other 1849
bad 3 good 24 ich 0 other 1987
bad 3 good 25 ich 0 other 1999
bad 3 good 26 ich 0 other 2064
bad 3 good 27 ich 0 other 2143
bad 3 good 28 ich 0 other 2189
bad 3 good 29 ich 0 other 2264
bad 3 good 30 ich 0 other 2299
bad 3 good 31 ich 0 other 2358
bad 4 good 31 ich 0 other 2449
bad 4 good 32 ich 0 other 2545
bad 4 good 33 ich 0 other 2551
bad 4 good 34 ich 1 other 2567
bad 4 good 35 ich 10 other 2648
bad 4 good 36 ich 14 other 2685
bad 4 good 37 ich 21 other 2781
bad 4 good 38 ich 21 other 2827
bad 4 good 39 ich 22 other 2896
bad 4 good 40 ich 22 other 2913
bad 4 good 41 ich 22 other 2925
bad 4 good 42 ich 27 other 3024
bad 4 good 43 ich 27 other 3039
bad 4 good 44 ich 27 other 3075
bad 4 good 45 ich 27 other 3077
bad 5 good 45 ich 27 other 3081
bad 5 good 46 ich 27 other 3081
bad 5 good 47 ich 47 other 3156
bad 5 good 48 ich 48 other 3163
bad 5 good 49 ich 53 other 3252
bad 5 good 50 ich 53 other 3269
bad 6 good 50 ich 53 other 3282
bad 7 good 50 ich 53 other 3327
bad 8 good 50 ich 53 other 3453
bad 8 good 51 ich 53 other 3530
bad 8 good 52 ich 53 other 3546
bad 8 good 53 ich 53 other 3573
bad 8 good 54 ich 53 other 3663
bad 8 good 55 ich 53 other 3676
bad 8 good 56 ich 53 other 3681
bad 8 good 57 ich 53 other 3690
bad 8 good 58 ich 53 other 3695
bad 8 good 59 ich 53 other 3745
bad 8 good 60 ich 53 other 3777
bad 9 good 60 ich 53 other 3798
bad 9 good 61 ich 53 other 3825
bad 9 good 62 ich 62 other 3892
bad 9 good 63 ich 62 other 3922
bad 10 good 63 ich 62 other 3995
bad 11 good 63 ich 62 other 4030
bad 11 good 64 ich 63 other 4082
bad 11 good 65 ich 63 other 4113
bad 11 good 66 ich 63 other 4194
bad 11 good 67 ich 63 other 4436
bad 11 good 68 ich 63 other 4545
bad 11 good 69 ich 63 other 4550
bad 11 good 70 ich 63 other 4579
bad 11 good 71 ich 63 other 4633
bad 11 good 72 ich 63 other 4667
bad 11 good 73 ich 63 other 4715
bad 11 good 74 ich 63 other 4770
bad 11 good 75 ich 63 other 4844
bad 11 good 76 ich 63 other 4846
bad 11 good 77 ich 63 other 4900
bad 11 good 78 ich 63 other 4900
bad 11 good 79 ich 63 other 4907
bad 11 good 80 ich 63 other 4941
bad 11 good 81 ich 63 other 4943
bad 11 good 82 ich 63 other 4945
bad 11 good 83 ich 63 other 5122
bad 11 good 84 ich 63 other 5145
bad 11 good 85 ich 63 other 5224
bad 12 good 85 ich 63 other 5371
bad 12 good 86 ich 63 other 5444
bad 12 good 87 ich 63 other 5547
bad 13 good 87 ich 63 other 5645
bad 13 good 88 ich 63 other 5720
bad 13 good 89 ich 63 other 5753
bad 13 good 90 ich 63 other 5765
bad 13 good 91 ich 63 other 5821
bad 13 good 92 ich 63 other 5890
bad 13 good 93 ich 63 other 5941
bad 13 good 94 ich 63 other 6016
bad 13 good 95 ich 63 other 6026
bad 13 good 96 ich 63 other 6129
bad 13 good 97 ich 63 other 6193
bad 13 good 98 ich 63 other 6236
bad 13 good 99 ich 63 other 6248
bad 13 good 100 ich 63 other 6292
bad 13 good 101 ich 63 other 6311
bad 14 good 101 ich 63 other 6322
bad 15 good 101 ich 63 other 6349
bad 15 good 102 ich 66 other 6368
bad 15 good 103 ich 84 other 6590
bad 15 good 104 ich 84 other 6600
bad 15 good 105 ich 84 other 6671
bad 15 good 106 ich 84 other 6681
bad 15 good 107 ich 84 other 6708
bad 15 good 108 ich 84 other 6714
bad 15 good 109 ich 84 other 6851
bad 15 good 110 ich 84 other 6967
bad 15 good 111 ich 84 other 7001
bad 15 good 112 ich 84 other 7007
bad 15 good 113 ich 84 other 7010
bad 15 good 114 ich 84 other 7055
bad 15 good 115 ich 84 other 7107
bad 15 good 116 ich 84 other 7134
bad 15 good 117 ich 84 other 7179
bad 15 good 118 ich 84 other 7190
bad 15 good 119 ich 84 other 7231
bad 15 good 120 ich 84 other 7288
bad 15 good 121 ich 85 other 7376
bad 15 good 122 ich 85 other 7392
bad 15 good 123 ich 85 other 7395
bad 15 good 124 ich 85 other 7439
bad 15 good 125 ich 85 other 7450
bad 15 good 126 ich 85 other 7524
bad 15 good 127 ich 85 other 7552
bad 15 good 128 ich 95 other 7641
bad 15 good 129 ich 95 other 7656
bad 15 good 130 ich 95 other 7659
bad 15 good 131 ich 95 other 7675
bad 15 good 132 ich 95 other 7753
bad 15 good 133 ich 96 other 7772
bad 15 good 134 ich 97 other 7813
bad 15 good 135 ich 97 other 7892
bad 15 good 136 ich 97 other 7918
bad 15 good 137 ich 97 other 7922
bad 15 good 138 ich 97 other 7933
bad 15 good 139 ich 97 other 7933
bad 15 good 140 ich 98 other 7963
bad 15 good 141 ich 102 other 8000
bad 15 good 142 ich 102 other 8026
bad 15 good 143 ich 102 other 8090

BTW, looking at how good and bad evolve around bursts of ICH,
I'm slowly getting convinced that ICH is best ignored when
calculating the ratio between good and bad. Does this make
sense ?

- Werner
Logged

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: Download part _X_ is Corrupt -- too often
« Reply #49 on: January 29, 2004, 09:13:13 AM »

Really, I'm reading this, paying attention and such. I have not yet what to say, anyway.
Logged

thepolish

  • Hero Member
  • *****
  • Karma: 2
  • Offline Offline
  • Posts: 896
Re: Download part _X_ is Corrupt -- too often
« Reply #50 on: January 29, 2004, 11:21:13 AM »

hello,

i m running the current cvs from 10.30 today and got a lot of corrupted files on different files i havent with previous version. Maybe a coincidence ?

2004-01-29 10:20:44: Downloaded part 43 is corrupt :(  (xxx.avi)
2004-01-29 10:23:37: Downloaded part 60 is corrupt :(  (xxx.avi)
2004-01-29 10:24:20: Downloaded part 13 is corrupt :(  (yyy.avi)
2004-01-29 10:26:51: Downloaded part 16 is corrupt :(  (zzz.AVI)
2004-01-29 10:28:45: Downloaded part 29 is corrupt :(  (aaa.AVI)
2004-01-29 10:31:10: Downloaded part 28 is corrupt :(  bbb.avi)
2004-01-29 10:37:31: Downloaded part 32 is corrupt :(  (ccc.avi)
2004-01-29 10:42:50: Downloaded part 14 is corrupt :(  qqq.avi)
2004-01-29 10:51:21: Downloaded part 31 is corrupt :(  sss.avi)
2004-01-29 10:57:46: Downloaded part 15 is corrupt :(  qqq.avi)
2004-01-29 11:09:23: Downloaded part 75 is corrupt :(  sss.avi)

something has changed in cvs ?

the polish
Logged
Only after the last tree has been cut down
Only after the last river has been poisoned
Only after the last fish has been caught
Only then you will find out that money cannot be eaten
(Cree Prophecy)

deltaHF

  • Evil Admin
  • Former Developer
  • Hero Member
  • *****
  • Karma: 6
  • Offline Offline
  • Posts: 3920
  • .. Legends may sleep, but they never die ..
    • http://www.amule.org
Re: Download part _X_ is Corrupt -- too often
« Reply #51 on: January 29, 2004, 11:27:38 AM »

Quote
2004-01-28 20:03:52: Creditfile loaded, 51801 clients are known
2004-01-28 20:03:52: Loaded ipfilter with 2518 IP addresses.
2004-01-28 20:03:52: 45 servers in server.met found
2004-01-28 20:03:53: Found 26 part files
2004-01-28 20:03:53: aMule CVS
2004-01-28 20:03:53: Found 10 known shared files
2004-01-28 20:03:53: *******************************.mpg
2004-01-28 20:03:53: *******************************.mpg
2004-01-28 20:04:00: Connecting to DateAttacke.de (62.241.53.17:4242)...
2004-01-28 20:04:00: Connected to DateAttacke.de (62.241.53.17:4242)
2004-01-28 20:04:15: Lost connection to DateAttacke.de (62.241.53.17:4242)
2004-01-28 20:04:25: Connection attempt to DateAttacke.de (62.241.53.17:4242 ) timed out
2004-01-28 20:04:26: Connecting to SILENTMAXX (217.85.99.150:8888)...
2004-01-28 20:04:27: Connecting to ### Die Saugstube ### (207.44.200.40:4242)...
2004-01-28 20:04:28: Connected to ### Die Saugstube ### (207.44.200.40:4242)
2004-01-28 20:04:28: Connection established on: ### Die Saugstube ###
2004-01-28 20:04:28: New clientid is 1762184766
2004-01-28 20:11:15: requested file not found
2004-01-28 20:32:52: requested file not found
2004-01-28 20:35:04: requested file not found
2004-01-28 20:38:42: Finished downloading *******************************.mpg :-)
2004-01-28 20:50:04: requested file not found
2004-01-28 20:54:33: requested file not found
2004-01-28 21:16:14: requested file not found
2004-01-28 21:16:25: requested file not found
2004-01-28 21:37:49: requested file not found
2004-01-28 21:59:32: requested file not found
2004-01-28 21:59:39: requested file not found
2004-01-28 22:21:30: requested file not found
2004-01-28 22:22:12: requested file not found
2004-01-28 22:32:26: Downloading ******************************.rar
2004-01-28 22:32:27: Downloading ******************************.zip
2004-01-28 22:32:28: Downloading ******************************.rar
2004-01-28 22:32:40: Downloading ******************************.rar
2004-01-28 22:32:41: Downloading ******************************.zip
2004-01-28 22:39:56: Downloading ******************************.rar
2004-01-28 22:43:09: requested file not found
2004-01-28 22:56:41: requested file not found
2004-01-28 22:58:18: requested file not found
2004-01-28 23:04:25: Downloaded part 7 is corrupt :(  (*********************************.avi)
2004-01-28 23:04:46: requested file not found
2004-01-28 23:05:39: requested file not found
2004-01-28 23:18:23: Finished downloading ********************************.mpg :-)
2004-01-28 23:26:28: requested file not found
2004-01-28 23:39:49: requested file not found
2004-01-28 23:47:22: requested file not found
2004-01-28 23:48:07: requested file not found
2004-01-28 23:48:52: requested file not found
2004-01-29 00:11:26: Downloaded part 42 is corrupt :(  (******************************.bin)
2004-01-29 00:31:07: Finished downloading *********************************.mpg :-)
2004-01-29 00:38:19: requested file not found
2004-01-29 01:54:21: Finished downloading *********************************.mpg :-)
2004-01-29 03:03:53: Finished downloading *********************************.mpg :-)
2004-01-29 03:31:03: Downloaded part 0 is corrupt :(  (*********************************.rar)
2004-01-29 03:44:56: Finished downloading *********************************.mpg :-)
2004-01-29 04:33:21: Finished downloading *********************************.mpg :-)
2004-01-29 04:45:03: Finished downloading *********************************.bin :-)
2004-01-29 05:04:01: ICH: Recovered corrupted part 0  (*********************.rar)
2004-01-29 05:30:50: ICH: Recovered corrupted part 42  (********************.bin)
2004-01-29 05:57:48: Finished downloading ********************************.mpg :-)
2004-01-29 06:18:40: Finished downloading **********************************.rar :-)
2004-01-29 06:48:32: Finished downloading ********************************.mpg :-)
2004-01-29 07:10:26: Finished downloading **********************************.rar :-)
2004-01-29 07:11:37: User ***** (2257992917) requested your sharedfiles-list -> denied
2004-01-29 07:12:23: Finished downloading *********************************.rar :-)
2004-01-29 07:21:30: Finished downloading *********************************.avi :-)
2004-01-29 07:26:36: Finished downloading *********************************.mpg :-)
2004-01-29 07:26:36: A file with that name already exists, the file has been saved as ***********.mpg
2004-01-29 07:33:39: Finished downloading *********************************.mpg :-)
2004-01-29 08:03:03: Finished downloading *********************************.mpg :-)
2004-01-29 08:25:26: Lost connection to ### Die Saugstube ### (207.44.200.40:4242)
2004-01-29 08:25:26: Disconnected
2004-01-29 08:25:26: Connecting to SILENTMAXX (217.85.99.150:8888)...
2004-01-29 08:25:51: Connection attempt to SILENTMAXX (217.85.99.150:8888 ) timed out
2004-01-29 08:25:51: Connecting to .S.E.D.G. (38.119.96.15:4661)...
2004-01-29 08:25:52: Fatal Error while trying to connect. Internet connection might be down
2004-01-29 08:25:52: Automatic connection to server will retry in 30 seconds
2004-01-29 08:25:52: Disconnected
2004-01-29 08:25:52: Disconnected
2004-01-29 08:26:22: Disconnected
2004-01-29 08:26:22: Connecting to SILENTMAXX (217.85.99.150:8888)...
2004-01-29 08:26:47: Connection attempt to SILENTMAXX (217.85.99.150:8888 ) timed out
2004-01-29 08:26:47: Connecting to .S.E.D.G. (38.119.96.15:4661)...
2004-01-29 08:26:47: Connected to .S.E.D.G. (38.119.96.15:4661)
2004-01-29 08:26:50: Connection established on: .S.E.D.G.
2004-01-29 08:26:50: New clientid is 1762184766
2004-01-29 09:16:19: Finished downloading ************************************.mpg :-)

Quote
deltaHF@deltahf:~/file/amule-cvs> aStats
aStats Script 1.6.3
aMule CVS has been running for 0 Day(s) and 15h27m05s
deltaHF[aMule] is Online .S.E.D.G. (38.119.96.15:4661) with HighID
Total Download: 22,42 GB, Upload: 71,67 GB
Sharing: 33 File(s) / Client(s) in queue: 2005
Download: 19,9 kB/s, Upload: 16,0 kB/s
Host Uptime: an 7 Tage 13:24,  1 Benutzer,  Durchschnittslast: 0,90, 0,88, 0,92
deltaHF@deltahf:~/file/amule-cvs>

whomnet

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 17
Re: Download part _X_ is Corrupt -- too often
« Reply #52 on: January 29, 2004, 12:37:02 PM »

I haven't tired the latest CVS, however let me show some stats of myself:

As an explanation, I will tell that I was downloading a file (just the last part) and about 6 fresh downloads more. I ran all night and in the morning the file was still missing the last chunk. aMule said it was corrupted a dozen of times!. I stopped the file, and so the good parts growth (those of the fresh downloads), with less corrupted parts found.
With aMule still running I used DonkeyDoctor to check the stopped download's chunks, and it said everyting was OK. Run it another time and DonkeyDoctor said the file had corrupted chunks (but I didn't do anything to the part.met file).
As the messages of the DonkeyDoctor were contradictory, I shutted down aMule and then ran the DonkeyDoctor  again. At the first check the program detected that the downloaded part marked as corrupt was OK, and so transferred the file to Incoming.

I can think on many answers of what is happening, but the might not be valid. Any idea, werner, Kry or DeltaHF?

Code: [Select]
BadGoodIchOther: 1 0 0 24 1
BadGoodIchOther: 1 0 1 24 1
BadGoodIchOther: 2 0 1 27 2
BadGoodIchOther: 3 0 1 28 3
BadGoodIchOther: 4 0 1 53 4
BadGoodIchOther: 5 0 1 172 5
BadGoodIchOther: 6 0 1 180 6
BadGoodIchOther: 6 1 1 245 7
BadGoodIchOther: 7 1 1 276 8
BadGoodIchOther: 7 2 1 307 9
BadGoodIchOther: 8 2 1 345 10
BadGoodIchOther: 9 2 1 365 11
BadGoodIchOther: 10 2 1 383 12
BadGoodIchOther: 11 2 1 395 13
BadGoodIchOther: 11 3 1 503 14
BadGoodIchOther: 11 4 1 718 15
BadGoodIchOther: 11 5 1 721 16
BadGoodIchOther: 11 5 2 778 16
BadGoodIchOther: 12 5 2 788 17
BadGoodIchOther: 12 6 2 912 18
BadGoodIchOther: 13 6 2 1013 19
BadGoodIchOther: 14 6 2 1039 20
BadGoodIchOther: 14 7 2 1059 21
BadGoodIchOther: 14 8 2 1118 22
BadGoodIchOther: 14 9 2 1146 23
BadGoodIchOther: 14 10 2 1159 24
BadGoodIchOther: 14 11 2 1237 25
BadGoodIchOther: 15 11 2 1239 26
BadGoodIchOther: 15 11 3 1409 26
BadGoodIchOther: 15 12 3 1461 27
BadGoodIchOther: 15 13 3 1496 28
BadGoodIchOther: 15 13 4 1497 28
BadGoodIchOther: 15 14 4 1516 29
BadGoodIchOther: 15 15 4 1574 30
BadGoodIchOther: 15 16 4 1640 31
BadGoodIchOther: 16 16 4 1662 32
BadGoodIchOther: 16 17 4 1693 33
BadGoodIchOther: 16 18 4 1745 34
BadGoodIchOther: 16 19 4 1767 35
BadGoodIchOther: 16 20 4 1790 36
BadGoodIchOther: 16 21 4 1836 37
BadGoodIchOther: 16 22 4 1843 38
BadGoodIchOther: 16 23 4 1903 39
BadGoodIchOther: 16 24 4 1961 40
BadGoodIchOther: 16 25 4 2025 41
BadGoodIchOther: 16 25 5 2117 41
BadGoodIchOther: 16 26 5 2237 42
BadGoodIchOther: 17 26 5 2418 43
BadGoodIchOther: 17 27 5 2487 44
BadGoodIchOther: 17 28 5 2583 45
BadGoodIchOther: 17 29 5 2592 46
BadGoodIchOther: 17 30 5 2633 47
BadGoodIchOther: 17 31 5 2670 48
BadGoodIchOther: 17 32 5 2833 49
BadGoodIchOther: 17 33 5 2850 50
BadGoodIchOther: 17 34 5 2915 51
BadGoodIchOther: 17 35 5 2965 52
BadGoodIchOther: 17 36 5 2990 53
BadGoodIchOther: 17 37 5 3139 54
BadGoodIchOther: 17 38 5 3281 55
BadGoodIchOther: 17 39 5 3531 56
BadGoodIchOther: 17 40 5 3686 57

Sorry they are in tab-separated rows to easily do the graphs using a spreadsheet.
Logged

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: Download part _X_ is Corrupt -- too often
« Reply #53 on: January 29, 2004, 02:12:33 PM »

Till we fix the buffer overflow corrupting lists on CVS, we cannot guess answers to that.
Logged

werner

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 47
Re: Download part _X_ is Corrupt -- too often
« Reply #54 on: January 30, 2004, 02:59:55 PM »

Quote
Originally posted by whomnet
aMule said it was corrupted a dozen of times!. I stopped the file, and so the good parts growth (those of the fresh downloads), with less corrupted parts found.

Hmm, could be aMule getting confused or a popular source offering
corrupt data.

Quote
With aMule still running I used DonkeyDoctor to check the stopped download's chunks, and it said everyting was OK.

Unfortunately, a run with aMule still caching data probably doesn't
mean much. BTW, is DonkeyDoctor reliable ? Does it use the same
code as some other incarnation of eDonkey ?

Quote
At the first check the program detected that the downloaded part
marked as corrupt was OK, and so transferred the file to Incoming.

Hmm. Could be either DD missing the corruption, aMule halluzinating
there was some, or a good fairy fixing the file while aMule was writing
it to disk :-)

What's interesting is that this was reproducible. So it could mean that
aMule somehow corrupted data that would not be saved to disk, or
that it is using data that may be saved to disk in an incorrect way.
(Or, if the problem is in DonkeyDoctor, it may do the same.)

One thing I'd look out for is the use of ranges. I've seen that aMule
frequently describes byte ranges with a (start, end) pair, there "start"
is the position of the first byte, and "end" the one of the last byte.
It's quite common to use a similar pair, where "end" is one byte
beyond the last byte, so length = end-start, and a loop
for (p = data+start; p != data+end; p++) will traverse the whole range.

A mixup of that kind could explain why aMule sometimes seems to
get lost. (Of course, it's probably something else. It's never the bugs
that seem logical and plausible :-)

- Werner
Logged

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: Download part _X_ is Corrupt -- too often
« Reply #55 on: January 30, 2004, 03:14:41 PM »

lol, well, I'm actually stuck at CList::AddTail saving corrupted data somehow, so I'm plenty of non-logical bugs
Logged

whomnet

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 17
Re: Download part _X_ is Corrupt -- too often
« Reply #56 on: January 30, 2004, 05:45:11 PM »

Maybe I have to introduce DonkeyDoctor (thought it was a long used program, but seems not). DonkeyDoctor can be found at http://usuarios.lycos.es/donkeydoctor/portal.php.
DonkeyDoctor is a Win32 application, I think it is not open source. It was first thought to recover the .part.met files after the common error of deleting them (that's it, when the backups does not work, etc). Now has increased its features, but every feature is related to chunk information: You can see the file information, or change its temp name (001.part to 004.part, i.e.).
 But the feature I use is the "Test met" feature. As I know, it does a complete re-hashing of the .part file, and checks if the .met file has marked as good blocks checked as bad (ie: when rebuilding a .met file you need to do this for not losing your download). If the .met is incongruent with the .part hashes, then you can update the .met with the new data or if the file is fully OK then it is transferred to the Incoming folder (as when completing on *mule). This feature is there since the beginning, and as far as I know it is quite stable.

In the other comments sense: Maybe a source offering corrupt parts will appear, bu this happens to me lots of times, I am beginning to think that it happens more as the file is more long and popular, but have no means to measure it. It does not happen always, but keeps repeating again and again.

I don't have too much time to look around the code, but I would bet that, n some point aMule doues the hashing over a invalid range (maybe the +-1 ibt problem). DonkeyDoctor has never failed me, and the last file (the one that repeater 12 times the bad chunk) was a .rar and all the files were valid, so DonkeyDoctor worked OK.
Logged

werner

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 47
Re: Download part _X_ is Corrupt -- too often
« Reply #57 on: January 30, 2004, 06:15:33 PM »

Quote
Originally posted by whomnet
Maybe I have to introduce DonkeyDoctor

Thanks. I've hard of DD, but, since it's for Windows, didn't pay much
attention to it. I had a look at the DD site, and indeed, there aren't many
signs of the source ...

Quote
This feature is there since the beginning, and as far as I know it is quite stable.

Okay, let's assume DD is doing the right thing.

Quote
Maybe a source offering corrupt parts will appear, bu this happens to me lots of times, I am beginning to think that it happens more as the file is more long and popular, but have no means to measure it. It does not happen always, but keeps repeating again and again.

That would be consistent with a broken client receiving good data but
sending out bad data. What would be interesting is to compare bad
chunks with the good ones, once the latter have been received. If e.g.
the corruption is in the last byte of some part/fragment/whatever it's
called, that would be a strong clue. (And also record where the data
came from, so one can check what the next hop client was. Given
enough data, once could eventually figure out who's the culprit.)

- Werner
Logged

werner

  • Approved Newbie
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 47
more data points
« Reply #58 on: February 08, 2004, 04:27:33 AM »

For each session, just the last entry. Still running 1.2.4.

bad 11 good 80 ich 0 other 4775
bad 4 good 18 ich 7 other 1420
bad 8 good 40 ich 35 other 2359
bad 11 good 14 ich 0 other 2061
bad 8 good 10 ich 0 other 1066
bad 46 good 329 ich 104 other 17922
bad 3 good 33 ich 0 other 2610
bad 4 good 108 ich 0 other 6363
bad 14 good 292 ich 33 other 14423
bad 3 good 57 ich 0 other 4183

So it seems that the good:bad ratio is steadily improving, although
there is a lot of noise.

If there is indeed an improvement, what may be the reason ? Well,
it could of course be that the world around me has decided to send
me better data. It could also be that there was one catastrophic
event in the past, and I'm slowly getting rid of the bad data.
Or ... the libc change did indeed bring some improvement

- Werner.
Logged

Kry

  • Ex-developer
  • Retired admin
  • Hero Member
  • *****
  • Karma: -665
  • Offline Offline
  • Posts: 5795
Re: Download part _X_ is Corrupt -- too often
« Reply #59 on: February 08, 2004, 10:07:44 AM »

Either of those.


Look foward for 1.2.5 in 1 or 2 days and test ;)
Logged
Pages: 1 2 3 [4] 5