aMule Forum
English => en_Bugs => Topic started by: jmccorm on January 09, 2004, 09:30:11 PM
-
Too often I am getting this error on files. Luckily, they are videos so I can watch most of them, but it is preventing these files from completing. I hate to put this in the bug forum, but I don't know if I've come across a bad batch of files, or I've got a compatibilitiy issue?
aMule 1.2.3 on Mandrake 9.2.
Example files:
ed2k://|file|XXXXXXXXXXXXX.avi|226400256|XXXXXXXXXXXXXXXXXXX|/
edit: no ed2k links plz
-
its not amules problem.. if u receiving corrupted files its related to the files
greets
delta
-
If it's really too often, more than could possiblity be possible reduce your packet size.
-
I've got the same problem, which I didn't use to have withe aMule 1.2.2 and previous.
aMule 1.2.3 (now 1.2.4, but I don't thing it will solve that problem) and mdk 9.2
-
No packet-handling code was seriously changed on 1.2.3 / 1.2.4 so I guess it's not version - related.
-
hello,
Could be a nice feature for next releases to be able to select in pref an option to ban people who send corrupted data to often to avoid dl and dl again useless data
the polish
-
Originally posted by thepolish
hello,
Could be a nice feature for next releases to be able to select in pref an option to ban people who send corrupted data to often to avoid dl and dl again useless data
the polish
Hi,
I'm not familiar with the underlying code of *mule, but I suspect that it would be hard to identify a malicious user, because generally many users contribute to one chunk. Corruption detection takes place when one chunk is finished, and usually there have been multiple uploaders for that chunk.
On the other hand: I (once again) don't really know how ICH works, but *if* it keeps track of the subparts of one chunk, then this data may be used to ban corrupt data uploaders.
Any devs to comment?
-fz
-
its "impossible" .. my 2cents ;)
-
Originally posted by emperor
If it's really too often, more than could possiblity be possible reduce your packet size.
How to reduce the packet size? I have the same too many corrupt chunks error. It is not dependant on the file, it happens since I can remember using amule. Also, using DonkeyDoctor (with wine) to chek files, most of the times the files are correctly downloaded, but aMule says not and throws away some chunks.
Note: I am Spanish, and reported this same error but in spanish in the spanish forum, I think Kry is looking for it... :))
-
How to reduce the packet size? I have the same too many corrupt chunks error. It is not dependant on the file, it happens since I can remember using amule. Also, using DonkeyDoctor (with wine) to chek files, most of the times the files are correctly downloaded, but aMule says not and throws away some chunks.
Note: I am Spanish, and reported this same error but in spanish in the spanish forum, I think Kry is looking for it... :))
Reducing the packet size only helps if the files really GET corrupted while transfering. If only amule says they are corrupted although they aren't it won't help.
-
Hi!
I've got the same problem, when I want to download a file of 80Mb, aMule (the same with xMule) transfer ~400Mb but complet only 40Mb (this file is not finish yet).
This is really boring.
I tried to install MLdonkey but downloads can't began at all, maybe it can help to find why it doesn't work.
I have Mandrake 9.2.
Please find how to make it run fine.
(Sorry for my poor english)
Thanx men.
-
the file itself is broken.. try to cancel it and dl again
-
I've got the same problem. I downloaded about 10 150MB files and there is no file without a corruption. Normal are about 3 corruptions per file.
-
All of my downloads are corrupts (15 downloads).
I tried to delete aMule and installed it again, also to clear all my downloads and download other files. It still doesn't work.
This is my log:
2004-01-19 18:42:07: Creditfile loaded, 2170 clients are known
2004-01-19 18:42:07: Loaded ipfilter with 0 IP addresses.
2004-01-19 18:42:07: 62 servers in server.met found
2004-01-19 18:42:07: Found 15 part files
2004-01-19 18:42:07: aMule 1.2.3
2004-01-19 18:42:07: Found 14 known shared files
2004-01-19 18:42:07: Connecting
2004-01-19 18:42:07: Disconnected
2004-01-19 18:42:07: Connecting to Fuckbird.com & Sexallinone.com Totally Free Porn Sites (66.111.43.80:4242)...
2004-01-19 18:42:07: Connected to Fuckbird.com & Sexallinone.com Totally Free Porn Sites (66.111.43.80:4242)
2004-01-19 18:42:07: Lost connection to Fuckbird.com & Sexallinone.com Totally Free Porn Sites (66.111.43.80:4242)
2004-01-19 18:42:08: Connecting to Razorback (195.245.244.243:4661)...
2004-01-19 18:42:08: Connected to Razorback (195.245.244.243:4661)
2004-01-19 18:42:08: Connection established on: Razorback
2004-01-19 18:42:08: New clientid is 3655798352
2004-01-19 18:42:08: Received 8 new servers
2004-01-19 18:44:50: Downloaded part 12 is corrupt :( (Le Monde De Nemo-French Dvdrip-By Skyw4Lkerteste Divxovore.avi)
2004-01-19 18:54:23: Found 14 known shared files
2004-01-19 19:18:25: Downloaded part 7 is corrupt :( (Le Monde De Nemo-French Dvdrip-By Skyw4Lkerteste Divxovore.avi)
2004-01-19 19:21:56: Downloaded part 11 is corrupt :( (Les.Nuls.L'integrule.FRENCH.DVDRip.XViD.Cd2.[by.madjix].teste.http://www.divxovore.com.avi)
2004-01-19 19:32:09: Downloaded part 28 is corrupt :( (12.08.2003.MATRIX.RELOADED.DVDRIP.FRENCH.XVID.1CD.REPACK-PYT.AVI)
2004-01-19 19:42:24: Downloaded part 1 is corrupt :( (Scary.Movie.3.FRENCH.TS.DivX.1.CD.REPACK-PYTEAMARENA_www.divxophile.fr.st.AVI)
2004-01-19 19:45:31: Downloaded part 19 is corrupt :( (12.08.2003.MATRIX.RELOADED.DVDRIP.FRENCH.XVID.1CD.REPACK-PYT.AVI)
2004-01-19 19:47:06: Downloaded part 67 is corrupt :( (Le Monde De Nemo-French Dvdrip-By Skyw4Lkerteste Divxovore.avi)
2004-01-19 20:06:39: Downloaded part 25 is corrupt :( (Scary.Movie.3.FRENCH.TS.DivX.1.CD.REPACK-PYTEAMARENA_www.divxophile.fr.st.AVI)
2004-01-19 20:11:29: Downloaded part 4 is corrupt :( (Les.Nuls.L'integrule.FRENCH.DVDRip.XViD.Cd2.[by.madjix].teste.http://www.divxovore.com.avi)
2004-01-19 20:15:09: Downloaded part 9 is corrupt :( (Le Monde De Nemo-French Dvdrip-By Skyw4Lkerteste Divxovore.avi)
2004-01-19 20:17:58: Downloaded part 7 is corrupt :( (Les.Nuls.L.integrule.FRENCH.DVDRip.XViD.Cd1.[by..madjix].avi)
2004-01-19 20:24:02: Downloaded part 5 is corrupt :( (Scary.Movie.3.FRENCH.TS.DivX.1.CD.REPACK-PYTEAMARENA_www.divxophile.fr.st.AVI)
2004-01-19 20:25:53: Downloaded part 11 is corrupt :( (12.08.2003.MATRIX.RELOADED.DVDRIP.FRENCH.XVID.1CD.REPACK-PYT.AVI)
2004-01-19 20:29:59: Downloaded part 20 is corrupt :( (Les.Nuls.L.integrule.FRENCH.DVDRip.XViD.Cd1.[by..madjix].avi)
2004-01-19 20:34:42: Downloaded part 29 is corrupt :( (dionysos.live.a.strasbourg.xvid.ac3.cd.1.avi)
2004-01-19 20:35:20: Downloaded part 48 is corrupt :( (Les.Nuls.L.integrule.FRENCH.DVDRip.XViD.Cd1.[by..madjix].avi)
2004-01-19 20:35:35: Downloaded part 11 is corrupt :( (Scary.Movie.3.FRENCH.TS.DivX.1.CD.REPACK-PYTEAMARENA_www.divxophile.fr.st.AVI)
2004-01-19 20:37:41: Downloaded part 2 is corrupt :( ([Album_MP3]_-_Dionysos_-_Western_sous_la_neige_-_2002_---_Covers_-_EAC_-_Lame_-_VBR_-_by_PatHibulaire.rar)
2004-01-19 21:04:32: Downloaded part 3 is corrupt :( (09.11.2003.MATRIX.REVOLUTION.FRENCH.TS.DIVX.1CD.REPACK-PYTEAMARENA.avi)
2004-01-19 21:06:25: Downloaded part 5 is corrupt :( (Les.Nuls.L'integrule.FRENCH.DVDRip.XViD.Cd-Bonus.[by.madjix].rar)
2004-01-19 21:09:00: Downloaded part 18 is corrupt :( (12.08.2003.MATRIX.RELOADED.DVDRIP.FRENCH.XVID.1CD.REPACK-PYT.AVI)
2004-01-19 21:11:44: Downloaded part 42 is corrupt :( (Les.Nuls.L.integrule.FRENCH.DVDRip.XViD.Cd1.[by..madjix].avi)
2004-01-19 21:12:02: Downloaded part 0 is corrupt :( (bernie_[Divx_Francais].avi)
2004-01-19 21:18:32: Downloaded part 4 is corrupt :( (Scary.Movie.3.FRENCH.TS.DivX.1.CD.REPACK-PYTEAMARENA_www.divxophile.fr.st.AVI)
2004-01-19 21:24:58: Downloaded part 18 is corrupt :( (09.11.2003.MATRIX.REVOLUTION.FRENCH.TS.DIVX.1CD.REPACK-PYTEAMARENA.avi)
2004-01-19 21:30:30: Downloaded part 9 is corrupt :( (Les.Nuls.L.integrule.FRENCH.DVDRip.XViD.Cd1.[by..madjix].avi)
2004-01-19 21:52:40: Downloaded part 6 is corrupt :( (Les.Nuls.L.integrule.FRENCH.DVDRip.XViD.Cd1.[by..madjix].avi)
2004-01-19 22:01:22: Downloaded part 13 is corrupt :( (12.08.2003.MATRIX.RELOADED.DVDRIP.FRENCH.XVID.1CD.REPACK-PYT.AVI)
2004-01-19 22:02:04: Downloaded part 21 is corrupt :( (Les.Nuls.L.integrule.FRENCH.DVDRip.XViD.Cd1.[by..madjix].avi)
2004-01-19 22:21:44: Downloaded part 32 is corrupt :( (09.11.2003.MATRIX.REVOLUTION.FRENCH.TS.DIVX.1CD.REPACK-PYTEAMARENA.avi)
2004-01-19 22:36:35: Downloaded part 0 is corrupt :( (Dolly - Je n'veux pas rester sage (live NPA 97).mpg)
2004-01-19 22:38:58: Downloaded part 50 is corrupt :( (Le Monde De Nemo-French Dvdrip-By Skyw4Lkerteste Divxovore.avi)
2004-01-19 22:39:46: Downloaded part 2 is corrupt :( (Les.Nuls.L.integrule.FRENCH.DVDRip.XViD.Cd1.[by..madjix].avi)
2004-01-19 22:42:36: Downloaded part 7 is corrupt :( ([Album_MP3]_-_Dionysos_-_Western_sous_la_neige_-_2002_---_Covers_-_EAC_-_Lame_-_VBR_-_by_PatHibulaire.rar)
2004-01-19 22:44:53: Downloaded part 45 is corrupt :( (dionysos.live.a.strasbourg.xvid.ac3.cd.1.avi)
2004-01-19 22:48:32: Downloaded part 3 is corrupt :( (bernie_[Divx_Francais].avi)
2004-01-19 22:57:36: Downloaded part 27 is corrupt :( (09.11.2003.MATRIX.REVOLUTION.FRENCH.TS.DIVX.1CD.REPACK-PYTEAMARENA.avi)
2004-01-19 23:00:12: Downloaded part 7 is corrupt :( (09.11.2003.MATRIX.REVOLUTION.FRENCH.TS.DIVX.1CD.REPACK-PYTEAMARENA.avi)
2004-01-19 23:14:35: Downloaded part 23 is corrupt :( (Le Monde De Nemo-French Dvdrip-By Skyw4Lkerteste Divxovore.avi)
Don't you think it is amazing?
I'm not the only one who have got this problem of corruption.
I tried all I can do.
-
r u sure u have enough free diskspace? maybe not and the chunks cannot be writed..
and it seems ur the only one with this probs.. cos noone else is getting so many corrupted parts..
btw.. WHY using 1.2.3 ??? we have 1.2.4 with a bug fix..
do u have ICH enabled ?
greets
-
ok, the corruptions do not happen so often to me as to him, but i have more than enough disk space, I.C.H. is enabled too but has recovered only 2 or three parts so far.
btw I'm using 1.2.4 self compiled with debian sid 2.4.22.
-
I'm also seeing a very large number of corruptions with aMule 1.2.4.
With xMule, this number is considerably lower. (Which may of course
be simply related to aMule moving more data. I didn't count corruptions
vs. amout of data transferred.)
Anyway, here are a few ideas for how this could be improved:
- if *Mule currently offers data from not yet verified chunks for upload,
it could stop doing so. That way, files spread more slowly, but
more importantly, the spread of corruption is slowed even more.
- I think the recovery strategy is to discard the entire chunk, and
to try again. It would be better to keep all downloaded parts,
keyed either by origin or by transfer session. aMule could then
try to combine fresh data with old data, to try to replace only the
corrupt part.
- if implementing the above, one could also rate sources by how
often/how much they contribute to producing a valid chunk. This
rating should be considered experimental, because there are
obviously several ways for subverting it.
- if data is keyed by origin, aMule could avoid asking sources for
parts of unknown validity) it has already received from them.
(Note: this assumes that data does not get corrupted during transfer.)
Keeping data of corrupt chunks around has the disadvantage that
disk space requirements increase unpredictably.
- Werner
-
Sorry man but there's a severe problem with the files you're downloading (you should reconsider, btw, not posting you're downloading copyrighted stuuf). If you do a global search of the names of the files, there are 1-2 results with lots of sources, then several dozens of 1 source elinks, which means the hash changed.
I really think is files related.
-
I also had the same problem with my amule the last week. I downlaoded some fiel 4times the amount of data it should be. Still they did not finisch.
After that I was a little frustrated placed my temp files on another computer and created a FAT partition on my SuSE.
afer that I run my temp files again an now I have a normal amound of currupt parts, and my files finally finish. ole!
maybe the people who have problem with currupt parts should consider creating a fat partition, at least it helped for me :D
stefanero
-
Hum. Then is ext2/3 related?
-
could be I don't know friend of mine uses amule fine with ext3. he had no problems at all. :rolleyes:
but I have to say he did not let it run very long. X( he went back to windoof (german)
in the beginning I had no problems also with currupt parts, but say after a week or so the problems seamed to increase.
don't know, have a fat partition now and everythign runs just great 8)
-
Originally posted by Kry
Hum. Then is ext2/3 related?
Maybe... I'm running it on JFS without any problems..
-
One more data point: I recompiled aMule 1.2.4 with
--enable-debug --disable-optimise, I'm running it now under gdb,
and I seem to be getting considerably less corruption.
This smells like either a compiler or library problem (*), or some
other data corruption in aMule that only hits vital data under
certain circumstances.
(*) E.g. https://rhn.redhat.com/errata/RHSA-2003-325.htm
System configuration:
aMule 1.2.4
wxWindows 2.4.1
GTK 1.2.10
gcc 3.2 20020903 (Red Hat Linux 8.0 3.2-7)
libc 2.2.93
libstdc++ 3.2 (libstdc++.so.5.0.1)
File system ext3
I've added some instrumentation to track the relative frequency of
corruption. Patch attached. (It's for 1.2.4, but is offset by one line,
because of some other changes not relevant to the issue at hand.)
Output so far (that's for about four hours of relatively low traffic,
around Jan 23, 03-07 UTC, with an increase of on-disk data of
about 20-50 MB/h):
bad 1 good 0 ich 0 other 175
bad 2 good 5 ich 0 other 648
bad 3 good 6 ich 0 other 745
bad 4 good 10 ich 0 other 1026
bad 5 good 10 ich 0 other 1040
bad 6 good 10 ich 0 other 1047
(The last three occurred within 5 minutes between "bad 4" and
"bad 6". After that, things were quiet for 30 minutes, at which time
I stopped the test.)
- Werner
-
aMule spit them out during shutdown:
bad 7 good 13 ich 0 other 1164
bad 8 good 16 ich 0 other 1177
Now preparing to run without gdb and default "configure"
settings.
- Werner
-
(*) E.g. https://rhn.redhat.com/errata/RHSA-2003-325.htm
Obviously, RH don't use 8+3 names, so make that
https://rhn.redhat.com/errata/RHSA-2003-325.html
The potential problem is the one at
http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=90077
Sorry,
- Werner
-
Well, it does NOT happen on my side so it might not be code's fault... or maybe yes.
:/
-
like i said.. not aMule fault cos i should have then the prob too .. actually i have 1 or 2 corrupted parts / week ..
thx for tracking werner :)
-
Originally posted by deltaHF
like i said.. not aMule fault cos i should have then the prob too ..
We'll see :-) If it's some memory corruption caused by aMule,
environmental factors can make a huge difference. Likewise,
if there is any possibility for a race condition in the hashing
part.
Nothing interesting to report from the non-gdb run: traffic is
very slow this morning at ~9kB/h, and no corruption so far.
- Werner
-
There is 60% sure a memory corruption is happening on amule, and I'm tracking it :)
-
A run without gdb, 5.5 hours, 08-13:30 UTC, 10-35 MB/h:
bad 1 good 2 ich 0 other 206
bad 2 good 4 ich 0 other 464
bad 3 good 5 ich 0 other 539
bad 4 good 6 ich 0 other 646
And during shutdown:
bad 5 good 8 ich 0 other 685
Now trying with a more recent glibc. (2.3.2-4.80.8 )
- Werner
-
Now trying with a more recent glibc. (2.3.2-4.80.8 )
Of course, upon realizing that I wasn't running gdb, and wasn't
watching the machine either, aMule crashed almost instantly :-(
With this glibc, the gdb setup procedure changes as follows:
Program received signal SIG32, Real-time event 32.
0x406dea35 in pthread_getconcurrency () from /lib/i686/libpthread.so.0
(gdb) ha SIG32 nostop noprint pass
- Werner
-
After the libc upgrade, 4.5h (16-20:30 UTC), 5.5-19 MB/h:
bad 1 good 0 ich 0 other 40
bad 1 good 1 ich 0 other 73
bad 1 good 2 ich 0 other 95
bad 1 good 3 ich 0 other 140
bad 1 good 4 ich 0 other 153
bad 1 good 5 ich 0 other 209
bad 1 good 6 ich 0 other 233
bad 1 good 7 ich 0 other 288
bad 2 good 7 ich 0 other 332
... and then aMule segfaulted. (I've changed aMule to also print
the counters when incrementing the "good" ones.)
This differs from the roughly 1:2:0:200 ratios I obtained with an older
libc. However, there's not enough data in this for me to claim that
the result is significant.
Note that, if indeed a libc bug caused data corruption, some of the
corrupt data may be on disk (aMule does commit unverified data
to disk, right ?), so even if the problem is fixed now, it may take a
while until the corrupt chunks drop to near-zero.
- Werner
-
A longer run (8.5h, 21-5:30 UTC, 9-35 MB/h) with the new libc
didn't show an improvement:
bad 1 good 0 ich 0 other 67
bad 2 good 0 ich 0 other 115
bad 3 good 0 ich 0 other 130
bad 4 good 0 ich 3 other 172
bad 4 good 1 ich 3 other 198
bad 4 good 2 ich 3 other 218
bad 4 good 3 ich 4 other 291
bad 4 good 4 ich 4 other 380
bad 4 good 5 ich 4 other 431
bad 5 good 5 ich 4 other 472
bad 5 good 6 ich 4 other 566
bad 5 good 7 ich 5 other 645
bad 5 good 8 ich 6 other 701
bad 5 good 9 ich 6 other 728
bad 5 good 10 ich 6 other 794
bad 5 good 11 ich 6 other 837
bad 5 good 12 ich 6 other 922
After shutdown:
bad 5 good 13 ich 7 other 983
bad 5 good 14 ich 7 other 985
bad 5 good 15 ich 7 other 992
bad 5 good 16 ich 7 other 996
bad 6 good 16 ich 7 other 1010
bad 7 good 16 ich 7 other 1011
Next is the upgrade from wxBase/wxGTK 2.4.1 to 2.4.2.
- Werner
-
First run, ca. 8h, ended by X session failing (weird. not sure what's
going on):
bad 16 good 41 ich 1 other 3148
Second run, ca. 4.5h, UTC 16:30-21, ca. 20 MB/h, ended by aMule
segfaulting (see posting in "Backtraces"):
bad 3 good 5 ich 0 other 397
We seem to be back to the 1:2:0:200 ratio. Of course, if my theory
that most of the corruption is still on disk is true, I'll need a few
hundred more good or bad hashes before there should be any
significant change in the numebers. We'll see.
Both runs with the latest glibc (without fopen data corruption bug),
wx 2.4.2. Note: aMule feels a lot less responsive since upgrading
to wx 2.4.2.
- Werner
-
werner, do you have any conclussions? I have the same problem, today I will move the "Temp" and "Incoming" directories to a FAT32 partition to se if it mades the change. I think it could be too some syncronization problem, but I don't know how to demonstrate my hipothesis (I only know that, although aMule says the parts are corrupt, the aren't when cheching with DonkeyDoctor).
As you can see (Kry, deltaHF) it is not an isolated problem.
"Seguiremos informando" 8)
-
I also have about 20-30 corrupted parts in 24h, having compiled wxGTK, xwBase and aMule by hand and running a AMD 770MHZ, SuSE 9 on 2.4.23 Kernel. Fiesystem is ReiserFS.
It it shurly not a single problem . . .
Thanks for making aMule, its greatful
Sebastian :]
-
well, i finished 7 dl's in last 24h ..
corrupted files i'm getting only by 1 file that isn't finished yet
-
Originally posted by whomnet
werner, do you have any conclussions?
Unfortunately not. I now have a fairly long run (19+ h so far,
~45 MB/h, started at UTC 21). The patten has changed, but
I'm not sure how to interpret it. The last entry:
bad 13 good 94 ich 61 other 5383
good/bad is considerably better than it used to be, but also
a lot of "ich" is happening, and I don't know how to factor this
in. Also, weekend traffic may have different properties than
workday traffic.
The good thing is that, if the corruption is also on disk, and one
of the changes I've made (libc upgrade, wx upgrade) has stopped
it, enough parts should have been hashed by tonight that future
test should yield a different good/bad ratio (well, as long as ICH
doesn't keep on adding noise).
Regarding FAT32: this seems very odd to me. About the only thing
that is different is that FAT32 internally doesn't have holes, so a
file always occupies the space indicated by its length. Now, while
it's true that aMule is a heavy user of holes, so are a lot of other
programs, and I'd be rather surprised if there was any kernel bug in
the handling of holes. For user space (libc, application), it's almost
transparent what the kernel does with holes.
- Werner
-
Saturday, 21 UTC to Sunday, ~21 UTC, ca. 46 MB/h, ended with a
segfault (see "Backtraces"), same system as before. Last entry:
bad 16 good 115 ich 61 other 6408
Full trace attached. What's special: lots of ICHs (a lot more than I used
to get), and a significantly better good/bad ratio (which may or may
not be significant).
The general trend seems to be for things to start poorly, and then to
improve. This could be just noise, it could be a normal property of
long-running sessions, it could be a weekend effect, or it could
mean that my files are getting more healthy.
BTW, I'd recommend in any case to upgrade libc if the current
version has the fopen bug. It's very subtle, and can cause weird
problems.
- Werner
-
From the run that ended in a very early (1-2h) crash:
bad 0 good 1 ich 0 other 55
bad 0 good 2 ich 0 other 91
bad 0 good 3 ich 0 other 118
What's interesting here is that this is the first time since taking these
traces that I had "good" count up without any "bad" hashes. This
would support the "libc" theory.
Things that don't support it:
- reports that independent (non-Unix :-( ) tools have verified the
correctness of on-disk files for other people
- reports that moving to FAT32 made the problems disappear
instantly
Of course, there's always the possibility that we're seeing the results
of more than one problem.
- Werner
-
9.25h, UTC 8-17:15, 65 MB/h, run ended with segfault:
bad 11 good 64 ich 31 other 3965
Less good than yesterday's 24h run, but apparently better than
the early ones. Lots of ICH, that may or may not skew the numbers.
A common pattern I notice is that, on each run, initially relatively
many corruptions are detected, and that things improve over time.
This is probably because of how aMule selects sources.
- Werner
bad 1 good 0 ich 0 other 107
bad 1 good 1 ich 0 other 156
bad 1 good 2 ich 0 other 203
bad 1 good 3 ich 0 other 235
bad 2 good 3 ich 0 other 335
bad 2 good 4 ich 0 other 378
bad 3 good 4 ich 0 other 387
bad 4 good 4 ich 0 other 405
bad 4 good 5 ich 0 other 466
bad 4 good 6 ich 0 other 471
bad 4 good 7 ich 0 other 472
bad 5 good 7 ich 0 other 491
bad 5 good 8 ich 0 other 569
bad 5 good 9 ich 0 other 710
bad 5 good 10 ich 4 other 935
bad 5 good 11 ich 4 other 1059
bad 5 good 12 ich 4 other 1108
bad 5 good 13 ich 4 other 1235
bad 5 good 14 ich 4 other 1301
bad 5 good 15 ich 4 other 1369
bad 5 good 16 ich 4 other 1524
bad 5 good 17 ich 4 other 1530
bad 5 good 18 ich 4 other 1554
bad 5 good 19 ich 4 other 1660
bad 5 good 20 ich 4 other 1699
bad 5 good 21 ich 4 other 1709
bad 5 good 22 ich 4 other 1775
bad 5 good 23 ich 4 other 1814
bad 5 good 24 ich 4 other 1815
bad 5 good 25 ich 4 other 1941
bad 6 good 25 ich 4 other 1960
bad 6 good 26 ich 4 other 2057
bad 6 good 27 ich 4 other 2059
bad 6 good 28 ich 4 other 2076
bad 6 good 29 ich 4 other 2084
bad 6 good 30 ich 5 other 2143
bad 6 good 31 ich 6 other 2151
bad 6 good 32 ich 6 other 2152
bad 6 good 33 ich 10 other 2183
bad 6 good 34 ich 15 other 2453
bad 6 good 35 ich 16 other 2481
bad 6 good 36 ich 17 other 2527
bad 6 good 37 ich 17 other 2548
bad 6 good 38 ich 17 other 2568
bad 6 good 39 ich 19 other 2637
bad 6 good 40 ich 20 other 2674
bad 6 good 41 ich 24 other 2720
bad 7 good 41 ich 25 other 2743
bad 7 good 42 ich 25 other 2789
bad 7 good 43 ich 25 other 2797
bad 7 good 44 ich 25 other 2812
bad 7 good 45 ich 25 other 2852
bad 7 good 46 ich 25 other 3032
bad 7 good 47 ich 25 other 3034
bad 7 good 48 ich 26 other 3180
bad 7 good 49 ich 26 other 3204
bad 8 good 49 ich 26 other 3329
bad 8 good 50 ich 26 other 3376
bad 8 good 51 ich 26 other 3395
bad 9 good 51 ich 26 other 3422
bad 9 good 52 ich 28 other 3517
bad 9 good 53 ich 31 other 3538
bad 9 good 54 ich 31 other 3551
bad 9 good 55 ich 31 other 3563
bad 9 good 56 ich 31 other 3577
bad 9 good 57 ich 31 other 3600
bad 10 good 57 ich 31 other 3615
bad 10 good 58 ich 31 other 3719
bad 10 good 59 ich 31 other 3827
bad 10 good 60 ich 31 other 3852
bad 11 good 60 ich 31 other 3856
bad 11 good 61 ich 31 other 3863
bad 11 good 62 ich 31 other 3923
bad 11 good 63 ich 31 other 3940
bad 11 good 64 ich 31 other 3965
Edited by BigBob to see inline without the mess to download the attachment ...
-
Using werner's data I have made plots of the evolution of the chunks, and get some (important?) notes:
1.- The corruted chunks have a somehow constant rate, the other types don't modify this rate.
2.- If considered "total good" chunks both the good and the ich recovered chunks, it can be seen that the ratio between total goods and bads fluctuates around 4:6, wich is pretty unacceptable.
I am going to patch my amule with the werner's patch and recompile it to check if my system has equal proportions, and also to try finding some (easy) solution...
[PD: edited to attach the resulting figures in zipped pdf format]
-
Originally posted by whomnet
2.- If considered "total good" chunks both the good and the ich recovered chunks, it can be seen that the ratio between total goods and bads fluctuates around 4:6, wich is pretty unacceptable.
Seems that you have good and bad reversed. Fortunately, I normally
don't get more bad than good chunks :-)
BTW, Monday night was a good night, UTC 22-5 or 6, ~65 MB/h:
bad 0 good 1 ich 0 other 16
bad 0 good 2 ich 0 other 277
bad 0 good 3 ich 0 other 555
bad 0 good 4 ich 0 other 680
bad 0 good 5 ich 0 other 740
bad 0 good 6 ich 0 other 789
bad 1 good 6 ich 0 other 826
bad 1 good 7 ich 0 other 908
bad 2 good 7 ich 0 other 930
bad 2 good 8 ich 0 other 937
bad 2 good 9 ich 0 other 1039
bad 2 good 10 ich 0 other 1129
bad 2 good 11 ich 0 other 1158
bad 2 good 12 ich 0 other 1187
bad 2 good 13 ich 0 other 1279
bad 2 good 14 ich 0 other 1296
bad 3 good 14 ich 0 other 1392
bad 3 good 15 ich 0 other 1447
bad 3 good 16 ich 0 other 1453
bad 4 good 16 ich 0 other 1458
bad 4 good 17 ich 0 other 1495
bad 4 good 18 ich 0 other 1567
bad 4 good 19 ich 0 other 1580
bad 4 good 20 ich 0 other 1653
bad 5 good 20 ich 3 other 1827
bad 6 good 20 ich 3 other 1837
bad 6 good 21 ich 3 other 1852
bad 6 good 22 ich 3 other 1854
bad 6 good 23 ich 3 other 1960
bad 6 good 24 ich 3 other 2057
bad 6 good 25 ich 3 other 2077
bad 6 good 26 ich 3 other 2136
bad 7 good 26 ich 3 other 2193
bad 7 good 27 ich 3 other 2393
bad 7 good 28 ich 3 other 2496
bad 7 good 29 ich 3 other 2628
bad 7 good 30 ich 3 other 2811
bad 7 good 31 ich 3 other 2853
bad 8 good 31 ich 3 other 2869
bad 8 good 32 ich 3 other 2894
bad 8 good 33 ich 3 other 2923
bad 8 good 34 ich 3 other 2983
bad 8 good 35 ich 3 other 2994
bad 9 good 35 ich 3 other 3017
bad 9 good 36 ich 3 other 3018
bad 9 good 37 ich 3 other 3360
bad 9 good 38 ich 3 other 3433
bad 9 good 39 ich 3 other 3455
bad 9 good 40 ich 3 other 3514
bad 9 good 41 ich 3 other 3554
bad 9 good 42 ich 3 other 3670
bad 9 good 43 ich 3 other 3799
bad 9 good 44 ich 3 other 3850
bad 9 good 45 ich 3 other 3856
bad 9 good 46 ich 3 other 3889
bad 10 good 46 ich 3 other 3933
bad 10 good 47 ich 3 other 3977
bad 10 good 48 ich 3 other 3984
bad 10 good 49 ich 3 other 4061
bad 10 good 50 ich 3 other 4174
bad 10 good 51 ich 3 other 4318
bad 10 good 52 ich 3 other 4353
bad 10 good 53 ich 3 other 4354
bad 10 good 54 ich 3 other 4457
bad 10 good 55 ich 3 other 4463
bad 10 good 56 ich 3 other 4518
bad 10 good 57 ich 3 other 4564
bad 11 good 57 ich 3 other 4617
bad 11 good 58 ich 3 other 4650
bad 12 good 58 ich 3 other 4653
bad 12 good 59 ich 3 other 4656
bad 12 good 60 ich 3 other 4718
bad 12 good 61 ich 3 other 4721
bad 12 good 62 ich 3 other 4740
bad 13 good 62 ich 3 other 4758
And during shutdown:
bad 13 good 63 ich 3 other 4814
bad 13 good 64 ich 3 other 4820
bad 13 good 65 ich 3 other 4826
bad 13 good 66 ich 3 other 4836
bad 14 good 66 ich 3 other 4846
bad 14 good 67 ich 3 other 4849
bad 15 good 67 ich 3 other 4850
bad 15 good 68 ich 3 other 4855
bad 15 good 69 ich 3 other 4869
- Werner
-
:] Oh, yes, I changed the good and bad numbers :S
hehehe, after noticing that, the correct graphics are the ones attached, and the conclussions are more logical :))
1- As before, the bad chunks follow a linear progression, as the good chunks. However the good blocks follow a steeper progression than the bad ones.
2- When ICH is taken into account, it seems that after some blocks the ICH is able to recover better the blocks (this is logical: I think the blocks were already well downloaded and so they are easy to be recovered). This makes the erroneus blocks to become less important since that time, but they keep at a rate of about 10:1 (10 goods, 1 bad).
I am trying a solution, and will use werner's patch to see what happens to my blocks. Let's see what numbers do I get tomorrow...
-
Originally posted by whomnet
2- When ICH is taken into account,
What confuses me with ICH is that there doesn't seem to be a direct
quantitative correlation between the message that ICH has done
something, and the code executing the ICH path. E.g. I once had
a few dozen (> 30, I think) ICHs counted in rapid succession, but
got only a single message.
So, quite clearly, there's something about ICH I don't understand :-(
- Werner
-
I have ran for 24 hours amule (ext filesystem, could'nt change to FAT32 though). The results are as follows:
What I have seen is that, when downloading a new file (about 500MB size), the bad chunks didn't appear (except for 2 bad after 60 good). Then, when the file is almost downloaded (1 or 2 chunks last), these last chunks are the erroneus ones, and aMule keeps saying they are bad although it downloads them many times (5 as my patience lasted).
Finally, I made a couple of experiments: the first one was to force a sync before test were done, in order to have the same data in the HD than in the cache... it seems that didn't work.
The second experiment was to pause the file (1 chunk was left to finish, but always being wrongly downloaded). As more files were being downloaded, the file begun downloading after the first (other) download finished and... ¡Oooh the chunk received was OK and the download finished!
My theory is that there are some desynchronized content, and it gets synchronized after a while. I will keep on testing possibilities to see if my theory is correct.
-
Originally posted by whomnet
Finally, I made a couple of experiments: the first one was to force a sync before test were done
As in typing "sync", or adding an fsync() call ? Unless you treat your
system extremely roughly (power off or hard reset without shutdown,
disconnect drive during operation, etc.), this won't make a difference.
BTW, does anyone know if aMule shares data from chunks that have
not been checksummed yet ? If it does, this would be a good thing to
make at least optional, to limit the spread of corrupt data and (more
importantly) to avoid downloading of good data into chunks already
containing bad data.
- Werner
-
Originally posted by werner
this would be a good thing to make at least optional
I'm no longer so sure about this. A little simulation shows that it
may actually be faster to also download bad data. The traffic
increases, but good data propagates considerably faster than
when waiting for verification.
I've attached the simulation program. See README for details
The .zip is of course .gz - silly forum won't let me pick the
extension I want. Warning: extracts into the current directory.
- Werner
-
For the data collectors, here is a ~37h run (Tuesday UTC 15 to
Thursday UTC 4). I don't have the exact average download rate,
but most of the time, it was probably not more than 5-15 kB/s.
bad 0 good 1 ich 0 other 110
bad 0 good 2 ich 0 other 391
bad 0 good 3 ich 0 other 398
bad 0 good 4 ich 0 other 539
bad 0 good 5 ich 0 other 653
bad 0 good 6 ich 0 other 728
bad 0 good 7 ich 0 other 829
bad 0 good 8 ich 0 other 878
bad 0 good 9 ich 0 other 901
bad 0 good 10 ich 0 other 1132
bad 0 good 11 ich 0 other 1149
bad 0 good 12 ich 0 other 1152
bad 0 good 13 ich 0 other 1168
bad 0 good 14 ich 0 other 1201
bad 0 good 15 ich 0 other 1223
bad 0 good 16 ich 0 other 1229
bad 0 good 17 ich 0 other 1257
bad 0 good 18 ich 0 other 1393
bad 1 good 18 ich 0 other 1423
bad 1 good 19 ich 0 other 1437
bad 1 good 20 ich 0 other 1454
bad 1 good 21 ich 0 other 1492
bad 1 good 22 ich 0 other 1662
bad 1 good 23 ich 0 other 1666
bad 2 good 23 ich 0 other 1785
bad 2 good 24 ich 0 other 1849
bad 3 good 24 ich 0 other 1987
bad 3 good 25 ich 0 other 1999
bad 3 good 26 ich 0 other 2064
bad 3 good 27 ich 0 other 2143
bad 3 good 28 ich 0 other 2189
bad 3 good 29 ich 0 other 2264
bad 3 good 30 ich 0 other 2299
bad 3 good 31 ich 0 other 2358
bad 4 good 31 ich 0 other 2449
bad 4 good 32 ich 0 other 2545
bad 4 good 33 ich 0 other 2551
bad 4 good 34 ich 1 other 2567
bad 4 good 35 ich 10 other 2648
bad 4 good 36 ich 14 other 2685
bad 4 good 37 ich 21 other 2781
bad 4 good 38 ich 21 other 2827
bad 4 good 39 ich 22 other 2896
bad 4 good 40 ich 22 other 2913
bad 4 good 41 ich 22 other 2925
bad 4 good 42 ich 27 other 3024
bad 4 good 43 ich 27 other 3039
bad 4 good 44 ich 27 other 3075
bad 4 good 45 ich 27 other 3077
bad 5 good 45 ich 27 other 3081
bad 5 good 46 ich 27 other 3081
bad 5 good 47 ich 47 other 3156
bad 5 good 48 ich 48 other 3163
bad 5 good 49 ich 53 other 3252
bad 5 good 50 ich 53 other 3269
bad 6 good 50 ich 53 other 3282
bad 7 good 50 ich 53 other 3327
bad 8 good 50 ich 53 other 3453
bad 8 good 51 ich 53 other 3530
bad 8 good 52 ich 53 other 3546
bad 8 good 53 ich 53 other 3573
bad 8 good 54 ich 53 other 3663
bad 8 good 55 ich 53 other 3676
bad 8 good 56 ich 53 other 3681
bad 8 good 57 ich 53 other 3690
bad 8 good 58 ich 53 other 3695
bad 8 good 59 ich 53 other 3745
bad 8 good 60 ich 53 other 3777
bad 9 good 60 ich 53 other 3798
bad 9 good 61 ich 53 other 3825
bad 9 good 62 ich 62 other 3892
bad 9 good 63 ich 62 other 3922
bad 10 good 63 ich 62 other 3995
bad 11 good 63 ich 62 other 4030
bad 11 good 64 ich 63 other 4082
bad 11 good 65 ich 63 other 4113
bad 11 good 66 ich 63 other 4194
bad 11 good 67 ich 63 other 4436
bad 11 good 68 ich 63 other 4545
bad 11 good 69 ich 63 other 4550
bad 11 good 70 ich 63 other 4579
bad 11 good 71 ich 63 other 4633
bad 11 good 72 ich 63 other 4667
bad 11 good 73 ich 63 other 4715
bad 11 good 74 ich 63 other 4770
bad 11 good 75 ich 63 other 4844
bad 11 good 76 ich 63 other 4846
bad 11 good 77 ich 63 other 4900
bad 11 good 78 ich 63 other 4900
bad 11 good 79 ich 63 other 4907
bad 11 good 80 ich 63 other 4941
bad 11 good 81 ich 63 other 4943
bad 11 good 82 ich 63 other 4945
bad 11 good 83 ich 63 other 5122
bad 11 good 84 ich 63 other 5145
bad 11 good 85 ich 63 other 5224
bad 12 good 85 ich 63 other 5371
bad 12 good 86 ich 63 other 5444
bad 12 good 87 ich 63 other 5547
bad 13 good 87 ich 63 other 5645
bad 13 good 88 ich 63 other 5720
bad 13 good 89 ich 63 other 5753
bad 13 good 90 ich 63 other 5765
bad 13 good 91 ich 63 other 5821
bad 13 good 92 ich 63 other 5890
bad 13 good 93 ich 63 other 5941
bad 13 good 94 ich 63 other 6016
bad 13 good 95 ich 63 other 6026
bad 13 good 96 ich 63 other 6129
bad 13 good 97 ich 63 other 6193
bad 13 good 98 ich 63 other 6236
bad 13 good 99 ich 63 other 6248
bad 13 good 100 ich 63 other 6292
bad 13 good 101 ich 63 other 6311
bad 14 good 101 ich 63 other 6322
bad 15 good 101 ich 63 other 6349
bad 15 good 102 ich 66 other 6368
bad 15 good 103 ich 84 other 6590
bad 15 good 104 ich 84 other 6600
bad 15 good 105 ich 84 other 6671
bad 15 good 106 ich 84 other 6681
bad 15 good 107 ich 84 other 6708
bad 15 good 108 ich 84 other 6714
bad 15 good 109 ich 84 other 6851
bad 15 good 110 ich 84 other 6967
bad 15 good 111 ich 84 other 7001
bad 15 good 112 ich 84 other 7007
bad 15 good 113 ich 84 other 7010
bad 15 good 114 ich 84 other 7055
bad 15 good 115 ich 84 other 7107
bad 15 good 116 ich 84 other 7134
bad 15 good 117 ich 84 other 7179
bad 15 good 118 ich 84 other 7190
bad 15 good 119 ich 84 other 7231
bad 15 good 120 ich 84 other 7288
bad 15 good 121 ich 85 other 7376
bad 15 good 122 ich 85 other 7392
bad 15 good 123 ich 85 other 7395
bad 15 good 124 ich 85 other 7439
bad 15 good 125 ich 85 other 7450
bad 15 good 126 ich 85 other 7524
bad 15 good 127 ich 85 other 7552
bad 15 good 128 ich 95 other 7641
bad 15 good 129 ich 95 other 7656
bad 15 good 130 ich 95 other 7659
bad 15 good 131 ich 95 other 7675
bad 15 good 132 ich 95 other 7753
bad 15 good 133 ich 96 other 7772
bad 15 good 134 ich 97 other 7813
bad 15 good 135 ich 97 other 7892
bad 15 good 136 ich 97 other 7918
bad 15 good 137 ich 97 other 7922
bad 15 good 138 ich 97 other 7933
bad 15 good 139 ich 97 other 7933
bad 15 good 140 ich 98 other 7963
bad 15 good 141 ich 102 other 8000
bad 15 good 142 ich 102 other 8026
bad 15 good 143 ich 102 other 8090
BTW, looking at how good and bad evolve around bursts of ICH,
I'm slowly getting convinced that ICH is best ignored when
calculating the ratio between good and bad. Does this make
sense ?
- Werner
-
Really, I'm reading this, paying attention and such. I have not yet what to say, anyway.
-
hello,
i m running the current cvs from 10.30 today and got a lot of corrupted files on different files i havent with previous version. Maybe a coincidence ?
2004-01-29 10:20:44: Downloaded part 43 is corrupt :( (xxx.avi)
2004-01-29 10:23:37: Downloaded part 60 is corrupt :( (xxx.avi)
2004-01-29 10:24:20: Downloaded part 13 is corrupt :( (yyy.avi)
2004-01-29 10:26:51: Downloaded part 16 is corrupt :( (zzz.AVI)
2004-01-29 10:28:45: Downloaded part 29 is corrupt :( (aaa.AVI)
2004-01-29 10:31:10: Downloaded part 28 is corrupt :( bbb.avi)
2004-01-29 10:37:31: Downloaded part 32 is corrupt :( (ccc.avi)
2004-01-29 10:42:50: Downloaded part 14 is corrupt :( qqq.avi)
2004-01-29 10:51:21: Downloaded part 31 is corrupt :( sss.avi)
2004-01-29 10:57:46: Downloaded part 15 is corrupt :( qqq.avi)
2004-01-29 11:09:23: Downloaded part 75 is corrupt :( sss.avi)
something has changed in cvs ?
the polish
-
2004-01-28 20:03:52: Creditfile loaded, 51801 clients are known
2004-01-28 20:03:52: Loaded ipfilter with 2518 IP addresses.
2004-01-28 20:03:52: 45 servers in server.met found
2004-01-28 20:03:53: Found 26 part files
2004-01-28 20:03:53: aMule CVS
2004-01-28 20:03:53: Found 10 known shared files
2004-01-28 20:03:53: *******************************.mpg
2004-01-28 20:03:53: *******************************.mpg
2004-01-28 20:04:00: Connecting to DateAttacke.de (62.241.53.17:4242)...
2004-01-28 20:04:00: Connected to DateAttacke.de (62.241.53.17:4242)
2004-01-28 20:04:15: Lost connection to DateAttacke.de (62.241.53.17:4242)
2004-01-28 20:04:25: Connection attempt to DateAttacke.de (62.241.53.17:4242 ) timed out
2004-01-28 20:04:26: Connecting to SILENTMAXX (217.85.99.150:8888)...
2004-01-28 20:04:27: Connecting to ### Die Saugstube ### (207.44.200.40:4242)...
2004-01-28 20:04:28: Connected to ### Die Saugstube ### (207.44.200.40:4242)
2004-01-28 20:04:28: Connection established on: ### Die Saugstube ###
2004-01-28 20:04:28: New clientid is 1762184766
2004-01-28 20:11:15: requested file not found
2004-01-28 20:32:52: requested file not found
2004-01-28 20:35:04: requested file not found
2004-01-28 20:38:42: Finished downloading *******************************.mpg :-)
2004-01-28 20:50:04: requested file not found
2004-01-28 20:54:33: requested file not found
2004-01-28 21:16:14: requested file not found
2004-01-28 21:16:25: requested file not found
2004-01-28 21:37:49: requested file not found
2004-01-28 21:59:32: requested file not found
2004-01-28 21:59:39: requested file not found
2004-01-28 22:21:30: requested file not found
2004-01-28 22:22:12: requested file not found
2004-01-28 22:32:26: Downloading ******************************.rar
2004-01-28 22:32:27: Downloading ******************************.zip
2004-01-28 22:32:28: Downloading ******************************.rar
2004-01-28 22:32:40: Downloading ******************************.rar
2004-01-28 22:32:41: Downloading ******************************.zip
2004-01-28 22:39:56: Downloading ******************************.rar
2004-01-28 22:43:09: requested file not found
2004-01-28 22:56:41: requested file not found
2004-01-28 22:58:18: requested file not found
2004-01-28 23:04:25: Downloaded part 7 is corrupt :( (*********************************.avi)
2004-01-28 23:04:46: requested file not found
2004-01-28 23:05:39: requested file not found
2004-01-28 23:18:23: Finished downloading ********************************.mpg :-)
2004-01-28 23:26:28: requested file not found
2004-01-28 23:39:49: requested file not found
2004-01-28 23:47:22: requested file not found
2004-01-28 23:48:07: requested file not found
2004-01-28 23:48:52: requested file not found
2004-01-29 00:11:26: Downloaded part 42 is corrupt :( (******************************.bin)
2004-01-29 00:31:07: Finished downloading *********************************.mpg :-)
2004-01-29 00:38:19: requested file not found
2004-01-29 01:54:21: Finished downloading *********************************.mpg :-)
2004-01-29 03:03:53: Finished downloading *********************************.mpg :-)
2004-01-29 03:31:03: Downloaded part 0 is corrupt :( (*********************************.rar)
2004-01-29 03:44:56: Finished downloading *********************************.mpg :-)
2004-01-29 04:33:21: Finished downloading *********************************.mpg :-)
2004-01-29 04:45:03: Finished downloading *********************************.bin :-)
2004-01-29 05:04:01: ICH: Recovered corrupted part 0 (*********************.rar)
2004-01-29 05:30:50: ICH: Recovered corrupted part 42 (********************.bin)
2004-01-29 05:57:48: Finished downloading ********************************.mpg :-)
2004-01-29 06:18:40: Finished downloading **********************************.rar :-)
2004-01-29 06:48:32: Finished downloading ********************************.mpg :-)
2004-01-29 07:10:26: Finished downloading **********************************.rar :-)
2004-01-29 07:11:37: User ***** (2257992917) requested your sharedfiles-list -> denied
2004-01-29 07:12:23: Finished downloading *********************************.rar :-)
2004-01-29 07:21:30: Finished downloading *********************************.avi :-)
2004-01-29 07:26:36: Finished downloading *********************************.mpg :-)
2004-01-29 07:26:36: A file with that name already exists, the file has been saved as ***********.mpg
2004-01-29 07:33:39: Finished downloading *********************************.mpg :-)
2004-01-29 08:03:03: Finished downloading *********************************.mpg :-)
2004-01-29 08:25:26: Lost connection to ### Die Saugstube ### (207.44.200.40:4242)
2004-01-29 08:25:26: Disconnected
2004-01-29 08:25:26: Connecting to SILENTMAXX (217.85.99.150:8888)...
2004-01-29 08:25:51: Connection attempt to SILENTMAXX (217.85.99.150:8888 ) timed out
2004-01-29 08:25:51: Connecting to .S.E.D.G. (38.119.96.15:4661)...
2004-01-29 08:25:52: Fatal Error while trying to connect. Internet connection might be down
2004-01-29 08:25:52: Automatic connection to server will retry in 30 seconds
2004-01-29 08:25:52: Disconnected
2004-01-29 08:25:52: Disconnected
2004-01-29 08:26:22: Disconnected
2004-01-29 08:26:22: Connecting to SILENTMAXX (217.85.99.150:8888)...
2004-01-29 08:26:47: Connection attempt to SILENTMAXX (217.85.99.150:8888 ) timed out
2004-01-29 08:26:47: Connecting to .S.E.D.G. (38.119.96.15:4661)...
2004-01-29 08:26:47: Connected to .S.E.D.G. (38.119.96.15:4661)
2004-01-29 08:26:50: Connection established on: .S.E.D.G.
2004-01-29 08:26:50: New clientid is 1762184766
2004-01-29 09:16:19: Finished downloading ************************************.mpg :-)
deltaHF@deltahf:~/file/amule-cvs> aStats
aStats Script 1.6.3
aMule CVS has been running for 0 Day(s) and 15h27m05s
deltaHF[aMule] is Online .S.E.D.G. (38.119.96.15:4661) with HighID
Total Download: 22,42 GB, Upload: 71,67 GB
Sharing: 33 File(s) / Client(s) in queue: 2005
Download: 19,9 kB/s, Upload: 16,0 kB/s
Host Uptime: an 7 Tage 13:24, 1 Benutzer, Durchschnittslast: 0,90, 0,88, 0,92
deltaHF@deltahf:~/file/amule-cvs>
-
I haven't tired the latest CVS, however let me show some stats of myself:
As an explanation, I will tell that I was downloading a file (just the last part) and about 6 fresh downloads more. I ran all night and in the morning the file was still missing the last chunk. aMule said it was corrupted a dozen of times!. I stopped the file, and so the good parts growth (those of the fresh downloads), with less corrupted parts found.
With aMule still running I used DonkeyDoctor to check the stopped download's chunks, and it said everyting was OK. Run it another time and DonkeyDoctor said the file had corrupted chunks (but I didn't do anything to the part.met file).
As the messages of the DonkeyDoctor were contradictory, I shutted down aMule and then ran the DonkeyDoctor again. At the first check the program detected that the downloaded part marked as corrupt was OK, and so transferred the file to Incoming.
I can think on many answers of what is happening, but the might not be valid. Any idea, werner, Kry or DeltaHF?
BadGoodIchOther: 1 0 0 24 1
BadGoodIchOther: 1 0 1 24 1
BadGoodIchOther: 2 0 1 27 2
BadGoodIchOther: 3 0 1 28 3
BadGoodIchOther: 4 0 1 53 4
BadGoodIchOther: 5 0 1 172 5
BadGoodIchOther: 6 0 1 180 6
BadGoodIchOther: 6 1 1 245 7
BadGoodIchOther: 7 1 1 276 8
BadGoodIchOther: 7 2 1 307 9
BadGoodIchOther: 8 2 1 345 10
BadGoodIchOther: 9 2 1 365 11
BadGoodIchOther: 10 2 1 383 12
BadGoodIchOther: 11 2 1 395 13
BadGoodIchOther: 11 3 1 503 14
BadGoodIchOther: 11 4 1 718 15
BadGoodIchOther: 11 5 1 721 16
BadGoodIchOther: 11 5 2 778 16
BadGoodIchOther: 12 5 2 788 17
BadGoodIchOther: 12 6 2 912 18
BadGoodIchOther: 13 6 2 1013 19
BadGoodIchOther: 14 6 2 1039 20
BadGoodIchOther: 14 7 2 1059 21
BadGoodIchOther: 14 8 2 1118 22
BadGoodIchOther: 14 9 2 1146 23
BadGoodIchOther: 14 10 2 1159 24
BadGoodIchOther: 14 11 2 1237 25
BadGoodIchOther: 15 11 2 1239 26
BadGoodIchOther: 15 11 3 1409 26
BadGoodIchOther: 15 12 3 1461 27
BadGoodIchOther: 15 13 3 1496 28
BadGoodIchOther: 15 13 4 1497 28
BadGoodIchOther: 15 14 4 1516 29
BadGoodIchOther: 15 15 4 1574 30
BadGoodIchOther: 15 16 4 1640 31
BadGoodIchOther: 16 16 4 1662 32
BadGoodIchOther: 16 17 4 1693 33
BadGoodIchOther: 16 18 4 1745 34
BadGoodIchOther: 16 19 4 1767 35
BadGoodIchOther: 16 20 4 1790 36
BadGoodIchOther: 16 21 4 1836 37
BadGoodIchOther: 16 22 4 1843 38
BadGoodIchOther: 16 23 4 1903 39
BadGoodIchOther: 16 24 4 1961 40
BadGoodIchOther: 16 25 4 2025 41
BadGoodIchOther: 16 25 5 2117 41
BadGoodIchOther: 16 26 5 2237 42
BadGoodIchOther: 17 26 5 2418 43
BadGoodIchOther: 17 27 5 2487 44
BadGoodIchOther: 17 28 5 2583 45
BadGoodIchOther: 17 29 5 2592 46
BadGoodIchOther: 17 30 5 2633 47
BadGoodIchOther: 17 31 5 2670 48
BadGoodIchOther: 17 32 5 2833 49
BadGoodIchOther: 17 33 5 2850 50
BadGoodIchOther: 17 34 5 2915 51
BadGoodIchOther: 17 35 5 2965 52
BadGoodIchOther: 17 36 5 2990 53
BadGoodIchOther: 17 37 5 3139 54
BadGoodIchOther: 17 38 5 3281 55
BadGoodIchOther: 17 39 5 3531 56
BadGoodIchOther: 17 40 5 3686 57
Sorry they are in tab-separated rows to easily do the graphs using a spreadsheet.
-
Till we fix the buffer overflow corrupting lists on CVS, we cannot guess answers to that.
-
Originally posted by whomnet
aMule said it was corrupted a dozen of times!. I stopped the file, and so the good parts growth (those of the fresh downloads), with less corrupted parts found.
Hmm, could be aMule getting confused or a popular source offering
corrupt data.
With aMule still running I used DonkeyDoctor to check the stopped download's chunks, and it said everyting was OK.
Unfortunately, a run with aMule still caching data probably doesn't
mean much. BTW, is DonkeyDoctor reliable ? Does it use the same
code as some other incarnation of eDonkey ?
At the first check the program detected that the downloaded part
marked as corrupt was OK, and so transferred the file to Incoming.
Hmm. Could be either DD missing the corruption, aMule halluzinating
there was some, or a good fairy fixing the file while aMule was writing
it to disk :-)
What's interesting is that this was reproducible. So it could mean that
aMule somehow corrupted data that would not be saved to disk, or
that it is using data that may be saved to disk in an incorrect way.
(Or, if the problem is in DonkeyDoctor, it may do the same.)
One thing I'd look out for is the use of ranges. I've seen that aMule
frequently describes byte ranges with a (start, end) pair, there "start"
is the position of the first byte, and "end" the one of the last byte.
It's quite common to use a similar pair, where "end" is one byte
beyond the last byte, so length = end-start, and a loop
for (p = data+start; p != data+end; p++) will traverse the whole range.
A mixup of that kind could explain why aMule sometimes seems to
get lost. (Of course, it's probably something else. It's never the bugs
that seem logical and plausible :-)
- Werner
-
lol, well, I'm actually stuck at CList::AddTail saving corrupted data somehow, so I'm plenty of non-logical bugs
-
Maybe I have to introduce DonkeyDoctor (thought it was a long used program, but seems not). DonkeyDoctor can be found at http://usuarios.lycos.es/donkeydoctor/portal.php.
DonkeyDoctor is a Win32 application, I think it is not open source. It was first thought to recover the .part.met files after the common error of deleting them (that's it, when the backups does not work, etc). Now has increased its features, but every feature is related to chunk information: You can see the file information, or change its temp name (001.part to 004.part, i.e.).
But the feature I use is the "Test met" feature. As I know, it does a complete re-hashing of the .part file, and checks if the .met file has marked as good blocks checked as bad (ie: when rebuilding a .met file you need to do this for not losing your download). If the .met is incongruent with the .part hashes, then you can update the .met with the new data or if the file is fully OK then it is transferred to the Incoming folder (as when completing on *mule). This feature is there since the beginning, and as far as I know it is quite stable.
In the other comments sense: Maybe a source offering corrupt parts will appear, bu this happens to me lots of times, I am beginning to think that it happens more as the file is more long and popular, but have no means to measure it. It does not happen always, but keeps repeating again and again.
I don't have too much time to look around the code, but I would bet that, n some point aMule doues the hashing over a invalid range (maybe the +-1 ibt problem). DonkeyDoctor has never failed me, and the last file (the one that repeater 12 times the bad chunk) was a .rar and all the files were valid, so DonkeyDoctor worked OK.
-
Originally posted by whomnet
Maybe I have to introduce DonkeyDoctor
Thanks. I've hard of DD, but, since it's for Windows, didn't pay much
attention to it. I had a look at the DD site, and indeed, there aren't many
signs of the source ...
This feature is there since the beginning, and as far as I know it is quite stable.
Okay, let's assume DD is doing the right thing.
Maybe a source offering corrupt parts will appear, bu this happens to me lots of times, I am beginning to think that it happens more as the file is more long and popular, but have no means to measure it. It does not happen always, but keeps repeating again and again.
That would be consistent with a broken client receiving good data but
sending out bad data. What would be interesting is to compare bad
chunks with the good ones, once the latter have been received. If e.g.
the corruption is in the last byte of some part/fragment/whatever it's
called, that would be a strong clue. (And also record where the data
came from, so one can check what the next hop client was. Given
enough data, once could eventually figure out who's the culprit.)
- Werner
-
For each session, just the last entry. Still running 1.2.4.
bad 11 good 80 ich 0 other 4775
bad 4 good 18 ich 7 other 1420
bad 8 good 40 ich 35 other 2359
bad 11 good 14 ich 0 other 2061
bad 8 good 10 ich 0 other 1066
bad 46 good 329 ich 104 other 17922
bad 3 good 33 ich 0 other 2610
bad 4 good 108 ich 0 other 6363
bad 14 good 292 ich 33 other 14423
bad 3 good 57 ich 0 other 4183
So it seems that the good:bad ratio is steadily improving, although
there is a lot of noise.
If there is indeed an improvement, what may be the reason ? Well,
it could of course be that the world around me has decided to send
me better data. It could also be that there was one catastrophic
event in the past, and I'm slowly getting rid of the bad data.
Or ... the libc change did indeed bring some improvement
- Werner.
-
Either of those.
Look foward for 1.2.5 in 1 or 2 days and test ;)
-
:baby: :baby: hope 1.2.5 will fix the problem
I updatet to FC 1 but everything stays the same. I'll wait till the 1.2.5 version
-
1.2.5 still gives me problems, not because many blocks are being marked corrupt, but because DonkeyDoctor reports some of them (not that many, though) as being actually correct.
I wish DonkeyDoctor was capable of running in batch... anyway, thanks to you for making aMule possible :)
Note that I didn't try 1.2.6, yet, but looking at the change log, I'm not very optimistical about it fixing this issue.
I was going to write a small tool to replace DonkeyDoctor based on aMule's code, but then I realized that it wouldn't be such a good idea. Anybody knows whether other eMule ports suffer from this same problem? I still think that such tool should be better left aside, but in the meanwhile...