aMule Forum
English => en_Bugs => Topic started by: stoatwblr on April 02, 2010, 08:37:25 PM
-
2 (minor) gui bugs (amule and amulegui)
1: There's a limit of 99 categories displayed but it's possible to create 100+ categories
2: If there are a lot of categories (eg 20), deleting one of the intermediate ones causes everything displayed to effectively shuffle right one tab
This has 2 effects:
a: Files now appear under the "wrong" category name
b: Files in the highest numbered category disappear from the tab display altogether
The only effective fix I can see is to renumber files vs categories each time a category is deleted.
-
1: There's a limit of 99 categories displayed but it's possible to create 100+ categories
I marvel what things people try. Noted, will be fixed asap (that means, when GUI gets unlocked).
I hope the crashes you reported weren't with >100 cats?
2: If there are a lot of categories (eg 20), deleting one of the intermediate ones causes everything displayed to effectively shuffle right one tab
Local or amulegui?
Edit: amuled with amulegui, right ?
-
2) is fixed in 10069
-
I marvel what things people try.
One cat per tv series... adds up fast. I've seen some people use one per season.
I hope the crashes you reported weren't with >100 cats?
99, of which about 40 are currently empty due to bug #2
2: If there are a lot of categories (eg 20), deleting one of the intermediate ones causes everything displayed to effectively shuffle right one tab
Local or amulegui?
Edit: amuled with amulegui, right ?
Both amule and amulegui. Ditto on bug #1
-
Missed bug #3 - amulegui
After some period of operation right clicking on the tabs stops working.
Usually this can be triggered by minimising amulegui and bringing it back or changing file download priorities.
Possibly related (#4?) Amulegui usually eats between 90-100% cpu no matter whether minimised or restored
(2.2GHz 3Gb athlon dual system).
Amuled only uses 5-10% occasionally jumping to (at most) 60%. This applies whether amulegui is running or not.
-
Amulegui usually eats between 90-100% cpu no matter whether minimised or restored
That's bad. How many downloads?
What happens if you disable progress bars, or change a different view (KAD window)?
-
That's bad. How many downloads?
0-50 downloading
100-300 waiting (more than 350 interrupts all downloading streams on completing one file)
900-1500 queued
Interestingly, using 10071 after a settling in period (a couple of hours) it's dropped right down to ~10-50% spiking to 100% for tab changes.
What happens if you disable progress bars, or change a different view (KAD window)?
In both 10071 and previous versions, no difference.
-
eeek!
10071 is using so little cpu because it's grown so large the machine's swapping like mad
currently 2600Mb and growing.
-
aMule's GUI (no matter if amule or amulegui) can't handle that many downloads at the moment I'm afraid.
I plan to look at the issue after the GUI change.
Try to cut down to < 500.
-
That's bad. How many downloads?
0-50 downloading
100-300 waiting (more than 350 interrupts all downloading streams on completing one file)
900-1500 queued
This, and 99 categories? You're abusing the program in almost every possible way. Why you're surprised with 100% CPU? For comparison - make following experiment - try to open 99 browser windows and issue 50 downloads.
Updating and drawing thousands of progress bars is very consuming task. Try to manage your downloads with web interface (amuleweb).
-
aMule's GUI (no matter if amule or amulegui) can't handle that many downloads at the moment I'm afraid.
Well, that's because colored progress/availability bar is quite expansive (resource wise). You can't do too match of such stuff without use of accelerated graphics.
-
Thousands of bars don't fit on the screen (only 30 or so), so it's a GUI problem if aMule does draw all of them all of the time even if they are invisible :P.
-
Thousands of bars don't fit on the screen (only 30 or so), so it's a GUI problem if aMule does draw all of them all of the time even if they are invisible :P.
Or even minimised. No point doing any drawing at all then.
One important point: Stu's non-compressed EC mod dropped cpu usage AND sped up synchronisation by 90%++ (seconds vs minutes)
I've made the request before to disable EC compression on non-WAN connections or make it switchable, but that mod underscores that EC compression is counterproductive on fast links (localhost/LAN connections)
(Yes, I know I'm abusing the program by driving this hard. Call it "stress testing" )
-
2) is fixed in 10069
Confirmed fixed :) Thanks
-
aMule's GUI (no matter if amule or amulegui) can't handle that many downloads at the moment I'm afraid.
Well, that's because colored progress/availability bar is quite expansive (resource wise). You can't do too match of such stuff without use of accelerated graphics.
Switching the progress bar off made no difference to CPU.
-
Stu's non-compressed EC mod dropped cpu usage AND sped up synchronisation by 90%++
What exactly do you mean? The hack I posted as a zip? That also had zlib compression.
If you want to try without compression, just raise EC_MAX_UNCOMPRESSED in ECSocket.cpp to like 10000000 and recompile amuled.
(Yes, I know I'm abusing the program by driving this hard. Call it "stress testing" )
You are generating valuable feedback. :)
Switching the progress bar off made no difference to CPU.
Good, so it's not that dumb. But there is still an O(n^2) somewhere in the download view I believe.
As I said - I'll investigate, but only after the big GUI update. Maybe the problem has already been fixed there (increasing the hype...).
-
One important point: Stu's non-compressed EC mod dropped cpu usage AND sped up synchronisation by 90%++ (seconds vs minutes)
I see. Apparently zlib calls are that costly.
(Yes, I know I'm abusing the program by driving this hard. Call it "stress testing" )
Stress testing is important, but tricky to do. Sometimes it's difficult to tell where limit is hit.
Switching the progress bar off made no difference to CPU.
So problem is probably not in drawing itself.
But there is still an O(n^2) somewhere in the download view I believe.
Not necessary. It can be O(n) with big multiplier (bad implementation).
-
What exactly do you mean? The hack I posted as a zip? That also had zlib compression.
Whatever you did in that hack, it was a lot faster to sync up.
I'll try your suggestion :)
-
I see. Apparently zlib calls are that costly.
Yikes! They are when you set this:
#define EC_COMPRESSION_LEVEL Z_BEST_COMPRESSION
in ECSocket.cpp (Thanks for the pointer Stu.)
That's equivalent to gzip -9 - which has a _massive_ CPU overhead for a small size gain over gzip's default level 6.
Here's what SSH's man page has to say about session compression:
-C Requests compression of all data (including stdin, stdout, stderr, and data for forwarded X11 and TCP connections). The com‐
pression algorithm is the same used by gzip(1), and the “level” can be controlled by the CompressionLevel option for protocol
version 1. Compression is desirable on modem lines and other slow connections, but will only slow down things on fast net‐
works. The default value can be set on a host-by-host basis in the configuration files; see the Compression option.
Gzip's man page says:
-# --fast --best
Regulate the speed of compression using the specified digit #, where -1 or --fast indicates the fastest compression method
(less compression) and -9 or --best indicates the slowest compression method (best compression). The default compression
level is -6 (that is, biased towards high compression at expense of speed).
After years of fiddling with long (slow) links, compression level 1 or 2 are about all I bother with even on transoceanic SSH sessions (we use SSH+rsync a lot at $orkplace to keep multi-Tb data archives up to date as well as to pull in stuff from observatories worldwide) .
There's a clear throughput penalty on local LAN with compression enabled - rsync initial sessions max out at 25-30Mb/s with it on (using i7 cpus and 8Gb+ ram) vs 60-90Mb/s with it off (disk speed is the limit with compression off on local links, vs CPU overhead with compression on)
I'm not totally dissing client-server compression - it's useful on slow links but it's always a juggling act to balance the latency+CPU overhead caused by compression vs link latency caused by uncompressed data - and care has to be taken to not try compressing data running across an already compressed link (such as a SSH tunnel, or a modem link (It's rare for modems to not include compression these days)). That's why I keep suggesting it be switchable.
Meantime I'll try setting compression to minimum (fastest) as a starting point and see how it behaves.
-
That's why I keep suggesting it be switchable.
So why don't you just try it out and switch it off and post the result here?
-
I have been.
It's better with min/no compression but works better still with your suggestion of increasing the min packet size - mainly because there's no latency in/out of zlib even though that's doing nothing (with compression disabled)
The CPU usage is still high but I suspect that's because I still have debugging dialled right up. Despite that it's still behaving better than on the stock compression/min packet settings - but not performing as well as your hack.
-
You know you can (and should in your case) use optimize and debug together.
The hack just replaced a large and complicated piece of code with a few wx calls. But the problem was elsewhere.
Anyone against that I add a setting to disable EC compression?
-
Anyone against that I add a setting to disable EC compression?
Not at all. And when enabled, it probably should use less aggressive setting.
P.S.
That's interesting discussion - _some_ time ago, I argued here that reworking code to save bandwidth is useless idea.
-
So why was zlib EC compression implemented in the first place (and with much effort looking at the code), if saving bandwidth is useless?
My approach is: what you don't transmit you don't have to compress. Maybe the effort to compress the data outweighs the bandwidth gain. Current amulegui EC transmits much less without compression than it did before with compression.
-
So why was zlib EC compression implemented in the first place (and with much effort looking at the code), if saving bandwidth is useless?
For 2 reasons - once upon a time, IIRC, EC was text based. So, zlib was really useful, them. I kept it, because without some kind of profiling cost of compression wasn't obvious.
Second reason is people with slow links, who want to control amule over DSL.
My approach is: what you don't transmit you don't have to compress.
You already have this idea implemented. Look for CObjTagMap in EC source. I guess it can be improved.
Maybe the effort to compress the data outweighs the bandwidth gain. Current amulegui EC transmits much less without compression than it did before with compression.
What you (or me at a time) do, is ALSO compression. Which cost both in memory (O(n) at least) and CPU. On embedded system (I tried amule on NSLU2 and AppleTV) this will be painful.
-
once upon a time, IIRC, EC was text based. So, zlib was really useful, them.
IIRC we didn't have any compression on ECv1.
-
But it certainly was text based.
-
You know you can (and should in your case) use optimize and debug together.
Have been playing with that. It's a helluvalot faster (similar 90-95% reduction in sync up time), but your hack is even faster still when compiled with optimization.
(Using -O2 --unroll-loops)
The latest incarnation (svn 10089) is partially losing contact with the server after a while - eg, throughput speeds don't update and stopping/resuming a file doesn't show any change, but restarting the GUI shows that the commands worked.
-
I'm preparing a version that allows to cleanly disable zlib compression for EC.
Anybody against permanently using a wxstream instead of the complicated buffering ECSocket is doing now? That's what my hack did. It appears to speed up things under load and greatly simplifies things.
If you're still losing contact with 10089 that's frustrating. :( Please enable EC debug logging and try to figure out at which point things start breaking.
-
It was ok with EC contact up to 10087 if that helps.
Debugging's on but nothing's being logged when contact is lost. Do you have any particular flag you want enabled in remote.conf other than these?
VerboseDebug=1
Cat_General=1
Cat_External\ Connect=1
-
FWIW when this manifests, exiting amulegui doesn't result in a clean shutdown.
I'm running it from commandline and end up having to ^C in order to get the shell prompt back.
If it's still in full contact with amuled then amulegui exits cleanly back to the shell prompt.
-
10091: EC ZLIB compression can now be turned off in remote.conf with EC/ZLIB=0
10092: Added debug logs to ECSocket.cpp
Please try 10092 with EC debug log on and look what messages you get when it starts hanging. Compare what the core sends and what the client receives.
-
If EC debugging is enabled on the server side, the clients can't establish a handshake (10098).
-
I found it (well - I found something). Please try 10100.
Thank you very much for your valuable testing!
-
can get ec debugging now.
On the cats front: while stuff is being renumbered, the last category isn't deleted.
It's not a big issue as the number of categories is correct, just leaves cruft at the end of the config file.
-
Fixed in 10114. (You have to add/remove/change a cat to trigger a cleanup.)
No more EC hangs?
-
Some hangs in 10100 onwards but none so far in 10112 (although I did spot a 3 minute delay in updating at one point after reload shared was issued)
Compiling 10114 at moment. Fingers crossed...
-
No more loss of communication between client and server, but after some time the ability to right click on a tab goes away.
-
Left click works and switches to the cat, but right click opens no context menu?
-
correct
-
Just to be clear, everything else is working, just not the right click context menus.
It looks like at least one trigger is starting/stopping groups of files.
-
There is a member variable set when a context menu opens, and unset when it closes. It appears you found a way to close the context menu in an improper way leaving the variable set and aMule thinking the menu is still open (and thus not opening another one).
Please watch yourself what exactly you were doing the last time a menu opened. Like closing it by minimizing aMule, doing a taskswitch, something like that.
-
It's quite possible I've been triggering it that way, but last time I was scuttling along the tabs, starting and stopping queued files via file context menus (which always work) and then found that tab context menus had stopped working - the gui was constantly in foreground.
-
found a possible trigger....
Going from a file context menu to right clicking on an adjacent tab seems to break things.
-
Can't reproduce it, neither in Windows nor in Linux. Try to describe in more detail what you are doing.
Funny though - in Windows both file context menus and cat tab menus pop up on right click and stay until you click somewhere else. In Linux only file menus stay up - cat menus vanish when you release the right mouse button.
-
In this particular instance, I was starting newly queued files in a "all others" view on Tab #0, then switching them into another category. I'd realised there wasn't a category for a batch of files, so right clicked on tab #1 to add one - and the context menu refused to come up - despite working 2-3 minutes previously.
-
Can't reproduce it here. :(
-
I would like to help, but I don't really understand the steps to take in order to reproduce it.
-
Neither do I obviously. There's something else involved, like having 99 cats.
-
Could be anything and I'm down under 99 cats (80 at moment)
Compiled with optimization & debugging, without profiling
My suspicion is it's related to the number of queued files (> 1000) as that makes the GUI sluggish and the context menus can take a while to appear a lot of the time (it otfen takes a number of right clicks on the taskbar icon to make that one appear. Gnome shows the wrong menu the first few times)
If a context menu is requested and the gui thinks one is already displayed, why not simply clear it and draw the newly requested one?
-
The thing that strikes me as most odd is that context menus on files work, just not on tabs.
-
If a context menu is requested and the gui thinks one is already displayed, why not simply clear it and draw the newly requested one?
That's not so simple. What happens (probably) is that the event handler is stuck in CTransferWnd::OnNMRclickDLtab and a new event handler handles things. No idea what happens if one simply deletes the menu then.
I suggest we look back at the issue after the GUI update. Stop making the GUI sluggish and the problem should go away. :)
-
The gui will always be sluggish on some hardware (old, slow, etc etc). Finding the causes now will likely be a lot easier than later.