aMule Forum
English => aMule Help => Topic started by: Crakem on May 08, 2008, 08:49:21 PM
-
Yeah! i have rode another post like this but nothing help me :'( Usually I have two crash per day of amuled (yes I have to setup gdb for send you backtraces, I promise you ;)) so I don't got SYN flood message. But somedays amuled don't crash and server run slow, so I log and reading on /var/log/messages I read SYN flooding on port... error. I have tested to decrease MaxConnectionsPerFiveSeconds to 10 (20->15->10) but didn't help. Works better, but problem persist. My config:
MaxSourcesPerFile=100
MaxConnections=3000And 30 files downloading (simultaneously)
I have tested all I found on forum (changing ports, changing params, but nothing help)
My server is an old computer and system become unusable with that problem so I have to stop my router for successfully logging in :-[
Anybody having same problem?
There is a picture I post time ago for same error from amuleweb statistics:
(http://img242.imageshack.us/img242/9113/graphwm7.jpg) (http://imageshack.us)
Thanks a lot
-
3000 connections is slightly crazy. ;) Try 100 instead. For me anything above 100 kills my router.
-
3000 connections is slightly crazy. ;) Try 100 instead. For me anything above 100 kills my router.
I'm thinking if I want to download 30 simultaneous files, I need
MaxConnections=MaxSourcesPerFile*30
Wrong reasoning? ::)
-
Reasoning is that NAT in your $100 router can't handle that amount of simultaneous connections (size of NAT table).
-
I'm thinking if I want to download 30 simultaneous files, I need
MaxConnections=MaxSourcesPerFile*30
Wrong reasoning? ::)
Obviously won't all 30 files download from 100 sources each at once. (It's probability is infinite near to zero.)
-
I have decreased limits :-[
For me anything above 100 kills my router
Obviously won't all 30 files download from 100 sources each at once
My router only hangs one time two month each but I got 'SYN flood' message on my server all days ::)
Reasoning is that NAT in your $100 router can't handle that amount of simultaneous connections
I'm very interested in knowing limits to aMule, I have to check (with netstat I think...) how many connections are established when router hangs, please how you know your NAT limits? Maybe your router bring that in specifications? Maybe size of NAT table is numeric and it's maximum number of connections you could get?
I'm going to test 30 files like this:
MaxSourcesPerFile=100
MaxConnections=500
MaxConnectionsPerFiveSeconds=20I'm check this week for number of simultaneous connections with:
netstat | grep -c <my amule port>
Now I have only 85 :o (I'm thinking amule never run below 180 as amuleweb statistics show me :-\)
-
please how you know your NAT limits?
Trial and error.
Maybe your router bring that in specifications? Maybe size of NAT table is numeric and it's maximum number of connections you could get?
Theoretically max number of NAT connections is 64K (port number is 16 bit). In practice, however, cheap home routers doesn't have table with 64K entries. Real size of the table is (of cause) not advertised. You may search the Google for your specific router model and firmware version.
-
I'm thinking if I want to download 30 simultaneous files, I need
MaxConnections=MaxSourcesPerFile*30
Wrong reasoning? ::)
Wrong reasoning. Simplified picture is:
AMule connects to all of the sources and asks to be put on the upload queue. At this stage it uses all the connections you configured (and kills the router if these are too many). If you lower the number of connections, this only takes a little while longer.
Afterwards, it's just waiting for a download slot. When one gets free on a source, the source connects to you.
-
Well I have tested like this:
MaxSourcesPerFile=100
MaxConnections=200
MaxConnectionsPerFiveSeconds=20But I continue getting SYN flood sometimes. Amule has not half-open connections limit, hasn't it?
I checked my kernel limit like this:
cat /proc/sys/net/ipv4/tcp_max_syn_backlogand got: 128
Could somebody post his value, please?
I don't know how I reach that limit (maybe an ISP P2P blocking feature?)
My router works well with 200 connections (tested with: netstat | grep -c <port>)
Thanks all for replies.
edit: I have read a little of SYN DoS attack and I think is a problem with half-open connections only. MaxConnections stop that too?
-
My debian says 1024
-
My debian says 1024
Thanks Kry, I have increased my sysctl param to 1024, too, like this (in /etc/sysctl.conf)
net.ipv4.tcp_max_syn_backlog = 1024and then running
# sysctl -pI'm going to check new value for a few days :)
-
Firstly, be aware that for tcp_max_syn_backlog to have any effect, tcp_syncookies must be set to 1 . Check that.
Yes, it is.
If tcp_max_syn_backlog was originally set to 128, it should be because you have less than 128Mbyte of RAM. Assuming this is true, with so few memory you can't expect to have hundreds of TCP simultaneous connections... the original hint of decreasing max amule connection parameters is valid.
Yeah, my server has (only :'() 64MB of RAM. Please, how many connections do you think I have to fix MaxConnections, please? netstat report me about 186 connections so I set 200 for that.
As all TCP buffers are stored in RAM, it is likely that under heavy (normal? ) load your tcp stack runs out of memory, hence the crashes. In this case, drastically increasing tcp_max_syn_backlog doesn't help .
lfroen told me I would have to find 'out of memory messages' but I never find something like this on /var/log/messages. How I could check that, please?
You may try to lower tcp_syn_retries and tcp_synack_retries from 5 (default) to, for example, 3, and set tcp_max_syn_backlog to a slightly higher value like 256.
As 'last resort' I will try revert to default all the changed tcp settings and then set 'tcp_abort_on_overflow' at 1.
Thanks iz0bbz I have to check it
-
I have look for params iz0bbz told me and I found this:
http://ipsysctl-tutorial.frozentux.net/chunkyhtml/tcpvariables.html (http://ipsysctl-tutorial.frozentux.net/chunkyhtml/tcpvariables.html)
as links say I don't must set on syncoockie protection because it's for servers under attack (which really I don't know because it could be true clients from ed2k network) so I have increased tcp_max_syn_backlog (228) over MaxConnectios (200) with a little offset, so aMule limiting connections and SYN flood protection don't warn. Please could somebody confirm aMule behavior work as I expected?
-
No, it didn't work, today I reached max connections and got SYN flood message. Why aMule didn't limit max number of connections as expected? This is statictis message with amulecmd
Max Connection Limit Reached: 569136 : 2008-05-16 18:45:57569136 can't be number of connections, isn't it?
-
By the way, have you tried the last resort (tcp_abort_on_overflow'= 1 ) ?
Thanks iz0bbz for your time.
I have read doing that I harm my clients (http://ipsysctl-tutorial.frozentux.net/chunkyhtml/tcpvariables.html) and I preferring do it that as last resort as you said. I'm uncomfortable with that solution.
Today I login into my server when it being under (possible) SYN flood attack and netstat show me 20 connections only. Some IP twice but changing all IPs. How I could reach more than 228 connections as 'SYN flooding' message said me in logs and netstat only showing no more than 21 connections (and all "SYN_RECV") ?? I have checked number of connections like this:
netstat | grep -c 1880
Maybe I have to set off SYN cookies for be able to measuring real number of connections?
edit: Output of netstat command [netstat -n --tcp | grep 1880 | sort]
tcp 0 0 10.1.1.20:1880 213.37.182.32:2649 SYN_RECV
tcp 0 0 10.1.1.20:1880 217.96.119.228:3608 SYN_RECV
tcp 0 0 10.1.1.20:1880 75.171.102.166:59113 SYN_RECV
tcp 0 0 10.1.1.20:1880 79.152.38.115:1318 SYN_RECV
tcp 0 0 10.1.1.20:1880 81.9.223.8:3510 SYN_RECV
tcp 0 0 10.1.1.20:1880 83.32.122.30:2154 SYN_RECV
tcp 0 0 10.1.1.20:1880 83.34.237.9:28360 SYN_RECV
tcp 0 0 10.1.1.20:1880 83.42.97.42:3001 SYN_RECV
tcp 0 0 10.1.1.20:1880 83.45.34.17:3596 SYN_RECV
tcp 0 0 10.1.1.20:1880 83.56.209.141:3634 SYN_RECV
tcp 0 0 10.1.1.20:1880 83.56.223.125:2099 SYN_RECV
tcp 0 0 10.1.1.20:1880 84.125.103.60:1218 SYN_RECV
tcp 0 0 10.1.1.20:1880 85.137.129.128:4203 SYN_RECV
tcp 0 0 10.1.1.20:1880 85.57.41.140:1842 SYN_RECV
tcp 0 0 10.1.1.20:1880 87.223.209.246:16724 SYN_RECV
tcp 0 0 10.1.1.20:1880 88.254.111.56:1589 SYN_RECV
tcp 108 0 10.1.1.20:1880 84.76.81.169:3928 CLOSE_WAIT
tcp 120 0 10.1.1.20:1880 84.102.1.16:3492 CLOSE_WAIT
tcp 123 0 10.1.1.20:1880 88.7.6.200:58101 CLOSE_WAIT
tcp 126 0 10.1.1.20:1880 88.15.23.112:1952 CLOSE_WAIT
tcp 133 0 10.1.1.20:1880 81.35.229.71:2988 CLOSE_WAIT
tcp 94 0 10.1.1.20:1880 81.208.31.212:63262 CLOSE_WAIT
-
Maybe you can try to set off SYN cookies and see if it just solves the problem
I just removed syncookies from TCP configuration.
Hard disk in my server has a standard partitioning schema, first partition is 1Gb swap (too much for this server, I know!) but server has only 64mb RAM, I'm building a cluster and always doing partitions like this way. Running normally I got, with top,
top - 00:58:43 up 26 days, 22:09, 1 user, load average: 0.98, 0.68, 0.62
Tasks: 48 total, 1 running, 47 sleeping, 0 stopped, 0 zombie
Cpu(s): 8.2%us, 11.5%sy, 0.7%ni, 76.6%id, 0.7%wa, 0.7%hi, 1.6%si, 0.0%st
Mem: 61288k total, 59508k used, 1780k free, 684k buffers
Swap: 1030160k total, 48284k used, 981876k free, 7852k cached
This server running only with amuled has always 60% load. I'm detecting SYN flood warnings, before seeing log, because load; it increase to 300%
Please what I have to look for?
Another question I didn't understand...
Why on SYN flood warning I only got 20 connections with netstat?
Last SYN flood warning I got cpu was spend 80%wa (amuled didn't reach its usual 50%)
-
Solutions:
1 ) Decrease your swap to max 2x real memory. More is useless, and often problematic.
Yes, easy
2) Free as much memory as possible: disable all useless daemons (perhaps some useful one too), tailor and recompile your kernel to your needs, make up something.
I couldn't free more resources here.
3) Buy more memory and install it.
I seriously thinking to update my server in few moths (It's a K6-II memory is SDRAM PC100 no available at normal prices)
I removed SYN cookies and system become more stable. System don't hang but I continue getting rare logs with amulecmd (statistics)
> Active Connections (estimate): 127
> Max Connection Limit Reached: 279543 : 2008-05-20 05:39:18
> Average Connections (estimate): 127.728
> Peak Connections (estimate): 5884
Thanks you iz0bbz