What causes a TCP/IP reset (RST) flag to be sent?
NetworkingTcpNetworking Problem Overview
I'm trying to figure out why my app's TCP/IP connection keeps hiccuping every 10 minutes (exactly, within 1-2 seconds). I ran Wireshark and discovered that after 10 minutes of inactivity the other end is sending a packet with the reset (RST) flag set. A google search tells me "the RESET flag signifies that the receiver has become confused and so wants to abort the connection" but that is a little short of the detail I need. What could be causing this? And is it possible that some router along the way is responsible for it or would this always come from the other endpoint?
Edit: There is a router (specifically a Linksys WRT-54G) sitting between my computer and the other endpoint -- is there anything I should look for in the router settings?
Networking Solutions
Solution 1 - Networking
A 'router' could be doing anything - particularly NAT, which might involve any amount of bug-ridden messing with traffic...
One reason a device will send a RST is in response to receiving a packet for a closed socket.
It's hard to give a firm but general answer, because every possible perversion has been visited on TCP since its inception, and all sorts of people might be inserting RSTs in an attempt to block traffic. (Some 'national firewalls' work like this, for example.)
Solution 2 - Networking
Run a packet sniffer (e.g., Wireshark) also on the peer to see whether it's the peer who's sending the RST or someone in the middle.
Solution 3 - Networking
I've just spent quite some time troubleshooting this very problem. None of the proposed solutions worked. Turned out that our sysadmin by mistake assigned the same static IP to two unrelated servers belonging to different groups, but sitting on the same network. The end results were intermittently dropped vnc connections, browser that had to be refreshed several times to fetch the web page, and other strange things.
Solution 4 - Networking
One thing to be aware of is that many Linux netfilter firewalls are misconfigured.
If you have something like:
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -p tcp -j REJECT --reject-with tcp-reset
then packet reordering can result in the firewall considering the packets invalid and thus generating resets which will then break otherwise healthy connections.
Reordering is particularly likely with a wireless network.
This should instead be:
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -m state --state INVALID -j DROP
-A FORWARD -p tcp -j REJECT --reject-with tcp-reset
Basically anytime you have:
... -m state --state RELATED,ESTABLISHED -j ACCEPT
it should immediately be followed by:
... -m state --state INVALID -j DROP
It's better to drop a packet then to generate a potentially protocol disrupting tcp reset. Resets are better when they're provably the correct thing to send... since this eliminates timeouts. But if there's any chance they're invalid then they can cause this sort of pain.
Solution 5 - Networking
RST is sent by the side doing the active close because it is the side which sends the last ACK. So if it receives FIN from the side doing the passive close in a wrong state, it sends a RST packet which indicates other side that an error has occured.
Solution 6 - Networking
If there is a router doing NAT, especially a low end router with few resources, it will age the oldest TCP sessions first. To do this it sets the RST flag in the packet that effectively tells the receiving station to (very ungracefully) close the connection. this is done to save resources.
Solution 7 - Networking
Some firewalls do that if a connection is idle for x number of minutes. Some ISPs set their routers to do that for various reasons as well.
In this day and age, you'll need to gracefully handle (re-establish as needed) that condition.
Solution 8 - Networking
This is because there is another process in the network sending RST to your TCP connection.
Normally RST would be sent in the following case
- A process close the socket when socket using SO_LINGER option is enabled
- OS is doing the resource cleanup when your process exit without closing socket.
In your case, it sounds like a process is connecting your connection(IP + port) and keeps sending RST after establish the connection.
Solution 9 - Networking
In most applications, the socket connection has a timeout. If there is no communication between the client and the server within the timeout, the connection is reset as you observe. A great example is a FTP server, if you connect to the server and just leave the connection without browsing or downloading files, the server will kick you off the connection, usually to allow other to be able to connect. I guess this is what you are experiencing with your connection. So take a look in the server application, if that is where you get the reset from, and see if it indeed has a timeout set for the connection in the source code.
Solution 10 - Networking
Here are some cases where a TCP reset could be sent.
- Non-Existence TCP endpoint: The client sends SYN to a non-existing TCP port or IP on the server-side. The server will send a reset to the client.
- SYN matches the existing TCP endpoint: The client sends SYN to an existing TCP endpoint, which means the same 5-tuple. The server will send a reset to the client.
- Accept Queue Full: When the accept queue is full on the server-side, and tcp_abort_on_overflow is set. The server will send a reset to the client.
- Half-Open Connections: When the server restarts itself. Then all connections before would receive reset from server side.
- Firewall: The firewall could send a reset to the client or server
- Time-Wait Assassination: When the client in the time-wait state, receives a message from the server-side, the client will send a reset to the server.
- Aborting Connection: When the client aborts the connection, it could send a reset to the server