What is the difference between Tomcat's BIO Connector and NIO Connector?
TomcatNioTomcat Problem Overview
I would like to know the internals of the tomcat NIO connector. How exactly are threads used when we create a servlet that implements CometProcessor?Is it still one thread per connection?
From what I read, the conversation goes like this
-
Client connects to a servlet
-
Servlet hangs on to the connection till there is any data available to the connected client
-
When data is ready , the server writes to the httpResponse and flushes it. This actually disconnects the connection?
-
Client sends another request which the server again hangs onto..
How many thread are used when this keeps happening?
Tomcat Solutions
Solution 1 - Tomcat
NIO and Comet are completely unrelated: you can mix-and-match them.
Using the NIO (or APR for that matter) connector allows you to handle more requests with fewer threads due to the threading model. See http://tomcat.apache.org/tomcat-7.0-doc/config/http.html#Connector_Comparison for a comparison between the Connectors.
Comet (and Websocket) have a completely different dispatch model which requires a different application architecture and achieves higher throughput in a different way.
The scenario you pose in your question is the typical blocking one-thread-per-request model. In step 4, the Java BIO connector (which is the default up through Tomcat 7) will continue to wait for additional requests on the existing connector -- for keepalive HTTP requests. If the client does not set Connection:close on the previous request and does not close the connection, the thread will hang until the keepalive timeout is reached. If you use the NIO connector, the thread will go back into the thread pool immediately after the response is sent and you won't "waste" a thread on keepalive requests that may never arrive.
Comet/Websocket works entirely differently by delivering a message to a specially-written servlet (and optional filters) and the threads are only used when there are messages to send or data to be written.
UPDATE 2016-08-19
Tomcat 8.5 and 9.0 have completely dropped the BIO connector. This is because many of the new APIs and technologies (e.g. Websocket) require non-blocking semantics, and building a non-blocking service on top of a blocking API is very very difficult. The code required to get the job done was making the rest of the Tomcat code very ugly, etc. and so the decision was made to drop the BIO connector completely. So for Tomcat 8.5 and onward, only NIO, NIO2, and the APR-based connectors are available.
Note that, also with Tomcat 8.5 and 9.0, support for Comet has been dropped. Uses of Comet should all be replaced with Websocket which is a more standard protocol.
Solution 2 - Tomcat
NIO use fewer thread, it means that the tcp/ip port use is fewer.
You know the port is 1 to 65534, so we can say that NIO can reach a higher TPS (Transactions Per Second) than BIO
I tested both protocol :HTTP/1.1 and org.apache.coyote.http11.Http11NioProtocol
with same web-project, same host, and same server.xml but the protocol.
Use jmeter for test.
I set 1000 thread to run request, when HTTP/1.1 in a few minutes, the host use port is more than 30000 and the TPS is 300 only!
When org.apache.coyote.http11.Http11NioProtocol, the max count of use port is never overstep 3000 and the tps is more than 1200!
Solution 3 - Tomcat
Adding late to this discussion - In this context of performance comparisons between blocking IO and Asynchronous NIO - an excellent read is "Old way of writing servers is new". In summary below thread per connection model was found to be better performing and easy to write as compared to the NIO version - contrary to the popular belief.
Solution 4 - Tomcat
Here are two good articles on the NIO connector in case this helps someone considering the differences between BIO (request processing is bound to accept thread) & NIO (request processing is passed onto another worker thread) in Tomcat.