Server for millions of concurrent connections

Question

I am creating a distributed system consisting of potentially millions of clients who all need to support an open (preferably HTTP) connection to wait for a command from the server (which works elsewhere). The load of messages / commands will not be very high, maybe one message / sec / 1000 clients, which means that it will be 1000 msg / sec @ 1 million clients. => mainly about parallel connections.

The requirements are also simple. One-way messaging (server-> client), only 1 client per channel.

I'm pretty open in terms of technology (xmpp / websockets / comet / ...). I use the Google App Engine as a server, but their “channels” will not work for me (too low quotas and lack of a Java client). XMPP was an option, but rather expensive. So far I have used URL Fetch and pubnub, but they have only just started charging connection fees (a lot of time).

So:

• Does anyone know of a service that can make this available to me in an accessible way? Most of them I found a restriction or a large connection fee.

• Any experience implementing such a server on your own? I actually already did this, and it works very well (based on Tomcat and NIO), but I still did not have time to set up an environment with heavy loads (partly because it is still a backup solution, I would preferred battle hardened server msg). Any experience how many users you get per GB? Any tough restrictions?

My architecture also allows fragmentation of msg servers, but I would like to maximize parallel connections because the msg message processor overhead is minimal.

Answer 1

In the meantime, I implemented my own message server using netty.io. Netty uses Java NIO and scales very well. For idle connections, I get 500 bytes of memory per connection. I only do very simple message forwarding (without caching, storage or other fancy things), but with this I get 1000 - 1500 msg / sec (every half kilobyte) on a small copy of amazon (1ECU / 1.6 GB).

Otherwise, if you are looking for a (paid) service, I can recommend spire.io (they do not charge for connection, but have a higher price for message) or pubnub (they charge for connection, but cheaper for each message).

Answer 2

You need to look more at the architecture for creating such an environment. First of all, if you write socket management yourself, do not use Thread for Client Socket. Use asynchronous methods to receive and send data. WebSockets can be too heavy if your messages are small. Since it implements framing, which must be applied to each message for each socket individually (caching can be used for different versions of the WebSockets protocols), which makes them process both directions more slowly: for receiving and sending, especially due to data masking.

You can create millions of sockets, but only the most advanced technologies can do this. Erlang is capable of handling millions of connections and is quite scalable. If you want to have millions of connections using other higher-level technologies, you need to think about clustering what you are trying to accomplish.

For example, using a gateway server that will monitor all processing servers. And to have data from them (IP, ports, download (if it is one internal network, a firewall and port forwarding can be convenient here). Client software connects to this gateway server, the gateway server checks the least loaded server and sends the ip and port to the client The client creates a connection directly to the production server using the provided address, so you will have a gateway that can also handle authorization and will not bind connections for a long time, so one of them may be enough And many employees who publish data and store links.

This is very related to your needs and may not be suitable for your decisions.