Socket server with epoll and threads

Question

I am trying to create a socket server in C for a collaborative text editor http://en.wikipedia.org/wiki/Collaborative_real-time_editor , but I do not know what the best server architecture is for it.

Firstly, I tried to use select for the socket server, but after that I read about epoll, and now I think that epoll is the best choice, because the client will send every letter that the user writes to textarea to the server, so the server will have dedicated data for processing.

Also, I want to use streams with epoll, but I don’t know exactly how to use them. I want to use threads, because I think it is better to use 2 or all processors on the target machine.

My plan

• create 2 threads at server startup

• The first stream will analyze new customers and prepare them for reading or sending

• the second stream will have a task for reading and sending data from / to clients

The problem is that these 2 threads will use while (1) with epoll_wait.

My questions: is the server architecture good for using epoll with threads? If not, what are my options?

EDIT: I cannot use libevent or libev or other libraries because this is a college project and I do not allow the use of external libraries.

Answer 1

I think you are trying to overdo it. Linux's epoll architecture was designed for situations where you have thousands of concurrent connections. In such cases, the overhead, as determined by the poll and select system calls, will be the main bottleneck on the server. The decision to use poll or select vs. epoll based on the number of connections, not the amount of data.

For what you are doing, it seems that the people in your editing system are going crazy after you press dozens of simultaneous editors. Using epoll will probably make you crazy; they play some tricks with the API to squeeze extra performance, and you have to carefully process the information you receive from calls.

Such an application looks like it will be network bound, not CPU bound. I would try to write it first as a single-threaded server with poll . When you receive new text, be careful when sending it, and then send it when the socket accepts write calls. Use non-blocking I / O; the only call you want to block is poll .

If you do a significant amount of data processing after receiving it, but before sending it back to clients, you can use multi-threaded. First write a single-threaded version, then if you are attached to the processor (check with top ), and most of the processor’s time is spent on the functions where you process the data (check with gprof ), add multithreading to the data processing.

If you want, you can use channels or sockets of the Unix domain inside the program to communicate between different threads --- this way, everything in the main thread can be event driven and processed through poll . Alternatively, with this model, you could even use multiple processes with fork instead of multiple threads.

Answer 2

You might want to use something like libev or libevent instead of writing your own implementation of event handling. this will give you a cross-platform event handler that will use any suitable (be it select , poll , epoll , kqueue or something else) and, most likely, with less overhead than passing two threads to work with each other.

Answer 3

Just start using libevent or libev and follow their examples. There are many examples - do not try to come up with something new here.