1. Depends on the web server (and sometimes on the configuration). Description of various models:
Apache with mpm_prefork (default on unix): processed per request. To minimize startup time, Apache maintains a pool of idle processes awaiting processing of new requests (which you are adjusting the size of). When a new request arrives, the master process delegates it to the available worker, otherwise a new one is created. If you had 100 requests, if you did not have 100 workers idling, you need to do some markup to handle the load. If the number of unoccupied processes exceeds the value of MaxSpare, some of them will be received after completion of the requests until there are enough of the total number of unoccupied processes.
Apache with mpm_event, mpm_worker, mpm_winnt: stream per request. Similarly, in most situations, apache supports a pool of idle threads and is also configurable. (A small detail, but functionally the same: mpm_worker starts several processes, each of which is multithreaded).
Nginx / Lighttpd: These are lightweight event-based servers that use select () / epoll () / poll () to multiplex multiple sockets without the need for multiple threads or processes. Due to the very careful encoding and use of non-blocking APIs, they can scale up to thousands of simultaneous requests for commercial equipment, providing affordable bandwidth and correctly configured file descriptor restrictions. The bottom line is that the implementation of traditional built-in scripting languages is almost impossible in the server context, this will negatively affect most of the benefits. Both support FastCGI, however, for external scripting languages.
2. Depending on the language or in some languages, which deployment model are you using. Some server configurations allow only specific deployment models.
Apache mod_php, mod_perl, mod_python: these modules run a separate interpreter for each apache worker. Most of them cannot work very well with mpm_worker (due to various thread safety issues in the client code), so they are mostly limited to forking models. This means that for each apache process you have a php / perl / python interpreter. This significantly increases the amount of memory: if a given apache-worker usually takes about 4 MB of memory on your system, one with PHP can take 15 MB, and one with Python can take 20-40 MB for an average application. Some of them will be shared memory between processes, but in general these models are very difficult to scale very large.
Apache (supported configurations), Lighttpd, CGI: this is basically a method of dying hosting. The problem with CGI is that you not only deploy a new request processing process, you do it for -all-request, and not only when you need to increase the load. Since dynamic languages today have a rather long startup time, this not only creates a lot of work for your web server, but also significantly increases the page load time. A small perl script may work just fine like CGI, but a large python, ruby or java application is rather cumbersome. In the case of Java, you can expect a second or more just to run the application, only to repeat it again on the next request.
All web servers, FastCGI / SCGI / AJP: this is the "external" model of hosting dynamic languages. There is a whole list of interesting options, but the bottom line is that your application listens on a socket and the web server processes the HTTP request and then sends it through a different protocol to the socket only for dynamic pages (static pages are usually processed directly by the web server).
This has many advantages because you will need less dynamic workers than you need the ability to handle connections. If for every 100 requests, half is for static files such as images, CSS, etc., And besides, if most dynamic requests are short, you can get 20 dynamic workers that will process 100 concurrent clients. That is, since the normal use of a supported connection to the server is 80% inactive, your dynamic interpreters can process requests from other clients. This is much better than the mod_php / python / perl approach, where when the user loads a CSS file or doesn't load anything at all, your interpreter sits there using memory and does nothing.
Apache mod_wsgi: This applies in particular to python hosting, but it takes advantage of several applications hosted on a web server (easy setup) and external hosting (process multiplexing). When you run it in daemon mode, mod_wsgi only delegates requests to your working daemons when necessary, and thus 4 daemons can handle 100 concurrent users (depending on your site and its workload).
Phusion Passenger: Passenger is an apache hosting system that is primarily designed to host ruby applications, and, like mod_wsgi, provides the benefits of both an external and a managed web server.
3. Again, I will analyze the issue based on hosting models, where applicable.
mod_php, mod_python, mod_perl: Only the common C libraries of your application will usually be shared between Apache workers. This is because apache first starts and then loads the dynamic code (which, due to the subtleties, basically cannot use shared pages). Translators do not communicate with each other within this model. Global variables are usually not shared. In the case of mod_python, you can have global variables between requests in a process, but not through processes. This can lead to some very strange behavior (browsers rarely support the same connection forever, and most open several for this website), so be very careful how you use global variables. Use something like memcached or a database or files for things like session storage and other cache bits to use.
FastCGI / SCGI / AJP / Proxied HTTP: since your application is essentially a server in itself, it depends on the language the server is written in (usually it is the same language as your code, but not always) and various factors. For example, most Java deployments use a stream per request. Python and its "flup" FastCGI library can work both in pre-sale and in streaming mode, but since Python and its GIL are limited, you are likely to get the best performance from prefork.
mod_wsgi / passenger: mod_wsgi in server mode you can configure how it processes things, but I would recommend you give it a fixed number of processes. You want to keep your python code in memory, deploy and get ready for work. This is the best approach for predictability and low latency.
In almost all of the models mentioned above, the process / thread lifetime is more than one request. Most of the settings follow some options in the apache model: Keep some spare workers around, if necessary, create more when they can be used, when there are too many of them, based on several customizable restrictions. Most of these settings do not kill the process after the request, although some can clear the application code (for example, in the case of PHP fastcgi).
4. If you say that “the web server can only process 100 requests”, it depends on whether you mean the web server itself or the dynamic part of the web server. There is also a difference between actual and functional limits.
In the case of Apache, for example, you configure the maximum number of workers (connections). If this number of connections was 100 and was reached, no connections will be accepted by apache until someone disconnects. With keep-alive enabled, these 100 connections can remain open for a long time, much longer than one request, and the remaining 900 people waiting for requests are likely to be absent.
If you have high enough limits, you can accept all these users. However, even with the lightest Apache, the cost is about 2-3 MB per worker, so only with apache can you talk about 3gb + memory only for processing connections, not to mention other possibly limited OS resources, such as process identifiers, descriptors files, and buffers, and this is before reviewing your application code.
For lighttpd / nginx, they can handle a large number of connections (thousands) in a tiny amount of memory, often just a few megabytes per thousand connections (depends on factors such as buffers and how async IO apis is configured). If we assume that most of your connections remain alive and 80% (or more) are inactive, this is very good, since you do not spend time on dynamic time or most of the memory.
In any external hosted model (mod_wsgi / fastcgi / ajp / proxied http), let's say you only have 10 employees and 1000 users make a request, your web server will post requests to your dynamic employees. This is ideal: if your queries return quickly, you can continue to work with a much greater user load without requiring more work. Typically, a premium is a memory or database connection, and in order of priority, you can serve many more users with the same resources, instead of denying some users.
Be careful: say that you have one page that builds a report or searches and takes a few seconds, and many users associate workers with this: someone who wants to load your first page can be queued for a few seconds while all these lengthy requests are completed. Alternatives use a separate pool of workers to process the URLs in your section of the reporting application or for a separate publication (for example, in a background job), and then to subsequently poll its completion. There are many options, but you need to think about your application.
5 .. Most people using apache, which need to handle many concurrent users, have disconnected due to the large amount of memory. Or Apache with support enabled, with a short standby time of, say, 10 seconds (so you can get your first page and images / CSS in one page load). If you really need to scale to 1000 connections or more, and you want to continue to live, you'll want to see Nginx / lighttpd and other lightweight event-based servers.
It can be noted that if you want to use apache (for the convenience of using the configuration or for placing certain settings), you can put Nginx in front of apache using an HTTP proxy. This will allow Nginx to handle keep-alive links (and preferably static files) and apache to handle only grunt work. Interestingly, Nginx is better than apache when writing log files. For production deployment, we were very pleased with nginx before apache (with mod_wsgi in this case). Apache does not perform access registration and does not process static files, which allows us to disable a large number of modules inside apache in order to save its small area.
I already answered this already, but no, if you have a long connection, it should not have anything to do with how long the interpreter works (while you are using an external hosted application, which by now should be much better). Therefore, if you want to use a comet and stay alive for a long time (which is usually good if you can handle it), consider nginx.
Bonus question FastCGI . You mentioned that fastcgi can be multiplexed in one connection. This is really supported by the protocol (I believe the concept is known as “channels”), so theoretically one socket can handle many connections. However, this is not a required feature of fastcgi developers, and in fact I do not believe that there is one server that uses this. Most fastcgi respondents do not use this feature because the implementation of this is very complicated. Most web servers will only make one request through a given fastcgi socket at a time, and then do the next one through that socket. This way, you often only have one fastcgi socket per process / thread.
Whether your fastcgi application is used for processing or streaming (and whether you implement it through the "main" process that accepts connections, delegation, or just a lot of processes, each of which does its job) is up to you; and depends on the capabilities of your programming language and OS. In most cases, everything that is used by default by the library should be fine, but be prepared for some benchmarking and setting parameters.
As for the general state, I recommend that you pretend that there are no traditional applications of the collaborative process in the process: even if they can work now, you may need to share your dynamic workers on several computers later. For states such as shopping trolleys, etc .; db may be the best option, session login information can be stored in protected files, and for a temporary state, something similar to memcached is pretty neat. The less you rely on functions that exchange data (the common-nothing approach), the more you can scale in the future.
Postscript : I wrote and deployed many dynamic applications in the entire settings area above: all the web servers listed above and everything in the PHP / Python / Ruby / Java range. I intensively tested (using both benchmarking and real-world observation) methods, and the results are sometimes surprising: less often more. After you have moved away from hosting your code in the web server process, you can often leave with a very small number of FastCGI / Mongrel / mod_wsgi / etc employees. It depends on how long your application stays in the database, but very often it happens that more processes than 2 * the number of processors will actually bring you nothing.