Erlang is very cheap to create processes (orders of magnitude compared to other systems).
Therefore, I recommend creating a new ProcessFileServer every time a new file for processing appears. When this is done, just complete the process with the goal of exiting normal .
I would suggest the following structure:
top_supervisor | +-----------------------+-------------------------+ | | directory_supervisor processing_supervisor | simple_one_for_one +----------+-----...-----+ | | | | starts children transient | | | | dir_watcher_1 dir_watcher_2 dir_watcher_n +-------------+------+---...----+ | | | proc_file_1 proc_file_2 proc_file_n
When a dir_watcher notes that a new file has appeared. It calls the processing_supervisor supervisor:start_child\2 function supervisor:start_child\2 , with an additional parameter to the pathe file, for example.
processing_supervisor should start its children with a transient reload policy.
Therefore, if one of the proc_file servers fails, it will be restarted, but when they fail with the normal exit reason, they will not restart. That way, you just exit normal when this is done, and crash when something else happens.
If you did not overdo it, the round-robin polling for files is Ok. If the system boots due to this poll, you can search kernel notification systems (for example, FreeBSD KQUEUE or higher-level services built on it in MacOSX) to send you a message when the file appears in the directory. However, these services are difficult because they need to give up if too many events happen (otherwise they will not be a performance improvement, but vice versa). Therefore, in any case, you will have to have a reliable survey solution.
Therefore, do not prematurely optimize and start by polling, adding improvements (which would be isolated on dir_watcher servers) when necessary.
Regarding the comment about what behavior should be used as dir_watcher , since it does not use most of gen_servers :
There is no problem using only part of the capabilities of gen_servers , in fact, very often do not use all of this. In your case, you only set up a timer in init and use handle_info to do your job. The rest of gen_server is just an immutable template.
If you later want to change parameters, such as the polling rate, it’s easy to add to this.
gen_fsm much less used since it is only suitable for a rather limited model and not very flexible. I use it only when it really meets 100% requirements (which it almost never does).
In the case when you just want a simple simple Erlang server, you can use the spawn functions in proc_lib to get just the minimum functionality to work under the supervisor.
An interesting way to write more natural Erlang code and still have the benefits of OTP plain_fsm , here you have the advantages of selective reception and flexible message processing, especially when processing protocols in combination with the good OTP features.
Having said all this: if I wrote dir_watcher , I would just use gen_server and use only what I need. Unused functionality really costs you nothing, and everyone understands what it does.