Monitoring and preventing system crashes for Mnesia on an Erlang system

Question

We have several cloud servers, and I'm creating a new one to check the provision of resources.

I get a fatal failure before I think I should, because some kind of system resource is running out.

=INFO REPORT==== 14-Feb-2010::12:40:14 === Setting up: "http://sub48.localhost:9000" as pirate Mnesia(' [e-mail] '): Data may be missing, Corrupt logfile deleted: "(...)/sub48.localhost&9000&styles.DCL", {file_error, "(...)/sub48.localhost&9000&styles.DCL", system_limit} =ERROR REPORT==== 14-Feb-2010::12:40:18 === Mnesia(' [e-mail] '): ** ERROR ** (could not write core file: system_limit) ** FATAL ** Cannot open log file "(...)/sub48.localhost&9000&styles.DCL": {file_error, "(...)/sub48.localhost&9000&styles.DCL", system_limit} 

The operating system is Ubunut 8.04 (LTS), but our others are Ubuntu 9.04 and Ubuntu 9.10 - I think we will have to standardize them :(

So my questions are:

• How to determine which resource is exhausted?
• What active monitoring steps can be taken to ensure that this does not happen again?
• What system resources, in general, can I exhaust with Erlang VM and what monitoring steps should I have for them?

Answer 1

There is an erlang module called os_mon that allows you to track various resources such as CPU utilization. Also check out the sasl OTP application, especially overload and alarm_handler.