Every couple of days we get a small number of MySql timeout errors that correspond to a large spike in the CPU and DB connections on our MySQL RDS instance. These are requests that are usually very fast (<5ms), which unexpectedly lose time.
At this point, database operations occur very slowly for a minute or so (probably due to the fact that new connections are allocated). The number of new connections often doubles and seems to correspond to the transition of the entire connection pool.
Timeouts do not seem to correspond to a heavy database load. The processor frequency is less than 7% when this happens to about 12%.
Once these connections are created, the old connections seem to remain for several hours.
There are some theories:
- Random network hiccups between EC2 and RDS
- Restarting the connection pool (is there such a thing?)
- Resource competition on a server that backs up all requests (no deadlocks)
Any help on debugging would be greatly appreciated.
System Information:
- Windows EC2 Instances
- .NET 4.5
- MySql Connector 6.8.3
- Entity Framework 6.0.2
- MySql.Data.Entities 6.8.3
- MySql 5.6.12 (hosted on Amazon RDS)
source
share