Combining java spring / stream and database access for mission-critical web applications

Question

I am developing an MVC spring web application and I would like to store my users actions (what they click, etc.) in a database for offline analysis. Let say that the action is a tuple (long userId, long actionId, Date timestamp). I am not interested in the actions of my users, but I take this as an example.

I expect a lot of action from a few (different) users in a couple of minutes (in seconds). Therefore, processing time is critical.

In my current implementation, I defined a data source with a connection pool to store actions in the database. I call the service from the controller request method, and this service calls the DAO, which stores the action in the database.

This implementation is inefficient because it expects a call from the controller and all the way to the database to be executed to return a response to the user. So I was thinking of wrapping this "save action" in a stream so that the response to the user would be faster. To receive the answer, the stream does not need to be completed.

I have no experience in these massive, parallel and mission critical applications. Therefore, any feedback / comments will be very helpful.

Now my questions are:

• How would you create such a system? Will you implement the service and then transfer it to the stream, called with each action?

• What should i use? I checked spring Batch and this JobLauncher, but I'm not sure if this is right for me.

• What happens when there are concurrent calls in the controller, service, DAO, and data source layer?

In more general terms, what are the best practices for designing such applications?

Thank you for your help!

Answer 1

Take the Singleton object @apps level and update it with every user action. This singleton object must have a Hashmap as a shared one, which should be updated periodically after it reaches the level of 10,000 points and stores it in DB, like a spring package.

Also, periodically, update it / clear to the last number of entries # each time it is processed. We can also reinitialize a singleton instance, weekly / monthly. Remember that this can cause the update to be updated if your applications are deployed to multiple JVMs. Thus, you need to implement an exception not supported by the clone in singleton.

Answer 2

It is generally considered the wrong form to create your own threads in a Java EE application.

The best approach would be to write to the local queue via JMS and then have a separate component, such as a message-driven bean (pretty simple with EJB or Spring) that is stored in the database.

Another approach would be to simply write to the log file and then process the log file and write it to the database once a day or ever.

What you need to consider: -

• How relevant do you need information?
• How important is the information, can you lose it?
• How reliable should order be?

All these factors will affect how many threads you process in the queue / log file, whether you need a constant JMS queue and whether your processing should be performed on the remote system in your main container.

Hope this answers your questions.

Answer 3

If you are interested in the actions your users take, you should understand this from the HTTP requests they send, so you might be better off not registering incoming requests on the Apache web server, which is forwarded to your application server. Putting a cluster of web servers in front of application servers is a common practice (they are good for serving static content), and they usually record requests anyway. Thus, logging will be fast, your application will not have to deal with it, and the biggest job will be to write a script to clone the logs into a database where you can analyze.

Answer 4

Here is what I did for this:

Used by aspectJ to mark all the actions of the user I wanted to collect. Then I sent it to log4j with asynchronous dbAppender ...

This allows you to enable or disable it using the log4j log.

works great.