Spring JPA and MySQL data: how to avoid a re-write error in a multi-threaded environment?

My web application database infrastructure is overlaid as follows:

  • Spring JPA Data
  • Jpa
  • Hibernate
  • Pool C3P0
  • MySQL

Some transactions take a lot of time, sometimes one minute for one large transaction (the goal of caching data to the database): an HTTP request from a user to my web server can start this transaction. Then my web server can request another remote third-party server for missing data. When all the data is collected, the transaction is completed, and all the collected data is written to the database.

It may happen that a user reloads my site during this lengthy transaction. This results in a different transaction for the same data in a separate stream. Since the purpose of this transaction is cached, and therefore this operation is idempotent in nature, I do not mind having the same calculations doubled. But in these situations, my web application sporadically encounters the following error.

20:12:46.392 container [Thread-24] WARN ohejdbc.spi.SqlExceptionHelper - SQL Error: 1062, SQLState: 23000 20:12:46.392 container [Thread-24] ERROR ohejdbc.spi.SqlExceptionHelper - Duplicate entry '146845042025054' for key 'PRIMARY' 20:12:46.602 container [Thread-24] INFO ohejbinternal.AbstractBatchImpl - HHH000010: On release of batch it still contained JDBC statements Exception in thread "Thread-24" org.springframework.dao.DataIntegrityViolationException: Duplicate entry '146845042025054' for key 'PRIMARY'; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: Duplicate entry '146845042025054' for key 'PRIMARY' at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:643) at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:104) at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:516) at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754) at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723) at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:394) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:155) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.data.jpa.repository.support.LockModeRepositoryPostProcessor$LockModePopulatingMethodIntercceptor.invoke(LockModeRepositoryPostProcessor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) at com.sun.proxy.$Proxy45.save(Unknown Source) at com.mycompany.MyJpaEntityDomainClass.setId(MyJpaEntityDomainClass.java:83) at com.mycompany.MyCustomRepository.findMyJpaEntityDomainClassInstance(MyCustomRepository.java:72) at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:155) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) at com.sun.proxy.$Proxy46.findMyJpaEntityDomainClassInstance(Unknown Source) at com.mycompany.AnotherDomainEntityClass.setAssociatedJpaEntityDomainInstances(AnotherDomainEntityClass.java:343) at com.mycompany.MyThreadInvokingService$SearcherThread.run(MyThreadInvokingService.java:106) at java.lang.Thread.run(Thread.java:722) Caused by: org.hibernate.exception.ConstraintViolationException: Duplicate entry '146845042025054' for key 'PRIMARY' at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:74) at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110) at org.hibernate.engine.jdbc.internal.proxy.AbstractStatementProxyHandler.continueInvocation(AbstractStatementProxyHandler.java:129) at org.hibernate.engine.jdbc.internal.proxy.AbstractProxyHandler.invoke(AbstractProxyHandler.java:81) at com.sun.proxy.$Proxy76.executeUpdate(Unknown Source) at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:56) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2962) at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3403) at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:88) at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:362) at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:354) at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:275) at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:326) at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:52) at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1210) at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:399) at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101) at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:175) at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:75) at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:512) ... 31 more Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '146845042025054' for key 'PRIMARY' at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) at com.mysql.jdbc.Util.getInstance(Util.java:386) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1040) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2629) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2719) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155) at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2450) at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2371) at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2355) at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:105) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.hibernate.engine.jdbc.internal.proxy.AbstractStatementProxyHandler.continueInvocation(AbstractStatementProxyHandler.java:122) ... 48 more 

This call to com.sun.proxy.$Proxy45.save is actually a call to the save method MyJpaEntityDomainClassRepository extends org.springframework.data.jpa.repository.JpaRepository<MyJpaEntityDomainClass, String> .

As you can infer from this stack trace, instances of MyJpaEntityDomainClass take care of creating and updating themselves in the repository.

I have the feeling that a duplicate write error occurs in situations of two parallel threads trying to write the same object to the database. I don’t understand why this error occurs at all, since org.springframework.data.jpa.repository.JpaRepositor.save( uniqueId ) in a slightly later stream should simply overwrite the object / record associated with uniqueId placed in the repository with a slightly earlier stream .

How can I avoid such transaction related errors related to transactions?

+4
source share
2 answers

A re-write error will occur when two records of the same object are inserted. all you need to check:

  • - This is the same JPA / Hibernate session shared by threads performing an update during an error.
  • How is a unique record identifier created? (myqsl auto-increment; generator code id. etc.), and if there is an identifier, the column is different from the actual Primary key (restriction) of the table.
+1
source

From your message, I can say that you are using transactions, so you need to synchronize the threads somehow. The simplest, but not the most portable or effective solution is to lock tables in which you can use the @Transactional annotation, you must add isolation of your transaction to isolation. SERIALIZABLE

 @Transactional(isolation = Isolation.SERIALIZABLE) 

If your application is running on the same JVM, another easy way to make a method available for only one thread is to use a synchronized keyword.

  public synchronized void method(){} 

You can create a common data structure and save it in memory and either synchronize it as a method, or use a framework, for example, a halo or terracotta, this answer goes deeper in this approach .

The last method I can think of is the most reliable, efficient, and difficult. And it synchronizes your application through JMS messages or something like that.

The problem may also be that the record already exists, so you can request the last identifier with something like:

 Entity findTop1ByOrderByIdDesc(); 

Then return the identifier and add it, this can leave holes if you remove your rights, and not the most effective ones, but complete the task.

See also the second diarmuid sentence. The primary key primary key in the definition of your JPA entity.

 @Id @GeneratedValue(strategy = GenerationType.AUTO) //This one @Column(name = "id", nullable = false) public int getId() { return id; } 

And double check that the id has auto-increment in the table definition. As a final recommendation, you can also check if your transaction does not have access to several entities, one of which may not have a GeneratedValue, and this may be the culprit of the message sent.

0
source

Source: https://habr.com/ru/post/1486934/


All Articles