Here is the repository with the code in this question setting out the error: https://github.com/agsimeonov/stream-bug
I am trying to pass query results using Spring data to JPA and Hibernate using the following code snippet (data.txt is a file with 3000 lines with a number in each line):
try (Stream<Customer> stream = repository.streamAll()) { stream.forEach(customer -> { try { File data = new File(getClass().getClassLoader().getResource("data.txt").getFile()); try (BufferedReader reader = new BufferedReader(new FileReader(data))) { while (reader.readLine() != null) {
Here is the domain object:
@Entity @Table(name = "customer") public class Customer { @Id @GeneratedValue(strategy = GenerationType.AUTO) private Long id; private String firstName; private String lastName; public Customer() {} public Customer(String firstName, String lastName) { this.firstName = firstName; this.lastName = lastName; } @Override public String toString() { return String.format("Customer[id=%d, firstName='%s', lastName='%s']", id, firstName, lastName); } }
And here is the repository:
public interface CustomerRepository extends JpaRepository<Customer, Long> { @Query("SELECT c FROM Customer c") Stream<Customer> streamAll(); }
Executing this result with the following error:
org.hibernate.exception.GenericJDBCException: could not advance using next() at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:47) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:95) at org.hibernate.internal.ScrollableResultsImpl.convert(ScrollableResultsImpl.java:69) at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:104) at org.springframework.data.jpa.provider.PersistenceProvider$HibernateScrollableResultsIterator.hasNext(PersistenceProvider.java:454) at java.util.Iterator.forEachRemaining(Iterator.java:115) at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) at stream.bug.StreamBugApplication.lambda$0(StreamBugApplication.java:34) at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784) at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:771) at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1186) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1175) at stream.bug.StreamBugApplication.main(StreamBugApplication.java:22) Caused by: org.h2.jdbc.JdbcSQLException: The object is already closed [90007-193] at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) at org.h2.message.DbException.get(DbException.java:179) at org.h2.message.DbException.get(DbException.java:155) at org.h2.message.DbException.get(DbException.java:144) at org.h2.jdbc.JdbcResultSet.checkClosed(JdbcResultSet.java:3202) at org.h2.jdbc.JdbcResultSet.next(JdbcResultSet.java:129) at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:99) ... 12 more
I spent a lot of time debugging, and I finally managed to create a small spring-boot example application showing a streaming error: https://github.com/agsimeonov/stream-bug
I know a few things for sure:
First one . This error has nothing to do with the database. Although I use H2 in the example project that I tried with Postgres and the error still occurs with a very similar error, note I using the tomcat connection pool in my other project, I tried different connection pools, so this is definitely not a connection pool or base database causing this. Here is an example of tracing with postgres and tomcat pools, as you may notice that this is very similar:
org.hibernate.exception.GenericJDBCException: could not advance using next() at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:47) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97) at org.hibernate.internal.ScrollableResultsImpl.convert(ScrollableResultsImpl.java:69) at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:104) at org.springframework.data.jpa.provider.PersistenceProvider$HibernateScrollableResultsIterator.hasNext(PersistenceProvider.java:454) at java.util.Iterator.forEachRemaining(Iterator.java:115) at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at com.trove.sunstone.attributefusion.services.impl.PhysicalServiceImpl.match(PhysicalServiceImpl.java:130) at com.trove.sunstone.attributefusion.AppRunner.main(AppRunner.java:31) Suppressed: java.lang.reflect.UndeclaredThrowableException at com.sun.proxy.$Proxy238.hashCode(Unknown Source) at java.util.HashMap.hash(HashMap.java:338) at java.util.HashMap.get(HashMap.java:556) at org.hibernate.resource.jdbc.internal.ResourceRegistryStandardImpl.release(ResourceRegistryStandardImpl.java:76) at org.hibernate.internal.AbstractScrollableResults.close(AbstractScrollableResults.java:104) at org.springframework.data.jpa.provider.PersistenceProvider$HibernateScrollableResultsIterator.close(PersistenceProvider.java:465) at org.springframework.data.util.StreamUtils$CloseableIteratorDisposingRunnable.run(StreamUtils.java:96) at java.util.stream.AbstractPipeline.close(AbstractPipeline.java:323) at com.trove.sunstone.attributefusion.services.impl.PhysicalServiceImpl.match(PhysicalServiceImpl.java:137) ... 1 more Caused by: java.sql.SQLException: Statement closed. at org.apache.tomcat.jdbc.pool.interceptor.AbstractQueryReport$StatementProxy.invoke(AbstractQueryReport.java:224) ... 10 more Caused by: org.postgresql.util.PSQLException: This ResultSet is closed. at org.postgresql.jdbc.PgResultSet.checkClosed(PgResultSet.java:2740) at org.postgresql.jdbc.PgResultSet.next(PgResultSet.java:1817) at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:99) ... 11 more
The second - the odd part is that removing the following lines from the forEach () stream in the stream leads to the correct processing of the stream. This makes me think it might be some kind of time issue, however I tried to play it using Thread.sleep () instead of reading the file without success. As a side note, data.txt is a file with 3,000 lines with a number on each line.
try { File data = new File(getClass().getClassLoader().getResource("data.txt").getFile()); try (BufferedReader reader = new BufferedReader(new FileReader(data))) { while (reader.readLine() != null) { // Do stuff for the current customer } } } catch (IOException e) {}
Third - Replacement:
Stream<Customer> stream = repository.streamAll()
WITH
Stream<Customer> stream = repository.findAll().stream()
The bug is fixed, so this is definitely a bug with streaming and / or ScrollableResults, since loading all the data into the list causes the application to end without errors, however for my current project I need to use streams directly using findAll () is not an option.
If someone has encountered this problem and was able to fix it, let me know. Also, please feel free to check the plug and / or change the code in the provided repository, which can help solve this problem. I created this project as a demo, which should be used to illustrate the error.