I am trying to load about 5 million objects retrieved from DB via hibernate into hashmap. I do this for two types of classes (A and B). I walk through the fields. The key is a field from pojo, and the value is pojo themselves.
1. For class A, the key is an integer field. I can download the map in less than 20 seconds.
For class B
2.a) Test 1, my key is a string field. When I try to load these objects into a new hashmap (a new attempt to restart the java process, so there is no attention with the GC yet), it takes about 30 seconds to load 100K objects onto the map.
2.b) Test 2, when I try to use another field of this class (integer type) and load the map, it works as the 1st one and loads in less than 20 seconds.
2.c) Test 3, I wondered if the problem was a data type. So, for class B, I tried a different approach to creating a string key using an integer field in # 2.b. (key = int_field + "") and loads in <20 seconds.
Another test, test 4, which I did for a class of type B, is the way I created the key. For 2.c, I created a key similar to this. map.put (pojo.getIntField () + "", pojo);
The result was as indicated above in 2.c
2.d) But when I created another getter in pojo, which returned int_field + "" and used it in a map placed as map.put (pojo.getIntFieldInStringForm (), pojo);
Performance degraded to about 30 seconds for 100K objects.
I know that the problem is with the keys because I checked the db fetch phase by adding the result objects to the list and loading it in <20 seconds for both types.
. - , . .
:
( /, ):
# 1
Map<String, ClassA> map = new HashMap<String, ClassA>();
Session session = sessionFactory.openNewSession();
try {
Iterator<ClassA> iterator = session.createQuery( "from ClassA" ).setFetchSize( 1000 ).iterate();
while ( iterator.hasNext() ) {
ClassB objClassA = iterator.next();
map.put( objClassB.getIntField(), objClassA );
}
}
catch (Exception e) {
e.printStackTrace();
}
finally {
session.close();
}
# 2.
Map<String, ClassB> map = new HashMap<String, ClassB>();
Session session = sessionFactory.openNewSession();
try {
Iterator<ClassB> iterator = session.createQuery( "from ClassB" ).setFetchSize( 1000 ).iterate();
while ( iterator.hasNext() ) {
ClassB objClassB = iterator.next();
map.put( objClassB.getStringField(), objClassB );
}
}
catch (Exception e) {
e.printStackTrace();
}
finally {
session.close();
}
# 2.b
Map<Integer, ClassB> map = new HashMap<Integer, ClassB>();
Session session = sessionFactory.openNewSession();
try {
Iterator<ClassB> iterator = session.createQuery( "from ClassB" ).setFetchSize( 1000 ).iterate();
while ( iterator.hasNext() ) {
ClassB objClassB = iterator.next();
map.put( objClassB.getIntField(), objClassB );
}
}
catch (Exception e) {
e.printStackTrace();
}
finally {
session.close();
}
# 2.c
Map<String, ClassB> map = new HashMap<String, ClassB>();
Session session = sessionFactory.openNewSession();
try {
Iterator<ClassB> iterator = session.createQuery( "from ClassB" ).setFetchSize( 1000 ).iterate();
while ( iterator.hasNext() ) {
ClassB objClassB = iterator.next();
map.put( objClassB.getIntField() + "", objClassB );
}
}
catch (Exception e) {
e.printStackTrace();
}
finally {
session.close();
}
# 2.d
Map<String, ClassB> map = new HashMap<String, ClassB>();
Session session = sessionFactory.openNewSession();
try {
Iterator<ClassB> iterator = session.createQuery( "from ClassB" ).setFetchSize( 1000 ).iterate();
while ( iterator.hasNext() ) {
ClassB objClassB = iterator.next();
map.put( objClassB.getIntFieldInStringForm() + "", objClassB );
}
}
catch (Exception e) {
e.printStackTrace();
}
finally {
session.close();
}