Avoiding a huge query result that throws an OutOfMemoryException

Question

Here is my situation:

From my Grails controller, I call a service that queries the read-only database, converts the result to JSON, and returns the result.

Specs: JDK 1.6, Tomcat 5.5, Grails 1.3.4, DB via JNDI

Tomcats MaxPermSize is set to 256 m and Xmx to 128 m.
EDIT: Increasing memory should be a last resort

Service Method:

  String queryDB(String queryString) {
    StringWriter writer = new StringWriter()
    JSonBuilder json = new JSonBuilder(writer)
    def queryResult = SomeDomain.findAllBySomePropIlike("%${queryString}%")

    json.whatever {
      results {
        queryResult.eachWithIndex { qr, i ->
          // insert domain w/ properties
        }
      }
    }
    queryResult = null
    return writer.toString()
  }

Now that queryString == 'a', the result set is huge, and I get the following:

[ERROR] 03/Nov/2010@09:46:39,604 [localhost].[/grails-app-0.1].[grails] - Servlet.service() for servlet grails threw exception
java.lang.OutOfMemoryError: GC overhead limit exceeded
    at org.codehaus.groovy.util.ComplexKeyHashMap.init(ComplexKeyHashMap.java:81)
    at org.codehaus.groovy.util.ComplexKeyHashMap.<init>(ComplexKeyHashMap.java:46)
    at org.codehaus.groovy.util.SingleKeyHashMap.<init>(SingleKeyHashMap.java:29)
    at groovy.lang.MetaClassImpl$Index.<init>(MetaClassImpl.java:3381)
    at groovy.lang.MetaClassImpl$MethodIndex.<init>(MetaClassImpl.java:3364)
    at groovy.lang.MetaClassImpl.<init>(MetaClassImpl.java:140)
    at groovy.lang.MetaClassImpl.<init>(MetaClassImpl.java:190)
    at groovy.lang.MetaClassImpl.<init>(MetaClassImpl.java:196)
    at groovy.lang.ExpandoMetaClass.<init>(ExpandoMetaClass.java:298)
    at groovy.lang.ExpandoMetaClass.<init>(ExpandoMetaClass.java:333)
    at groovy.lang.ExpandoMetaClassCreationHandle.createNormalMetaClass(ExpandoMetaClassCreationHandle.java:46)
    at groovy.lang.MetaClassRegistry$MetaClassCreationHandle.createWithCustomLookup(MetaClassRegistry.java:139)
    at groovy.lang.MetaClassRegistry$MetaClassCreationHandle.create(MetaClassRegistry.java:122)
    at org.codehaus.groovy.reflection.ClassInfo.getMetaClassUnderLock(ClassInfo.java:165)
    at org.codehaus.groovy.reflection.ClassInfo.getMetaClass(ClassInfo.java:182)
    at org.codehaus.groovy.runtime.callsite.ClassMetaClassGetPropertySite.<init>(ClassMetaClassGetPropertySite.java:35)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.createClassMetaClassGetPropertySite(AbstractCallSite.java:308)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.createGetPropertySite(AbstractCallSite.java:258)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.acceptGetProperty(AbstractCallSite.java:245)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:237)
    at org.codehaus.groovy.grails.plugins.web.filters.FilterToHandlerAdapter.accept(FilterToHandlerAdapter.groovy:196)
    at org.codehaus.groovy.grails.plugins.web.filters.FilterToHandlerAdapter$accept.callCurrent(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:44)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:143)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:159)
    at org.codehaus.groovy.grails.plugins.web.filters.FilterToHandlerAdapter.preHandle(FilterToHandlerAdapter.groovy:107)
    at org.springframework.web.servlet.HandlerInterceptor$preHandle.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
    at org.codehaus.groovy.grails.plugins.web.filters.CompositeInterceptor.preHandle(CompositeInterceptor.groovy:42)
    at org.codehaus.groovy.grails.web.servlet.GrailsDispatcherServlet.doDispatch(GrailsDispatcherServlet.java:282)

, , Hibernate , . , , ( ), GORM.

? -?

EDIT: , , finder. , GORM , , ? , , greenhorn, .

Answer 1

Sun ( ) OutOfMemoryError :

/ OutOfMemoryError, : 98% 2% , OutOfMemoryError . . , -XX: -UseGCOverheadLimit .

, , (, , ). , , .

Answer 2

. , . , :

def offset = 0
def max = 50
while(stillMoreResults) {
    def batch = SomeDomain.findAllBySomePropIlike("%${queryString}%", [max: max, offset: offset])
    appendBatchToJsonResult(batch)
    offset += max
}

. .

, , . , , , , , .

Answer 3

, , , . , - /; , , . , , - ?

, , , .. , , , JSON. .

Answer 4

, , , .

JSON comsume. , "-"

Answer 5

In a Grails application using a MySQL database (MySQL was installed through Homebrew), I got the same problem, oddly enough, only by running the application without first starting the MySQL server. This way it just does

mysql.server start

fixed the problem for me.