Scala can't deal with huge objects?

Question

I have a syntactically correct Scala source file containing a huge data structure. The code is as follows:

object ChiSquareAlpha { val ChiSquareToAlphaIndexTable = Map( 1 -> Array(0.00016,0.00063,0.00393,0.01579,0.06418,0.14847,0.45494,1.07419,1.64237,2.70554,3.84146,5.41189,6.63490,10.82757), .. 9998 other entries .. 10000 -> Array(9673.95,9711.71,9768.53,9819.19,9880.79,9925.36,9999.33,10073.68,10118.82,10181.66,10233.75,10292.58,10331.93,10442.73) ) } 

So, you can see that the object contains a huge search table, a map [Int, Array [Double]] with 10,000 elements. Compilation takes about a minute, but ends without warning or error. But at runtime, it fails:

 java.lang.ClassFormatError: Invalid this class index 4280 in constant pool in class file bm/statistic/ChiSquareAlpha$ at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:791) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.scala$tools$nsc$util$ScalaClassLoader$$super$findClass(ScalaClassLoader.scala:88) at scala.tools.nsc.util.ScalaClassLoader$class.findClass(ScalaClassLoader.scala:44) at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.findClass(ScalaClassLoader.scala:88) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.scala$tools$nsc$util$ScalaClassLoader$$super$loadClass(ScalaClassLoader.scala:88) at scala.tools.nsc.util.ScalaClassLoader$class.loadClass(ScalaClassLoader.scala:50) at scala.tools.nsc.util.ScalaClassLoader$URLClassLoader.loadClass(ScalaClassLoader.scala:88) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at bm.statistic.ChiSquareTest.testOfIndependence(ChiSquareTest.scala:323) at ... 

I tested this with Scala 2.9.2 and Scala 2.10. The same result. When the object is much smaller - say, 500 entries in the map - the program works. Therefore, I assume that Scala cannot deal with such large files. Any suggestions for a solution?

Answer 1

You are right, your syntax looks fine.

We apologize for the disappointing answer, this is an open bug with Scala:

Scala SI-6543 programming language scala compile a crash with a simple but long repeating script (ClassNotFound and / or StackOverflowError)

The priority of fixation is major (high), and you can find additional information about this (they explain why this happens).

You can track the progress. In the meantime, you can create this information in a flat file and read the data from it as a temporary fix.