Force serial processing in Haskell Data.Binary.Get

Question

After trying to import the Java rt.jar Runtime core library using the language-java-classfile, I found that it was using huge amounts of memory.

I reduced the program demonstrating the problem to 100 lines and loaded it into hpaste . Without forcing streamme to evaluate on line 94, I have no chance of ever running it, because it absorbs all my memory. Forced streambefore passing it to getClass, but still uses huge amounts of memory:

  34,302,587,664 bytes allocated in the heap
  32,583,990,728 bytes copied during GC
     139,810,024 bytes maximum residency (398 sample(s))
      29,142,240 bytes maximum slop
             281 MB total memory in use (4 MB lost due to fragmentation)

  Generation 0: 64992 collections,     0 parallel, 38.07s, 37.94s elapsed
  Generation 1:   398 collections,     0 parallel, 25.87s, 27.78s elapsed

  INIT  time    0.01s  (  0.00s elapsed)
  MUT   time   37.22s  ( 36.85s elapsed)
  GC    time   63.94s  ( 65.72s elapsed)
  RP    time    0.00s  (  0.00s elapsed)
  PROF  time   13.00s  ( 13.18s elapsed)
  EXIT  time    0.00s  (  0.00s elapsed)
  Total time  114.17s  (115.76s elapsed)

  %GC time      56.0%  (56.8% elapsed)

  Alloc rate    921,369,531 bytes per MUT second

  Productivity  32.6% of total user, 32.2% of total elapsed

I thought the problem was that it ConstTableremained, so I tried to force it clson line # 94. But this only worsens the memory consumption and runtime:

  34,300,700,520 bytes allocated in the heap
  23,579,794,624 bytes copied during GC
     487,798,904 bytes maximum residency (423 sample(s))
      36,312,104 bytes maximum slop
             554 MB total memory in use (10 MB lost due to fragmentation)

  Generation 0: 64983 collections,     0 parallel, 71.19s, 71.48s elapsed
  Generation 1:   423 collections,     0 parallel, 344.74s, 353.01s elapsed

  INIT  time    0.01s  (  0.00s elapsed)
  MUT   time   40.60s  ( 42.38s elapsed)
  GC    time  415.93s  (424.49s elapsed)
  RP    time    0.00s  (  0.00s elapsed)
  PROF  time   56.53s  ( 57.71s elapsed)
  EXIT  time    0.00s  (  0.00s elapsed)
  Total time  513.07s  (524.58s elapsed)

  %GC time      81.1%  (80.9% elapsed)

  Alloc rate    844,636,801 bytes per MUT second

  Productivity   7.9% of total user, 7.7% of total elapsed

, , , (cls)?

Answer 1

2: , :

stream <- BL.pack <$> fileContents [] classfile

. pack . , pack ByteString.

, , , .

, , .

, cls , ? 94

cls `seq` return cls

deepseq seq, , seq .

, , mapM_ mapM. , ( ) , , , . :

main = do 
  withArchive [CheckConsFlag] jarPath $ do
    classfiles <- filter isClassfile <$> fileNames []
    forM_ classfiles $ \classfile -> do 
      stream <- BL.pack <$> fileContents [] classfile
      let cls = runGet getClass stream
      lift $ print cls

print , forM_ . cls , , GC'd forM_.

, .

: , iteratees .

Answer 2

cls 94 . , , . paste , . 40 220 .

, cls, rnf cls. . :

rnf cls `seq` return cls

Control.Exception.evaluate: $rnf cls return cls

Answer 3

.

, .jar - , .jar , 50 .

, , , libzip libzip?