Java is the right way to scale an image

Question

Context

I am looking at some legacy Java code that is used in a server application to scale images. Until recently, it was primarily used with input images that had resolutions of 1024x768 or lower, and in this context, it seems to have worked well (or, at least, quite well).

Now, however, many image manipulations have resolutions from 2592x1936 (8MP) to 6000x4000 (24MP). This led to the appearance of frequent OutOfMemoryError messages on the server, which I believe were caused by image manipulation code.

Testcase

I put together a simple test application that calls the image processing code in the same way the server does (with one caveat, the test file is single-threaded, and the server environment is clearly not):

 public static void main(String[] args) throws Exception { File file = new File("test_fullres.png"); //8MP test image, PNG format for (int index = 0; index < 1000; index++) { File output = new File("target/images/img_" + index + ".jpg"); verifyImageIsValidAndScale(file, output, 1024, 768, true, 80); if (index % 10 == 0) { String memStats = "Memstats after " + index + " iterations:"; memStats += "\tFree Memory: " + (Runtime.getRuntime().freeMemory() / 1024.0 / 1024.0) + " MBytes\n"; memStats += "\tTotal Memory: " + (Runtime.getRuntime().totalMemory() / 1024.0 / 1024.0) + " MBytes\n"; memStats += "\tMax Memory: " + (Runtime.getRuntime().maxMemory() / 1024.0 / 1024.0) + " MBytes\n"; System.out.println(memStats); } } } 

So, it’s quite simple, just loops 1000 times and scales the input image at each iteration, displaying some memory statistics every 10 iterations. I also set the following JVM flags:

-verbose:gc -XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:GCPauseIntervalMillis=3000

For the control, I used 1024x768 JPEG as the input. For the actual test, I am using an 8 megapixel PNG image.

code

The rest of the test code:

 public static boolean verifyImageIsValidAndScale(File in, File out, int width, int height, boolean aspectFit, int quality){ boolean valid = verifyImageIsValid(in); aspectFitImage(in, out, width, height, quality); return valid; } public static boolean verifyImageIsValid(File file){ InputStream stream = null; try { String path = file.getAbsolutePath(); stream = new FileInputStream(path); byte[] buffer = new byte[11]; int numRead = stream.read(buffer); if(numRead > 3 && buffer[1] == 'P' && buffer[2] == 'N' && buffer[3] == 'G') { stream.close(); buffer = null; LOG.debug("Valid PNG"); return ImageIO.read(new File(path)) != null; //PNG } else if (numRead > 10 && ((buffer[6] == 'J' && buffer[7] == 'F' && buffer[8] == 'I' && buffer[9] == 'F' && buffer[10] == 0) || (buffer[6] == 'E' && buffer[7] == 'x' && buffer[8] == 'i' && buffer[9] == 'f' && buffer[10] == 0))){ stream.close(); buffer = null; LOG.debug("Valid JPEG"); return ImageIO.read(new File(path)) != null; //JPEG } buffer = null; } catch (Exception e) { return false; } finally { try { stream.close(); } catch (Exception ignored) {} } LOG.debug("Invalid Image"); return false; } public static Image aspectFitImage(File uploadedFile, File outputFile, int maxWidth, int maxHeight, int quality) { if (quality < 0) { quality = 0; } Image scaledImage = null; try { BufferedImage image = ImageIO.read(uploadedFile); double scaleX = image.getWidth() > maxWidth ? maxWidth / (double)image.getWidth() : 1.0; double scaleY = image.getHeight() > maxHeight ? maxHeight / (double)image.getHeight() : 1.0; double useScale = scaleX < scaleY ? scaleX : scaleY; scaledImage = image.getScaledInstance((int)(image.getWidth() * useScale), (int)(image.getHeight() * useScale), Image.SCALE_SMOOTH); //XXX: getting bizarre results where overwriting a pre-existing output file doesn't properly overwrite the existing file; explicitly deleting the output file when it already exists seems to solve the issue if (outputFile.exists()) { outputFile.delete(); } ImageOutputStream imageOut = ImageIO.createImageOutputStream(outputFile); ImageWriter writer = ImageIO.getImageWritersByFormatName("jpeg").next(); ImageWriteParam options = writer.getDefaultWriteParam(); options.setCompressionMode(ImageWriteParam.MODE_EXPLICIT); options.setCompressionQuality(quality / 100.0f); PixelGrabber pg = new PixelGrabber(scaledImage, 0, 0, -1, -1, true); pg.grabPixels(); DataBuffer buffer = new DataBufferInt((int[]) pg.getPixels(), pg.getWidth() * pg.getHeight()); WritableRaster raster = Raster.createPackedRaster(buffer, pg.getWidth(), pg.getHeight(), pg.getWidth(), RGB_MASKS, null); BufferedImage bi = new BufferedImage(RGB_OPAQUE, raster, false, null); writer.setOutput(imageOut); writer.write(null, new IIOImage(bi, null, null), options); imageOut.close(); writer.dispose(); } catch (Exception e) { uploadedFile.delete(); return null; } return scaledImage; } 

In addition, I found that if I start commenting on the code until there is only one non-trivial line left:

BufferedImage image = ImageIO.read(uploadedFile);

... the results are still basically the same in terms of the total amounts of memory.

results

Below, if the final result of each run, both from GC logs and from the built-in memory logging statements:

 #Source @ 1024x768 (control) [GC pause (young) 1333M->874M(1612M), 0.0016051 secs] Memstats after 990 iterations: Free Memory: 505.5521469116211 MBytes Total Memory: 1612.0 MBytes Max Memory: 6144.0 MBytes #Source @ 2592x1936 (test): [GC pause (young) 2161M->1610M(2632M), 0.0025038 secs] Memstats after 990 iterations: Free Memory: 425.37239837646484 MBytes Total Memory: 2632.0 MBytes Max Memory: 6144.0 MBytes 

Both seem taller than they should be. And what is not fixed here is that during the test run, the garbage collector often returned up 1 GB of memory at a time.

It may also be worth noting that the value of "full memory" gradually increases and grows as the mileage progresses, as if the garbage collector cannot recover all the memory that is being whipped. Perhaps this means a memory leak?

On average, the garbage collector runs every 2-3 iterations. And in any case, the amount of outflow seems rather unreasonable, given that I scale the images one at a time in one thread. This should not be what burns through gigabytes and gigabytes of memory. Even if behind the scenes Java decompresses every image into a 32-bit bitmap, I would not expect that memory usage will grow as fast as it was observed.

Questions

• What is the correct / most memory efficient way to scale high resolution images in Java?
• Is there an obvious mistake in the legacy code, or should I just abandon it and rewrite it?
• Would it be better to get rid of the external process to perform image manipulation (for example, if Java, for some reason, simply does not fit this use case)?