Limited image processing on the server

Question

We have a server application (Java EE), it will perform some tasks on image processing based on user request. Such as converting the image format (for example, TIFF to JPEG), converting the color of the image (for example, RGB to gray in BW), resizing (resizing) the image. Some customers from the printing industry use a very large image, for example, 2000 dpi, 6 * 8 inches, 4 color components that occupy a memory of 6 * 2000 * 8 * 2000 * 4 = 768 MB. The server cannot store this large image in memory, so we decided to make the process lane lane. The problem is that it still does not work, because there can be many clients at the same time. Do you have an idea on how to implement image processing with a limited amount of memory? Or, do you know if there is any paper / article, we can provide us with solutions.

Thanks,

Answer 1

I would suggest considering moving the image processing part to a separate JVM that you are communicating with from the main application using RMI or similar.

This allows you to configure JVM processing separately from the main JVM and, possibly, even create a distributed system, if necessary, on several machines. It can also allow you to manage conversions so that only a few individual images are allowed at the same time.

Are there any restrictions that would refrain from this?

As a last resort, I would suggest moving the actual image conversions to native programs, such as ImageMagick on Linux, which is then called by your program, performs the conversion, and allows your JVM to return the result to the user. This will probably be faster than cpu-wise and require less memory.

Answer 2

Well, the first thing you should know is that the amount of memory you can allocate does not depend on how much RAM you have. The OS manages virtual memory, and any allocation is performed in virtual memory. Then the pieces are unloaded into the RAM / Disk OS. You have no control over this. You might think that you allocate 5 MB in RAM, but it depends on the OS to support it in RAM or to get it from your disk into your program when you need it. On Windows 32 bit, you have 2 GB in user space to work with (the rest is used by the kernel). In 64 bit OS, this number is much larger.

However, just because you have 2 GB of user space that you can allocate (in a 32-bit OS or more in the 64-bit version), this does not mean that you can always allocate a large piece, because for memory allocation contiguous blocks of memory are required. Thus, memory fragmentation can be a problem.

Third, the JVM limits the amount of memory that you can allocate. Therefore, you need to increase them using the -Xms -Xmx parameters that limit the heap size.

Other than these comments, I really have no solution for you.

Answer 3

You can use tiled image processing methods. IPP, for example, supports tiled image processing.