How to compile a program so that it can use> 4 GB of memory on 32-bit Linux?

Question

All code is written in C, C ++ and Fortran. Is it possible to use more than 4 GB of memory. Now it always crashes when it reaches 3 GB of memory.

If possible, how to configure compilation options (or configure flags)?

We can use gcc, g ++, ... or Intel compilers

our OS: Fedora 12 x32

cat /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority bogomips : 5319.72 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual 

Answer 1

Unfortunately, you have reached the limit, which

 R> 2^32 [1] 4294967296 

which is your limit of 4 gb. Thus, you simply cannot index beyond the limits of one application, no matter what the OS.

This is one of the reasons why many of us have switched to 64-bit versions of our OS. Linux has supported this since the late 1990s. Just switch to FC (or Ubuntu or ...) at 64 bit.

One possible alternative is to install more RAM (which will be processed by Linux) and segment your task into multiple instances of the application, effectively executing it in parallel. But it may not be worth it ...

Answer 2

I believe that if you put the file in the tmpfs (or hugetlbfs) file system and your program card has small (1 - 2 GB) parts at a time, you can work with more than 4 GB of data at the same time.

Card operations are not so fast, so you can get a performance hit if you accidentally go through your memory too.

Answer 3

You need to break your working set into pieces <3 GB, and process each piece in a separate process. Connect processes through pipes or sockets.

This is very similar to developing it in a network / cluster application, which might not be a bad idea if you want scalability.

Answer 4

32-bit integers cannot be considered higher than 4 GB:

http://www.linuxquestions.org/questions/linux-general-1/32-bit-os-and-4gb-memory-limit-707762/