Work with large arrays - OutOfRam

I have an algorithm where I create two two-dimensional arrays, for example:

  TYPE
   TPtrMatrixLine = array of byte;
   TCurMatrixLine = array of integer;
   TPtrMatrix     = array of TPtrMatrixLine;                                    
   TCurMatrix     = array of TCurMatrixLine;  


  function x
  var  
     PtrsMX: TPtrMatrix;
     CurMx : TCurMatrix;         
  begin 
   { Try to allocate RAM }
   SetLength(PtrsMX, RowNr+1, ColNr+1);                              
   SetLength(CurMx , RowNr+1, ColNr+1);
   for all rows do 
    for all cols do 
     FillMatrixWithData; <------- CPU intensive task. It could take up to 10-20 min
  end;

Two matrices always have the same dimension. Typically, there are only 2,000 rows and 2,000 columns in a matrix, but sometimes it can reach 25000x6000, so for both matrices I need something like 146.5 + 586.2 = 732.8 MB of RAM. The problem is that the two blocks must be adjacent, so in most cases, even if 500-600 MB of free RAM does not seem to be much on a modern computer, I run out of RAM.

The algorithm fills the cells of the array with data based on the neighbors of this cell. Operations are simply additions and subtractions.

TCurMatrixLine - , , . , , Word . SmallInt ( SmallInt, Word). , - , , / . , , .

? [ Delphi 7]


- , . (), , TList. . , . , .

TCurMatrixLine = array of integer;                                   
TCurMatrix     = array of TCurMatrixLine; 

, TCurMatrix= array of array of integer (- , ). , .

+3
3

2- .

, exe LARGE_ADDRESS_AWARE, {$SetPEFlags IMAGE_FILE_LARGE_ADDRESS_AWARE} .dpr . , 64- Windows 2 4 .

, Delphi 7 (, D7), FastMM, Borland . $SetPEFlags , exe EDITBIN.

, , - . , .

, , , , , .

+4

CurMx , . 1 .

+2

, ?
... , ?

Using heap can actually increase speed and reduce memory usage, since you can avoid copying the entire array from one memory segment to another memory segment. (For example, if your is FillMatrixWithDatadeclared using the non-const open array parameter).

+1
source

Source: https://habr.com/ru/post/1790715/


All Articles