How to store a large amount of text data in memory?

Question

I am working on a parser c and wondering how an expert manages a large amount of text / line (> 100mb) for storage in memory? content is expected to be available all the time at a fast pace. bg: redhat / gcc / libc

one char array would be out of bounds causing a segmentation error ... any idea or experience is welcome to share / discuss ...

Answer 1

mmap - The best way to process a large amount of data that is stored in a file if you want to get random access to this data.

mmap , , . , . , , , , , ( ) .

:

#include <sys/types.h>
#include <sys/stat.h>
#include <sys/mman.h> /* the header where mmap is defined */
#include <fcntl.h>

int file;
char *contents;
struct stat statbuf;
off_t len;

file = open("path/to/file", O_RDONLY);
if (file < 0)
  exit(1); /* or otherwise handle the error */

if (fstat(file, &statbuf) < 0)
  exit(1);

len = statbuf.st_size;

contents = mmap(0, len, PROT_READ, MAP_SHARED, file, 0);
if (contents == MAP_FAILED)
  exit(1);

// Now you can use contents as a pointer to the contents of the file

// When you're done, unmap and close the file.

munmap(contents, len);
close(file);

Answer 2

mmap (2) .

Answer 3

" char , " - , . , . , 2-3 32- 64- .

char, , , , - .

? c-? : , .

Answer 4

C-, ( , ), . . (#includes), 100 - , ?

Answer 5

• , .
• ( ), .

, .

, / , :

• .

Answer 6

<100 char , , , . /, , (google " " )

, , static char. , malloc(). ( , a - , C99, . OTOH C malloc() .)

Answer 7

, , ( ). 50%.

, . , , , .

Answer 8

, pbm, .

int a = 10000000; char content2[a]; content2[0] = 'a';

, ( xml) , , ,