How to write data to a> 4 GB file with C ++?

Question

I am trying to write a large file, but ran into a problem.

I use a long time to find a place to record, but I can not write a file larger than 4.2 GB. What did I forget?

More: I open the 4Gb file:

ifstream ifs(db_name.c_str(), ios_base::binary); if (!ifs) throw Bad_archive(); ifs.read(as_bytes(seg_size), sizeof(int)); ifs.read(as_bytes(last_idx), sizeof(int)); ifs.read(as_bytes(free_segs), sizeof(int)); if (free_segs > 0) { long long seek_g = 3 * sizeof(int) + (long long)last_idx * seg_size; ifs.seekg(seek_g, ios_base::beg); for (int i = 0; i < free_segs; i++) { int tmp = 0; ifs.read(as_bytes(tmp), sizeof(int)); free_spaces.push_back(tmp); } } ifs.close(); 

After that, I read the 400Mb file that I want to add to db. And write (here is a short code):

 // write object ofstream ofs(db_name.c_str(), ios_base::binary | ios_base::in | ios_base::ate); for (int i = 0; ; i++) { // set stream position long long write_position = sizeof(int) * 3; ... write_position += (long long) seg_size * last_idx; ofs.seekp(write_position, ios::beg); ... // write sizeof object if (i == 0) ofs.write(as_bytes(object_size), sizeof(object_size)); else { int null = 0; ofs.write(as_bytes(null), sizeof(null)); } // write data for (int j = 0; j < (seg_size - sizeof(int) * 2); j++) { ofs.write(as_bytes(obj_content[j + i * (seg_size - sizeof(int) * 2)]), sizeof(char)); } if (write_new_seg) last_idx++; ... ofs.write(as_bytes(cont_seg), sizeof(cont_seg)); } ofs.close(); 

After that I save the db info:

 if (last_idx == 0) { ofstream ofs(db_name.c_str()); ofs.close(); } ofstream ofs(db_name.c_str(), ios_base::binary | ios_base::in | ios_base::out | ios_base::ate); long long seek_p = 0; ofs.seekp(seek_p, ios_base::beg); ofs.write(as_bytes(seg_size), sizeof(seg_size)); ofs.write(as_bytes(last_idx), sizeof(last_idx)); ofs.write(as_bytes(free_segs), sizeof(free_segs)); ofs.close(); 

But this code works:

 ofstream ofs2("huge2"); ofs2.close(); ofstream ofs("huge2", ios_base::in | ios_base::ate); long long sp = 10000000000; ofs.seekp(10000000000, ios_base::beg); ofs.write(as_bytes(sp), sizeof(sp)); ofs.close(); 

Answer 1

  fseeko64 (Filename, 0, SEEK_END); long size = ftello64 (Filename); fclose (Filename); 

You cannot use fseek to access large files; instead, switch to fseeko64 ();

Answer 2

There is a function here called _lseeki64 () on Win32 and lseek64 () on * nix. However, the iostream library does not directly bind them, you need to somehow restore the file descriptor and do something about buffering.

Answer 3

Many file systems do not support files larger than 4 GB. In addition, you should keep in mind that your int type may have a range that ends in 2^32 , i.e. 4.29497e + 09.

Answer 4

Here is the summary for Linux:

http://www.suse.de/~aj/linux_lfs.html

And a more specific detail from RedHat aimed at RHEL, these problems are typically for 32-bit applications that access 64-bit files.

http://people.redhat.com/berrange/notes/largefile.html

Wikipedia actually has artile for supporting large files , but is not very informative.