Multithreaded application in C ++

Question

I am working on a multi-threaded application programmed in C ++. I use some temporary files to transfer data between my streams. One stream writes data to be processed into files in the directory. Another thread checks the directory for working files and reads the files and processes them further, and then deletes these files. I have to use these files because if my application is killed, when I have to save data that has not been processed yet.

But I hate using multiple files. I just want to use one file. One stream continuously writes to a file and another stream, reading data and deleting data that has been read. As the ship is filled up and down, I can get and delete data from the ship. How to do this efficiently in C ++, first there is a way ...?

Answer 1

As suggested in the comments on your questions, using a database such as SQLite can be a very good solution. However, if you insist on using the file, this is of course possible.

I did it myself once - created a permanent queue on disk using a file.

Here are some guidelines for achieving this:

• The file should contain a header that indicates the next raw record (record) and the next available recording location.
• If the entries are of variable length, each entry should contain a heading that indicates the length of the entry.
• You can add a flag to each record that indicates whether the record has been processed.
• file locking can be used to ensure that no one reads part of the file that is written to
• Use low I / O - do not use buffered streams of any type, use direct write semantics

And here are the schemes for reading and writing (perhaps with some small logical errors, but you should be able to do this from there):

READER

• Lock the file header and read it and open it.
• Go to the last recording position
• Read the post title and post
• Record the title of the record with the processing flag enabled.
• If you are not at the end of the file Lock the header and write down a new location for the next raw record, add another label to indicate that there are no more records to process
• Make sure the next record to record indicates the correct location.

You may also want the reader to compress the file for you for a while:

• Lock entire file
• Copy all raw records to the beginning of the file (you might want to keep some logic in order not to overwrite your raw records - perhaps compactly, only if the processed space is larger than the raw space)
• Update title
• Unlock file

WRITER

• Lock the file header and see where the next record will be recorded, then unlock it
• Lock the file from the place to be written to the recording length
• Record and unlock
• Lock the title, if the raw record mark indicates that there are no records to process, to indicate a new record, unlock the title

Hope this brings you to the recording track

Answer 2

The win32Api CreateFileMapping() function allows processes to exchange data, and several processes can use memory mapped files stored in the system’s paging file.

Some good links:

http://msdn.microsoft.com/en-us/library/aa366551(VS.85).aspx

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366551(v=vs.85).aspx

http://www.codeproject.com/Articles/34073/Inter-Process-Communication-IPC-Introduction-and-S

http://www.codeproject.com/Articles/19531/A-Wrapped-Class-of-Share-Memory

http://www.bogotobogo.com/cplusplus/multithreaded2C.php

Answer 3

you can write data that was a process line per line and a divider for each line, indicate whether this record is being processed or not