I ran into the following problem: I am trying to implement a supply chain simulator. This will lead to a large number of EPCIS events (events that occur in RFID readers). Then these events should be written to the csv file in order to load them into any database and run analytical algorithms on them.
The simulator is implemented using python and works great. What I'm trying to do now is to buffer the recording of events to a file in order to reduce the time required to access the disk. After looking through the python documentation, I stumbled upon an io.BufferedWriter, which sounds exactly like what I was looking for. Anyway, I can't get it to work.
Here is what I have done so far. I implemented my CsvWriter class, which inherits RawIOBase and controls the file descriptor. When it is created, it will create a BufferedWriter , passing in itself, as the raw parameter (maybe this is a problem already?)
class CsvWriter(AbstractWriter): def __init__(self, filename): self.filename = filename self.file = self.openFile() self.buffer = BufferedWriter(self, settings.WRITE_THRESHOLD)
When I know, I want to write something, I call write_buffered buffered, which looks like this:
def write_buffered(self, data_dict): self.buffer.write(b';'.join(map(str, data_dict.values())) + '\n')
The actual write methods that (as I understand it) should be implemented in the CsvWriter class are as follows:
def write(self, data): if self.file.closed: self.file = self.openFile() return self.file.write(data)
The problem is that when I try to run the simulator, I get the following error:
IOError: raw write() returned invalid length -1 (should have been between 0 and 78)
Do any of you have a clue for me how to fix this?