How to use Avro to process a stream that I cannot find?

Question

I am using Avro 1.4.0 to read some data from S3 using the avro Python bundles and the S3 boto library. When I open the avro.datafile.DataFileReader file in the file, like the objects returned by boto, it immediately fails when it tries to search (). While I'm working on this, reading S3 objects into temporary files.

I would like to be able to pass through any python object that supports read (). Can anyone give some advice?

Answer 1

I do not really understand this, and this may not be the answer. I got the impression that

diter = datafile.DataFileReader(..) 

,

for data in diter:
    ....

, .

:

, datafile.DataFileReader , .

avro.io.BinaryDecoder, .

class BinaryDecoder(object):
    """Read leaf values."""
    def __init__(self, reader):
        """
    reader is a Python object on which we can call read, seek, and tell.
    """
    self._reader = reader

, , - , , boto S3 .