How to build a TarFile object in memory from a byte buffer in Python 3?

Question

Is it possible to create a TarFile object in memory using a buffer containing tar data without having to write the TarFile to disk and reopen it? We get bytes sent over the socket.

Something like that:

import tarfile byte_array = client.read_bytes() tar = tarfile.open(byte_array) # how to do this? # use "tar" as a regular TarFile object for member in tar.getmembers(): f = tar.extractfile(member) print(f) 

Note. One of the reasons for this is that we ultimately want to be able to do this with multiple threads at the same time, so the use of a temporary file can be overridden if two threads try to do this at the same time.

Thanks for any help!

Answer 1

BytesIO () from the IO module does exactly what you need.

 import tarfile, io byte_array = client.read_bytes() file_like_object = io.BytesIO(byte_array) tar = tarfile.open(fileobj=file_like_object) # use "tar" as a regular TarFile object for member in tar.getmembers(): f = tar.extractfile(member) print(f) 

Answer 2

Of course, something like this:

 import io io_bytes = io.BytesIO(byte_array) tar = tarfile.open(fileobj=io_bytes, mode='r') 

(Set mode to match the format of your tar file, for example, `mode = 'r: gz', etc.)