All geek questions in one place

C and closing files in Python

I read that a file opened in this way closes automatically when exiting a block:

with open("x.txt") as f: data = f.read() do something with data

but when opening from the internet I need this:

 from contextlib import closing from urllib.request import urlopen with closing(urlopen('http://www.python.org')) as page: for line in page: print(line)

why and what is the difference? (I am using Python3)

+6
source share
2 answers

The details are a little technical, so let's start with a simple version:

Some types know how to use them in a with statement. An example of this type are file objects, for example, what you return from open . As it turned out, the objects returned from urllib.request.urlopen are also an example of this type, so your second example can be written in the same way as the first.

But some types do not know how to use them in a with statement. The closing function is intended to wrap such types, if they have a close method, it is called by their close method when exiting the with statement.

Of course, some types do not know how to use the with statement, nor can they be used with closing because their cleaning method is not called close (or because it clears them more difficult than just closing them). In this case, you need to write your own context manager. But even this is usually not that difficult.

In technical terms:

Operator

A with requires a context manager , an object with __enter__ and __exit__ methods. It will call the __enter__ method and give you the value returned by this method in the as clause, and then call the __exit__ method at the end of the with statement.

File objects inherit from io.IOBase , which is the context manager, the __enter__ method returns itself and whose __exit__ calls self.close() .

The object returned by urlopen (assuming an http or https URL) is an HTTPResponse , which, as the docs say, "can be used with the with statement."

closing function:

Return the context manager that closes the object after the block completes. This is basically equivalent to:

 @contextmanager def closing(thing): try: yield thing finally: thing.close()

This is not always 100% clear in documents whose types are context managers and which are not. Moreover, since the creation of the main drive with 3.1, everything has been done that could be a context manager in one (and for that matter, do everything that is basically file-like in the actual IOBase , if that makes sense), but this is still not 100% compared to 3.4.

You can always just try and see. If you get AttributeError: __exit__ , then the object cannot be used as a context manager. If you think it should be, report an error indicating a change. If you do not receive this error, but the documents do not mention that it is legal, indicate the error that suggests updating the documents.

+6
source

No. urlopen('http://www.python.org') also returns a context manager:

 with urlopen('http://www.python.org') as page:

This is described on urllib.request.urlopen() :

For URLs, files, and URL data and requests executed by the old URLopener and FancyURLopener , this function returns a urllib.response.addinfourl object that can act as a context manager ..].

The emphasis is mine. For HTTP responses, an http.client.HTTPResponse() object is returned, which is also the context manager:

The answer is an iterable object, and can be used in operator c .

The Examples section also uses the object as a context manager:

Since python.org uses utf-8 encoding as indicated in its meta tag, we will use the same to decode the byte object.

 >>> with urllib.request.urlopen('http://www.python.org/') as f: ... print(f.read(100).decode('utf-8')) ... <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtm

The objects returned by open() are also context managers ; they implement the special methods object.__enter__() and object.__exit__() .

The documentation for contextlib.closing() uses the contextlib.closing() example, which is deprecated; in Python 2, the predecessor for urllib.request.urlopen() did not create a context manager, and you needed to use this tool to automatically close the connection to the context manager. This has been fixed with issues 5418 and 12365 , but this example has not been updated. I created question 22755 asking for another example.

+7
source

Source: https://habr.com/ru/post/977383/

More articles:


All Articles