Python queries - How to use system ca certificates (debian / ubuntu)?

Question

I installed the self-signed root ca cert in debian /usr/share/ca-certificates/local and installed them using sudo dpkg-reconfigure ca-certificates . Currently true | gnutls-cli mysite.local true | gnutls-cli mysite.local happy and true | openssl s_client -connect mysite.local:443 true | openssl s_client -connect mysite.local:443 happy, but the python2 and python3 request module insists that it is not happy with the certificate.

python2:

 Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 70, in get return request('get', url, params=params, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 56, in request return session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 488, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 609, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 497, in send raise SSLError(e, request=request) requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",) 

python3

 Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/bin/python3.5/site-packages/requests/api.py", line 70, in get return request('get', url, params=params, **kwargs) File "/usr/local/bin/python3.5/site-packages/requests/api.py", line 56, in request return session.request(method=method, url=url, **kwargs) File "/usr/local/bin/python3.5/site-packages/requests/sessions.py", line 488, in request resp = self.send(prep, **send_kwargs) File "/usr/local/bin/python3.5/site-packages/requests/sessions.py", line 609, in send r = adapter.send(request, **kwargs) File "/usr/local/bin/python3.5/site-packages/requests/adapters.py", line 497, in send raise SSLError(e, request=request) requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",) 

Why does python ignore the system ca-certificate package and how to integrate it?

Answer 1

From fooobar.com/questions/97505 / ...

To make python requests use the system ca-certificate package, you need to say to use it in your built-in package

 export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt 

---

Queries insert their bundles here for reference:

 /usr/local/lib/python2.7/site-packages/requests/cacert.pem /usr/lib/python3/dist-packages/requests/cacert.pem 

Answer 2

I struggled with this for a week or so recently. I finally found a way to verify a self-signed or personally signed certificate in Python. You need to create your own certificate package file. There is no need to renew unknown certificate packages each time the library is updated or something is added to the system certificate store.

Start by running the openssl command that you ran earlier, but add -showcerts. openssl s_client -connect mysite.local:443 -showcerts This will give you a long output, and at the top you will see the entire certificate chain. Typically, these are three certificates, a website certificate, an intermediate certificate, and a root certificate in that order. We need to put only the root and intermediate certificates in the next file in reverse order.

Copy the last certificate, the root certificate, into a new text file. Take only what is in between, including:

 -----BEGIN CERTIFICATE----- ... -----END CERTIFICATE----- 

Copy the middle certificate (aka intermediate certificate) into a new text file under the root certificate. Again, take the strings of the Start and End certificate and everything in between.

Save this text file to the directory where your Python script is located. I recommend calling it CertBundle.pem . (If you give it a different name or place it somewhere else in the structure of your folder, make sure that the verification line reflects this.) Update your script to refer to the new certificate package:

 response = requests.post("https://www.example.com/", headers=headerContents, json=bodyContents, verify="CertBundle.pem") 

And that is it. If you have only a root or only an intermediate certificate, then Python will not be able to verify the entire certificate chain. But if you include both certificates in the certificate set you created, then Python can verify that the intermediate was signed by the root, and then, when accessing the website, it can verify that the website’s certificate was signed with the intermediate certificate.,

edit: fixed file extension for a set of certificates. Also fixed a couple of grammatical errors.

Answer 3

Certificate tuple transfer should work:

 certificate_path = os.path.join(CERT_PATH, 'cacert.pem') certificate_key_path = os.path.join(CERT_PATH, 'cacert.key') re = requests.get(next_url, cert=(certificate_path, certificate_key_path)) 

I use this approach;)