How do you login to the internet in a python script?

Question

I am trying to connect to the internet with my python script but the proxy requires a username and password. I have both of these, but I do not know how to sign into the proxy during the code.

The response to me trying to connect without a password is this;

import pip
pip.main(['install', 'matplotlib.pyplot'])
Collecting matplotlib.pyplot
  Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required',))': /simple/matplotlib-pyplot/
  Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required',))': /simple/matplotlib-pyplot/
  Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required',))': /simple/matplotlib-pyplot/
  Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required',))': /simple/matplotlib-pyplot/
  Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required',))': /simple/matplotlib-pyplot/
  Could not find a version that satisfies the requirement matplotlib.pyplot (from versions: )
No matching distribution found for matplotlib.pyplot
Out[3]: 1

I can elaborate on anything if need be. Thanks.

Just saying, this is on a school internet system (NSW) so there may be actions that are restricted.

Followed bastelflp's instructions and got this result; Traceback (most recent call last): File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 141, in _new_conn (self.host, self.port), self.timeout, **extra_kw) File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/packages/urllib3/util/connection.py", line 83, in create_connection raise err File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) TimeoutError: [Errno 60] Operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 594, in urlopen self._prepare_proxy(conn) File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 810, in _prepare_proxy conn.connect() File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 284, in connect conn = self._new_conn() File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) requests.packages.urllib3.exceptions.NewConnectionError: <requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x118900908>: Failed to establish a new connection: [Errno 60] Operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/adapters.py", line 438, in send timeout=timeout File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py", line 649, in urlopen _stacktrace=sys.exc_info()[2]) File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/packages/urllib3/util/retry.py", line 388, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='finance.yahoo.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x118900908>: Failed to establish a new connection: [Errno 60] Operation timed out',))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/euanoflynn/anaconda/tests/Tests.py", line 22, in <module> r = requests.get("https://finance.yahoo.com", proxies=proxies) File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/api.py", line 72, in get return request('get', url, params=params, **kwargs) File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/api.py", line 58, in request return session.request(method=method, url=url, **kwargs) File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/sessions.py", line 518, in request resp = self.send(prep, **send_kwargs) File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/sessions.py", line 639, in send r = adapter.send(request, **kwargs) File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/adapters.py", line 500, in send raise ProxyError(e, request=request) requests.exceptions.ProxyError: HTTPSConnectionPool(host='finance.yahoo.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x118900908>: Failed to establish a new connection: [Errno 60] Operation timed out',)))

@EleshNorn Why are your using pip from inside a Python script and not directly from the command line? Have a look at [Using pip behind a proxy](https://stackoverflow.com/q/14149422/5276734). — bastelflp, Nov 22 '17 at 08:34
Sorry for my ignorance, but... what does this mean in basic (2 months) pythonspeak? — Elesh Norn, Nov 23 '17 at 03:21

score 0 · Answer 1 · edited Nov 11 '17 at 17:49

0

Pip install library requests. Use requests to post login event. Keep the session and cookies and you can send new request just like you are login-ed.

edited Nov 11 '17 at 17:49

bastelflp

9,362
7
32
67

answered Nov 09 '17 at 03:45

W.Perrin

4,217
32
31

I'm pretty new... how do you post the login event? Had a look at [link] (http://docs.python-requests.org/en/master/user/quickstart/) but couldn't understand – Elesh Norn Nov 09 '17 at 03:51
1

Use chrome, enter develop mode, login the website and find out params send to the website server while login. Then try to send the same data using requests module. – W.Perrin Nov 09 '17 at 04:19
params meaning parameters? – Elesh Norn Dec 06 '17 at 00:52

score 0 · Answer 2 · edited Nov 22 '17 at 02:04

0

If you just want to install Python packages, try using pip directly from the command line, i.e.

pip --proxy your_proxy:port matplotlib

with your_proxy:port beeing your proxy, e.g. proxy.example.com:8080. You can add authentication with the following syntax: user:password@your_proxy:port. If you have special characters in your user or password, you may have to escape them. More options for pip behind a proxy can be found here.

If you want to access the Internet via Python, use the Python package requests, which can also handle proxies:

import requests

proxies = {
  "http": "your_proxy:port",
  "https": "your_proxy:port",
}

r = requests.get("http://example.org", proxies=proxies)
r.status_code

You can find a more detailed explanation on requests with proxies here.

edited Nov 22 '17 at 02:04

Elesh Norn

165
1
8

answered Nov 09 '17 at 06:50

bastelflp

9,362
7
32
67

Well, this has helped; the program can overcome the proxy. The problem, though, is that it says the host is down even though I can connect to the internet just fine (I am writing this at the same time as the program says the host was down, from the same place). This is a NSW school internet, though, so some actions might be restricted. – Elesh Norn Nov 20 '17 at 02:00
I have updated the question to include the school internet factor. – Elesh Norn Nov 20 '17 at 02:00
`File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/adapters.py", line 500, in send raise ProxyError(e, request=request) requests.exceptions.ProxyError: HTTPConnectionPool(host='10.181.157.13', port=80): Max retries exceeded with url: http://finance.yahoo.com/ (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError(': Failed to establish a new connection: [Errno 64] Host is down',)))` – Elesh Norn Nov 20 '17 at 02:10
`File "/Users/euanoflynn/anaconda/lib/python3.6/site-packages/requests/adapters.py", line 500, in send raise ProxyError(e, request=request) requests.exceptions.ProxyError: HTTPSConnectionPool(host='finance.yahoo.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError(': Failed to establish a new connection: [Errno 60] Operation timed out',)))` – Elesh Norn Nov 20 '17 at 02:35

How do you login to the internet in a python script?

2 Answers2