How to download under proxy with python

Proxies are headache for many people. They restrict you from many things you love to do. In this short article I will tell you how to write scripts to work under proxy in python. Its really simple and you don’t need to know anything exceptional except a bit of python programming language. Lets see how to download under proxy with python

How to download under proxy with python
Simple diagram to show what is proxy. Taken from https://www.drupal.org/files/project-images/proxy.png

How to download under proxy with python

For download, in python people generally use two libraries. First one is requests and the other one is urllib.

We will discuss the script in both the libraries.

urllib

import urllib
proxies = {'http': 'http://YOUR_USERNAME:YOURPASSWORD@192.168.1.107:3128',
           'https': 'https://YOUR_USRNAME:YOURPASSOWRD@192.168.1.107:3128'}
res=urllib.urlopen("http://www.py4inf.com/cover.jpg",proxies=proxies)
con=res.read()
outf=open("imgname.jpg",'wb')
outf.write(con)
outf.close()

What we did here:

We import the library, set the proxies in dictionary, open the url with proxies as parameters. Read the content of the url then write it down in a file. Here we saved an image and it is done in no time. Really simple isn’t it.

Requests

import requests
proxies = {'http': 'http://YOUR_USERNAME:YOURPASSWORD@192.168.1.107:3128',
           'https': 'https://YOUR_USRNAME:YOURPASSOWRD@192.168.1.107:3128'}
res=requests.get("http://www.py4inf.com/cover.jpg",proxies=proxies)
con=res.content
outf=open("imgname.jpg",'wb')
outf.write(con)
outf.close()

Yes exactly the same there is a very little change which anyone can understand so I don’t need to explain it.

So this is the end of article How to download under proxy with python.


Gaurav Yadav

Gaurav is cloud infrastructure engineer and a full stack web developer and blogger. Sportsperson by heart and loves football. Scale is something he loves to work for and always keen to learn new tech. Experienced with CI/CD, distributed cloud infrastructure, build systems and lot of SRE Stuff.

3 COMMENTS
  • خرید پروکسی
    Reply

    Thanks for the marvelous posting! Valuable information .I will make sure to bookmark your blog and will often come back at some point. I want to encourage you to ultimately continue your great posts, I bookmarked it.!

  • Nono-London
    Reply

    Is there a way to use multiple proxies for a same file? So effectively splitting the file per numbe of proxies. I am trying to download a big file and using just 1 proxy it takes ages.. Thanks for the help

    1. Gaurav Yadav
      Reply

      You can try splitting the file in multipart. And use the same proxy multiple times to get in parallel and download. Since they can be seperate instance of python program you can use different proxy also

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.