python pycurl模块网络编程入门例子

发布时间:2020-12-06编辑:脚本学堂
python pycurl模块实现网络编程的二个例子,pycurl模块自动处理cookie,pycurl模块实现post方法等。

python网络编程中,使用python urllib模块时经常会死掉,以前debug过,由于没有设置 timing out导致超时而死掉。

本节内容:
1、python pycurl模块用法
2、pycurl模块自动处理cookie
3、pycurl模块实现post方法

可以使用curl的扩展库pycurl模块,curl是非常强劲的一个工具。

链接:http://pycurl.sourceforge.net/ 下载最新的PycURL。

例1,python pycurl模块用法:
 

复制代码 代码示例:

#!/usr/bin/env python

import pycurl
import StringIO
 
url = "http://www.jb200.com/"
crl = pycurl.Curl()
crl.setopt(pycurl.VERBOSE,1)
crl.setopt(pycurl.FOLLOWLOCATION, 1)
crl.setopt(pycurl.MAXREDIRS, 5)
crl.fp = StringIO.StringIO()
crl.setopt(pycurl.URL, url)
crl.setopt(crl.WRITEFUNCTION, crl.fp.write)
crl.perform()
print crl.fp.getvalue()

例2,pycurl模块自动处理cookie。
 

复制代码 代码示例:

#!/usr/bin/env python

import pycurl
import StringIO
 
url = "http://www.plcxue.com/"
crl = pycurl.Curl()
crl.setopt(pycurl.VERBOSE,1)
crl.setopt(pycurl.FOLLOWLOCATION, 1)
crl.setopt(pycurl.MAXREDIRS, 5)
crl.fp = StringIO.StringIO()
crl.setopt(pycurl.URL, url)
crl.setopt(crl.WRITEFUNCTION, crl.fp.write)
 
# Option -b/--cookie <name=string/file> Cookie string or file to read cookies from
# Note: must be a string, not a file object.
crl.setopt(pycurl.COOKIEFILE, "cookie_file_name")
 
# Option -c/--cookie-jar <file> Write cookies to this file after operation
# Note: must be a string, not a file object.
crl.setopt(pycurl.COOKIEJAR, "cookie_file_name")
 
crl.perform()
print crl.fp.getvalue()

例3,pycurl模块实现post方法。
 

复制代码 代码示例:

#!/usr/bin/env python

import pycurl
import StringIO
import urllib
 
url = "http://www.plcxue.com/"
post_data_dic = {"name":"value"}
crl = pycurl.Curl()
crl.setopt(pycurl.VERBOSE,1)
crl.setopt(pycurl.FOLLOWLOCATION, 1)
crl.setopt(pycurl.MAXREDIRS, 5)
#crl.setopt(pycurl.AUTOREFERER,1)
 
crl.setopt(pycurl.CONNECTTIMEOUT, 60)
crl.setopt(pycurl.TIMEOUT, 300)
#crl.setopt(pycurl.PROXY,proxy)
crl.setopt(pycurl.HTTPPROXYTUNNEL,1)
#crl.setopt(pycurl.NOSIGNAL, 1)
crl.fp = StringIO.StringIO()
crl.setopt(pycurl.USERAGENT, "dhgu hoho")
 
# Option -d/--data <data>   HTTP POST data
crl.setopt(crl.POSTFIELDS,  urllib.urlencode(post_data_dic))
 
crl.setopt(pycurl.URL, url)
crl.setopt(crl.WRITEFUNCTION, crl.fp.write)
crl.perform()
 
print crl.fp.getvalue()

附,urllib 超时设置
 

import socket
socket.setdefaulttimeout(5.0)