代码(python pycurl模块用法):
#!/usr/bin/env python
import pycurl
c = pycurl.Curl()
c.setopt(pycurl.URL, ‘http://feeds.feedburner.com/solidot’)
import StringIO
b = StringIO.StringIO()
c.setopt(pycurl.WRITEFUNCTION, b.write)
c.setopt(pycurl.FOLLOWLOCATION, 1)
c.setopt(pycurl.MAXREDIRS, 5)
#c.setopt(pycurl.PROXY, ‘http://11.11.11.11:8080′)
#c.setopt(pycurl.PROXYUSERPWD, ‘aaa:aaa’)
c.perform()
print b.getvalue()
以上代码会抓取某网站的(Solidot)数据,有代理服务器的情况,需要修改下注释的两行。