搜索资源列表
获取代理案例
- 利用scrapy框架写的python爬虫程序，使用爬取代理的案例来讲解的。(Scrapy framework is used to program crawler procedures in Python.)
agentip.tar
- 在爬虫中，有些网站可能为了防止爬虫，会记录每个IP的访问次数，那么我们就需要访问一次换一个IP。我们可以用爬虫爬IP。声明：仅学习交流，切勿用作商业用途等(In reptiles, some websites might be able to keep track of the number of visits per IP in order to prevent crawlers, so we need to visit and change a IP. We can climb IP wit
gotoweb
- 利用python语言，实现从IP代理网站获取IP，并用此IP重复访问指定网页(Using the python language, the IP is obtained from the IP proxy site, and the specified page is repeatedly accessed with this IP)
xici_proxy
- 爬取西刺前10页(可自行修改参数total_page来管理爬取的页数)有效期大于1天的高匿代理IP,并测试其有效性,最后保存为Proxies.json文件(Unicode),使用时导入文件随机选取一个代理ip使用即可.(Crawl up to 10 pages before the Western thorn, which can modify the parameter total_page to manage the page number of climbing. The high hid
多线程爬虫
- 一个简单健壮的多线程爬虫程序，可以添加代理IP池、url池。并且已经构造好请求头和异常处理(A simple and robust crawler program, which can add agent IP pool and URL pool. And have constructed the request header and exception handling)
ProxyPool-master
- python爬虫代理的实现,即自动检测redis中存储的代理服务(python crawl proxy(agent))