写 Python 爬虫的助手
文章目录
写爬虫变成了这样:
from gspider.base import BaseSpider
from gspider.fields import PQField
class PythonDocSpider(BaseSpider):
title = PQField('h1') # select `h1` element from the page, you can use other css selectors too
content = PQField('p')
spider = PythonDocSpider(
'https://docs.python.org/3/library/base64.html'
)
>>> print(spider.title)
>>> 19.6. base64 — Base16, Base32, Base64, Base85 Data Encodings ¶
安装:
git clone https://github.com/ericls/gspider
cd gspider
python setup.py install
或者直接 clone 进自己的脚本目录,再解决requests和pyquery两个依赖
更多信息:
Project Name: Gspider. (Pretty random name, isn't it?)
GitHub Repo: __https://github.com/ericls/gspider__
评论
登录后参与技术讨论,分享你的经验与见解。
登录 / 注册暂无评论