AirSpider-An Easy-to-use and Fast Python Spider Framework-Liu Xunzhuo
发布:2020-12-22 14:50:11.623476
作者:Liu Xunzhuo
### 作者邮箱:mixdeers@gmail.com
### 首页:https://pypi.org/project/airspider/
### 文档:None
### 下载链接
<p align="center">
<img src="https://picreso.oss-cn-beijing.aliyuncs.com/airs.png" width="150px">
Distributed - Asynchronous - Easy-to-use
An Easy-to-use and Fast Python Spider Framework
Overview
- AirSpider is a high-performance asynchronous crawler framework for developers
- Based on Redis: task distribution, task deduplication, and distributed ☁️
Requirements
- Python 3.6
- Works on Linux, Windows, macOS
Features
- Quick to Start
- Low Coupling
- High Cohesion
- Easy Expansion
- Orderly Workflow
Installation
# For Linux && MacOS
pip3 install airspider
# For Windows
pip3 install airspider
## Documents
Introduction:Introduction to AirSpider
Tutorials: Quick to program a Spider By AirSpider
Plugins:Program extensions for AirSpider
Topics
Item:Define Target in HTML
Selector:Select Target from HTML
Request:Request Part
Response:Response Part
Middleware:Make Airspider Support to Extensions
Spider:Entrance for AirSpider
TODO
- Complete Plugins of Redis
- Complete Distributed Architecture
Contributing
AirSpider🕷️ is still under Developing
Feel free to open issues💬 and pull requests
- Report or Fix bugs
- Build Powerful plugins
- Make documentation Better
- Add Examples of Spiders
Copy from pypi.org
查询时间:71.264ms
渲染时间:71.403ms
本文同步分享在 博客“zhenruyan”(other)。
如有侵权,请联系 support@oschina.cn 删除。
本文参与“OSC源创计划”,欢迎正在阅读的你也加入,一起分享。