AirSpider

AirSpider-An Easy-to-use and Fast Python Spider Framework-Liu Xunzhuo

发布:2020-12-22 14:50:11.623476

作者:Liu Xunzhuo

### 作者邮箱:mixdeers@gmail.com

### 首页:https://pypi.org/project/airspider/

### 文档:None

### 下载链接 

<p align="center">
<img src="https://picreso.oss-cn-beijing.aliyuncs.com/airs.png" width="150px">

Distributed - Asynchronous - Easy-to-use

An Easy-to-use and Fast Python Spider Framework


Overview

  • AirSpider is a high-performance asynchronous crawler framework for developers
  • Based on Redis: task distribution, task deduplication, and distributed ☁️

Requirements

  • Python 3.6
  • Works on Linux, Windows, macOS

Features

  • Quick to Start
  • Low Coupling
  • High Cohesion
  • Easy Expansion
  • Orderly Workflow

Installation

# For Linux && MacOS
pip3 install airspider


# For Windows
pip3 install airspider

## Documents

TODO

  • Complete Plugins of Redis
  • Complete Distributed Architecture

Contributing

AirSpider🕷️ is still under Developing

Feel free to open issues💬 and pull requests

  • Report or Fix bugs
  • Build Powerful plugins
  • Make documentation Better
  • Add Examples of Spiders

 Copy from pypi.org

查询时间:71.264ms

渲染时间:71.403ms

本文同步分享在 博客“zhenruyan”(other)。
如有侵权,请联系 support@oschina.cn 删除。
本文参与“OSC源创计划”,欢迎正在阅读的你也加入,一起分享。

评论

暂无评论