Contents

Scrapy 2.11.2

0

A high-level Web Crawling and Web Scraping framework

A high-level Web Crawling and Web Scraping framework

Stars: 51396, Watchers: 51396, Forks: 10383, Open Issues: 657

The scrapy/scrapy repo was created 14 years ago and the last code push was 21 hours ago.
The project is extremely popular with a mindblowing 51396 github stars!

How to Install scrapy

You can install scrapy using pip

pip install scrapy

or add it to a project with poetry

poetry add scrapy

Package Details

Author
Scrapy developers
License
BSD
Homepage
https://scrapy.org
PyPi:
https://pypi.org/project/Scrapy/
Documentation:
https://docs.scrapy.org/
GitHub Repo:
https://github.com/scrapy/scrapy

Classifiers

  • Internet/WWW/HTTP
  • Software Development/Libraries/Application Frameworks
  • Software Development/Libraries/Python Modules
No  scrapy  pypi packages just yet.

Errors

A list of common scrapy errors.

Code Examples

Here are some scrapy code examples and snippets.

GitHub Issues

The scrapy package has 657 open issues on GitHub

  • Proxy + invalid domain makes Downloader stuck
  • Fix command parse unhandled error :AttributeError: 'NoneType' object has no attribute 'start_requests'(#3264)
  • Issue #5366 Move from optparse to argparse
  • remove pywin32 docs section
  • deprecate scrapy.pipelines.images.NoimagesDrop
  • Remove scrapy.pipelines.images.NoimagesDrop
  • Removed the pywin32 docs section
  • Remove the pywin32 docs section
  • Deprecate scrapy.pipelines.images.NoimagesDrop
  • Move from optparse to argparse
  • Create SECURITY.md
  • Add SECURITY.md
  • Use coverage data from Windows runs in addition to Ubuntu ones
  • Document coroutines for signals.
  • Updated overview for video intro

See more issues on GitHub

Related Packages & Articles

qiling 1.4.6

Qiling is an advanced binary emulation framework that cross-platform-architecture