Scrapy 2.7.1


A high-level Web Crawling and Web Scraping framework

A high-level Web Crawling and Web Scraping framework

Stars: 45466, Watchers: 45466, Forks: 9814, Open Issues: 769

The scrapy/scrapy repo was created 12 years ago and was last updated 3 hours ago.
The project is extremely popular with a mindblowing 45466 github stars!

How to Install scrapy

You can install scrapy using pip

pip install scrapy

or add it to a project with poetry

poetry add scrapy

Package Details

Scrapy developers
GitHub Repo


  • Internet/WWW/HTTP
  • Software Development/Libraries/Application Frameworks
  • Software Development/Libraries/Python Modules
No  scrapy  pypi packages just yet.


A list of common scrapy errors.

Code Examples

Here are some scrapy code examples and snippets.

GitHub Issues

The scrapy package has 769 open issues on GitHub

  • Proxy + invalid domain makes Downloader stuck
  • Fix command parse unhandled error :AttributeError: 'NoneType' object has no attribute 'start_requests'(#3264)
  • Issue #5366 Move from optparse to argparse
  • remove pywin32 docs section
  • deprecate scrapy.pipelines.images.NoimagesDrop
  • Remove scrapy.pipelines.images.NoimagesDrop
  • Removed the pywin32 docs section
  • Remove the pywin32 docs section
  • Deprecate scrapy.pipelines.images.NoimagesDrop
  • Move from optparse to argparse
  • Create
  • Add
  • Use coverage data from Windows runs in addition to Ubuntu ones
  • Document coroutines for signals.
  • Updated overview for video intro

See more issues on GitHub

Related Packages & Articles

qiling 1.4.4

Qiling is an advanced binary emulation framework that cross-platform-architecture

fastapi-framework 1.5.2

A FastAPI Framework for things like Database, Redis, Logging, JWT Authentication and Rate Limits