Scrapy

I Use This!

High Activity

Analyzed about 2 hours ago. based on code collected 1 day ago.

Project Summary

Scrapy is a fast high-level scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

Badges

In a Nutshell, Scrapy...

...
has had 11,152 commits made by 754 contributors
representing 61,283 lines of code
...
is mostly written in Python
with a low number of source code comments
...
has a well established, mature codebase
maintained by a very large development team
with stable Y-O-Y commits
...
took an estimated 16 years of effort (COCOMO model)
starting with its first commit in June, 2008
ending with its most recent commit 2 days ago

Quick Reference

Project Links:

Homepage
Download

Code Locations:

https://github.com/scrapy/scrapy

Similar Projects:

Managers:

Shane Evans, Daniel Graña, and Pablo Hoffman

Licenses

BSD 3-clause "New" or "Revised" License

Permitted

Commercial Use

Modify

Distribute

Place Warranty

Forbidden

Hold Liable

Use Trademarks

Required

Include Copyright

Include License

These details are provided for information only. No information here is legal advice and should not be used as such.

All Licenses

Project Security

Vulnerabilities per Version ( last 10 releases )

Project Vulnerability Report

Security Confidence Index

Poor security track-record

Favorable security track-record

Vulnerability Exposure Index

Many reported vulnerabilities

Few reported vulnerabilities

About Project Vulnerability Report

Did You Know...

...
65% of companies leverage OSS to speed application development in 2016
...
data presented on the Open Hub is available through our API
...
in 2016, 47% of companies did not have formal process in place to track OS code
...
compare projects before you chose one to use