“Web scraping”, also called crawling or spidering, is the automated gathering of data from someone else’s website.
Scraping is an essential part of how the Internet functions, and it has traditionally been the backbone of many companies’ marketing, lead generation, and market intelligence efforts.
In fact, many online services, large and small, use scraping to build their databases, all of which are worth hundreds of billions of dollars. This article will answer the question – is web scraping legal?
Even though web scraping is ubiquitous, a plethora of legal issues remains fraught with the process. The following analysis explains the position of the law on web scraping in the world today.
Under the GDPR
Under the EU’s General Data Protection Regulation (or GDPR) web scraping does not apply to a person or company except such an entity is scrapping personal data of people within the European Economic Area, including Iceland, Liechtenstein, and Norway.
Examples of personal data include a person’s Name, Physical Address, Email Address, Phone Number, Credit Card Details, Bank Details, IP Address, Date of Birth, Employment Information, Social Security Number, Medical Information and Video/Audio Recording.
Use this guide to web scraping legal issues to ensure your web scraping is GDPR compliant.
However, exceptions can be made for the scraping of the personal data of people within the EEA. The five types of lawful reasons are Consent, Contract, Compliance (with a legal obligation), Vital Interest/Public Interest/Official Authority, Legitimate Interest.
Is scraping e-commerce sites legal? In 1999, the first web scraping case between eBay and Bidder’s Edge was heard in a US court. Therein, Bidder’s Edge accessed eBay’s approximately 100,000 times without the latter’s authorization.
The case was settled out of court when Bidder’s Edge paid Ebay an undisclosed amount and agreed not to access Ebay’s data. The legal principle has been amended in the US, among a plethora of cases, the latest of which will be addressed in subsequent paragraphs.
Is Scraping Social Media Sites Legal? In late 2019, the US Court of Appeals denied LinkedIn’s request to prevent HiQ, an analytics company, from scraping its data. Consequently, the historic decision showed that any data that is publicly available and is not copyrighted is legally valid for the use of web crawlers.
The decision did not, however, grant HiQ or other web crawlers the freedom to use data obtained for unlimited commercial purposes. For example, a web crawler would be allowed to search YouTube for video titles, but it could not re-post the Youtube videos on its own site since the videos are copyrighted.
To keep your scraping activities ethical and also to avoid any problems follow the best practices, like limiting your requests to every target site so that they will not feel invaded and their servers will not overload. Also, the proxy provider and type of proxy IP you choose is also very important.
Check out this guide to web scraping proxies and find out which proxy providers offer the best combination of reliability and price.
Existing Legal Issues With Web Scraping
Copyright Infringement: In most jurisdictions, web scraping is legal, but using copyright data contains certain restrictions.
Violation of the Computer Fraud and Abuse Act (CFAA): This law, enacted to prevent computer hackers, prevents fetching data by getting unauthorized access to a page.
Trespass to Chattel: Here, a chattel (or data) is violated if the website server is hurt in any way. Thus, trespass to chattel is violated if the server slows or stops because of the scraping.
Promoted
