Web Scraping Legalities

9 Replies

Hello all,

I have a good list of addresses in the zip codes I am targeting for my direct mailing campaign. I am an engineer and have the software knowledge to make a program to scrape all the relevant information I need. However, I'm concerned on the legalities of scraping utilizing the county's website. I know with scraping before my IP address was literally banned from certain websites due to violating their terms & conditions (non-real estate related). Is there a better way to go about getting the information I need in an ethical & convenient manner?

Most people buy the information from a 3rd party. If you do not want to get the data yourself and use your software knowledge a couple of companies offer API access to call the data you are looking for

Originally posted by @Nettles Mason :

Most people buy the information from a 3rd party. If you do not want to get the data yourself and use your software knowledge a couple of companies offer API access to call the data you are looking for

 Any recommendations on 3rd partys? 

Melisa data has an api that you can send your address and they will append data. 

Scrapping can be a business issue (terms of services) or a legal issue (data protection). It depends on who owns the data and what you are collecting. The county data is likely publically owned so you might have a legal right to make a copy. You can ask.

You also need to understand the laws for direct mail (physical or electronic). The more you know about someone, the more they might have rights or protections against you using the info.

Originally posted by @John Corey :

Scrapping can be a business issue (terms of services) or a legal issue (data protection). It depends on who owns the data and what you are collecting. The county data is likely publically owned so you might have a legal right to make a copy. You can ask.

You also need to understand the laws for direct mail (physical or electronic). The more you know about someone, the more they might have rights or protections against you using the info.

 Getting the information from a dozen of addresses by doing a search on the county's website is perfectly legal but for me to scrape using 30k addresses violates their terms of conditions.

Then that makes ie legally clear Markos. You can not use the county data in bulk (fair use policy is a way to label this). Plan B or break the law are the options. As others have said, there are other sources of the data if you want to pay for it. I would not be surprised that the County will sell it to you under the right conditions.

Is the legal issue collecting data that is open to the public in bulk or collecting the data using software, or the speed at which you collect the data? In other words does it still violate any terms if you only scrape a few properties a day or does it not matter?

@Markos Gerges

It's almost a year old, but why don't you use their open data portal? I think this is exactly what you need.

https://data-wayne.opendata.arcgis.com/

https://data-wayne.opendata.arcgis.com/datasets/5bffcfce048e4b33a4b49027c47b5b66_0

Maybe you'd like to collaborate on software ideas? I'm currently doing this in another market right now.

Originally posted by @Michael Hajduk :

@Markos Gerges

It's almost a year old, but why don't you use their open data portal? I think this is exactly what you need.

https://data-wayne.opendata.arcgis.com/

https://data-wayne.opendata.arcgis.com/datasets/5b...

Maybe you'd like to collaborate on software ideas? I'm currently doing this in another market right now.

Yes this would be good if Wayne County was my target. I'll reach out to you and give you an idea of what i'm trying to do!

Create Lasting Wealth Through Real Estate

Join the millions of people achieving financial freedom through the power of real estate investing

Start here