Data Mining......Any Easier Way

10 Replies

Guys I have my assistant pulling data from MLS, county records and land records to come up with my investor data file for the county I invest in the most. Basically I pulled all the cash purchases for the last 2 years and I have my assistant pulling the owner records and deed/deed of trust records for this purchase. She is then entering the data into my master file that has a number of sortable fields that are included when you export from the MLS. Basically she is entering 5 columns worth of data that pertain to who bought the house for cash, their contact info, did they have a Deed of Trust, if so how much and with who, and did this investor flip the house or rent it out.

She is doing a great job and got through 100 houses in the first day as most of this is copying and pasting from the county office and the land records office but is there any easier way? I've heard that you can buy data files from the county but I am not sure if this will help our cause much as she would still have to sort through that file to copy and paste data to my master file.

Also is there any other information while we are going through this process that might be helpful? I am thinking of adding a column for the rehabbers where it list their end buyer price for the flip. Could be useful to look at spreads people are getting, average rehabber sale in a subdivision and maybe the finishes on these spreads. Any thoughts?

Account Closed - the key is to find a computer programmer (like a college kid), and have him write some scripts to compile all the data, and then export it into a excel sheet or build a simple tool to analyze the data.

Years back I wrote a script for someone that crawled local court cases (via a RSS feed) that were people with homes that were about to be foreclosed on and put them into a excel file - instead of hours of sorting through the data after a few simple clicks what took days could be done in seconds.

What exactly are you trying to understand? It sounds like you're gathering info on flippers, but what is the purpose for doing so?

Sorry - I didn't really provide any help there. Just trying to understand what you're going for to maybe offer some better advice. If you know what you're going after you may be able to narrow down the scope.

As @John K. mentioned, doing some sort of scraping would be the way to go. Depending on the amount of data, it may take longer to write a scraper than to manually pull the data, but once you've go the scraper you can easily update in the future and modify for other uses if needed.

I'd recommend finding somebody who is familiar with AutoIt. You might even be able to find somebody on the forum there to automate the data pulling for you.

I've pulled large data using an AutoIt script and also via copying raw data to excel and then parsing what was needed out of it. If you're having to click to get into the deeds and then view a scanned copy of the deed, I don't know that there is an easy way to automate that as you would need some kind of text recognition built into the software (very complex and probably not cheap).

The data will be useful for a list of cash buyer's and contact info, specifically cash buyer's for flips and rentals. But more importantly those cash buyer's who are using bank money or private money will typically have a deed of trust filed which will have their lender and often times terms. In my opinion this information will be invaluable as capital is necessary to grow businesses. You can also sort through a number of other data sources typically found in the MLS again which could be very useful.

Thanks guys I will look into it. I am unfamiliar with scripting but would it be able to pull the data from the county records for addresses I provide? Also deeds of trust are pdf files and some of the info I need is included in it. Does that change anything?

Account Closed 

Hi Derek,

It sounds like what you need is a GIS (geographic information system). Your county should provide full data files of property records and tax data. Although usually you do have to pay for these. The files are quite onerous to maintain. I did a quick perusal of the Prince George's County website and it looks like these are located under the planning department. With these files you can load them into the GIS (the program I use is called ArcMap, made by ESRI) and then map all the properties in your county. If you are able to download data from your MLS (say into an Excel spreadsheet), you can then load that data into the GIS and join the two separate pieces of data (MLS data and county property data) to create a complete database. After that all you need is a couple of precise SQL queries and you'll have what you need.

I'm a bit of a mapping/GIS nerd so let me know if I can help you at all.


@Jaclyn B.  Thanks for the response.  I noticed that the county record site sells the data on SpecPrint which says that I can pull the data out in an ASCII format.  I tried to load an example up to see if it is something that I can easily convert to an excel format and somehow merge with my document but my computer won't allow me to open an ASCII file.  Is there a way to convert ASCII to Excel easily?  

Thanks for the recommendation to look into the GIS Data under my planning department.  This looks to be map data.  In this map data would it have the raw data I am looking for  that I can pull out?  

Account Closed 

Sometimes the easiest way to open an ASCII file is to open it in a text editor like Notepad, figure out how it's delimited (comma/tab etc), and then import the data from Excel (where you can specify how it's delimited) rather than just opening the file. It's a tad onerous but it's worked for me in the past.

The map data is the raw data, you're just able to map it. Which can be very helpful if you're targeting a specific area. If you can put the map data (usually in geodatabase format) into a GIS system (I use ArcMap) you can run a query on it and extract the specify cases you're looking for. It might take some manipulation of the raw data (for example, creating a new column that flags a certain purchase date range), but querying the data isn't too complex. If you don't have access to a GIS system, I would definitely look into it. I will say that many of these systems have a steep learning curve. Depending on how you value your time, you can learn it yourself or find someone who has access and pay them to do the work for you.


Okay thanks Jaclyn.  I appreciate the feedback/insight.

Free eBook from BiggerPockets!

Ultimate Beginner's Guide Book Cover

Join BiggerPockets and get The Ultimate Beginner's Guide to Real Estate Investing for FREE - read by more than 100,000 people - AND get exclusive real estate investing tips, tricks and techniques delivered straight to your inbox twice weekly!

  • Actionable advice for getting started,
  • Discover the 10 Most Lucrative Real Estate Niches,
  • Learn how to get started with or without money,
  • Explore Real-Life Strategies for Building Wealth,
  • And a LOT more.

Lock We hate spam just as much as you

Create Lasting Wealth Through Real Estate

Join the millions of people achieving financial freedom through the power of real estate investing

Start here