Using Software Engineer to Aggregate Data?

Question

I'm looking to use large-scale data scraping to find deals.  Has anyone done this before or would be interested in doing it?  I'd love to hear ideas on what parameters would provide value.  I'd also love to discuss what automation features would add value to searches.

Lane Kawaoka · Answer

I am and work with a lot of engineer investors. Remember rei is a relationship business. I see so many smart people wasting their time on this data stuff which is just a subconscious mechanism to not talk to people.

Neil Polehn · Answer

I used to be a data analyst so I was working on this, but the issue I couldnt resolve was that data is historic and the real estate market is always changing with some new factor (lower interest rates, etc). I think you could come up with some good performance ratios like the 1% guideline though.  I am pondering that.

Walter Holmes · Answer

im very intrigued about the Decision tree methodology algorithm , sorry my expertise is not the tech part of the process in my firm but I understand the "idea" of this process and im very intrigued if applied properly I think it has huge potential to be applied in lead generation.

My advice is to find an expert in whatever field or market so he can lead the tech team in the process of gathering data, sorting, analyzing and have a final input.

The biggest problem with data analysts and projects like this which can lead to huge waste of time is hoping to "find something " in data (commonalities , trends,  ect) to get a "magic " final output that is valuable. There are just too many variables to account for and if 1 is missed the results are just not worth the time.

so for example you want to use data to find short-sale leads get a short-sale expert , when i say expert i mean someone who has work in lending that knows the process of how  short-sale offers are approved or denied....match those characteristics with properties in public records and you'll be on your way on getting solid leads. Ofcourse this is easier said than done, it takes good engineering but the key is the "arquitec" that knows exactly what those characteristics are..

I'm experienced in loss mit and did those analitics for lenders biggest problem I had was the amount of short-sale inquiries I would get from brokers for underwater  properties with second mortgages,  huge hoa debt and other liens ...99% of the time those were denied almost on the spot...

220k appraised property where a 1st position is owed 250k , the sales contract comes at $180k but after paying other liens the 1st position  ends up receiving $140k (thats 70k difference)... The lender would rather proceed with fcl, wipe the liens and safe harbor the hoa debt paying a smaller % of the debt or simply high bid at auction $180 for an investor to buy and collect an additional 40k.

so, if i want a good lead for short-sales I find properties with no additional mortgages or liens .

Again , I know what im looking for and its up to the engineer team to put the tech to work and find those properties.

I think real estate is in the stone age when data is concerned and i dont mean finding data , but how to apply that data.

Neil Polehn · Answer

I use to be a data analysis and love looking at closed listings to pull out trends.  When I have time I plan to work on this project more, but life happens:)  I have some questions I want to answer.  I think properties that had multi offers are interesting..what caused this?  Price per sq foot in the same areas, what makes a property go for more then another (floor plan fixings etc)?  Does advanced marketing (virtual tours, online bidding, etc)  result in more offers?

Walter Holmes · Answer

Wow,  sorry but that is not only a horrible generalization but also so short sighed..

REI a "relationship business "?What kind of "relationship" are we talking about here?

What does process  improvement have to do with "talking to people"?

with all due respect but you are either blind to the huge problems in real estate or have no clue about this subject....or both.

Real estate is the biggest market in the world by far and arguably the most archaic in their processes.

And the only people who pay the price are consumers ...And this is true in both macro and micro.

Or did 2008 teach you nothing?

To this day I see people being victims of others who "talked them into a relationship business "

This is a business like any other business...money in money out  don't try to romanticize it!

Now "this data stuff" im guessing  you are "old-school " and "still get the newspaper delivered because you like how it feels in your hands" or rather call than email or text "because is so much personal" ?

You can argue whatever you want about the way someone is implementing data and automation and i would agree but saying is a waste of time is throwing the baby with the bathwater.

Trevor Ewen · Answer

@Scott Wilkonson

I am a software engineer and have done this several times. Most notably, I did one to analyze undervalued properties near military bases. More recently, I worked with a client to create a database of trailer parks across the country & Canada, using different online sources.

My advice is to hyper-target what you are doing and make sure you can take action in the short-term. As the situation on the ground changes (as COVID has made obvious to anyone) insights gathered several months ago may be irrelevant at the current juncture.

I also recommend staying away from people that over-hype machine learning and advanced methods. If they can't produce a spreadsheet / chart of insights, quickly, then it's not going to be of great use unless you are a large entity with the funding and timeline that come with a proper machine learning operation.

Walter Holmes · Answer

It depends what your goal is and what deals you are looking for.

in my case i wanted to cut the time it took me to do due diligence on properties sold at auction so that I could go through as many properties as posible to be able to compete with funds that had teams doing this.

My problem was not how to do due diligence or wanting to cut corners on the quality of work but the long hours I spent working on properties ONLY to disqualify them as a deal. Unfortunately the only way to disqualify them was going through the process until I found the property was a second position, hoa, foreclose was deficient, surviving liens etc..

Creating a report with only relevant data, grouped and ordered a certain way would allow me to see disqualifying characteristics very quickly...so that what i set out to build I started with only a few fields (case number, folio, address) used that data to look for other data like owner names , then that data to check for liens, mortgages, court dockets , violations, tax bills, permits,  etc..

it was definitely a challenge to say the least.

Data is vast and its spread out in many sources all with different  challenges i.e search parameters, integrity and availability To name a few...

"Finding deals" is a broad term that needs to be defined first so that the data can be curated and presented in a way that makes sense for the person who is looking for those specific deals otherwise is like building a puzzle with missing pieces and pieces from other puzzles all combined.I started backwards, I knew exactly what data I needed, how, and where to find it so that i could quickly recognize missing pieces or pieces from other puzzles For my target deals.

Imo the mayor problems in our industry is the lack of centralized data sources, the level of complexity on deals in each niche "sub-market" (i.e distressed properties can be segmentedo in short-sales,  auction, refi, wholesale etc) each needing a level of expertise.

So imo the starting point should be a deal or sub-market you are an expert on choose a starting point and build backwards you'll see the parameters that add value as you start gathering data.

I found this to be true with my build..

i.e

Distressed properties are my market.

Starting point ?  properties been sold at auction (due diligence specifically ) so i did something like this.

I gather only relevant data>identified parameters that added value (disqualifying characteristics to filter prospects>identify prospects >attempt to close deal.

What quickly came afloat was that the "disqualifying" characteristics for some properties were only disqualifying them for an investor wanting to buy at auction but making it a positive characteristic for other type of deal.

i.e

one disqualifying characteristic for properties at auction are properties with equity , these have a high possibility of cancelation these properties are good prospects for a short-sales.(ofcourse There's a little more to it)

When i work with investors I explain to them that my job is to tell them where NOT to invest or what strategy fits best any given scenario. I do that by looking at disqualifying characteristics for each strategy i.e a property worth 100k with a mortgage payoff of 120k is a good prospect for a short-sale but if there are high hoa fees, a second mortgage or other liens it makes it near to imposible for the short-sales to be materialize.

The first position will continue through foreclosure in order to remove the other liens, apply the safe harbor for the hoa and mitigate their losses.

So data wise concentrating in disqualifying characteristics will allow you to assign the investment strategy to each scenario.

Its definitely a fun challenge but very time consuming in my opinion real-estate is in desperate need for innovation.

Using Software Engineer to Aggregate Data?

Most Popular Reply