Market Analysis - City Level Data
Hey guys,
I am trying to do some bottoms-up market analysis. Figured you guys could probably point me towards a data provider that would have city level data exportable to excel so you can manipulate it. I don't mind if it is paid/subscription but I need to be able to run some analysis on my own. Hoping you can point me in the right direction.
Most Popular Reply
Hello @Luis Fontenoy,
You did not state the purpose for the data so I will assume you are looking for a good investment city. So we are on the same page, my definition of a good investment city is one where:
- Rents outpace inflation
- Persistent income - You will not outlive the rental income
Before I proceed, Some background information about me. I am an engineer working for an investor services company in Las Vegas. My role involves data science, property analysis, and software development. I developed software tools and processes that enable us to identify and evaluate properties based on individual client's specific needs and goals. To date, we have delivered over 480 properties to more than 180 clients worldwide, with a repeat business rate exceeding 90%. I believe my experience makes me qualified to offer opinions on the use of analytics.
Raw Data
You asked about city-level data sources. What data you find will likely vary in terms of quality, consistency, and completeness. Even if you find high-quality data, interpreting the data is always a challenge. There is a better way than using raw data, and that is using indicators. An example will illustrate the point.
Suppose you are tasked with determining which country, Country A or Country B, is more desirable for people from both countries to live in.
- Data-based approach - Assuming you have all the data you need, you can analyze it to determine which country is more desirable. However, your conclusion is unlikely to be correct because you must make assumptions about what matters to the citizens of both countries. Such information is not likely to be found in raw data.
- Indicator-based approach - What if you sat on the border and observed how many people moved between Country A and B? If by headcount, you determine significantly more people move from Country B to A, you can assume that, to the citizens of the two countries, Country A is the more favorable. This approach used the indicator, the actual movement of people, instead of raw data.
The use of indicators provides highly accurate results. This is how our data mining software works; it evaluates properties based on historical tenant behaviors.
Indicators for a Good Investment City
Repeating my earlier opinion of what is a good investment city:
- Rents outpace inflation
- Income persistence - You will not outlive the rental income
What is driving these two criteria?
Rents Outpace Inflation
In real estate, prices and rents are determined by the imbalance between the number of buyers and sellers. When there are more buyers than sellers, prices rise until the number of buyers and sellers is balanced. On the other hand, when there are more sellers, prices drop until the balance is restored. Rental rates follow prices. When prices are high, demand for rental properties increases and rents rise; when prices are low, demand for rental properties decreases and rents fall.
What causes the imbalance between buyers and sellers? Population change. And, if the population increases fast enough, rents will rise faster than inflation. So, we have the first city selection indicator.
✅ Significant and sustained population growth. Use Wikipedia
Income Persistence
Income persistence depends on your tenants remaining employed at similar wages for a long time. However, all private sector jobs are short-lived. The average lifespan of a company is about 10 years. Even S&P 500 companies only have an average lifespan of about 18 years. So, whether your income persists is not only about the present jobs, it is also about future replacement jobs.
What conditions are necessary for existing companies to invest in expansions, and for new companies to relocate to a city? In my opinion:
✅ Economic stability. This requires a metro population >1M. Smaller cities tend to be dependent on a single company or market sector. Wikipedia
✅ Low operating costs: The three most apparent costs for investors are income taxes, property taxes, and insurance. Tax Foundation, Insurance - ValuePenguin, State Property Tax Rates - Rocket Mortgage
✅ Low crime rate: Companies depend on attracting talented workers. Talented workers will not move to a high-crime city. Do not invest in any city on Neighborhood Scouts’ list of the 100 most dangerous US cities.
✅ Low risk of a natural disaster
✅ Pro-business environment
✅ No rent control of any kind. Rent control is a strong indicator of an intrusive government
There is data for each of these indicators, which I will show shortly. For now, the question is how to use these indicators.
Do Not Select, Eliminate
Start with the easiest criteria and then eliminate cities based on additional criteria, as illustrated below.
The criteria and sources:
- Cities with a metro population greater than 1M. Wikipedia
- Significant and sustained population growth. Wikipedia
- Low crime rate. Never invest in any city on Neighborhood Scout’s 100 most dangerous cities list.
- Low risk of a natural disaster - Homeowners insurance rates are a good indicator of the potential for a natural disaster to occur. The higher the rate, the higher the probability of a natural disaster. Insurance - ValuePenguin
- Low operating costs - The three most apparent costs for investors are income taxes, property taxes, and insurance. Tax Foundation, Insurance - ValuePenguin, State Property Tax Rates - Rocket Mortgage
- Pro-business environment - Google search
- No rent control of any kind - Google search
You Need an Investment Team
One additional criterion I recommend is an experienced local investment team. Podcasts, books, seminars, and websites only provide general information. You will buy a specific property, in a specific city, subject to specific local conditions. The only source for the skills, processes, and experience needed to identify, validate, renovate, and manage properties is a local investment team. By working with a team, you also gain real-world investment training.
Also, working with an investment team usually does not cost more. For instance, we have delivered over 480 properties and only charged our clients a fee on only four or five of them, and these were exceptional circumstances. In all other cases, our fees were paid by the listing agent of the seller, not by our client.
- Eric Fernwood
- [email protected]
- 702-358-8884
#1 Starting Out Contributor
- Flipper/Rehabber
- Pittsburgh
- 5,566
- Votes |
- 6,220
- Posts
#1 Starting Out Contributor
- Flipper/Rehabber
- Pittsburgh
what are you trying to accomplish?
Hello @Luis Fontenoy,
You did not state the purpose for the data so I will assume you are looking for a good investment city. So we are on the same page, my definition of a good investment city is one where:
- Rents outpace inflation
- Persistent income - You will not outlive the rental income
Before I proceed, Some background information about me. I am an engineer working for an investor services company in Las Vegas. My role involves data science, property analysis, and software development. I developed software tools and processes that enable us to identify and evaluate properties based on individual client's specific needs and goals. To date, we have delivered over 480 properties to more than 180 clients worldwide, with a repeat business rate exceeding 90%. I believe my experience makes me qualified to offer opinions on the use of analytics.
Raw Data
You asked about city-level data sources. What data you find will likely vary in terms of quality, consistency, and completeness. Even if you find high-quality data, interpreting the data is always a challenge. There is a better way than using raw data, and that is using indicators. An example will illustrate the point.
Suppose you are tasked with determining which country, Country A or Country B, is more desirable for people from both countries to live in.
- Data-based approach - Assuming you have all the data you need, you can analyze it to determine which country is more desirable. However, your conclusion is unlikely to be correct because you must make assumptions about what matters to the citizens of both countries. Such information is not likely to be found in raw data.
- Indicator-based approach - What if you sat on the border and observed how many people moved between Country A and B? If by headcount, you determine significantly more people move from Country B to A, you can assume that, to the citizens of the two countries, Country A is the more favorable. This approach used the indicator, the actual movement of people, instead of raw data.
The use of indicators provides highly accurate results. This is how our data mining software works; it evaluates properties based on historical tenant behaviors.
Indicators for a Good Investment City
Repeating my earlier opinion of what is a good investment city:
- Rents outpace inflation
- Income persistence - You will not outlive the rental income
What is driving these two criteria?
Rents Outpace Inflation
In real estate, prices and rents are determined by the imbalance between the number of buyers and sellers. When there are more buyers than sellers, prices rise until the number of buyers and sellers is balanced. On the other hand, when there are more sellers, prices drop until the balance is restored. Rental rates follow prices. When prices are high, demand for rental properties increases and rents rise; when prices are low, demand for rental properties decreases and rents fall.
What causes the imbalance between buyers and sellers? Population change. And, if the population increases fast enough, rents will rise faster than inflation. So, we have the first city selection indicator.
✅ Significant and sustained population growth. Use Wikipedia
Income Persistence
Income persistence depends on your tenants remaining employed at similar wages for a long time. However, all private sector jobs are short-lived. The average lifespan of a company is about 10 years. Even S&P 500 companies only have an average lifespan of about 18 years. So, whether your income persists is not only about the present jobs, it is also about future replacement jobs.
What conditions are necessary for existing companies to invest in expansions, and for new companies to relocate to a city? In my opinion:
✅ Economic stability. This requires a metro population >1M. Smaller cities tend to be dependent on a single company or market sector. Wikipedia
✅ Low operating costs: The three most apparent costs for investors are income taxes, property taxes, and insurance. Tax Foundation, Insurance - ValuePenguin, State Property Tax Rates - Rocket Mortgage
✅ Low crime rate: Companies depend on attracting talented workers. Talented workers will not move to a high-crime city. Do not invest in any city on Neighborhood Scouts’ list of the 100 most dangerous US cities.
✅ Low risk of a natural disaster
✅ Pro-business environment
✅ No rent control of any kind. Rent control is a strong indicator of an intrusive government
There is data for each of these indicators, which I will show shortly. For now, the question is how to use these indicators.
Do Not Select, Eliminate
Start with the easiest criteria and then eliminate cities based on additional criteria, as illustrated below.
The criteria and sources:
- Cities with a metro population greater than 1M. Wikipedia
- Significant and sustained population growth. Wikipedia
- Low crime rate. Never invest in any city on Neighborhood Scout’s 100 most dangerous cities list.
- Low risk of a natural disaster - Homeowners insurance rates are a good indicator of the potential for a natural disaster to occur. The higher the rate, the higher the probability of a natural disaster. Insurance - ValuePenguin
- Low operating costs - The three most apparent costs for investors are income taxes, property taxes, and insurance. Tax Foundation, Insurance - ValuePenguin, State Property Tax Rates - Rocket Mortgage
- Pro-business environment - Google search
- No rent control of any kind - Google search
You Need an Investment Team
One additional criterion I recommend is an experienced local investment team. Podcasts, books, seminars, and websites only provide general information. You will buy a specific property, in a specific city, subject to specific local conditions. The only source for the skills, processes, and experience needed to identify, validate, renovate, and manage properties is a local investment team. By working with a team, you also gain real-world investment training.
Also, working with an investment team usually does not cost more. For instance, we have delivered over 480 properties and only charged our clients a fee on only four or five of them, and these were exceptional circumstances. In all other cases, our fees were paid by the listing agent of the seller, not by our client.
- Eric Fernwood
- [email protected]
- 702-358-8884
Eric,
That was one of the most valuable posts i've seen on this forum.
Thank you.
Eric, your post is so valuable. I'm going to save it for my own market research. As a data nerd myself, here are some other data I also look at.
- Job growth and Unemployment rate trends: BLS.gov (can be drilled down to city level)
- Median listing price per sqft with trends: FRED
- Owner-occupied housing unit rate: I didn't know this existed but it's an interesting metric to look at. If a city has high owner-occupied rate, the demand for rentals might be low? Census
- Census has some other population metrics like median household income.
Hello Thuy,
Thank you and @Andrew Romano for your kind remarks. When I began investing in real estate many years ago, I made numerous mistakes. However, I was fortunate enough to receive help from others, and now I aim to repay their kindness by assisting others on their own journeys.
Data science and the study of demographics formed the foundation of our investor services business. The software and processes we have today are the result of many years of mistakes. One of the most important lessons I learned from my mistakes is that correctly interpreting raw data for real-world applications is very difficult. The combination of incomplete and unnormalized data, unpredictable human behaviors, and the necessity to make assumptions based on insufficient information makes accurate predictions challenging. An example may help.
Suppose your job is to determine whether Country A or Country B is the more desirable place to live for the citizens of the two countries. Assume every type of data you could want is available for both countries. The problem is that you will have to make assumptions about the data concerning what is desirable to the people of countries A and B. The odds of getting this right with only data are low. However, there is another way.
Suppose you stationed observers on the border between the two countries and they keep count for an extended period of time which way people were moving. If significantly more people moved from country A to Country B, you know that in the eyes of the people of these countries, Country B is more desirable. This is the advantage of observations over raw data.
Another example. When I moved to Las Vegas in 2005 to start an investor services business, I wanted to find a tenant segment with a high concentration of reliable tenants. These tenants stay many years, pay rent on schedule, and take care of the property. Despite spending over a month analyzing data, I couldn't draw any useful conclusions and eventually gave up on this approach.
I downloaded approximately 10 years of MLS rental data and analyzed historical tenant behaviors. I discovered several subdivisions where the average tenant stay was longer than five years and ran correlations between them.
From the rental rates of these properties, I estimated the tenants' income range. I then used this range to determine the types of jobs they held. I found that those at the lower end of the income range largely relied on tourism-related employment, so I raised the lower threshold. I also noticed a decrease in tenant tenure for properties with rent at the upper end of the income range, so I lowered the upper threshold. This research enabled me to identify the tenant segment we have been targeting for 15+ years with great success.
Thuy, based on my experience, I recommend focusing on observations and indicators as opposed to raw data. Raw data, although abundant, is challenging to use for drawing reliable conclusions.
I wish you success,
…Eric
- Eric Fernwood
- [email protected]
- 702-358-8884