What it takes to build a Family Office From Scratch

A few weeks ago Alina spoke with Noah Rosenfarb on the topic of building a Family Office. Here is the full interview content: 

Alina: Very nice meeting you, Noah! Thanks a lot for agreeing to be interviewed. Can you please tell our audience a little bit about yourself, your family, your passions in life?

Noah: Sure, so I’m a third generation CPA. My grandfather, my father, and myself. Actually, my wife’s father as well. You know, a lot of accounting in the family. I started my career at my dad’s accounting firm, but left after about a decade after becoming a partner to start my own family office. My wife and I’ve been married for just now 20 years, and we met when we were teenagers. And we’ve got two young children that are still in grade school, and we live in South Florida, although I was born and raised in New Jersey.

Alina: What made you move to Florida?

Noah: Just the weather. When we had our first child, we would spend a couple of weeks during the winter in Florida, and we really loved it. And my grandfather was here; my father’s father, I enjoyed spending time with him. We had a place at the beach that we could stay at. So, it’s just a nice relaxing way to enjoy part of the winter and get away from the snow. And when we looked at each other, my wife and I, one day walking on the boardwalk after we bought a lottery ticket, and we said, you know, if we win we’d move here, and part of our philosophy has always been to do what we would do if we would win the lottery, but just do it without winning the lottery, so that motivated us to get down here.

Alina: Noah, you mentioned the topic that we’re going to cover today, and that is family office. As you said, you started off your own family office. Can you define for our audience, what family office is?

Noah: Yeah, I guess there’s no, you know, the definition has been somewhat bastardized over the last decade. So historically, family offices were created by families that had significant wealth to manage their own money, and they’d hire their own team to build their own family office just to manage their own family’s money. What happened over time is simply we had this evolution away from stockbrokers to financial advisors. That evolution continued from financial advisors to becoming so-called wealth managers, and now you have essentially a lot of wealth managers that are becoming family office advisors. I’ve kind of grown up that way, so I used to distinguish myself from financial advisors by being a wealth manager. Now I distinguish myself from being a wealth manager, as being a family office advisor. What that really means is that we’re managing the total balance sheet for affluent families. So, our business is a multi-family office as some would call it, which is not dissimilar to a wealth management firm. The main distinction, at least for us, for what we’re doing, is that we’re looking at the total balance sheet, so we’re not necessarily taking a position where we’re only getting paid on stocks and bonds that we’re directly managing, which is really what most wealth management firms derive the vast majority of their compensation from. As a family office we’re really focused on how do we advise a family around all things related to their wealth, and how do we charge them a fee that compensates both of us fairly for the effort and the results that we provide.

Alina: Excellent! And to expand on that definition, would you say that wealth advisors primarily work with the person that engages with them only, where family office concentrates more on the family overall? The person, who engages with you, maybe has children or plans to leave a legacy in some sort of way? Can you expand on that and can you cover those other areas as well?

Noah: It depends on who’s hiring us and what their goals are. So a lot of our new clients come to us when they’re thinking about selling their business, and oftentimes those entrepreneurs are in their 40s 50s, or 60s, and depending on where they’re at in their life spectrum and how old their kids are, it may or may not be that relevant to help with the family education around money. What we find is that usually, as the demographics of our clients get older, and they start to have grandchildren, now there’s a reason for us to be interacting with their children who are maybe in their late 20s early 30s, that are building families of their own, starting to educate them about money, not just the money that maybe mom and dad are going to leave to them, but also the money they’re earning themselves; how to protect themselves from taxes, from creditors, asset protection. And then also, you know, if something were to happen to them, what would they want to happen for their children.

Alina: Sure. So, essentially, prepping for the family office entails all financial assets; so, it’s managing their balance sheet, doing tax compliance, tax strategy and asset protection for them. And then, helping them to not only invest their funds in Wall Street, but also be able to diversify beyond Wall Street into other financial vehicles, that are definitely not available through the typical wealth managers that simply trade on Wall Street.

Noah: That, and to add to that, especially for the clients that we deal with. So, most of our clients are entrepreneurs, they’re creators. They’ve created value through the building of a business, and our focus for them is figuring out how to maximize the value of that company, whether they want to own it, or they want to transfer it, so we don’t necessarily only deal with entrepreneurs after they’ve sold a business and they’ve got a big pile of cash that they need to put into Wall Street products like stocks and bonds. We’re working with clients where the vast majority of their wealth might still be tied up in their business. And, you know, just because it’s in a business doesn’t mean they don’t need advice around how to properly manage it. And the same is true of private equity investments that they may come across that we’re not sourcing directly for them. So, a lot of our clients have a great network of friends, colleagues that are doing other amazing things in their businesses, and sometimes they’re looking for capital, and we’re helping them decide what to allocate to, how much allocate, how to select investments as well.

Alina: Interesting. So, in a way, it becomes sort of a club, if you will. Maybe not officially, but unofficially where people can bring their potential investment ideas. And have you and your team help them to research and do due diligence and advice on those investments.

Noah: Yes, we have a prospective client who’s sold a medical device business, and he’s been thinking about getting into another medical device business. And what we showed him is that if he works with our firm, how he could structure his investment, not just to invest his own capital that he could get a return on, but also how to aggregate capital from other people that already trust him. And then potentially from other families that we work with, where he could raise additional capital to make money for syndicating and putting that deal together and overseeing an investment, so he could leverage, not just the opportunity that he sees in front of him but it also leverages his time, so that if he does help to produce results for that investment, he’ll receive rewards not just on his own money but on the money that he’s able to raise behind them.

Alina: Excellent, that’s a great idea! I love that! Majority of our audience are investing in real estate syndications, so a lot of them are familiar with real estate investments. I am curious, is real estate one of the vehicles, investment vehicles you offer to your clients and, if so, how often do you offer it – is that more common than, let’s say, other investments outside of real estate?

Noah: Yeah, I like to think of my family as a real estate family, we own interests in 3500 different apartment units around the country, and a half a million square feet of retail and office space. We’ve done 33 transactions. And, you know, that’s how I like to invest most of my liquid wealth. I believe it’s super-efficient, it’s got a great risk profile. And so, I’m sourcing deals for my family, and then I’m bringing in other families with us, our platform for that is called “Invest With Our Family” because that’s essentially what people are doing. We closed on about 1000 doors in the past twelve months. We’ll probably close on another thousand doors in the next 12 months. Usually that’s broken up between anywhere from four to eight deals. And we’ll work with new investors that want to invest with our family just in one real estate project as a way to build a relationship. They don’t have to be a family office client, not everyone that wants to invest in real estate really has the need for the full family office services that we provide. But then all of our family office clients typically want access to the real estate investments that we’re making.

Alina: Sure. So, you mentioned that one of your real estate investments is office space. Do you still believe in this asset class? Do you think it has a future?

Noah: No, we bought our office buildings early in the cycle so we’ve got a 200,000 square foot building out in Oak Brook Illinois that we acquired in 2012 at a discount to the note value. So, we basically took it over from a bank and there was a lot of built-in equity when we took over that acquisition, we’ve gotten 70% of our money out of it already. Over the course of the last eight years. And, you know, there were some risks in the deal that were unique to it we had a single tenant, we were trying to figure out if they would renew their lease, and if they did it would be a homerun, if they didn’t, you know, maybe it would be a single or double, so they ended up renewing two floors, not all three floors and you know we’re working on renting it out now. You know, it’s a great location, it’s a great suburb, it’s a good asset. I don’t know if that will ever be worthless, and I don’t know that we’d ever lose our equity, but the play when we went in, was that just based on the existing lease term and the purchase price, we were going to get out the vast majority of our money just from the cash flow, so even if at the end of the day, the business was worthless and we had to give the keys back to the bank, we really weren’t going to lose that much money. We’ve got two other office buildings that we bought with similar stories, you know, unique situations where we felt like from a risk-reward standpoint, our capital was pretty secure, and, you know, it was early in the cycle. I would say by 2014 we didn’t acquire any more office space; the pandemic has certainly shown what sensitivity there is in the office space environment. And I’m glad I hadn’t been allocating to this space for the last six years, and I don’t foresee allocating in the future.

Alina: It makes sense. Going forward you said you plan to acquire another thousand doors this year, and so forth moving forward. What type of real estate investments would be on your horizon? Knowing the situation that we’re in today, where we’re expecting presidential elections to happen, we don’t yet have a vaccine for COVID, and we don’t know which way the economy is going to turn out down the road, how do you proceed in selecting your next real estate asset class?

Noah: We’ve been heavily allocating to what’s been commonly referred to as workforce housing. So, assets that are essentially C class assets sometimes they might be in a, b minus neighborhood or maybe we’re buying a b minus asset in the C neighborhood, but usually we’re buying assets that are built in the 60s 70s or 80s. They have a need for improvements; these places that you know most of our investors would never want their kids to live. But what we do is we go in, we buy them with as high loan-to-cost financing as we can get, that’s part of our model that we believe in, with low interest rates that we have today, we go in and we rehab them, we renovate them, and then we refinance them, and we you know we refer to that as our Infinite Return Model because typically if we execute well, we’re able to pull in our capital for the closing within you know 12 months maybe even 18 months at the long side, we’re getting the vast majority of our capital back directly, and then indirectly through the tax benefits. So, we basically recoup all of our capital, we still own the asset, we still get cash flow. So that’s really the playbook that we’ve been using for a number of years, that I’ve gotten very comfortable with in today’s interest rate environment. I think the playbooks change over time based on what’s happening in the marketplace. Right now, we’re recording this in the fall of 2020 and going into this winter that I think there’s some uncertainty as to what rent collections might be, you know, the stimulus has run out, a lot of our tenants are in that $30,000 and $60,000 a year income range. So, some of them had been displaced by COVID above the restaurant and retail worker level, but below the kind of white-collar workers that are working remotely pretty effectively. So, you know a lot of nurses, and managers, and mid-level people. We believe that there’s a chance that in the assets that we own or the assets that we acquire, we’re going to take a hit in the winter with our rent collections, but the flip side to that is that there’s a lot of capital that’s leaving the northern New England states, that’s leaving California, because of the fear of the tax rates, because of the fear of the policies that they’re implementing and on, you know, the workers, and that capital is fleeing to the markets that we like exposure in, which is mostly the Sunbelt states and the low tax states. So, I think the cap rate compression, that we might see over the next six months to twelve months is going to more than make up for any of the cash that we miss out on collecting in that same time frame.

Alina: Noah, that’s an excellent strategy! I love it! That’s amazing! And what would you say are your main goals when it comes to investing? Would you say it’s more of capital preservation or is it something else?

Noah: I tend to be a higher risk investor in the real estate investments that we’d like to make. I think there’s, you know, luck favors and bold or fortune favors the bold whatever that term is. I am young, you know not all of my clients have the same risk profile as me. But, but for my family you know we’ve got more cash flow than we need to pay for our lifestyle expenses, so we can take risks that other families can’t afford to take. And by producing these syndicated investments that have very high loan-to-cost financing, we really reduced the amount of equity, that’s required to get the deal closed so we just bought a 475-unit asset for $30 million. We only had to put up three and a half million of equity. So, you know, that’s a really skinny equity check to acquire such a large asset. Now, in the worst-case scenario, we lose all of our money, right? Now, that’s a risk I’m willing to take because in the kind of best case scenario based on what we think are some relatively reasonable assumptions we could probably do five or six times our money in the next 12 months, just through the capital improvement plan: the value add planning, refinancing, getting our money out. So, I’m willing to take the risk of loss in exchange for that upside. Because if I’m wrong on most of the deals that I’m doing, and we’re not able to refinance our bridge we’re not able to get up our rent, if we had 20% equity at closing instead of 15% or 10%, we probably would be in almost the same position. So, I think, at least in my mind I could always inject that additional equity after the fact if I feel like it’s a good bet, but I can’t get that equity out if it was a bad bet, so I might as well go into the deal with as little equity as possible. And then if we need to rescue it for whatever reason, we have that option and if we want to walk away, we know exactly what our losses are.

Alina: Brilliant. You mentioned that, because you’re young, your investment profile is probably a little bit riskier than some of your clients, who might be older and are not willing to take as much risk. So, when you’re advising them, do you still offer them this kind of riskier investment or do you tend to offer them less risky more stabilized investments?

Noah: A lot of times they’ll make an allocation to every deal that we do. Their allocation might be different than mine. So, you know, they might have twice as much money as me but they’re investing half as much. And so that’s because they do have a fixed income portfolio whereas I don’t, I don’t have any fixed income, I don’t believe in fixed income for someone like me. I have a private debt fund. So a lot of times, you know, I am making investments in private debt that are certainly much more conservative from a risk standpoint, a lot of that is actually mezzanine debt or preferred equity in real estate deals where we’re taking a different portion of the capital stack. So somebody is acquiring a multi-family property. They’re getting an agency loan. Maybe it’s 65 or 75% loan to value. We will come in with some additional money on top of that, in a preferred equity position where our total return is going to be capped at 12% or 14%, but we’re getting a current pay of, you know, anywhere from 8 to 10%. And we’re protected by that other 15 or 20% of equity that’s coming in behind us. I’m really comfortable with those investments. Again, my clients may have more exposure to that than me, because while I like that risk profile, I still would rather be on the equity side of that deal.

Alina: Sure, it makes sense. So, now what does it take someone to build a family office?

Noah: So, I think there’s, you know there’re two ways to answer the question. One is, if you’re an entrepreneur that you know has built a really successful business and you now want to find a way to manage your money the same way you’ve grown your company by building your own team, putting people together that are smarter than you can have expertise that’s greater than yours. You know usually you’re going to look at, probably a minimum budget of a million dollars a year to run that family office. So, you have to decide what’s the delta, that you need to get in returns to make up for that cost, or the value of control enough to pay the million dollars of fees to build your own team. And that’s really on the low end, if the question is really geared towards – “hey I’m a wealth manager, I have a couple of clients that have you know $5 million, $10 million, $20 million. Should I try and move up market to a family office”? It’s a little bit of a different question I think; it really depends on who you like to work with, what you’d like to do for them. Where does your passion live? And you know, do you really enjoy the total balance sheet for a family, and some of that off balance sheet stuff, which is interacting with the kids, coming up with the estate planning strategies, the family conversations around money, the more holistic approach to how is a family going to utilize their wealth over generations and incorporate philanthropy and there’re some broader discussions that are outside of analyzing investment portfolio.

Alina: Got it. And, you know, in addition to having that business profile you have to have the skill set. And if you don’t have that necessary skill set your family wants, and it sounds like it’s rather unique skill set as the person has to be approachable, easy-going, social and be able to work with family members and by the same token, they have to have the expertise necessary to offer the right advice.

Yeah.

We’re coming up to the end of our interview and we typically like to end by asking our interviewees this question. What if you were to look back in time when you were starting out: what would you recommend to your younger self. In other words, what would you do differently if you were starting out a family office again, knowing everything that you know today?

Noah: Yeah. I probably would have gotten started with real estate syndications sooner. So my wife and I bought our first two family house in 2000 – 20 years ago when we were first getting married, and we lived in half and we rented half and then, you know, a couple years later we took out our equity and we bought another two family house and we ran through that and then a couple years later we took out the equity and we did it again, and the plan was for me to acquire 20 units, and you know, build my career in accounting and 20 units of multifamily housing, pay off the mortgages, as we were working, and then I’d retire on 20 units worth of rental income. And I think that was a great plan, but it wasn’t a great plan for me. And really my plan should have started with real estate syndication. It’s much better suited to my skills. I ended up getting started in syndications around 2011-2012. So, I would have had 12 more years of experience in syndication, and I think that was a missed opportunity.

Alina: Have you heard of real estate syndications in 2000, when you were starting out?

Noah: Yeah, I mean. So I started out with the real estate, a concept you know in part from the books I was reading, in part from our accounting office: my clients and the clients that my dad had worked with when I was growing up and you know you kind of knew, like, oh so and so you know he’s a wealthy guy, he owns a bunch of apartments, you know, and I started seeing like a lot of the wealthy guys have apartment buildings, okay there must be something to it, but none of them were in the syndication business. So, I didn’t see it from a professional standpoint, but I just knew of it, but I didn’t really connect the dots as to like I had a client who was in the syndication business. And that’s how they became wealthy.

Alina: Yeah, yeah. And the point I was trying to bring across, I don’t think syndications have been as common in the early 2000’s as they are today. I think the trend really picked up maybe in the last five to ten years.

Noah, this is amazing content! Thank you so much. This concludes our interview for the day. Thank you for your input. I’m sure our audience appreciates all the knowledge that you shared with us today.

Noah: Yeah, well, thank you for reaching out to me on LinkedIn and getting me on and I look forward to seeing the results of the interview.

Alina: Thank you for your invaluable interview Noah!