Your best bet would be to look at smaller credit unions or banks who would have more flexible loan guidelines.

@Adam KuszczakIf you're firm on moving out of the property in 2-5 years you could run into issues if the market takes a turn.Timing is everything and flexibility will save you.

A patient one that is proactive in answering your questions and has flexible loan products to fit your needs.

I agree with Henry's point about having the flexibility to combine units for one user because of the clear heights. 

In this way you keep your flexibility to separate if desired. 

This is a deal with some I know personally and are flexible.  

Since then, I've determined that I'm going to forgo the live-in property (not by choice, necessarily) and shoot for a pure investment property.With that said, I'm pretty flexible on my criteria in order to achieve the highest cashflow property possible.

Utilizing ADU laws, have some flexibility in the area you're targeting will go a long way.

You will find more flexibility on the commercial side, but rates will be higher, lower LTV, and often shorter amortization and term.  

I think being flexible with our strategy and focusing on the type of deal we are looking at is such great advice!