I purchased my first home using a VA loan. It is a duplex and I live in half while renting out the other. I have been doing this for about 3 years.

I want to expand my portfolio by using another government loan to buy a 4 plex(also a VA that my lender found a way to qualify me for). I am supposed to live there but my question is can I avoid living in it as long as I keep one unit vacant or some other means?

Other ideas I have come up with are getting utilities, mail, insurance, and even moving some things into a unit. 

I know the spirit of the law implies I physically live there, but does the letter of the law say where I have to sleep as long as it is set up for me ?

It also says only myself OR a spouse have to live there. so if I "lived" there and just visited my wife a lot is that fraud?

I know you all are not stupid and neither am I, I am trying to LEGALLY do this. I am not interested in committing fraud but as long as I am skirting the right side of the law I am ok with it.

If I switch jobs and am forced to drive an hour or so would this also be a way to avoid living there?

How I am reading it, and I very well could be wrong, as long as I dont rent one out I could be ok.

Thanks for your help.