Second Primary Residence Loan

[Michael Herr]

I live in a home purchased(before starting investing) , titled and mortgaged jointly with my wife.

We are moving to a neighboring County.

I would like to leave the existing(owner occ) financing in place when I rent it out.

I would like my wife to be the sole owner of the new house and get the loan in her name only. (so it doesn't count towards my fannie/freddie property limit) - she is on board.

For IRS purposes the new house will become our primary residence.

Is there anything illegal/unethical about having two houses with primary residence financing? Or am I just at risk for the bank calling the note due on sale clause?

I don't want to ask either bank in case its a dont ask don't tell sort of thing.

Note: new loan is a different bank different county.