Modular Home - Build to Sell

I am exploring the possibility of purchasing an off market plot of land, building a modular home, and then selling it. This would be in central Jersey which is still a hot market as a result of record low inventory.

I am planning to do this strategy since I want to put low money down (10%) on a construction loan and achieve a quick build to lower my holding fees. 

One of my concerns is that lenders won't lend to potential buyers because it is a modular home. Has anybody tried this strategy? Are there any gotchas I should be aware of?