100% seller financing, where is the downpayment kept?

[Dave S.]

I'm looking to purchase a few units with 100% seller financing. The deal is that once the principle hits the necessary 20%, that I would refi out. My question is where should the downpayment be kept till then? Should he hold onto it? What happens if he spends it? Or at closing he spells me the property at x $, would he be able to put money down on the deal for me? Not sure how that would work? Please advise Thanks