Deal Question - Seller Financing: prepayment penalty

I have been working this deal for about 5 months. It is finally closing on Friday. The seller just realized that I could pay this off next month at a great price. He was also offering seller financing as a tax strategy. Now the seller wants to add a prepayment penalty of 3%, 2% and 1% for the 1st, 2nd and 3rd year, respectively. Which is really just $1,800, $1,200 and $600. 

Our initial contract calls for no prepayment penalty so I can probably hold him to it and he could try to blow the deal up for another reason. In reality, I expect he will grudgingly agree. However, I am signing up for a potential ten year relationship with this individual so I do not want to start with an arm wrestle.

I am just trying to get an outside perspective. For what it is worth I expect to hold this property for 5 -10 years, but we all know things change. 

I am leaning towards agreeing because I do not expect to sell within three years and the fee will not break the bank, but I remember from a podcast that the guest (I think Brian Burke) said to get the best deal you can and not to let up just because you have a good deal.

What are your thoughts?