Has anyone had any experience working with small neighborhood associations to get covenants and restrictions amended?

We have a piece of land in a great location of San Antonio under contract to purchase, it's approx. 1.5 acres.  It has never been developed due to no infrastructure (sewer and a street need to be put in) but home values have increased significantly in the area and the project would now be viable even with this massive cost. 

Our title co. found a set of restrictions and covenants put in place in 2011 when an old developer owned the property.  The parties on the documents are the declarant (developer) and a tiny neighborhood association that appears to be just one guy.  His name is all over it saying nothing can change without his approval.  

The biggest kicker is that it says the lots have to be double the size of what we expected, cutting our total number of buildable units in half and thus making the expense far outweigh the benefit.  

We've called the guy multiple times and he's only answered once, with a gruff and whatever attitude toward the whole thing.  We went by his house just a block from this land and tried to have a talk with him but he just wanted us to leave a bunch of paperwork with him and he'd get to it "in a few weeks".  

Any ideas on how to motivate someone in this scenario that has no real motive or interest in helping us?  

The prior developers should never have agreed to these restrictions when they rezoned from MF-33 to residential zoning as the numbers simply do not pan out.  That much is obvious because they never built and eventually they were foreclosed on and had a huge lien on the property from the old engineer they worked with. 

I've consulted with a few attorneys and there really is nothing we can do outside of working with this individual.