1-4 Family Rider zoning change

Hello BP! Hoping someone out there has some similar experience w/ this, although I recognize this may be a unique situation. About 4 years ago my wife & I purchased a 6 unit building in chicago using a FHA 203k loan - full rehab. In order to use the 203k we had to de-convert the bldg to 4 units. At the end of the day we were left w/ a gutted unit that is being used as a storage area/laundry room. 2 years ago we re-fi'd into a conventional loan to lose the PMI. We recently looked into converting that storage space back into a unit & we have a path forward, but it would require a zoning change. Our bldg is currently zoned manufacturing, but is & has always been a residential bldg. I was recently reviewing the loan doc's & realized that there was a clause on the 1-4 family rider that said that we can not change the zoning on the property w/out the permission of the lender. Has anyone ever done this? Is it a big deal to get this permission? Would converting the bldg to 5 units create any issues for our conventional loan? Thanks & looking forward to the feedback!