Duplex multifamily on R-4 zoning

Hi experts, 

i'm planning to purchase my first property and which is multifamily property. Currently the property having two units with 600 SF each units with lot size of 7500 SF. Property zoning is R-4 which allow to build upto 4 units. 
If property value is $950K and getting current rent of $4K month and by adding two more units rent could be at $10K a month. Is it worth it to construct two units? 
What is your recommendations and this is my first property so, I'm little bit nervous.