If you have an old furnace with existing duct-work then likely a 1 for 1 replacement of the furnace makes the most sense. 2.

@Jenny Gremillion is it forced heat right now and you have ductwork?  

There is no ductwork and no evidence of hvac of any kind having been there.

Also the overhead piping and duct work.

This is for the unit only, not sure about duct work.

However as I understand it, the fittings don't restrict flow as much as would be assumed by first glance and it's sort of calculated similar to air flow in duct work.... a 90 degree elbow fitting can be roughly accounted as the same as x ft of straight run in terms of how much pressure is lost.

White cloth tape at ductworks, mastic, 9x9 vct tile are items to look for that contain asbestos.

Total price is $3,900 for a 3 ton unit with copper, furnace, compressor, ductwork and everything and 10 year warranty.

I cannot afford to replace duct works every few years.And as I am only have to provide heat to the unit, I am looking to substitute the HVAC with a gas wall heater. 

If you have no radiators and can't use space heaters, you'll have to install radiators or figure out another radiant heating system -- I assume that you don't want to add duct work and reconfigure furnances/boilers.