You are not allowed to use arrest records to determine a rental applicant’s qualifications.According to the U.S Department of Housing and Urban Development: Arrests records are not a valid reason to deny a rental applicant housing.Convicted criminals may be denied housing if the reason for their convictions clearly demonstrates that the safety of your residents and/or property are at risk.Blanket terms in your screening criteria that say “Any criminal convictions will be denied” are now considered discriminatory and in violation of the Fair Housing Act.If you deny a rental applicant due to information found on a public record (like a criminal background report or eviction search) you need to provide the applicant with a denial letter and instructions on how they can obtain a copy of the report.

To make matters worse, when I went to report him to the Sheriff, they sent notice to the building department, which has since sent me a letter ordering me to cease work on the apartment until I obtain necessary permits.

I would call your local building department to find out.  

(also make sure there isn't any hidden surprises on the property, my local Code Enforcement department, Planning and Development department... 

Having no security deposit nor a lease with this tenant I requested that he depart by the end of the month and I would pro rate the rent upon his departure.  

Brandon West - while Bram Spiero is correct that you can get another FHA loan if you refi your existing property out of FHA, there is a big caveat to that: you cannot use the rental income from the departing income as qualified income unless the new property is more than 100 miles away.

I would add a paragraph in the letter going to the departing resident that it is your understanding they will be leaving by ____, and ask them to notify you when they are ready for the walkthrough,  Remind them that you will need a forwarding address for sending the security deposit accounting.  

I would start by talking to the local zoning department to determine if legally you can subdivide and build a second house. 

We’ll be arriving April 28th, and departing May 5th on a cruise.

As far as the State lien you will have to go the revenue department for the state of CA and see what is required for them to release their right to redeem.