You have the deed? You have ownership. Who knows when that sign was put up?
Can you find the current address of the previous owner? Talk to a local real estate attorney about starting an eviction for anyone currently living there.
Be safe, take at least one intimidating friend if you decide to knock on the door. Good luck!