Shane, welcome to BP, like seeing all the new folks tonight!
The property is most likely zoned agricultural, and with the acreage that property is not a single family secondary market financing property, it's ag/commercial. Now, you could survey off the house and say 5 acres, what might be customary for that area for an appraisal to like properties and finance that in the secondary market.
Another option is to look at USDA, the house and acreage will likely fit thier lending requirements, similar financing to the residential loans you illuded to. Programs vary from state to state, so you'll need to check with them.
As to the property and calling it a second home, second homes are basically considered when two issues can be applied. 1. There is some recreational activity in the area that draws people for vacations, they are deemed to be vacation areas, but some exceptions can be justified, and 2. They are more than fifty miles away from your pricnipal residence. This factors are things underwriters will look at to justify the second home nature of the loan to ensure that someone is not claiming a second home as an investment property. Exceptions can be made, like the kids to live in while going to college or near elderly parents for example. I'dsuggest you not try for the second home ploy, that can backfire on you later on. You could have plans to move in after repairs and sell your home, or rent it out.
Your property also needs repairs, getting a good appraisal may be hard without repairs knocked out.
Another issue using the home as an anchor for the loan is that the value of the land exceeds 25% of the home, in fact I'm sure from what you said, the land value exceeds the value of all improvements, unless that is really some barn! So, again, it is an Ag loan.
Seller financing is customary for properties like the one you're interested in, mainly due to the fact that financing can be hard to obtain, but more so because of taxes for the seller.
Farms are generally held for longer terms, I did a deal on a farm that had been in the family since the civil war and was successful when hundreds of other agents, developers and buyer were not because of my seller financing and tax avoidance tactics. Never mind me, point is that they will likely have a very low basis and if they sell for cash the tax man will have his hand out. By seller financing the deal, the sellers pay taxes as the profit from the deal is received, not at the time of the sale. And then there is the fact that the loan will probably be at a higher rate of interest than they can probably get in another investment with similar risk and management concerns.
So, in this case, I'd suggest one of two things or a combination of the two, USDA financing and/or seller financing. You can read here on BP concerning seller financing issues for both buyer and seller.
And, then you could always head to a bank for a straight Ag loan, but I wouldn't go there unless USDA can't do the deal. Even with the bank, you can still incorporate seller financing.
Good luck... :)