Buying investment Property after Foreclosure(4+ years)

I have foreclosed my house in Apr'2010. I am looking for buy an investment property in the range of around 250-300k. I am ready to put down 20% and I have sufficient income to pay the mortgage and other expenses for the investment property. 

Every bank/broker, I talked to say that, I have to wait for "7-years", whereas it is only 4.5 years since my foreclosure.

As per the research, my understanding is that, my only option will be with "portfolio loan lenders"

What are my options for me to buy an investment property, considering the foreclosure. 

Also, I am first-time rental property buyer.

Any suggestions?

Thanks