Borrowing Against an IRA???

[Dan Barman]

I've been hearing some murmurs lately about people using their IRAs as collateral (?) to "borrow" funds to use for down payments / rehab financing for real estate projects. A family member of mine passed away a few years ago and willed to me part of their IRA and Pension (now a "beneficiary IRA").

Is there was a way to borrow against / use these funds for real-estate projects without incurring the usual penalty for early withdrawal? What other tax issues / pitfalls should I look out for / be aware of if this is something I look to do? 

Thanks so much everyone!

-Dan