It's a numbers game.
Leverage = risk; what is your risk tolerance?
Leverage can amplify your returns or losses.
IMO, you want your money to be working for you. Equity is is generally dead money. If you refi to 30 years at 3.75% (loan constant around 5.5%), then take cash out to buy another income producing asset. Based on the above example, that asset should earn more then 5.5% plus a risk premium to make it worth while for you to do so. Or you could possibly take that money and work with a hard money broker to loan your money and make 9-12% interest. Is making 3.5%+ on the spread a worthwhile risk premium?
Also, make sure you consider all your assets in your portfolio and determine if it's within your overall risk tolerance. IE, stocks, bonds, money market, real estate, life insurance, etc. Take a wholistic approach. If buy taking on additional risk, you can't sleep at night then its not worth it.
I'm 36 and had some of the same concerns you do. This place had really helped me to put together a game plan. Good luck!