I would like to chime in because this is near and dear to my heart. I made the mistake of not touching my 457B for many years. I was misguided and under the worker bee delusion that many 9-5 workers are institutionalized. I kick myself in the butt because I could have deployed it a decade ago when the market was ripe for the picking, but I believed the finance guys the employers provided at their seminars.
I have done it. I regret not having done it sooner. Sometimes, it is that little extra that gets you over the hump to acquire/control a greater asset than you would have without withdrawing those funds.
Take a moment and think about all those numbers that have been thrown around. The funds you put into your 457B were pretaxed. Those funds grew in value on top of the regular allocations from your paycheck. All of it is yours. You borrow from YOURSELF. You pay back with interest TO YOU, granted they are posttax funds, but does it matter? You gained interest/value on pre-taxed funds that you just borrowed from. Then, on top of all that, you just controlled or improved some real estate with it. Unless you totally threw all REI strategies out the window, which I highly doubt since you are getting into the weeds of Def Comp, you will blow that 457B out of the water in due time. That $50K likely elevated your value by $100K-$500K, based on your payback term. On a side note, any tax you pay later in life when you draw out your 457B is really meaningless, because you still WIN either way. You are taxed on money that should have been taxed and you are taxed on gains of the untaxed dollars... you made out with a 457B account!
Here is my example that I just did, which I wish I had done years ago. At the end of last summer, I pulled $50K from my Def Comp (my max). I also had my wife pull $25K from hers (her max). I used it to assist with my downpayment and costs to stabilize 3 parcels, all acquired at the same time, totaling 9 units. I used 80% LTV. Improved the asset with some cosmetics, such as better signs for address numbers, locking mailboxes, minor landscape, and trash clean up. Just got some stuff from Home Depot and some scrap wood around the house. Shaped the wood really nicely and had my wife paint it so she feels like she contributed, which made her feel important/needed (HIGHEST ROI!!!!!!). I did have to install a few clean-outs, but its was necessary. I increased the rents as well as spruced up the curb appeal. Currently in the process to refi and cash out roughly $200K, ON TOP OF the original $280 I put down. In fact, $100K of that original $280K was not even my money, but that's another story.
All that occurred because I pulled some 457B loan out, from myself to myself. All within 1 year. There is no way that 457B can perform close to that. Don't even use a 30 year timeline for 457B, because in 30 years of reinvesting that money that was pulled out, there is a high likelihood that I would have turned it into $Ms through 1031s and depreciations, etc...