Ok, I'm really kind of disturbed by some of the discussion here because its so darn black and white. Yes there are people who it was the right decision to leave the military and invest, and there are those that stayed in and retired out, but there is absolutely no reason that you can't do both. At the end of the day, Edsiaren Ignacio, you need to ask yourself why you joined the military. If it was for "A" job, then what life situation changed that you don't need a job anymore and are able to take on REI full time? If it was for service to the country, then when you enlisted, how long did you see yourself serving.
As far as retirement goes, while it may not be a full time jobs worth, for 20 years of service, retiring at 41 is HUGE! It's literally free money for the rest of your life, yes you paid into that by working hard hours now, but its a really good deal, there are very few jobs that have a comparable retirement option anymore. Then their are the benefits, I think Bill Gulley adequately covered the additional benefits that you get that while inconvenient at times, are none the less benefits you get you otherwise wouldn't have.
Finally, Why can't you do both? As an E-4 I get that you might not be able to stay off base (assuming you aren't on sea duty) but since you are married I believe you are permitted to stay off base, why not have your non-taxable BAH work for you in the form of a mortgage, or even get creative and get a VA loan, get a 4 plex, become a landlord and have that supplement your income, all the while using your BAH to cover your housing costs. There is absolutely no reason you can't invest while you are in the military. It opens you up to many different markets, it allows you to travel if that is your thing, you can homestead if you find a place you want to call home for a while. I just really don't get why you have this preconceived notion that REI and Military are two separate and mutually exclusive things.
Caveat - I'm an officer, but all that means is I went to school first and am able to afford more from my base pay when looking at housing. If you are creative with your financing, and use the tools (like your VA loan) to your advantage, you have every opportunity regardless of rank, and for the most part us O-types don't get down-payment sized reenlistment bonuses for key career fields, so you might actually have a leg up in getting started on me.
Bottom line take aways - Edsiaren, you actually answered yourself in your post if you read between the lines. You asked "When should I start investing?" I think everyone here can agree with me when I say, "NOW". Investing isn't a job, it's saving for a purpose. If you are good at managing the money you have, you can live without a job, but that requires money to start out, though not always your own. Ultimately, you need money to invest, unless you get really creative, but even then, while you build your business, you need something to get you by. So I think the bigger question is "What is wrong with the pasture you are in?" because every pasture looks greener from where you are at, but right now you have a job, and there are a lot of folks that don't and would love for an opportunity to have what you do, but I agree, there are circumstances where it just isn't smart to keep on doing the status quot. I really hope you find what you are looking for, but either way, keep using BP as a tool and community to guide your path. This entire post can be taken as either a push for you to stay in, or a push for you to get out, but I ask you not to take it as either. Use it more to ask "why can't I do both?" and then the follow on, "why did I join the military, and why do I continue to serve?" and never forget, what you are doing ultimately is serving for a higher purpose, this isn't your typical 9-5 job.
Good luck!