What should it cost to setup a Solo 401k? And what should the annual fees be?
Prices vary quite a bit. I'd suggest calling a few providers to see what seems like a fair price to you given the company's expertise, experience, and reputation in the industry.
Is this something I could setup myself with a brokerage house such as TD Ameritrade?
Yes, you could open a brokerage Solo 401k plan if you do not want to invest into alternative assets and don't want many features in the plan. Many brokerages don't offer Roth contributions or participant loans, for example. After-tax contributions and mega backdoor Roth capability typically only come with plans setup by specialty firms. Same with the ability to invest in real estate.
Unless you want to self-direct the investments into alternative investments, opening a solo 401k plan with a typical brokerage firm will suffice and they will charge fees based on the underlying mutual fund fees which can amount to hundreds of dollars depending on the value of your account.