Recommendations on building a more useful baluster for these stairs

We replaced the shoddy work of the previous homeowner and built these lovely steps.  From floor to top of landing is 32.5".  Rental compliance requires me to have a 34" Baluster on the right side of these steps.  My concern is its narrow already and putting a permanent baluster setup is going to make maneuvering furniture around the corner even more difficult.  Is there a good removable setup that is strong enough to maintain the structural integrity?   If not, the only other good and bullet proof option we have come up with is a permanent 2x4 wall with drywall with yellow pine cap.

I am aware that the gap needs to be closed as well.  A wall will solve this problem obviously and I can easily finish this out with sanded plywood.  Everything will be painted eventually.

Thoughts??