It seems like all of the multi family houses I’m looking at are being won by all cash offers (not confirmed but a guess by my REA). Any tips on how to beat them out if I’m using traditional financing via a mortgage.

It could be they are just offering less.  That doesn't mean you should though.

Strong loan commitment letter from reputable lender with history of closing, offer fast close, offer to pay closing costs and transfer taxes, forego inspections, try to develop some personal level interaction with seller.

Forgot larger than normal EMD.