A listing Broker, like real estate agents, will frequently split the commission with the buyers broker. I've heard of some brokers that will not actively sell the boat, but instead for a reduced commission list the boat and handle escrow. This is effectively a FSBO deal and they will take a few %. As stated, like in million dollar real-estate, yachts will sometimes work on a lower commission rate.
For a typical recreational boat, I would be cautious about attempting to negotiate the brokerage rate. It would need to be done at the time you're signing a contract with the listing broker. If he agrees to a reduced rate, I wouldn't expect him to work very hard on the sale.