Boat running a bit rough

Boat: 2007 Sanger V230 5.7 Black Scorpion engine with 125.6 hours.

Problem: It's idling at lower RPM's (600 instead of around 800) then normal, there is new smell to the exhaust (maybe unburned fuel?), and at surfing speeds (2300-2500RPM) it just feels different (almost like a car with fouled plugs)

These are just small things and the boat is still running OK, but I would like to get to the bottom of this problem(s). The differences I was noticing today are so slight that no one else caught on until I told them and they drove it.

Thanks in advance!

I have the same motor and it normally idles about 640 when warm. The differences you describe can be ignition or fuel, or a sensor starting to go. You can eliminate a few possibilities by some maintenance that won't hurt or cost much.
Add some injection cleaner and change the fuel filter to clear up a possible stuck or fouled injector or water in the fuel. Check the rotor and cap for any signs of arcing, or just replace the rotor. Look for loose plugs on sensors, and check the voltage while the engine is running. It should be above 13.7 and below 15.3. Get someone to run the ECM to look for codes.
Change the plugs is the next on the list and make sure they're gapped properly.

I had that engine in a 05 Centurion, 2 different times the distributor cap had corrosion that caused similar symptoms.