From Rigs of Rods Wiki
What is a "Boat Engine"?
This engine is called a "Screwprop", which runs the Propeller, which is submerged to create a push, thus moving the boat
Setting up the chassis
To create a chassis, or setup for the screwprop, you must have 3 nodes in a triangular shape. Like so:
screwprops ;prop node,back node,top node,power 88,93,91,100000.0 89,92,90,100000.0
The settings entered there are borrowed from the Truck Description File section of the wiki
Once you have finished making the three nodes in a triangle and stabalizing it with beams, You should enter the "screwprops" section into your .boat file, then, you can have your own sea exploration! :-)