Some, like water striders, use appendages they insert below the surface for propulsion. At 0:49 in the montage above, you can see flow visualization of the vortices generated by a stroke.