A common problem encountered when making boat-related games or really any games that contain a big water surface is hiding that water surface when something is floating on it. I’ll describe the approach I use in my game Sail Forth, in Unity3D terms, but the technique should be applicable anywhere.
Since water in most games is just a big plane, it makes sense that if anything was floating in it, the water surface would intersect it!
So how do we fix this? There are 2 main approaches I know of: one is based around deforming the water mesh below the hull of the boat, and another involves masking out the water surface within the boat interior. We’ll be focusing on the second approach because it is the one I know how to do!
There are 3 components to this solution:
- Making a ‘mask’ mesh for each boat
- Writing a shader for the ‘mask’ mesh
- Modifying the water shader to use the mask
The Mask Mesh
First we need a mesh for our boat that we can use to mask out the water surface. I hand-author a mesh for each boat, depending on your situation you may be able to use a single generic mesh.
A good way to create this mesh is to take the edges along the rim of your boat’s interior and fill it in. It’s important that this mesh be either a separate object or have its own material in the engine, so that we can assign it the masking material.
The Mask Shader
Now that we have the mask mesh, we need a shader and material that will hide the water surface wherever the mask is drawn. At first, we’ll accomplish this purely with the depth buffer.
For now, this is the whole shader for the water mask! To sum it up: it renders after all opaque geometry (our boat, for example), before the water, and it doesn’t write any colors but does write to depth. This last part means you won’t be able to see it, but it does occlude things behind it. It doesn’t occlude the boat itself because it is drawn after the boat.
If we apply that shader to our mask mesh, assuming the water’s render queue is after the mask render queue, you’ll see it’s already working!
All that’s happening here is the water is being obscured by our invisible mask mesh, in the same way it’s obscured by the other parts of the boat.
This on it’s own might be enough of a solution for your situation, and it was for me for a few years of development. However, there is one problem:
What’s happening here? As our boat bobs up and down, sometimes part of the waves rise above the top of the boat. This means the water is closer to the camera than the mask, so it passes the Z test and gets rendered.
You might consider this to be technically correct, as that part of the boat is literally underwater, so perhaps the water should be shown! You might also argue that the physics should be fixed such that the boat never goes below the water surface, but this can be quite tricky to tune. Fortunately there’s a way to fix our shader so that we never see the water inside the boat again!
The Stencil Buffer
If you’re unfamiliar with the Stencil Buffer, you can think of it as basically another screen you can draw to that contains just integers instead of colors. Shaders can specify what value to write to the stencil buffer, and also can specify a comparison operation that prevents the shader from drawing unless the stencil buffer matches the referenced value. It’s like an auxiliary depth buffer that you can choose any value to write to.
We’re going to modify our masking shader to utilize the stencil buffer, which will also require modifying our water shader.
This is the exact same shader, but with the addition of the stencil block at the bottom. The stencil syntax can be kind of confusing to understand, so I’ll try to sum up what’s happening there:
Stencil - This is just signifying that we’ll be doing a stencil operation in this shader pass Ref 1 - The stencil value we'll be referencing is 1Comp always - When we look at the current stencil value, our shader should always draw its pixel regardless of the stencil value.Pass replace - When we draw a pixel, we should replace the current stencil value with our value, aka '1'.
So, the result of this shader running should be that the stencil buffer will contain ‘1’s at each pixel where our object was drawn.
Now, we need to use this stencil information in our water shader.
Masking in the Water Shader
Water shaders can be pretty complicated, so I’m going to leave out everything related to that except for the relevant stencil part. Presumably whatever you’re using for your water has some kind of custom shader associated with it, so you’ll just need to edit that shader and insert this Stencil block.
This is pretty similar to the stencil code we had in our masking shader, but with slightly different parameters.
We’re still referencing the value of 1, but our goal here is to not render the water if the stencil buffer is equal to 1, because we know that it is 1 everywhere our masking object is.
For that reason, we make the Comp parameter ‘notequal’, which should be pretty self explanatory. If the stencil value is not 1, the stencil test will pass.
It doesn’t really matter what we do with the stencil value if the test passes, as we aren’t using it anywhere else, so I specified to ‘keep’ the stencil value if the test passes.
With that change, you can see the stencil buffer in action! Here I’m moving the boat up and down, far below the water surface, but you can see the water is never drawn over the boat interior.
This does bring up another issue: What if the boat sinks? My solution to this is to disable the water mask renderer as soon as the boat begins sinking. Otherwise, I assume there is no reason why you’d want to see the water drawn over the boat’s interior.
Edit #1: Another issue, similar to the sinking one, is brought up in this nice article: https://simonschreibt.de/gat/black-flag-waterplane/. A tall wave could get between the boat and the camera, which would then be incorrectly stenciled out causing an ugly artifact. Depending on your use case this may never occur, or might be very noticeable.
My solution to this for now, is to toggle between the initial depth-mask implementation and the stencil-mask based on distance to the camera. This reduces the problem decently, depending on your use case.
At close camera distances, it’s very unlikely that a wave could get between the boat and the camera, so the more accurate stencil approach works well.
At far camera distances, a wave might be between the camera and the boat, so the depth-mask approach is needed to avoid ugly artifacts. The bright side here is that at farther distances it’s less important to prevent any water showing over the top of the boat as it will naturally be smaller on the screen and less noticeable!
That’s it for this tutorial, hopefully this has helped to explain the power of the stencil buffer!