Water in OpenGL and GLES 2.0: Part 4 - Blending it all together
If you've been following the previous three parts of this article then by now you must be itching to see how the fruits of your labour are going to look, so let's dive right in. To get a hint of the final outcome we can modify the watersample.frag file so that gl_FragColor is a straight blend of the reflection and refraction images:
gl_FragColour = mix(refractionColour, reflectionColour, 0.5);
This performs a 50/50 mix of the two images, with a final result which looks like a slightly odd frozen lake.
Adding Detail
This is nice, but we can do better! For one thing the amount of blending each fragment receives should vary based on the perceived angle of the camera's eye position relative to any given point on the water plane. That is, the more directly we look at the water, the more transparent, and the more of the refraction image should be shown, and, conversely, the shallower the angle of observation the more reflective the surface should be. This is done by approximating the Fresnel term, a floating point number calculated for any given fragment, which replaces the constant 0.5 value in the mix() function. There are a variety of methods of doing this, all of which (as far as I can tell) require a normal vector representing the water's surface normal at any given point - so that we can measure the angle between the camera's viewpoint and the fragment by taking the dot product of the eye position with the normal vector. To start with we could use a single up facing vector which represents the entirety of the plane, but here is a good opportunity to add some extra detail to the water's surface.
Using a normal map we can store a whole range of normal vectors, mapped across the surface of the plane, each representing a slightly different angle producing a perturbation of the surface. As an added bonus the red and green channels of the normal map can be used to create a slight distortion in both the reflection and refraction images, adding another level of detail.
Normal Mapping
To map the normal texture to the water plane we need to do some modifications to the watersample shaders. First we need to add the texture coordinate attribute a_texCoord to the vertex shader, which is automatically passed in by Gameplay. Then in the main() function pass the value directly to a new varying variable v_texCoord so that it is available in the fragment shader. As well as adding the new v_texCoord to the fragment shader, we also need to add a sampler uniform u_normalMap so that we can pass in the normal texture. To bind the actual texture to the uniform we don't actually need to do anything in the project's code. Gameplay provides a nice auto-binding mechanism, allowing us to pass the texture in simply by editing the watersample.material file. Add
sampler u_normalMap
{
mipmap = true
wrapS = REPEAT
wrapT = REPEAT
minFilter = LINEAR_MIPMAP_LINEAR
magFilter = LINEAR
path = res/images/water_normal.png
}
to the material water definition, or look at the article source code for part four. Assuming the path points to a valid image file the texture will automatically be loaded and bound to the shader when the program starts. Once this is all set up we can return to the fragment shader, and start using the normal data stored in the texture.
Immediately in the main() function we sample the normal map, and convert it to normalised values:
vec4 normal = texture2D(u_normalMap, v_texCoord * textureRepeat);
normal = normalize(normal * 2.0 - 1.0);
textureRepeat is a constant value which allows tiling of the texture to better fit the water plane. Set it to 2.0 to make the texture repeat twice in both the S and T direction, 12.5 to make it repeat 12.5 times and so on. Before we start calculating any reflection and blend parameters, let's add some distortion to the output.
//distortion offset
vec4 dudv = normal * distortAmount;
//refraction sample
vec2 textureCoord = fromClipSpace(v_vertexRefractionPosition) + dudv.rg;
textureCoord = clamp(textureCoord, 0.001, 0.999);
distortAmount reduces the amount of distortion added, as too much can easily ruin the effect, and is typically a small number such as 0.05. The red and green values of dudv are then added to the texture coordinates, offsetting them slightly, before clamping the coordinates within a reasonable range. The refraction texture is then sampled in the normal way with the newly offset coordinates, and the process repeated for the reflection texture. The output should now be a nice wavy distorted image (assuming you're using the normal map texture supplied with the article source. You can use any normal map texture you like).
Fresnel Calculation
After the reflection and refraction textures have been sampled, we are now ready to approximate the fresnel value, and use it to blend the textures together. To do this we need the eye position relative to the current vertex, so we can take at dot product of it with the current normal value. The watersample vertex shader needs two new uniform variables
uniform mat4 u_worldMatrix;
uniform vec3 u_cameraPosition;
and a new varying
varying vec3 v_eyePosition;
so that the calculated position can be passed along to the fragment shader. Gameplay provides the worldMatrix and cameraPosition values for us as standard, and we can auto bind these in the material file the same way as we did the normal map, which saves having to modify the project code:
u_worldMatrix = WORLD_MATRIX
u_cameraPosition = CAMERA_WORLD_POSITION
Then, in the main() function of the vertex shader, we can calculate the eye position
v_eyePosition = u_cameraPosition - (u_worldMatrix * a_position).xyz;
With the eye position available in the fragment shader we can begin to use it to calculate the fresnel value. Before we can use it, however, the eye position needs to be converted to the tangent space coordinates used by the normal map (or we could just use an object space normal texture - but that would upset the distortion factor). Due to the fact the water plane is fixed horizontally we can use a set of constant vectors to represent the plane's normal, tangent and bitangent vectors (if the plane was oriented in any other way we'd probably have to pass these values in either as an attribute or a uniform value), and use them to move the eye position into tangent space
const vec4 tangent = vec4(1.0, 0.0, 0.0, 0.0);
const vec4 viewNormal = vec4(0.0, 1.0, 0.0, 0.0);
const vec4 bitangent = vec4(0.0, 0.0, 1.0, 0.0);
vec4 viewDir = normalize(vec4(v_eyePosition, 1.0));
vec4 viewTanSpace = normalize(vec4(dot(viewDir, tangent), dot(viewDir, bitangent), dot(viewDir, viewNormal), 1.0));
then create a reflected vector of the view and dot it with the normal to get our approximated fresnel term
vec4 viewReflection = normalize(reflect(-1.0 * viewTanSpace, normal));
float fresnel = dot(normal, viewReflection);
we now have our value to feed into the mix function:
gl_FragColor = mix(reflectionColour, refractionColour, fresnel);
Load up the scene and you should see the water really beginning to take shape. Moving around the scene you'll notice the blending of the reflection and refraction map change to match your view.
Animating the Surface
One thing is still not right though, and that is the fact that the water is still apparently frozen. We can change this with a simple new uniform in the fragment shader
uniform float u_time;
This is simply going to be a floating point value which increases over time. In the article's source folder there is a small utility class called Timer, which abstracts the Gameplay clock, although you can use getGameTime() directly if you prefer. Create a private const function to return its value, preferably divided by some amount (else the animation will run waaay too fast), and use it to bind the elapsed time to the new shader uniform. In the fragment shader add the time to the coordinates of the normal map look up.
vec4 normal = texture2D(u_normalMap, v_texCoord * textureRepeat + u_time);
This will have the effect of offsetting the normal map texture, scrolling it across the surface of the plane, and creating a simple yet pleasing animation. If you get odd stretched lines across the surface make sure to check that the sampler settings in your water material have wrapS and wrapT set to repeat.
Conclusion
That pretty much sums up what I set out to describe in this article, but there is plenty more which could be added to improve the effect. For instance no lighting is taken into account in the fragment shader, which, once added, could also be used in conjunction with the normal map to calculate specular highlights on the surface of the water. The water also looks very clean too. It is entirely possible to calculate the depth of the water and blend it with a colour so that it appears darker and murkier the the deeper you go.
Here's a short video of the final version of the project, and the water effect running on my Moto G with Android 4.4.2
References:
Eric Pacelli
Lauris Kaplinski
Riemer's XNA page
Source Code:
Github page
Previous Parts:
Part One
Part Two
Part Three
gl_FragColour = mix(refractionColour, reflectionColour, 0.5);
This performs a 50/50 mix of the two images, with a final result which looks like a slightly odd frozen lake.
Adding Detail
This is nice, but we can do better! For one thing the amount of blending each fragment receives should vary based on the perceived angle of the camera's eye position relative to any given point on the water plane. That is, the more directly we look at the water, the more transparent, and the more of the refraction image should be shown, and, conversely, the shallower the angle of observation the more reflective the surface should be. This is done by approximating the Fresnel term, a floating point number calculated for any given fragment, which replaces the constant 0.5 value in the mix() function. There are a variety of methods of doing this, all of which (as far as I can tell) require a normal vector representing the water's surface normal at any given point - so that we can measure the angle between the camera's viewpoint and the fragment by taking the dot product of the eye position with the normal vector. To start with we could use a single up facing vector which represents the entirety of the plane, but here is a good opportunity to add some extra detail to the water's surface.
Using a normal map we can store a whole range of normal vectors, mapped across the surface of the plane, each representing a slightly different angle producing a perturbation of the surface. As an added bonus the red and green channels of the normal map can be used to create a slight distortion in both the reflection and refraction images, adding another level of detail.
Normal Mapping
To map the normal texture to the water plane we need to do some modifications to the watersample shaders. First we need to add the texture coordinate attribute a_texCoord to the vertex shader, which is automatically passed in by Gameplay. Then in the main() function pass the value directly to a new varying variable v_texCoord so that it is available in the fragment shader. As well as adding the new v_texCoord to the fragment shader, we also need to add a sampler uniform u_normalMap so that we can pass in the normal texture. To bind the actual texture to the uniform we don't actually need to do anything in the project's code. Gameplay provides a nice auto-binding mechanism, allowing us to pass the texture in simply by editing the watersample.material file. Add
sampler u_normalMap
{
mipmap = true
wrapS = REPEAT
wrapT = REPEAT
minFilter = LINEAR_MIPMAP_LINEAR
magFilter = LINEAR
path = res/images/water_normal.png
}
to the material water definition, or look at the article source code for part four. Assuming the path points to a valid image file the texture will automatically be loaded and bound to the shader when the program starts. Once this is all set up we can return to the fragment shader, and start using the normal data stored in the texture.
Immediately in the main() function we sample the normal map, and convert it to normalised values:
vec4 normal = texture2D(u_normalMap, v_texCoord * textureRepeat);
normal = normalize(normal * 2.0 - 1.0);
textureRepeat is a constant value which allows tiling of the texture to better fit the water plane. Set it to 2.0 to make the texture repeat twice in both the S and T direction, 12.5 to make it repeat 12.5 times and so on. Before we start calculating any reflection and blend parameters, let's add some distortion to the output.
//distortion offset
vec4 dudv = normal * distortAmount;
//refraction sample
vec2 textureCoord = fromClipSpace(v_vertexRefractionPosition) + dudv.rg;
textureCoord = clamp(textureCoord, 0.001, 0.999);
distortAmount reduces the amount of distortion added, as too much can easily ruin the effect, and is typically a small number such as 0.05. The red and green values of dudv are then added to the texture coordinates, offsetting them slightly, before clamping the coordinates within a reasonable range. The refraction texture is then sampled in the normal way with the newly offset coordinates, and the process repeated for the reflection texture. The output should now be a nice wavy distorted image (assuming you're using the normal map texture supplied with the article source. You can use any normal map texture you like).
Fresnel Calculation
After the reflection and refraction textures have been sampled, we are now ready to approximate the fresnel value, and use it to blend the textures together. To do this we need the eye position relative to the current vertex, so we can take at dot product of it with the current normal value. The watersample vertex shader needs two new uniform variables
uniform mat4 u_worldMatrix;
uniform vec3 u_cameraPosition;
and a new varying
varying vec3 v_eyePosition;
so that the calculated position can be passed along to the fragment shader. Gameplay provides the worldMatrix and cameraPosition values for us as standard, and we can auto bind these in the material file the same way as we did the normal map, which saves having to modify the project code:
u_worldMatrix = WORLD_MATRIX
u_cameraPosition = CAMERA_WORLD_POSITION
Then, in the main() function of the vertex shader, we can calculate the eye position
v_eyePosition = u_cameraPosition - (u_worldMatrix * a_position).xyz;
With the eye position available in the fragment shader we can begin to use it to calculate the fresnel value. Before we can use it, however, the eye position needs to be converted to the tangent space coordinates used by the normal map (or we could just use an object space normal texture - but that would upset the distortion factor). Due to the fact the water plane is fixed horizontally we can use a set of constant vectors to represent the plane's normal, tangent and bitangent vectors (if the plane was oriented in any other way we'd probably have to pass these values in either as an attribute or a uniform value), and use them to move the eye position into tangent space
const vec4 tangent = vec4(1.0, 0.0, 0.0, 0.0);
const vec4 viewNormal = vec4(0.0, 1.0, 0.0, 0.0);
const vec4 bitangent = vec4(0.0, 0.0, 1.0, 0.0);
vec4 viewDir = normalize(vec4(v_eyePosition, 1.0));
vec4 viewTanSpace = normalize(vec4(dot(viewDir, tangent), dot(viewDir, bitangent), dot(viewDir, viewNormal), 1.0));
then create a reflected vector of the view and dot it with the normal to get our approximated fresnel term
vec4 viewReflection = normalize(reflect(-1.0 * viewTanSpace, normal));
float fresnel = dot(normal, viewReflection);
we now have our value to feed into the mix function:
gl_FragColor = mix(reflectionColour, refractionColour, fresnel);
Load up the scene and you should see the water really beginning to take shape. Moving around the scene you'll notice the blending of the reflection and refraction map change to match your view.
Animating the Surface
One thing is still not right though, and that is the fact that the water is still apparently frozen. We can change this with a simple new uniform in the fragment shader
uniform float u_time;
This is simply going to be a floating point value which increases over time. In the article's source folder there is a small utility class called Timer, which abstracts the Gameplay clock, although you can use getGameTime() directly if you prefer. Create a private const function to return its value, preferably divided by some amount (else the animation will run waaay too fast), and use it to bind the elapsed time to the new shader uniform. In the fragment shader add the time to the coordinates of the normal map look up.
vec4 normal = texture2D(u_normalMap, v_texCoord * textureRepeat + u_time);
This will have the effect of offsetting the normal map texture, scrolling it across the surface of the plane, and creating a simple yet pleasing animation. If you get odd stretched lines across the surface make sure to check that the sampler settings in your water material have wrapS and wrapT set to repeat.
Conclusion
That pretty much sums up what I set out to describe in this article, but there is plenty more which could be added to improve the effect. For instance no lighting is taken into account in the fragment shader, which, once added, could also be used in conjunction with the normal map to calculate specular highlights on the surface of the water. The water also looks very clean too. It is entirely possible to calculate the depth of the water and blend it with a colour so that it appears darker and murkier the the deeper you go.
Here's a short video of the final version of the project, and the water effect running on my Moto G with Android 4.4.2
References:
Eric Pacelli
Lauris Kaplinski
Riemer's XNA page
Source Code:
Github page
Previous Parts:
Part One
Part Two
Part Three
Comments
Post a Comment