19 Tessellation & Terrains Written by Marius Horga & Caroline Begbie

You can unlock the rest of this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.

So far, you’ve used normal map trickery in the fragment function to show the fine details of your low poly models. To achieve a similar level of detail without using normal maps requires a change of model geometry by adding more vertices. The problem with adding more vertices is that when you send them to the GPU, it chokes up the pipeline. A hardware tessellator in the GPU can create vertices on the fly, adding a greater level of detail and thereby using fewer resources.

In this chapter, you’ll create a detailed terrain using a small number of points. You’ll send a flat ground plane with a grayscale texture describing the height, and the tessellator will create as many vertices as needed. The vertex function will then read the texture and displace (move) these new vertices vertically.

In this example, on the left side are the control points. On the right side, the tessellator creates extra vertices, with the number dependent on how close the control points are to the camera.

Tessellation

For tessellation, instead of sending vertices to the GPU, you send patches. These patches are made up of control points — a minimum of three for a triangle patch, or four for a quad patch. The tessellator can convert each quad patch into a certain number of triangles: up to 4,096 triangles on a recent iMac and 256 triangles on an iPhone that’s capable of tessellation.

Note: Tessellation is available on all Macs since 2012 and on iOS 10 GPU Family 3 and up. This includes the iPhone 6s and newer devices. However, Tessellation is not available on the iOS simulator.

With tessellation, you can:

• Send less data to the GPU. Because the GPU doesn’t store tessellated vertices in graphics memory, it’s more efficient on resources.
• Make low poly objects look less low poly by curving patches.
• Displace vertices for fine detail instead of using normal maps to fake it.
• Decide on the level of detail based on the distance from the camera. The closer an object is to the camera, the more vertices it contains.

The Starter Project

So that you can more easily understand the difference between rendering patches and rendering vertices, the starter project is a simplified renderer. All the rendering code is in Renderer.swift, with the pipeline state setup in Pipelines.swift.

Tessellation Patches

A patch consists of a certain number of control points, generally:

Tessellation Factors

For each patch, you need to specify inside edge factors and outside edge factors. The four-point patch in the following image shows different edge factors for each edge — specified as `[2, 4, 8, 16]` — and two different inside factors — specified as `[8, 16]`, for horizontal and vertical respectively.

``````let patches = (horizontal: 1, vertical: 1)
var patchCount: Int {
patches.horizontal * patches.vertical
}
``````
``````var edgeFactors: [Float] = [4]
var insideFactors: [Float] = [4]
``````
``````lazy var tessellationFactorsBuffer: MTLBuffer? = {
// 1
let count = patchCount * (4 + 2)
// 2
let size = count * MemoryLayout<Float>.size / 2
return Renderer.device.makeBuffer(
length: size,
options: .storageModePrivate)
}()
``````

Setting Up the Patch Data

Instead of an array of six vertices, you’ll create a four-point patch with control points at the corners.

``````var controlPointsBuffer: MTLBuffer?
``````
``````let controlPoints = Quad.createControlPoints(
patches: patches,
size: (2, 2))
controlPointsBuffer =
Renderer.device.makeBuffer(
bytes: controlPoints,
length: MemoryLayout<float3>.stride * controlPoints.count)
``````

Set Up the Render Pipeline State

You can configure the tessellator by changing the pipeline state properties. Until now, you’ve processed only vertices with the vertex descriptor. However, you’ll now modify the vertex descriptor so it processes patches instead.

``````vertexDescriptor.layouts[0].stepFunction = .perPatchControlPoint
``````

The Tessellation Kernel

To calculate the number of edge and inside factors, you’ll set up a compute pipeline state object that points to the tessellation kernel shader function.

``````var tessellationPipelineState: MTLComputePipelineState
``````
``````tessellationPipelineState =
PipelineStates.createComputePSO(function: "tessellation_main")
``````

Compute Pass

You now have a compute pipeline state and an `MTLBuffer` containing the patch data. You also created an empty buffer which the tessellation kernel will fill with the edge and inside factors. Next, you need to create the compute command encoder to dispatch the tessellation kernel.

``````guard let computeEncoder =
commandBuffer.makeComputeCommandEncoder() else { return }
computeEncoder.setComputePipelineState(
tessellationPipelineState)
computeEncoder.setBytes(
&edgeFactors,
length: MemoryLayout<Float>.size * edgeFactors.count,
index: 0)
computeEncoder.setBytes(
&insideFactors,
length: MemoryLayout<Float>.size * insideFactors.count,
index: 1)
computeEncoder.setBuffer(
tessellationFactorsBuffer,
offset: 0,
index: 2)
``````
``````let width = min(
patchCount,
let gridSize =
MTLSize(width: patchCount, height: 1, depth: 1)
MTLSize(width: width, height: 1, depth: 1)
gridSize,
computeEncoder.endEncoding()
``````

The Tessellation Kernel Function

➤ Create a new Metal file named Tessellation.metal, and add this:

``````#import "Common.h"

kernel void
tessellation_main(
constant float *edge_factors [[buffer(0)]],
constant float *inside_factors [[buffer(1)]],
*factors [[buffer(2)]],
{
}
``````
``````factors[pid].edgeTessellationFactor[0] = edge_factors[0];
factors[pid].edgeTessellationFactor[1] = edge_factors[0];
factors[pid].edgeTessellationFactor[2] = edge_factors[0];
factors[pid].edgeTessellationFactor[3] = edge_factors[0];

factors[pid].insideTessellationFactor[0] = inside_factors[0];
factors[pid].insideTessellationFactor[1] = inside_factors[0];
``````

The Render Pass

Before doing the render, you need to tell the render encoder about the tessellation factors buffer that you updated during the compute pass.

``````renderEncoder.setTessellationFactorBuffer(
tessellationFactorsBuffer,
offset: 0,
instanceStride: 0)
``````
``````renderEncoder.setVertexBuffer(
offset: 0,
index: 0)
``````
``````renderEncoder.setVertexBuffer(
controlPointsBuffer,
offset: 0,
index: 0)
``````
``````renderEncoder.drawPatches(
numberOfPatchControlPoints: 4,
patchStart: 0,
patchCount: patchCount,
patchIndexBuffer: nil,
patchIndexBufferOffset: 0,
instanceCount: 1,
baseInstance: 0)
``````

The Post-Tessellation Vertex Function

``````// 1
// 2
vertex VertexOut
vertex_main(
// 3
patch_control_point<ControlPoint> control_points [[stage_in]],
// 4
constant Uniforms &uniforms [[buffer(BufferIndexUniforms)]],
// 5
float2 patch_coord [[position_in_patch]])
{
}
``````
``````float u = patch_coord.x;
float v = patch_coord.y;

VertexOut out;
out.position = float4(u, v, 0, 1);
out.color = float4(u, v, 0, 1);
return out;
``````

``````float2 top = mix(
control_points[0].position.xz,
control_points[1].position.xz,
u);
float2 bottom = mix(
control_points[3].position.xz,
control_points[2].position.xz,
u);
``````

``````out.position = float4(u, v, 0, 1);
``````
``````float2 interpolated = mix(top, bottom, v);
float4 position = float4(
interpolated.x, 0.0,
interpolated.y, 1.0);
out.position = uniforms.mvp * position;
``````

Multiple Patches

Now that you know how to tessellate one patch, you can tile the patches and choose edge factors that depend on dynamic factors, such as distance.

``````let patches = (horizontal: 2, vertical: 2)
``````

``````uint patchID [[patch_id]]
``````
``````out.color = float4(0);
if (patchID == 0) {
out.color = float4(1, 0, 0, 1);
}
``````

Tessellation By Distance

In this section, you’re going to create a terrain with patches that are tessellated according to the distance from the camera. When you’re close to a mountain, you need to see more detail; when you’re farther away, less. Having the ability to dial in the level of detail is where tessellation comes into its own. By setting the level of detail, you save on how many vertices the GPU has to process in any given situation.

``````typedef struct {
vector_float2 size;
float height;
uint maxTessellation;
} Terrain;
``````
``````static let maxTessellation = 16
``````
``````var terrain = Terrain(
size: [2, 2],
height: 1,
maxTessellation: UInt32(Renderer.maxTessellation))
``````
``````let controlPoints = Quad.createControlPoints(
patches: patches,
size: (width: terrain.size.x, height: terrain.size.y))
``````
``````var cameraPosition = float4(camera.position, 0)
computeEncoder.setBytes(
&cameraPosition,
length: MemoryLayout<float4>.stride,
index: 3)
var matrix = modelMatrix
computeEncoder.setBytes(
&matrix,
length: MemoryLayout<float4x4>.stride,
index: 4)
computeEncoder.setBuffer(
controlPointsBuffer,
offset: 0,
index: 5)
computeEncoder.setBytes(
&terrain,
length: MemoryLayout<Terrain>.stride,
index: 6)
``````
``````constant float4 &camera_position [[buffer(3)]],
constant float4x4 &modelMatrix   [[buffer(4)]],
constant float3* control_points  [[buffer(5)]],
constant Terrain &terrain        [[buffer(6)]],
``````

``````float calc_distance(
float3 pointA, float3 pointB,
float3 camera_position,
float4x4 modelMatrix)
{
float3 positionA = (modelMatrix * float4(pointA, 1)).xyz;
float3 positionB = (modelMatrix * float4(pointB, 1)).xyz;
float3 midpoint = (positionA + positionB) * 0.5;

float camera_distance = distance(camera_position, midpoint);
return camera_distance;
}
``````
``````uint index = pid * 4;
``````
``````float totalTessellation = 0;
``````
``````for (int i = 0; i < 4; i++) {
int pointAIndex = i;
int pointBIndex = i + 1;
if (pointAIndex == 3) {
pointBIndex = 0;
}
int edgeIndex = pointBIndex;
}
``````
``````float cameraDistance =
calc_distance(
control_points[pointAIndex + index],
control_points[pointBIndex + index],
camera_position.xyz,
modelMatrix);
``````
``````float tessellation =
factors[pid].edgeTessellationFactor[edgeIndex] = tessellation;
totalTessellation += tessellation;
``````
``````factors[pid].insideTessellationFactor[0] =
totalTessellation * 0.25;
factors[pid].insideTessellationFactor[1] =
totalTessellation * 0.25;
``````
``````// 1
pipelineDescriptor.tessellationFactorStepFunction = .perPatch
// 2
pipelineDescriptor.maxTessellationFactor = Renderer.maxTessellation
// 3
pipelineDescriptor.tessellationPartitionMode = .fractionalEven
``````

Displacement

You’ve used textures for various purposes in earlier chapters. Now you’ll use a height map to change the height of each vertex. Height maps are grayscale images where you can use the texel value for the `Y` vertex position, with white being high and black being low. There are several height maps in Textures.xcassets you can experiment with.

``````let heightMap: MTLTexture!
``````
``````do {
} catch {
fatalError(error.localizedDescription)
}
``````
``````renderEncoder.setVertexTexture(heightMap, index: 0)
renderEncoder.setVertexBytes(
&terrain,
length: MemoryLayout<Terrain>.stride,
index: 6)
``````
``````texture2d<float> heightMap [[texture(0)]],
constant Terrain &terrain [[buffer(6)]],
``````
``````// 1
float2 xy = (position.xz + terrain.size / 2.0) / terrain.size;
// 2
constexpr sampler sample;
float4 color = heightMap.sample(sample, xy);
out.color = float4(color.r);
// 3
float height = (color.r * 2 - 1) * terrain.height;
position.y = height;
``````
``````out.color = float4(0);
if (patchID == 0) {
out.color = float4(1, 0, 0, 1);
}
``````
``````let rotation = float3(Float(-20).degreesToRadians, 0, 0)
``````

``````static let maxTessellation: Int = {
#if os(macOS)
return 64
#else
return 16
#endif
}()
``````
``````let patches = (horizontal: 6, vertical: 6)
var terrain = Terrain(
size: [8, 8],
height: 1,
maxTessellation: UInt32(Renderer.maxTessellation))
``````

In the previous section, you sampled the height map in the vertex function, and the colors are interpolated when sent to the fragment function. For maximum color detail, you need to sample from textures per fragment, not per vertex.

``````let cliffTexture: MTLTexture?
let snowTexture: MTLTexture?
let grassTexture: MTLTexture?
``````
``````cliffTexture =
snowTexture =
grassTexture =
``````
``````renderEncoder.setFragmentTexture(cliffTexture, index: 1)
renderEncoder.setFragmentTexture(snowTexture, index: 2)
renderEncoder.setFragmentTexture(grassTexture, index: 3)
``````
``````float height;
float2 uv;
``````
``````out.uv = xy;
out.height = height;
``````
``````texture2d<float> cliffTexture [[texture(1)]],
texture2d<float> snowTexture  [[texture(2)]],
texture2d<float> grassTexture [[texture(3)]]
``````
``````constexpr sampler sample(filter::linear, address::repeat);
float tiling = 16.0;
float4 color;
if (in.height < -0.5) {
color = grassTexture.sample(sample, in.uv * tiling);
} else if (in.height < 0.3) {
color = cliffTexture.sample(sample, in.uv * tiling);
} else {
color = snowTexture.sample(sample, in.uv * tiling);
}
return color;
``````

``````pipelineDescriptor.tessellationPartitionMode = .pow2
``````

The snow line in your previous render is unrealistic. By checking the slope of the mountain, you can show the snow texture in flatter areas, and show the cliff texture where the slope is steep.

The Metal Performance Shaders framework contains many useful, highly optimized shaders for image processing, matrix multiplication, machine learning and raytracing. You’ll read more about them in Chapter 30, “Metal Performance Shaders.”

``````import MetalPerformanceShaders
``````
``````static func heightToSlope(source: MTLTexture) -> MTLTexture {
}
``````
``````let descriptor =
MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: source.pixelFormat,
width: source.width,
height: source.height,
mipmapped: false)
``````
``````guard let destination =
Renderer.device.makeTexture(descriptor: descriptor),
let commandBuffer = Renderer.commandQueue.makeCommandBuffer()
else {
fatalError("Error creating Sobel texture")
}
``````
``````let shader = MPSImageSobel(device: Renderer.device)
commandBuffer: commandBuffer,
sourceTexture: source,
destinationTexture: destination)
commandBuffer.commit()
return destination
``````
``````let terrainSlope: MTLTexture
``````
``````terrainSlope = Renderer.heightToSlope(source: heightMap)
``````

Challenge

Your challenge for this chapter is to use the slope texture from the Sobel filter to place snow on the mountain on the parts that aren’t steep. Because you don’t need pixel perfect accuracy, you can read the slope image in the vertex function and send that value to the fragment function. This is more efficient as there will be fewer texture reads in the vertex function than in the fragment function.

Key Points

• Tessellation utilizes a tessellator chip on the GPU to create extra vertices.
• You send patches to the GPU rather than vertices. The tessellator then breaks down these patches to smaller triangles.
• A patch can be either a triangle or a quad.
• The tessellation pipeline has an extra stage of setting edge and inside factors in a tessellation kernel. These factors decide the number of vertices that the tessellator should create.
• The vertex shader handles the vertices created by the tessellator.
• Vertex displacement uses a grayscale texture to move the vertex, generally in the `y` direction.
• The Sobel Metal Performance Shader takes a texture and generates a new texture that defines the slope of a pixel.

Where to Go From Here?

With very steep displacement, there can be lots of texture stretching between vertices. There are various algorithms to overcome this, and you can find one in Apple’s excellent sample code: Dynamic Terrain with Argument Buffers at https://developer.apple.com/documentation/metal/fundamental_components/gpu_resources/dynamic_terrain_with_argument_buffers. This is a complex project that showcases argument buffers, but the dynamic terrain portion is interesting.

Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.