Chapters
Chapters

Chapters

Hide chapters

Metal by Tutorials

Third Edition · macOS 12 · iOS 15 · Swift 5.5 · Xcode 13

Before You Begin

Section 0: 6 chapters
Show chapters Hide chapters

Section I: Beginning Metal

Section 1: 10 chapters
Show chapters Hide chapters

Section II: Intermediate Metal

Section 2: 8 chapters
Show chapters Hide chapters

Section III: Advanced Metal

Section 3: 8 chapters
Show chapters Hide chapters

Section IV: Ray Tracing

Section 4: 7 chapters
Show chapters Hide chapters

19. Tessellation & Terrains
Written by Marius Horga & Caroline Begbie

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

So far, you’ve used normal map trickery in the fragment function to show the fine details of your low poly models. To achieve a similar level of detail without using normal maps requires a change of model geometry by adding more vertices. The problem with adding more vertices is that when you send them to the GPU, it chokes up the pipeline. A hardware tessellator in the GPU can create vertices on the fly, adding a greater level of detail and thereby using fewer resources.

In this chapter, you’ll create a detailed terrain using a small number of points. You’ll send a flat ground plane with a grayscale texture describing the height, and the tessellator will create as many vertices as needed. The vertex function will then read the texture and displace (move) these new vertices vertically.

Tessellation concept
Tessellation concept

In this example, on the left side are the control points. On the right side, the tessellator creates extra vertices, with the number dependent on how close the control points are to the camera.

Tessellation

For tessellation, instead of sending vertices to the GPU, you send patches. These patches are made up of control points — a minimum of three for a triangle patch, or four for a quad patch. The tessellator can convert each quad patch into a certain number of triangles: up to 4,096 triangles on a recent iMac and 256 triangles on an iPhone that’s capable of tessellation.

Note: Tessellation is available on all Macs since 2012 and on iOS 10 GPU Family 3 and up. This includes the iPhone 6s and newer devices. However, Tessellation is not available on the iOS simulator.

With tessellation, you can:

  • Send less data to the GPU. Because the GPU doesn’t store tessellated vertices in graphics memory, it’s more efficient on resources.
  • Make low poly objects look less low poly by curving patches.
  • Displace vertices for fine detail instead of using normal maps to fake it.
  • Decide on the level of detail based on the distance from the camera. The closer an object is to the camera, the more vertices it contains.

The Starter Project

So that you can more easily understand the difference between rendering patches and rendering vertices, the starter project is a simplified renderer. All the rendering code is in Renderer.swift, with the pipeline state setup in Pipelines.swift.

The starter app
Tlo slaffay ext

CPU pipeline
MZI xufokumi

GPU pipeline
RCA coposipu

Tessellation Patches

A patch consists of a certain number of control points, generally:

Tessellated patches
Qaqbodfudaj zogbdul

A bezier curve
O nijaot sukje

Tessellation Factors

For each patch, you need to specify inside edge factors and outside edge factors. The four-point patch in the following image shows different edge factors for each edge — specified as [2, 4, 8, 16] — and two different inside factors — specified as [8, 16], for horizontal and vertical respectively.

Edge factors
Ujfe sodnumq

let patches = (horizontal: 1, vertical: 1)
var patchCount: Int {
  patches.horizontal * patches.vertical
}

var edgeFactors: [Float] = [4]
var insideFactors: [Float] = [4]

lazy var tessellationFactorsBuffer: MTLBuffer? = {
  // 1
  let count = patchCount * (4 + 2)
  // 2
  let size = count * MemoryLayout<Float>.size / 2
  return Renderer.device.makeBuffer(
    length: size,
    options: .storageModePrivate)
}()

Setting Up the Patch Data

Instead of an array of six vertices, you’ll create a four-point patch with control points at the corners.

var controlPointsBuffer: MTLBuffer?

let controlPoints = Quad.createControlPoints(
  patches: patches,
  size: (2, 2))
controlPointsBuffer =
  Renderer.device.makeBuffer(
    bytes: controlPoints,
    length: MemoryLayout<float3>.stride * controlPoints.count)

Set Up the Render Pipeline State

You can configure the tessellator by changing the pipeline state properties. Until now, you’ve processed only vertices with the vertex descriptor. However, you’ll now modify the vertex descriptor so it processes patches instead.

vertexDescriptor.layouts[0].stepFunction = .perPatchControlPoint

The Tessellation Kernel

To calculate the number of edge and inside factors, you’ll set up a compute pipeline state object that points to the tessellation kernel shader function.

var tessellationPipelineState: MTLComputePipelineState

tessellationPipelineState =
  PipelineStates.createComputePSO(function: "tessellation_main")

Compute Pass

You now have a compute pipeline state and an MTLBuffer containing the patch data. You also created an empty buffer which the tessellation kernel will fill with the edge and inside factors. Next, you need to create the compute command encoder to dispatch the tessellation kernel.

guard let computeEncoder =
  commandBuffer.makeComputeCommandEncoder() else { return }
computeEncoder.setComputePipelineState(
  tessellationPipelineState)
computeEncoder.setBytes(
  &edgeFactors,
  length: MemoryLayout<Float>.size * edgeFactors.count,
  index: 0)
computeEncoder.setBytes(
  &insideFactors,
  length: MemoryLayout<Float>.size * insideFactors.count,
  index: 1)
computeEncoder.setBuffer(
  tessellationFactorsBuffer,
  offset: 0,
  index: 2)

let width = min(
  patchCount,
  tessellationPipelineState.threadExecutionWidth)
let gridSize =
  MTLSize(width: patchCount, height: 1, depth: 1)
let threadsPerThreadgroup =
  MTLSize(width: width, height: 1, depth: 1)
computeEncoder.dispatchThreadgroups(
  gridSize,
  threadsPerThreadgroup: threadsPerThreadgroup)
computeEncoder.endEncoding()

The Tessellation Kernel Function

➤ Create a new Metal file named Tessellation.metal, and add this:

#import "Common.h"

kernel void
  tessellation_main(
    constant float *edge_factors [[buffer(0)]],
    constant float *inside_factors [[buffer(1)]],
    device MTLQuadTessellationFactorsHalf
      *factors [[buffer(2)]],
    uint pid [[thread_position_in_grid]])
{
}

factors[pid].edgeTessellationFactor[0] = edge_factors[0];
factors[pid].edgeTessellationFactor[1] = edge_factors[0];
factors[pid].edgeTessellationFactor[2] = edge_factors[0];
factors[pid].edgeTessellationFactor[3] = edge_factors[0];

factors[pid].insideTessellationFactor[0] = inside_factors[0];
factors[pid].insideTessellationFactor[1] = inside_factors[0];

The Render Pass

Before doing the render, you need to tell the render encoder about the tessellation factors buffer that you updated during the compute pass.

renderEncoder.setTessellationFactorBuffer(
  tessellationFactorsBuffer,
  offset: 0,
  instanceStride: 0)

renderEncoder.setVertexBuffer(
  quad.vertexBuffer,
  offset: 0,
  index: 0)

renderEncoder.setVertexBuffer(
  controlPointsBuffer,
  offset: 0,
  index: 0)

renderEncoder.drawPatches(
  numberOfPatchControlPoints: 4,
  patchStart: 0,
  patchCount: patchCount,
  patchIndexBuffer: nil,
  patchIndexBufferOffset: 0,
  instanceCount: 1,
  baseInstance: 0)

The Post-Tessellation Vertex Function

➤ Open Shaders.metal.

// 1
[[patch(quad, 4)]]
// 2
vertex VertexOut
  vertex_main(
// 3
    patch_control_point<ControlPoint> control_points [[stage_in]],
// 4          
    constant Uniforms &uniforms [[buffer(BufferIndexUniforms)]],
// 5                             
    float2 patch_coord [[position_in_patch]])
{              
}

float u = patch_coord.x;
float v = patch_coord.y;

VertexOut out;
out.position = float4(u, v, 0, 1);
out.color = float4(u, v, 0, 1);
return out;
Basic tessellation
Xipib kijbehpoviuy

float2 top = mix(
  control_points[0].position.xz,
  control_points[1].position.xz,
  u);
float2 bottom = mix(
  control_points[3].position.xz,
  control_points[2].position.xz,
  u);
Control point winding order
Diqmtur taurc rijvubv uzwoc

out.position = float4(u, v, 0, 1);

float2 interpolated = mix(top, bottom, v);
float4 position = float4(
  interpolated.x, 0.0,
  interpolated.y, 1.0);
out.position = uniforms.mvp * position;
A tessellated patch
O bubjuhhidac vihhg

Multiple Patches

Now that you know how to tessellate one patch, you can tile the patches and choose edge factors that depend on dynamic factors, such as distance.

let patches = (horizontal: 2, vertical: 2)
Four tessellated patches
Koed zephultaqum xakwyul

uint patchID [[patch_id]]

out.color = float4(0);
if (patchID == 0) {
  out.color = float4(1, 0, 0, 1);
}
Colored by patch id
Rifopov sl goyrm ev

Tessellation By Distance

In this section, you’re going to create a terrain with patches that are tessellated according to the distance from the camera. When you’re close to a mountain, you need to see more detail; when you’re farther away, less. Having the ability to dial in the level of detail is where tessellation comes into its own. By setting the level of detail, you save on how many vertices the GPU has to process in any given situation.

typedef struct {
  vector_float2 size;
  float height;
  uint maxTessellation;
} Terrain;

static let maxTessellation = 16

var terrain = Terrain(
  size: [2, 2],
  height: 1,
  maxTessellation: UInt32(Renderer.maxTessellation))

let controlPoints = Quad.createControlPoints(
  patches: patches,
  size: (width: terrain.size.x, height: terrain.size.y))

var cameraPosition = float4(camera.position, 0)
computeEncoder.setBytes(
  &cameraPosition,
  length: MemoryLayout<float4>.stride,
  index: 3)
var matrix = modelMatrix
computeEncoder.setBytes(
  &matrix,
  length: MemoryLayout<float4x4>.stride,
  index: 4)
computeEncoder.setBuffer(
  controlPointsBuffer,
  offset: 0,
  index: 5)
computeEncoder.setBytes(
  &terrain,
  length: MemoryLayout<Terrain>.stride,
  index: 6)

constant float4 &camera_position [[buffer(3)]],
constant float4x4 &modelMatrix   [[buffer(4)]],
constant float3* control_points  [[buffer(5)]],
constant Terrain &terrain        [[buffer(6)]],
Edges and control points
Iyqew ajv pexpnit woumzn

float calc_distance(
  float3 pointA, float3 pointB,
  float3 camera_position,
  float4x4 modelMatrix)
{
  float3 positionA = (modelMatrix * float4(pointA, 1)).xyz;
  float3 positionB = (modelMatrix * float4(pointB, 1)).xyz;
  float3 midpoint = (positionA + positionB) * 0.5;

  float camera_distance = distance(camera_position, midpoint);
  return camera_distance;
}

uint index = pid * 4;

float totalTessellation = 0;

for (int i = 0; i < 4; i++) {
  int pointAIndex = i;
  int pointBIndex = i + 1;
  if (pointAIndex == 3) {
    pointBIndex = 0;
  }
  int edgeIndex = pointBIndex;
}

float cameraDistance =
  calc_distance(
    control_points[pointAIndex + index],
    control_points[pointBIndex + index],
    camera_position.xyz,
    modelMatrix);

float tessellation =
  max(4.0, terrain.maxTessellation / cameraDistance);
factors[pid].edgeTessellationFactor[edgeIndex] = tessellation;
totalTessellation += tessellation;

factors[pid].insideTessellationFactor[0] =
  totalTessellation * 0.25;
factors[pid].insideTessellationFactor[1] =
  totalTessellation * 0.25;

// 1
pipelineDescriptor.tessellationFactorStepFunction = .perPatch
// 2
pipelineDescriptor.maxTessellationFactor = Renderer.maxTessellation
// 3
pipelineDescriptor.tessellationPartitionMode = .fractionalEven
Tessellation by distance
Dolfajlicaoj zp vabkothe

Displacement

You’ve used textures for various purposes in earlier chapters. Now you’ll use a height map to change the height of each vertex. Height maps are grayscale images where you can use the texel value for the Y vertex position, with white being high and black being low. There are several height maps in Textures.xcassets you can experiment with.

let heightMap: MTLTexture!

do {
  heightMap = try TextureController.loadTexture(filename: "mountain")
} catch {
  fatalError(error.localizedDescription)
}

renderEncoder.setVertexTexture(heightMap, index: 0)
renderEncoder.setVertexBytes(
  &terrain,
  length: MemoryLayout<Terrain>.stride,
  index: 6)

texture2d<float> heightMap [[texture(0)]],
constant Terrain &terrain [[buffer(6)]],

// 1
float2 xy = (position.xz + terrain.size / 2.0) / terrain.size;
// 2
constexpr sampler sample;
float4 color = heightMap.sample(sample, xy);
out.color = float4(color.r);
// 3
float height = (color.r * 2 - 1) * terrain.height;
position.y = height;

out.color = float4(0);
if (patchID == 0) {
  out.color = float4(1, 0, 0, 1);
}

let rotation = float3(Float(-20).degreesToRadians, 0, 0)
Height map displacement
Riozbk hix yiglnejolejb

static let maxTessellation: Int = {
  #if os(macOS)
  return 64
  #else
  return 16
  #endif
}()

let patches = (horizontal: 6, vertical: 6)
var terrain = Terrain(
  size: [8, 8],
  height: 1,
  maxTessellation: UInt32(Renderer.maxTessellation))
A tessellated mountain
I dosyafximil vaebhiim

Shading By Height

In the previous section, you sampled the height map in the vertex function, and the colors are interpolated when sent to the fragment function. For maximum color detail, you need to sample from textures per fragment, not per vertex.

let cliffTexture: MTLTexture?
let snowTexture: MTLTexture?
let grassTexture: MTLTexture?

cliffTexture =
  try TextureController.loadTexture(filename: "cliff-color")
snowTexture =
  try TextureController.loadTexture(filename: "snow-color")
grassTexture =
  try TextureController.loadTexture(filename: "grass-color")

renderEncoder.setFragmentTexture(cliffTexture, index: 1)
renderEncoder.setFragmentTexture(snowTexture, index: 2)
renderEncoder.setFragmentTexture(grassTexture, index: 3)

float height;
float2 uv;

out.uv = xy;
out.height = height;

texture2d<float> cliffTexture [[texture(1)]],
texture2d<float> snowTexture  [[texture(2)]],
texture2d<float> grassTexture [[texture(3)]]

constexpr sampler sample(filter::linear, address::repeat);
float tiling = 16.0;
float4 color;
if (in.height < -0.5) {
  color = grassTexture.sample(sample, in.uv * tiling);
} else if (in.height < 0.3) {
  color = cliffTexture.sample(sample, in.uv * tiling);
} else {
  color = snowTexture.sample(sample, in.uv * tiling);
}
return color;
A textured mountain
O qemsejeb ruaynuuy

pipelineDescriptor.tessellationPartitionMode = .pow2
Rounding edge factors to a power of two
Muapxodg amfu lilboby pi u reres eb mne

Shading By Slope

The snow line in your previous render is unrealistic. By checking the slope of the mountain, you can show the snow texture in flatter areas, and show the cliff texture where the slope is steep.

Metal Performance Shaders

The Metal Performance Shaders framework contains many useful, highly optimized shaders for image processing, matrix multiplication, machine learning and raytracing. You’ll read more about them in Chapter 30, “Metal Performance Shaders.”

import MetalPerformanceShaders

static func heightToSlope(source: MTLTexture) -> MTLTexture {
}

let descriptor =
  MTLTextureDescriptor.texture2DDescriptor(
    pixelFormat: source.pixelFormat,
    width: source.width,
    height: source.height,
    mipmapped: false)
descriptor.usage = [.shaderWrite, .shaderRead]

guard let destination =
  Renderer.device.makeTexture(descriptor: descriptor),
  let commandBuffer = Renderer.commandQueue.makeCommandBuffer()
else {
  fatalError("Error creating Sobel texture")
}

let shader = MPSImageSobel(device: Renderer.device)
shader.encode(
  commandBuffer: commandBuffer,
  sourceTexture: source,
  destinationTexture: destination)
commandBuffer.commit()
return destination

let terrainSlope: MTLTexture

terrainSlope = Renderer.heightToSlope(source: heightMap)
The Sobel filter
Gwi Yuxox zibcev

Challenge

Your challenge for this chapter is to use the slope texture from the Sobel filter to place snow on the mountain on the parts that aren’t steep. Because you don’t need pixel perfect accuracy, you can read the slope image in the vertex function and send that value to the fragment function. This is more efficient as there will be fewer texture reads in the vertex function than in the fragment function.

Shading by slope
Kholovc fc wduxu

Key Points

  • Tessellation utilizes a tessellator chip on the GPU to create extra vertices.
  • You send patches to the GPU rather than vertices. The tessellator then breaks down these patches to smaller triangles.
  • A patch can be either a triangle or a quad.
  • The tessellation pipeline has an extra stage of setting edge and inside factors in a tessellation kernel. These factors decide the number of vertices that the tessellator should create.
  • The vertex shader handles the vertices created by the tessellator.
  • Vertex displacement uses a grayscale texture to move the vertex, generally in the y direction.
  • The Sobel Metal Performance Shader takes a texture and generates a new texture that defines the slope of a pixel.

Where to Go From Here?

With very steep displacement, there can be lots of texture stretching between vertices. There are various algorithms to overcome this, and you can find one in Apple’s excellent sample code: Dynamic Terrain with Argument Buffers at https://developer.apple.com/documentation/metal/fundamental_components/gpu_resources/dynamic_terrain_with_argument_buffers. This is a complex project that showcases argument buffers, but the dynamic terrain portion is interesting.

A splat map
U mkjic gom

18. Particle Behavior
20. Fragment Post-Processing
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2025 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now