Home iOS & Swift Books Metal by Tutorials

11
Maps & Materials Written by Marius Horga & Caroline Begbie

Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as scrambled text.

You can unlock the rest of this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.

Unlock Now

In the previous chapter, you set up a simple Phong lighting model. In recent years, researchers have made great steps forward with Physically Based Rendering (PBR). PBR attempts to accurately represent real-world shading, where the amount of light leaving a surface is less that the amount the surface receives. In the real world, the surfaces of objects are not completely flat, as yours have been so far. If you look at the objects around you, you’ll notice how their basic color changes according to how light falls on them. Some objects have a smooth surface, and some have a rough surface. Heck, some might even be shiny metal!

In this chapter, you’ll find out how to use material groups to describe a surface, and how to design textures for micro detail.

Normal Maps

The following example best describes normal maps:

An object rendered with a normal map
An object rendered with a normal map

On the left, there’s a lit cube with a color texture. On the right, there’s the same low-poly cube with the identical color texture and lighting. The only difference is that the cube on the right also has a second texture applied to it known as a normal map. This normal map makes it appear as if the cube is a high-poly model with lots of nooks and crannies. In truth, these high-end details are just an illusion.

For this illusion to work, the model needs a texture, like this:

A normal map texture
A normal map texture

All models have normals that stick out perpendicular to each face. A cube has six faces, and the normal for each face points in a different direction. Also, each face is flat. If you wanted to create the illusion of bumpiness, you’d need to change a normal in the fragment shader.

Look at the following image. On the left is a flat surface with normals in the fragment shader. On the right, you see perturbed normals. The texels in a normal map supply the direction vectors of these normals through the RGB channels.

Normals
Normals

Now, look at this single brick split out into the red, green and blue channels that make up an RGB image.

Normal map channels
Normal map channels

Each channel has a value between 0 and 1, and you generally visualize them in grayscale as it’s easier to read color values. For example, in the red channel, a value of 0 is no red at all, while a value of 1 is full red. When you convert 0 to an RGB color (0, 0, 0), the result is black. On the opposite spectrum, (1, 1, 1) is white. And in the middle, you have (0.5, 0.5, 0.5), which is mid-gray. In grayscale, all three RGB values are the same, so you only need to refer to a grayscale value by a single float.

Take a closer look at the edges of the red channel’s brick. Look at the left and right edges in the grayscale image. The red channel has the darkest color where the normal values of that fragment should point left (-X, 0, 0), and the lightest color where they should point right (+X, 0, 0).

Now look at the green channel. The left and right edges have equal value but are different for the top and bottom edges of the brick. The green channel in the grayscale image has darkest for pointing down (0, -Y, 0) and lightest for pointing up (0, +Y, 0).

Finally, the blue channel is mostly white in the grayscale image because the brick — except for a few irregularities in the texture — points outward. The edges of the brick are the only places where the normals should point away.

Note: Normal maps can be either right-handed or left-handed. Your renderer will expect positive y to be up, but some apps will generate normal maps with positive y down. To fix this, you can take the normal map into Photoshop and invert the green channel.

The base color of a normal map — where all normals are “normal” (orthogonal to the face) — is (0.5, 0.5, 1).

A flat normal map
A flat normal map

This is an attractive color but was not chosen arbitrarily. RGB colors have values between 0 and 1, whereas a model’s normal values are between -1 and 1. A color value of 0.5 in a normal map translates to a model normal of 0. The result of reading a flat texel from a normal map should be a z value of 1 and the x and y values as 0. Converting these values (0, 0, 1) into the colorspace of a normal map results in the color (0.5, 0.5, 1). This is why most normal maps appear bluish.

Creating Normal Maps

To create successful normal maps, you need a specialized app. You’ve already learned about texturing apps, such as Adobe Substance Designer and Mari in Chapter 8, “Textures”. Both of these apps are procedural and will generate normal maps as well as base color textures. In fact, the brick texture in the image at the start of the chapter was created in Adobe Substance Designer.

A cross photographed and converted into a normal map
I xcokn wxafuvsetleb alv gesdewkim erfo i boxhet guq

Tangent Space

To render with a normal map texture, you send it to the fragment function in the same way as a color texture, and you extract the normal values using the same UVs. However, you can’t directly apply your normal map values onto your model’s current normals. In your fragment shader, the model’s normals are in world space, and the normal map normals are in tangent space. Tangent space is a little hard to wrap your head around. Think of the brick cube with all its six faces pointing in different directions. Now think of the normal map with all the bricks the same color on all the six faces.

Normals on a sphere
Zohpavv ap e llfuca

Visualizing normals in world space
Teseozivafx ranricp ag nurqk fyece

The TBN matrix
Xdo YPB xivnam

The Starter App

➤ In Xcode, open the starter project for this chapter.

A cartoon cottage
U monrees tabquto

Using Normal Maps

➤ In the Models ▸ Cottage group, open cottage1.mtl in a text editor.

map_tangentSpaceNormal cottage-normal
map_Kd cottage-color

let normal: MTLTexture?

normal = property(with: .tangentSpaceNormal)

NormalTexture = 1

encoder.setFragmentTexture(
  submesh.textures.normal,
  index: NormalTexture.index)

texture2d<float> normalTexture [[texture(NormalTexture)]]

float3 normal;
if (is_null_texture(normalTexture)) {
  normal = in.worldNormal;
} else {
  normal = normalTexture.sample(
  textureSampler,
  in.uv * params.tiling).rgb;
}
normal = normalize(normal);
return float4(normal, 1);
The normal map applied as a color texture
Jsi takpik rej ovpceaz et e timax bowbiwa

return float4(normal, 1);
A normal map overlaid with UVs
I rawyoq dog iqerhaun racz ODc

1. Load Tangents and Bitangents

➤ Open VertexDescriptor.swift, and look at MDLVertexDescriptor’s defaultLayout. Here, you tell the vertex descriptor that there are normal values in the attribute named MDLVertexAttributeNormal.

Flat vs smooth shaded
Klam zl szoohp xhituh

let (mdlMeshes, mtkMeshes) = try! MTKMesh.newMeshes(
  asset: asset,
  device: Renderer.device)

var mtkMeshes: [MTKMesh] = []
let mdlMeshes =
  asset.childObjects(of: MDLMesh.self) as? [MDLMesh] ?? []
_ = mdlMeshes.map { mdlMesh in
  mdlMesh.addNormals(
    withAttributeNamed: MDLVertexAttributeNormal,
    creaseThreshold: 1.0)
  mtkMeshes.append(
    try! MTKMesh(
      mesh: mdlMesh,
      device: Renderer.device))
}
An unsmoothed model
Aw ohcpeisnad biqip

mdlMesh.addNormals(
  withAttributeNamed: MDLVertexAttributeNormal,
  creaseThreshold: 1.0)

mdlMesh.addTangentBasis(
  forTextureCoordinateAttributeNamed:
    MDLVertexAttributeTextureCoordinate,
  tangentAttributeNamed: MDLVertexAttributeTangent,
  bitangentAttributeNamed: MDLVertexAttributeBitangent)

Tangent = 4,
Bitangent = 5

TangentBuffer = 3,
BitangentBuffer = 4,

vertexDescriptor.attributes[Tangent.index] =
  MDLVertexAttribute(
    name: MDLVertexAttributeTangent,
    format: .float3,
    offset: 0,
    bufferIndex: TangentBuffer.index)
vertexDescriptor.layouts[TangentBuffer.index]
  = MDLVertexBufferLayout(stride: MemoryLayout<float3>.stride)
vertexDescriptor.attributes[Bitangent.index] =
  MDLVertexAttribute(
    name: MDLVertexAttributeBitangent,
    format: .float3,
    offset: 0,
    bufferIndex: BitangentBuffer.index)
vertexDescriptor.layouts[BitangentBuffer.index]
  = MDLVertexBufferLayout(stride: MemoryLayout<float3>.stride)
The cottage - no changes yet
Qvo qofnefe - ya cyontej new

2. Send Tangent and Bitangent Values to the GPU

➤ Open Model.swift, and in render(encoder:uniforms:params:), locate for mesh in meshes.

for (index, vertexBuffer) in mesh.vertexBuffers.enumerated() {
  encoder.setVertexBuffer(
    vertexBuffer,
    offset: 0,
    index: index)
}

3. Convert Tangent and Bitangent Values to World Space

Just as you converted the model’s normals to world space, you need to convert the tangents and bitangents to world space in the vertex function.

float3 tangent [[attribute(Tangent)]];
float3 bitangent [[attribute(Bitangent)]];

float3 worldTangent;
float3 worldBitangent;

.worldTangent = uniforms.normalMatrix * in.tangent,
.worldBitangent = uniforms.normalMatrix * in.bitangent

4. Calculate the New Normal

Now that you have everything in place, it’ll be a simple matter to calculate the new normal.

normal = normal * 2 - 1;

normal = float3x3(
  in.worldTangent,
  in.worldBitangent,
  in.worldNormal) * normal;

float3 color = phongLighting(
  normal,
  in.worldPosition,
  params,
  lights,
  baseColor
);

float3 normalDirection = normalize(in.worldNormal);
The cottage with a normal map applied
Qcu napmaya fifn u favfok maz ivlfiix

Other Texture Map Types

Normal maps are not the only way of changing a model’s surface. There are other texture maps:

Materials

Not all models have textures. For example, the train you rendered earlier in the book has different material groups that specify a color instead of using a texture.

typedef struct {
  vector_float3 baseColor;
  vector_float3 specularColor;
  float roughness;
  float metallic;
  float ambientOcclusion;
  float shininess;
} Material;

let material: Material

private extension Material {
  init(material: MDLMaterial?) {
    self.init()
    if let baseColor = material?.property(with: .baseColor),
      baseColor.type == .float3 {
      self.baseColor = baseColor.float3Value
    }
  }
}

if let specular = material?.property(with: .specular),
  specular.type == .float3 {
  self.specularColor = specular.float3Value
}
if let shininess = material?.property(with: .specularExponent),
  shininess.type == .float {
  self.shininess = shininess.floatValue
}
self.ambientOcclusion = 1

material = Material(material: mdlSubmesh.material)

MaterialBuffer = 14

var material = submesh.material
encoder.setFragmentBytes(
  &material,
  length: MemoryLayout<Material>.stride,
  index: MaterialBuffer.index)

constant Material &_material [[buffer(MaterialBuffer)]],

Material material = _material;

float3 baseColor;
if (is_null_texture(baseColorTexture)) {
  baseColor = in.color;
} else {
  baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv * params.tiling).rgb;
}

if (!is_null_texture(baseColorTexture)) {
  material.baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv * params.tiling).rgb;
}

float3 color = phongLighting(
  normal,
  in.worldPosition,
  params,
  lights,
  material
);

Material material

Material material

float3 baseColor = material.baseColor;

float materialShininess = material.shininess;
float3 materialSpecularColor = material.specularColor;
Using color from the surface material
Agilp hobof bhel wme febwifa todebiap

With and without specular highlight
Goxj irv votwoaf wpahopog mammwujmh

Physically Based Rendering (PBR)

To achieve spectacular scenes, you need to have good textures, but shading plays an even more significant role. In recent years, the concept of PBR has replaced the simplistic Phong shading model. As its name suggests, PBR attempts physically realistic interaction of light with surfaces. Now that Augmented Reality has become part of our lives, it’s even more important to render your models to match their physical surroundings.

PBR Workflow

First, change the fragment function to use the PBR calculations.

Target membership
Sadmag nehcoqhmuj

let roughness: MTLTexture?

roughness = property(with: .roughness)

if let roughness = material?.property(with: .roughness),
  roughness.type == .float3 {
  self.roughness = roughness.floatValue
}

encoder.setFragmentTexture(
  submesh.textures.roughness,
  index: 2)

camera.distance = 3.5
camera.target = .zero
A cube with albedo map applied
A nufa sopd ophamo sed azmgoos

A cube with normal and albedo maps applied
O litu ragr baktox asm encoza nirs izjveul

The cube's texture maps
Qcu lucu'f yuvsogo nemj

The final rendered cube
Hno cexeb jalqurof zexe

Channel Packing

Later, you’ll be using the PBR fragment function for rendering. Even if you don’t understand the mathematics, understand the layout of the function and the concepts used.

roughness = roughnessTexture.sample(textureSampler, in.uv).r;
Different channels in Photoshop
Kiptimadw yhurtubv ub Knavisfib

Challenge

In the resources folder for this chapter is a fabulous helmet model from Malopolska’s Virtual Museums collection at sketchfab.com. Your challenge is to render this model. There are five textures that you’ll load into the asset catalog. Don’t forget to change Interpretation from Color to Data, so the textures don’t load as sRGB.

Rendering a helmet
Vumfizecq i suwwoz

Where to Go From Here?

Now that you’ve whet your appetite for physically based rendering, explore the fantastic links in references.markdown, which you’ll find in the resources folder for this chapter. Some of the links are highly mathematical, while others explain with gorgeous photo-like images.

12. Render Passes 10. Lighting Fundamentals

Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.

© 2022 Razeware LLC

You're reading for free, with parts of this chapter shown as scrambled text. Unlock this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.

Unlock Now

To highlight or take notes, you’ll need to own this book in a subscription or purchased by itself.

Read for Free with a Subscription

Buy this book for $59.99