Chapters
Chapters

Chapters

Hide chapters

Apple Augmented Reality by Tutorials

Before You Begin

Section 0: 5 chapters
Show chapters Hide chapters

Section I: Reality Composer

Section 1: 5 chapters
Show chapters Hide chapters

Section II: Reality Composer & Image Tracking

Section 2: 2 chapters
Show chapters Hide chapters

Section III: Reality Composer & Object Tracking

Section 3: 1 chapter
Show chapters Hide chapters

Section IV: RealityKit & Face Tracking

Section 4: 3 chapters
Show chapters Hide chapters

Section V: ARKit & SpriteKit

Section 5: 3 chapters
Show chapters Hide chapters

Section VI: ARKit & SceneKit

Section 6: 2 chapters
Show chapters Hide chapters

Section VII: ECS & Collaborative Experiences (Bonus Section)

Section 7: 1 chapter
Show chapters Hide chapters

Section VIII: Creating 3D Models With Object Capture (Bonus Section))

Section 8: 2 chapters
Show chapters Hide chapters

13. ARKit & SpriteKit
Written by Chris Language

Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as scrambled text.

You can unlock the rest of this book, and our entire catalogue of books and videos, with a kodeco.com Professional subscription.

Unlock now

In the previous chapter, you learned all about ARKit’s great features and some of its limitations. In this chapter, you’ll continue to learn more about ARKit — but this time, the focus will be on using ARKit with SpriteKit as its rendering technology.

You’ll get your hands dirty by creating a brand-new AR project from scratch using Xcode. You’ll create a fun AR experience that uses 2D-based emoji graphics. Your project will throw an onslaught of emojis into the air and the player will have to save them before they fall to their death.

Keen on seeing emojis fall to their death? Then what are you waiting for? Jump in and get those hands dirty!

What is SpriteKit?

SpriteKit is Apple’s general-purpose 2D graphics framework. You can use it to draw shapes, particles, text, sprites and video.

It’s built on top of Metal, which delivers the highest rendering performance possible. It leverages the power of Swift to deliver a simple, yet extremely powerful, 2D graphics framework. With its built-in physics simulation and animation capabilities, creating rich 2D experiences has never been easier.

Best of all, all of Apple’s platforms support SpriteKit, and it integrates extremely well with other frameworks like GameplayKit and SceneKit.

So start Xcode, it’s time to create the project.

Creating a SpriteKit AR Project

Create a new project in Xcode. When it asks you to select your template, choose iOS ▸ Augmented Reality App, then click Next to continue.

Exploring the Project

In Xcode, with the project open, explore the important components that Xcode generated for you based on the SpriteKit Augmented Reality Template project.

AppDelegate.swift

This is the standard starting point of your app.

LaunchScreen.storyboard

The launch screen is another standard part of every app. It’s the first thing the user sees when they launch your app.

Main.storyboard

The main storyboard is the view component of your AR app, containing the app’s UI. This is a good place to put buttons and heads-up displays, for example.

ViewController.swift

The view controller contains the code behind the entire AR experience, specifically for the main storyboard.

Scene.sks

This defines an empty SpriteKit scene.

Scene.swift

This contains the code behind the SpriteKit scene.

Assets.xcassets

Here, you’ll find your stock-standard app assets like your app icon, for example.

Info.plist

When your app runs for the first time, it has to ask for permission to access the camera. ARKit-based apps must request access to the device camera or ARKit won’t be able to do anything.

ARSKView & ARSession

The ARSKView (Augmented Reality SpriteKit View) is a special class used to create 2D SpiteKit AR experiences. It allows you to place 2D content into 3D space within the camera view.

Creating a Heads-Up Display (HUD)

For this particular AR experience, you’ll need a basic Heads-Up Display (HUD) to show the player important information.

Updating the HUD

With the HUD in place, you need a way to update the displayed message while the game is running.

func updateHUD(_ message: String) {
  guard let sceneView = self.view as? ARSKView else {
    return
  }
  let viewController = sceneView.delegate as! ViewController
  viewController.hudLabel.text = message
}

Adding Game State

A good way to control the game is to add some kind of game state management. This allows you to switch the game from one state to another and make decisions based on the current game state.

public enum GameState {
  case Init
  case TapToStart
  case Playing
  case GameOver
}

Declaring Game Variables

Other than the game state, you’ll use a few other variables to control important aspects of your game.

var gameState = GameState.Init
var anchor: ARAnchor?
var emojis = "😁😂😛😝😋😜🤪😎🤓🤖🎃💀🤡"
var spawnTime : TimeInterval = 0
var score : Int = 0
var lives : Int = 10


public func startGame() {
  gameState = .TapToStart
  updateHUD("- TAP TO START -")
}

public func playGame() {
  gameState = .Playing
  score = 0
  lives = 10
  spawnTime = 0
}

public func stopGame() {
  gameState = .GameOver
  updateHUD("GAME OVER! SCORE: " + String(score))
}

switch (gameState)
{
  case .Init:
    break
      
  case .TapToStart:
    playGame()
    break
      
  case .Playing:
    //checkTouches(touches)
    break
      
  case .GameOver:
    startGame()
    break
}

Creating a Spawn Point

With all that in place, it’s time to start the game. When the app starts, the view controller will load Scene.sks. Once loaded, the app presents the scene to the user and calls didMove(to:). This is a great place to start the game.

startGame()

func addAnchor() {
  // 1
  guard let sceneView = self.view as? ARSKView else {
    return
  }
  // 2
  if let currentFrame = sceneView.session.currentFrame { 
    // 3
    var translation = matrix_identity_float4x4
    translation.columns.3.z = -0.5
    let transform = simd_mul(currentFrame.camera.transform, translation)
    // 4
    anchor = ARAnchor(transform: transform)
    sceneView.session.add(anchor: anchor!)
  }
}

addAnchor()

func removeAnchor() {
  guard let sceneView = self.view as? ARSKView else {
    return
  }
  if anchor != nil {
    sceneView.session.remove(anchor: anchor!)
  }
}

removeAnchor()

ARSKViewDelegate

If you recall, the ViewController adopted the ARSKViewDelegate protocol. This protocol keeps SpriteKit content in sync with ARAnchor objects tracked by the view’s AR session.

Adding a Spawn Point

After the app creates the AR anchor, you’ll use the delegate to provide a SKNode for the new anchor. This SpriteKit node acts as the Spawn Point for the game.

// 1
let spawnNode = SKNode()
spawnNode.name = "SpawnPoint"
// 2
let boxNode = SKLabelNode(text: "🆘")
boxNode.verticalAlignmentMode = .center
boxNode.horizontalAlignmentMode = .center
boxNode.zPosition = 100
boxNode.setScale(1.5)
spawnNode.addChild(boxNode)
// 3    
return spawnNode

Handling Problems with the AR Session

Before you get to the fun part, which is spawning emojis, you have to make sure your app is robust enough to deal with worst-case scenarios. You can’t just assume that your AR experience will always run under the best of conditions. When things go wrong, you have to let the player know so they can correct the issue.

func showAlert(_ title: String, _ message: String) {
  let alert = UIAlertController(title: title, message: message, 
    preferredStyle: .alert)
  alert.addAction(UIAlertAction(title: "OK",
    style: UIAlertAction.Style.default, handler: nil)) 
  self.present(alert, animated: true, completion: nil)
}

Handling AR Session Failures

The only thing you really can do when an AR session fails is to inform the player of the issue.

showAlert("Session Failure", error.localizedDescription)

Handling Camera Tracking Issues

When the AR tracking conditions degrade, you can check a few things to try to determine what the problem is. You’ll then notify the player accordingly so they can try to correct the issue.

func session(_ session: ARSession, 
  cameraDidChangeTrackingState camera: ARCamera) {
// 1
switch camera.trackingState {
  case .normal: break
  case .notAvailable:
    showAlert("Tracking Limited", "AR not available")
    break
  // 2
  case .limited(let reason):
    switch reason {
    case .initializing, .relocalizing: break
    case .excessiveMotion:
      showAlert("Tracking Limited", "Excessive motion!")
      break
    case .insufficientFeatures:
      showAlert("Tracking Limited", "Insufficient features!")
      break
    default: break
    }
  }
}

Handling AR Session Interruptions

If something like a phone call or switching to another app interrupts the AR session, there’s a good chance you’ll have to restart everything. Luckily, there are delegates that help you handle this.

showAlert("AR Session", "Session was interrupted!")

let scene = sceneView.scene as! Scene
scene.startGame()

Key Points

Fantastic, you’ve reached the end of this chapter. You can find a copy of the project in its current state under final/EmojiPop.

12. ARKit
14. Raycasting & Physics
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2023 Kodeco Inc.

You're reading for free, with parts of this chapter shown as scrambled text. Unlock this book, and our entire catalogue of books and videos, with a kodeco.com Professional subscription.

Unlock now

Access this book

To highlight or take notes, you'll need to own this book in a subscription or purchased by itself.

Read for Free with a Subscription

Buy this book for $59.99