Beginning ARKit

In this course, you will explore ARKit from getting to know how it works, to adding new furniture in your room and identifying 3D objects from the camera. By Joey deVilla.

5 (5) · 1 Review

Save for later
Comments
Share

Who is this for?

This course is for advanced iOS developers. Experience with SpriteKit or SceneKit is helpful, but not necessary.

Covered concepts

Students will learn:

  • The general principles of augmented reality
  • The "augmented" part of ARKit in the form of SceneKit
  • How to draw simple and rendered 3D shapes
  • How to superimpose shapes over camera images
  • The basics of the "reality" part of ARKit
  • How to sense the iDevice’s position and orientation in space
  • How to detect and interact with horizontal and vertical surfaces
  • How to respond to the users taps and relate them to real and virtual objects in AR space
  • How to recognize known 2D images
  • How to draw AR text in 3D space
  • How to extend iOS's AR capabilities by combining ARKit and Core ML

Part 1: Hello, Augmented World!

01
Toggle description

An overview of the Beginning ARKit course, as well as what you’ll be learning in this part of the course.

Toggle description

Find out what augmented reality is, from the century-old perspective of a book written by the author of The Wizard of Oz.

Toggle description

Learn about ARKit, its history to date, the hardware and software you’ll need to develop and use ARKit apps, and what options ARKit provides for rendering virtual objects.

Toggle description

It’s time to dive right into ARKit programming by starting with the ARKit app template that Xcode provides.

Toggle description

A look at SceneKit and some key concepts, including 3D coordinates, aligning those coordinates with the real world, SceneKit geometries, and SceneKit’s node graph.

Toggle description

It may be a simple one, but it’s the first step to drawing more complex SceneKit scenes.

Toggle description

You just drew a dull object. Now you’ll make it shiny, and learn about diffuse and specular reflection while doing so.

Toggle description

You know how to draw a sphere, and make it shiny and blue. Can you draw a shiny green pyramid?

Earth 1:21
Toggle description

Let's turn that shiny blue sphere into our home planet by covering it with a photo of the Earth’s surface.

Toggle description

Take what you’ve learned from the previous lesson and draw a flat surface with a cat picture on it.

Toggle description

They’re pronounced “oiler”, not “yoo-ler”, and they’re how you can tilt objects at jaunty angles.

Toggle description

Now that you know about Euler angles, apply them to that cat picture.

Actions 3:34
Toggle description

Actions are objects that make changes to SceneKit objects. They can move objects, rotate them, grow and shrink them, change their appearance and more. Learn how to use them to put a little spin on the Earth!

Toggle description

All this time, you’ve been drawing simple shapes using SceneKit geometries. Let’s draw something fancier.

Conclusion 0:48
Toggle description

You’ve just completed a crash course in SceneKit and written some basic AR app. Let's review where you are and see what comes next.

Part 2: Happy AR Painter

Toggle description

Let's review what you'll be learning in this part of the course, and review the history of its inspiration, the legendary Bob Ross.

Toggle description

Before you begin with this exercise, you’ll need to learn a couple of matrix math concepts, as well as an AR SceneKit property called pointOfView.

Toggle description

So far, you’ve been writing apps that simply draw images over the real world. It’s now time to write apps that respond the to real world. Your first step: finding out where in the real world the device is, and which way its camera is facing.

Toggle description

What’s with the other tab in this app? You’ll find out here.

Toggle description

Now that you can determine the device’s position and orientation in the real world, you can use that information to leave a trail of virtual objects, effectively “painting” in 3D space.

Toggle description

Your “paint” trails are SceneKit objects, and SceneKit objects can have SceneKit actions applied to them. Let’s liven them up by animating them.

Conclusion 0:43
Toggle description

You’ve just written a 3D painting app! Let's review where you are and see what comes next.

Part 3: Raykea

Toggle description

In this part of the course, you’ll build an app similar to one made by everyone’s favorite Swedish semi-disposable furniture store.

Two Worlds 3:52
Toggle description

A look at the “two worlds” theory of the real and virtual world, as well as the important concept of AR anchors.

Toggle description

In this episode, you’ll configure the app to detect horizontal and vertical planar surfaces in the real world, and get it to tell you where they are.

Toggle description

It often helps to understand the theory behind how something works. Let’s take a look at how ARKit detects horizontal planes (relatively easy) and vertical ones (a little trickier).

Now that we now how to detect real-world planes, let’s draw AR planes over them. We’ll draw a grid that says “place furniture here” over detected horizontal surfaces, and a poster of Ray over detected vertical surfaces.

Toggle description

We return to the “two worlds” theory to talk about the fact that as an AR session goes on, ARKit learns more about the real world, and updates its virtual world to match.

Now that you know that ARKit revises its internal map of the world as it learns more about it, you’ll take advantage of that learning to revise the planes you draw in AR space.

Hit Tests 1:42
Toggle description

In Raykea, the user taps on the “place furniture here” grid on the screen, and that tap specifies where the furniture goes. But how do we translate those taps into a 3-dimensional location for the virtual furniture? With hit tests.

Toggle description

Let’s use hit tests to see if a detected horizontal surface is onscreen at the moment.

Toggle description

Putting it all together, you'll take everything we’ve covered and finish building the app.

Conclusion 0:35
Toggle description

Congratulations — you’ve written a scaled-down version of one of the most popular AR apps! Let's review where you are and see what comes next.

Part 4: BaedekAR

Toggle description

In this section, you’ll be building an AR guidebook — a Baedeker — for paintings and photos in galleries, museums, or any place that displays 2D art.

Toggle description

The first step in building BaedekAR is providing it with the set of images that it should detect and recognize. You’ll do that in this step.

With the reference images imported into the app, it’s time to take the first step and detect those images in the real world, making note of their location and size.

Toggle description

Now that the app can detect the reference images in the real world, let’s make it so that the user can tap them to find out more about them.

Toggle description

Let’s improve the app by displaying the name of the detected image using AR text. We’ll also make use of billboard constraints to make sure that the text is always facing the user, so that it’s easy to read.

Toggle description

What if there were images that we wanted to block from the user?

Toggle description

Let’s protect the user from scary clown images by blocking them with a soothing picture of Ray.

Conclusion 0:26
Toggle description

Congratulations — you’ve written an AR app that can detect known real-world images and block scary ones! Let's review where you are and see what comes next.

Part 5: Into the Third Dimension

Toggle description

In this part of the course, you’ll take BaedekAR into the third dimension by using the same concepts to detect known 3D objects instead of 2D ones.

Toggle description

Before you can detect 3D objects in the real world, you’ll need to provide reference objects, which means that you’ll have to scan them first. You’ll download Apple’s app for scanning 3D objects and put it to use.

Toggle description

Now that you’ve scanned your reference objects, it’s time to import them into the app.

With the reference images imported into the app, now you can detect their real-world counterparts.

Conclusion 8:01
Toggle description

Congratulations — you’ve written an AR app that can detect known real-world objects! You’ve also completed the course. Let's review where you are and see where to go from here.