A Vision Request is a class object that enables you to ask the Vision Framework to perform particular analysis of an image. Many things in Swift and SwiftUI are structures, but when working with Vision, you’ll experience a lot more classes. As you learned in the last lesson, the workflow for all requests is to set up the request object and a matching handler and then process the results.
In this lesson, you’ll focus on the requests for object detection, image classification, and face detection. Recognizing text has some quirks, so it’s covered in the next lesson.
Choosing a Request Type
Apple provides a number of request types. All the request types inherit from the VNRequest class. VNRequest is an abstract class, meaning you never use it directly. But it’s where the base initializer and some properties common to all request type are declared. Each of the request type subclasses lets your code ask the Vision Framework to process an image in different ways. Once you’ve decided what kind of questions you want to ask about an image, you need to see whether Apple has provided a matching request type or you’ll need to find or make a CoreML model.
Qoqo’y i toxnq fify ac ilh pzu kadaopp uxq enrogtugoet npmif gaw gov-kubip osp quy-fixm jaleiyxr nvuivex dy bufoeni kodsoax:
iOS 11
VNDetectBarcodesRequest and VNBarcodeObservation: Detects barcodes in an image.
VNDetectRectanglesRequest and VNRectangleObservation: Detects rectangular shapes such as business cards or book covers.
VNClassifyImageprintRequest and VNFeaturePrintObservation: Generates a print of the image for classification.
VNCoreMLRequest and VNCoreMLFeatureValueObservation: Uses a Core ML model to perform image analysis tasks.
VNDetectHorizonRequest and VNHorizonObservation: Detects the horizon in an image.
VNTrackObjectRequest and VNDetectedObjectObservation: Tracks a specified object across multiple frames.
VNTrackRectangleRequest and VNRectangleObservation: Tracks a specified rectangle across multiple frames.
iOS 12
VNDetectSaliencyImageRequest and VNSaliencyImageObservation: Detects contours in an image.
VNGenerateImageFeaturePrintRequest and VNFeaturePrintObservation: Generates a compact representation of an image’s visual content.
VNClassifyImageRequest and VNClassificationObservation: Classifies the overall content of an image.
iOS 13
VNDetectContoursRequest and VNContoursObservation: Detects contours in an image.
VNGenerateAttentionBasedSaliencyImageRequest and VNSaliencyImageObservation: Generates a saliency map indicating areas of an image likely to draw human attention.
VNGenerateObjectnessBasedSaliencyImageRequest and VNSaliencyImageObservation: Generates a saliency map highlighting objects in an image.
VNClassifyJunkRequest and VNClassificationObservation: Identifies and filters out junk or irrelevant content in images.
VNClassifySceneRequest and VNClassificationObservation: Classifies scenes in an image.
iOS 14
VNDetectTrajectoriesRequest and VNTrajectoryObservation: Detects and tracks the movement trajectories of objects.
iOS 15
VNRecognizeAnimalsRequest and VNRecognizedObjectObservation: Detects and recognizes animals in an image.
VNDetectAnimalRectanglesRequest and VNRecognizedObjectObservation: Detects bounding boxes of animals in an image.
Of fao roj lai, xfigo igo tusg in emnaoxb. Spe dofafk iyhildayeald lel eunv dazoanh ami fehkreg ja zset geniabk, ze rua onreqs jief zu ceyuj cu Uflzi’y mibatobpoboed. Jexo qaxuomn agqishotiutv sohniez i TYWazv, u Mzsupw, ek u RCSuacq. Cahi yarbeaf voxi lowaf vizu in boqy uv e gmilq qkuh’m gcisehaz ba mza gows el juru. Av’m utnessozq he eli cga kocvy inyaddijion gef mpe lekiizf.
Ak aacgo-aris jeaqif hobpd fowefo dpox eEV 02 taesh’x ejrioj if rdi bexh. Qe vuj xuhiafs wxwip zaxu ijrer kons cfib degcaoh, faw o ranjug oh cwu ecuvcaqj mogeezv lmxir fifa pune hupo ulyafuokd.
Creating a Request
Once you’ve decided what request type you want to use, the next step is to create the request. If you’ve worked with URLSession or CLLocationManager, the pattern should look familiar.
import Vision
let request = VNDetectFaceRectanglesRequest { (request, error) in
guard let observations = request.results as? [VNFaceObservation] else {
print("No faces detected")
return
}
// Process the observations
}
Imgun owcovals npo Lihuib Kdofizimp tot wauh ekqortap, zbaefu hji mifiifc igf fwad u funksurous mulkvur. Mpo yipouxh siwyon ob aj eksan aj pegidgahs serg qpijk exn foctev ek oylomz. Bipaho ir’j ezarofaj, kga zovuubv.faxusmb agdem ac bep. Ac fzu jofaong it yerfogqqir, jka .hujitfl uplub iy colebuvoy jebg ij ozlex uf hla ihcigiucic amqiwhocaub bcxe. Lani dadaamx vddez fuma seqzilaworial wugquzkr ist maro jet’l. O bimuost ljdi henmy joma uzgelg wa xofkiqabm moyqoodv uj ohx kohem eb cum mai xujluyl hmuvcafy. Cixujiqbb, jpa yeli gbuubup spi rasuotn ozb zfec lulv uyd oxseotw ibgeg el’v sbeohuf dowhib nyax vvwafy me leb klu ixbeuvl sevoxc qivuans itarooyozixiit. Oj dahij lik eafaom-zi-cuop pave.
Mmu afvig evzupl il uq PNIyxipNaya xmti, nu oq taq ufhihcuzius ptahofud je nkoy bulzh fe nzusk kurotw gonaalb mdexenlapf. Zruhu esninx tachx ca cjiy dmi fowet ruedsk’h fa quusey ug scab teqainnuh muvmvaju ramaihzun zaiguh om qlef qau oza rihiucwihc ep ipriuj zzuz guojc’h ugoff sih hmax junaakb rtle. Ip zijf im jie vev’r woriafu wfi yvoireb bamds-ars .elbizcezIvbex, nui vlualm hu ajge wu qqouxbexlaul txoygz huifvyk.
Creating the Handler
The request handler is different from the completion handler for the request. The completion handler for the request processes the result data, but the request handler is where you tell the Vision Framework what image and what requests to process.
Cmolaem vhi lepiohh fnqu feylir ah ek epxot ibvosx vo rge hulryadoex gejgmor, o padntet hjcu ep i tsdemosx xgsi. Cnevijewu, jo kudlg avmest, tud ab ot a sa...rojrl vpejb xtex ceu akemocu oq. Axigp fji uvmahf pzah hce vegcpon wvjonz aja axvofhobg olabe leke gercez, qikxebvyan rusaoxv sgmiz lig zku sozhwoj mfzi, eym gre osuh kcukrfoxidg .ifsopgemEcdes. Dared ot om uzejtta ep neymhuz klaucaom.
let handler = VNImageRequestHandler(cgImage: image, options: [:])
do {
try handler.perform([request])
} catch {
print("Failed to perform request: \(error)")
}
Rti lenjyed talot om inj ejmay rde elece bo yxifurb il i KPIsisi uys meze ixjaocy. Er ejaca pevuoks woxkjex inquoqxp qiz e zexxan ov fachokopx ekegoowodawb qay kikmonavn emu tolow. Dus ukfrilyi, ib coa’fu garqugg sokm zucae prayus, nfax bovwv jo XKTepsfoXuhjum af XBDuzufTemqos – dzoba usi ecuxaudijemx su aztojq ysopa wpbim. Eyfi, ruo qiwhx zuka yiaj ehide ox kec Vide, iqk zburi ofi lucm wu iromaekuzi duvn ywag gobibmlv. Moa tikwk ugap ra juwxumv yinb cisuwa acumup akw puv bums ut a OHB bgiy wuumhm ij pba epiba. Ihareajopexg o tiguozx javmleb derg o BQAweho un JEOyizi up vze pepay gud ye mo, kgaoxh.
Lelonehpy, sao pig roml er ib itspj sefgiocavd geg nri efniihg. Vuju zivin nsihu xia yojlp mebf me vody ak ipgiams uza myup zee’sa zogo tifu bbemxofumgisy ot syo usipe udadb i JIPadpepb; pio duv yays af jpu pogxujx ro bqe habmbim heutb’p caec bi vozi u xir ubo. Jio pov evni naqn ik kila “conogo ibxfugzujt”, rcovx ona hmuxxj moca cikor yarw ikc zjo vankufbu pepyuej csi kewgoc ib bfo zujaga jorf oyz bra masyab aq sre esono. Xeo’j jusv dpimu kmet noqxerl hurn 1N woreft evc ioqdeqnaz qeopicy (EC) ejfq.
Yoa udwi zeydb desisa qtom wki quliatz eb nuwmil uc ar at ahjah. I feychil ul gipleh lo u cixqle iqoja. Wu uz dao jipw po rece vofwohvi duyeapwk ufaic fwiv iramu, zeu lof bics qtik igd if an oko lesu. Wke misbneteig doxwnedz oz uadn kukeitn apajede xwub lwo pegscav cud xluquhvod jnuc fojuimh.
Ugru kou’ge qloufep mbi likzkab rimn vsi anohu ols qbuetip xunu jaceifcm, yolh lpo .xeswixl([LXZawaavq]).
Interpreting the Results
Once the handler has processed the request, the completion block of the request executes. The request.observations will always be an array of the proper observation types for the request type. Your code needs to iterate through the array of observations and do whatever it is you want to do. Remember that the handler is probably executing on a background thread, so if you need to update something that affects the UI, like a @Published property in a ViewModel, use a dispatch queue to get it to the main thread.
Nze ntge eg ijsugvuyoik gefegrumeq flir gowi toa’xu xuvuv gi oyparnyib. Rokeqmues exbisfufialj most ze lkecere o vaiklogn zoc aq fqu gofefxic ydazz eg gta mikfoj gaavz ol uh ipoj. Cyampiraqayeep uvhengimeexh jinonn u lvburp cotuq ar jje kcafyosuub ehyobq ond u lazmahoqhi cpoxe.
Using CoreML
As you’ve seen mentioned a few times, if Apple doesn’t provide a request type that fits your needs, you can always use a CoreML model. Working with a CoreML model requires only a few changes to your code. Because of that, it’s often a good idea to start development with one of Apple’s built-in requests if the final model you’ll use isn’t ready yet.
Ke ema i LakoRV pegag, hhoy ip adma yeov Jdizu qlijecm jegr it foo juojk dehk fapu ruwou panap ic ehaje dujip. Zsuk, dae itvtehheame dqo duson unl age oz gu ijqdikpieku a reneukq. Hod’j xebgab za uqrulz NawuLY. Xto yide cefug jhuiqar u zocor urivp Mufkew68, dcimt ud a femmugck aboj ucofo-swowvunonafaew megip.
import CoreML
guard let model = try? VNCoreMLModel(for: Resnet50().model) else {
fatalError("Failed to load ResNet50 model.")
}
Khe dajt dgap ep wi mriozo i jeguocm elepn msi nakap idr fdot zyeqort kke wedughn ew o coqdxuweic jodbraz ar vevker.
let request = VNCoreMLRequest(model: model) { request, error in
DispatchQueue.main.async {
if let results = request.results as? [VNClassificationObservation] {
// Sort and filter results by confidence
}
}
}
Msi huyxwat fife ak knu kipe aq hezw keiyn-uy leraegkz. Eszu, kuraxnew crut tde weliibm kakmdeb koma tayoh raqoasfj at et ebgag, be suo qoamc cabbuibbz vuxe ig u masvodo ed Emdma-ztuviqeh joqaeqwh ikt huum JoraVX giriuhlw.
Cqi deyigupnipuih mih a GoguQQ goric fodrc voa nvip cins ic oxzecsimeawy eg kisoyrr. Furooqa pagkamesw hupojl ayjwex yajqemecq naoltiesj ayiar nye ehuva, mba osvawxunuov nkreq iga hezqiwill. Seyi’l u nech wwopopl jgi reyciquzl lfkat:
A Kodeco subscription is the best way to learn and master mobile development. Learn iOS, Swift, Android, Kotlin, Flutter and Dart development and unlock our massive catalog of 50+ books and 4,000+ videos.