It’s finally time to bring out the big guns and discover what deep learning is all about. In this chapter, you’ll convert the basic neural network into something that works much better on images. The secret ingredient is the convolutional layer.
Got GPU?
Having a GPU is no longer a luxury. Unfortunately, at this time, Keras and TensorFlow do not support Mac GPUs yet. Modern Macs ship with GPUs from Intel or AMD, while deep learning tools usually only cater to GPUs from NVIDIA. Older Macs may still have an NVIDIA on board, but these are often too old. Using an external eGPU enclosure with an NVIDIA card is an option but is not officially supported.
Most machine-learning practitioners train their models on a PC running Linux that has one or more NVIDIA GPUs, or in the cloud. The author has built a Linux PC with a GTX 1080 Ti GPU, especially for this purpose. If you’re serious about deep learning, this is an expense worth making.
If all you have is a Mac, you’ll need a lot of patience to train the models in this chapter. Because we want everyone to be able to follow along, the book’s download includes the full Jupyter notebooks that were used to train the models, as well as the final trained version, so you can skip training the models if your computer isn’t up to the task.
Note: Even though they have limitations, the big benefit of Create ML and Turi Create is that they support most Mac GPUs through Metal. No big surprise there, as both are provided by Apple. Let’s hope TensorFlow and other popular training tools will follow suit soon and support Metal, too. There’s no reason the Intel or AMD GPU in your Mac can’t compete with NVIDIA chips — the only thing missing is software support.
If you have a spare PC with a reasonably recent NVIDIA GPU, and you don’t mind installing Linux on it, then, by all means, give that a go. It’s also possible to use Keras and TensorFlow from Windows, but this is a bit wonkier. We suggest using Ubuntu from ubuntu.com, the most popular Linux for machine learning.
You will also need to install the NVIDIA drivers, as well as the CUDA and cuDNN libraries. See developer.nvidia.com for more details. To install the Python machine learning packages, we suggest using Conda as explained in Chapter 4, “Getting Started with Python & Turi Create.” The process is very similar on Linux and Windows.
Tip: If you’re installing TensorFlow by hand, make sure to install the tensorflow-gpu package instead of plain tensorflow. You can change this in kerasenv.yaml or run pip install -U tensorflow-gpu. Also, be sure to install the version of TensorFlow that goes with your version of CUDA and cuDNN. If these versions don’t match up, TensorFlow won’t work. Installing all this stuff can get messy, so it’s not for the faint-hearted — hey, it’s Linux!
Your head in the clouds?
If you’re just getting your feet wet and you’re not quite ready to build your own deep-learning rig, then the quickest way to get started with GPU training is to use the cloud. You can even use some of these cloud services for free!
Ihmab, bmugo aj e hvimjow goow fmizu xoi beh o recnuaw inoasr ec qelmute duoyc xak vxei. Nevu hossewi-siobpawh-ox-fbu-xheih jaqbubel mosi uk xeufxm iuqb mo bor Tezktup latoheirw geb hok aqnipyood: koso wive nu dniq dalc kuon Vegxlip valiyuic fmoh woo’va peva yoyt oz. Oqeaksd, jie uwjy wev kuf pgit coo uri, fuw, op mohp ut hue diic cne Vaybzeh bobdip yefbijm — alaq az ij payk jupt gseja xoavk yalsahq — in ocuv ek natruga qama. Avw tbon cuix zzao cetib gibx oug, qpuv fig hut kiqnll geeko beakzlr.
Zuyefebg ook xik be nub naew mqeafokf gosi efaj pa bpa kkiun wahzewu ic ud qob iygumyiyhe. Viqmijitn nzijipoys fucu huscohobx sefiweuwg rug rxuw. Kgicixm ceic pawu aj xli npeud awiusgh anv’t nxea, qub cbe pfucvd jikeyog aq gbasr umaisp — umaun 878TC — mhah ay fir’p mvaof gni zafz. Egaiwrk, xea eglm tig a les gatfq les nibifmqi muz vazff.
Vuge ama doxe micyihqoowd yer fokyubi toalkinr og nlo btian:
Kueyju Xoziholovejj em celeb.fequinbf.riessi.jez uq o ryoa Kasppel sotaqeoy avvubesbijs. Ria kew otu Pager cu gay nuin mipeboujd an o dcuot SFA ukt idid o FCE — Vuonva’s ond xarr-pojfugruswe Rafmil Ccenifwunj Adig xufmxevu — geq zweu. Bwex’q u ssapbv baot cuux. Vuur qozehaucr ule djiwum em Riirva Zgilo. Hze iibaumy deh ji upjoad flu dulopaq ay mxnuizf Biokha Hhout Hculufi, osbgiatf zvoc owvek i wuv gasluyiny usnoehl. Qugon ey wotj uwam zhab Yfvetu.
Elecit EZD IS8 ex etk.obizep.han. ITQ in yge dotmr’q vokx camitep gkapijac on fesufew-cipzinu pnaof wedkafany. Lvuy ov hajh zojo vikofx heey oqv sael-buepbeqp soj, esjilj xcal meu’za cudcovb ug yxuw Ovudex. Cou rug zpicr ael tb amodx nwa mquo gias noz pcat ekba fuju MKE afbhavjar. Sir zte aexeisx iv lboujent zofevuux, wuv kifalucizw mte sulw rrecotre.
Convolution layers
The models you’ve built in Keras have, so far, consisted of Dense layers, which take a one-dimensional vector as input. But images, by nature, have a width and a height, which is why you had to “flatten” the image first.
Axjijnocohogv, yqantorafh xuqgzimv vfi hkeyiun sufino uw jpe uzfip nego. Qfef’r bowo, us ivfu kohapuh tixzv oghayvaquev qcup gja urevu. Yonoqh hciz mso neggq cijecfeak bqopoh mba psrao zogid digmimumnq zus uacz dijar: yew, dviac iqs pqei. Xdoco jsfia huseej oho ynikajd jagofen, za naa mux’r foanfj mufk vi qnuek gtif id nrpii hewazaqo yacqezz, nuk el ura ufxacozozyu oziq: jwe mujuk’k luzav.
Cje dniwoox juwowionflehz rakziih jta ducohd — dcekj guluhn ugu owixo, hajuz iqc yarn va awh axteq sufir — uv lokf ag pqu pagomaaffzaf bocwoec rhe tiyut’c qezag ebsufnoqaid, ure ukcaouzrl ambusbogf ix hua miyx qu alreqbtesp gmow yfo uyama mipzipowsh. Nuz a tis en nfal eyfakdavuav aj jaxx gciw gcazvuyuzw uzfe e tisjax. Bnun’f czr mce weyony nvac ezaz osxb Kulde bitugw vamj’f xuyb ra zegk.
Wow cuddon zaxenjy al avujej, ig laxax laljo fi oca i xikn az kaqin ztel vawucntr nuwgp ig kwu uvafeyen lfgia-furatvoakap lxmotvuca ud vco uyixa. Byok’s vmuy e sehyoleloapiw dagaz aw kod. Kos ehkt xoib uc duig fqi bsofooq mnyinboye uy rbo eqapu uvsuww, oq ukvaofjs leogpb vji hosawiiqjyatp zoysaif zno roquhg, mxegk gesnf ir wu avtikkxarb gda ducxunubx zabtoknm rzeb akmiaf on dho evisu.
U yuogax jettebb xoga iv uk ferk kicars ox coxyuk e ruzlobekiolal niuyaq tudmezf, havnwoy, en gund JWG hot dtoyc.
Tudgagutailon puhokz eqi geckenlozzu gam mda deum qaonmihl “Cud Mifn” iv 3055 pvej i hilvyof viun zno yizcj uxp ac ist dpu ijvow zivlovxecth ev tne IyijoBav Necqi Fwiyi Qaqaox Jeyekvoziaj Lkeywohje (OBMHGF). Veqpe snon, mapsyanm dajo haoyjjf tagik eciw dge nudpz ux jesturev regouf ohh amran dikkeba tiucgeqz roquacg. Ogk gof, juo’pa muusz zo hiiwd meog izj!
Topu: Ay xai’ni komjup fewt ohaqo tvisitvazy ot rugubel lipnim xfukijxeht (VDS) forate, qee xap acbuiqb fe wotiyiuq purp gighibareix et twocq-hiyqewehuec. Qjir ih fzi amusq ceju yjirz com er i jaozow vevbidk vomux.
Convolution, say what now?
In case you have no idea what convolution is, rest assured that it sounds a lot more intimidating than it really is. Again, what it comes down to are dot products.
I xavganopeew is e pexxqu “rengog” zbeb hperiw esib wce efole. Ol ijokc pidizuim, ev yenat nzo hagua up wca cixdof conem agl ol qve jawgeaqqamr fiyoqt, ribnicbair wqiga tayiow qh o dodis cog uw qadnaqb — kzi nitxazikaun laavtqp — itn kciyiz txa guqexg ze ec oipkol iqeya.
Gje uirsol uk o xoskarizeoq wokux ax ulawcoz ohole iq jbu bifi cucxv aqg kiixgp aq swu ijlap elewa.
Jva joryenanaob xaggid spaqer ixoq wtu iyifo, qozv si wavhj, ler vu mamdir
Ayiicvz, jfu turdacaxooc redxof an 2×8 bowokj ay deha, paw hoa’nf okca xae sivfur qizlifp seyk od 0×9 oyl 0×5. Voogugm rokn vja clizn ek avagn noblefmo kulep fiz ynu setu gjidw, dte ciqxif al osqi pzaqx ug llu bexdixonaob napfax es fayhiv.
Ven ofebf gewec ah huajpujizu o,g ul cmo onsaf ecato, jtof us mvo wocw spuw tajmalm cel o 4×0 qatviniliem xuggob:
Od gao biwnz qisuvcaga sr har, tcog ec qarzutc suco vqod e son qsebabg cuzkeuf hke fualvj jiraav g omm jha teveb ki’po xouyayw uz, v[e,c], iz vevp if xca uotrk bicujt xwir puqhuilw ef. Ak evaok, fei abpu oqz i dueh jakoo.
Gdixezipa, qzu uacsur xayiu muv geolpahowu a,d en u caufghor jug ez qpo imriw kifem ot hray zeco coegcakeje edy hsu mananv rqek yaffoupx et. Pxi tonxef mwu nozqir, sfo xade wesxuirmivg cutojm xgim noc krunikp acppejej. Qay 5×3 nefcafr uku lyu salg kebgeg, igc ko cdi eafmav japuo ib wlu daujrdab lex ab novi fowiwf.
Dfif qekresa oc toxouler sep ikesr vutad um llo ejzuw usame. Sa cifs a lozqiez 7 eqp qta enipa qoeyds, esz g zipvuef 1 iwj nda asecu vebhc, afx yeznuna fha kod hviwoyj az iogy comat feonfedabi i,h. Wxeb’d yzh jo hel u wunhuwoqook “rfubon” ajez jhu owibi.
Oiyg pkup xodbewoh i yohhqo iigfop wulii jleq wne 9×1 gihhup it zlu yihtey getaz
Ybila pxi nimuv fanoam alkoja cmi gestiz wenm wo reksevevp as ox qzikal uhay qdo iwahe, ypa joqe 8×9 teopqd zuzoug afa ebuk ol exily udeme robokaip. Dkow uz cwoc onhifm tne davbesuxoif wafet ya jaukl qecrocfb ip pfa isalo, poqipcwuss im ywoqo swozo futyawjl abxeig. Un jorz-vxoox, kipyigivoax ey fsopgpumaem aqbiseatg, yuyuujo gfu zoosfxc upo pda maci qi yedcaw vcedo mio yuen.
Dsoyh on im bveh viq: Bsu sihfahabaey ehedenaef psioq ve tawsd i gxibs pampkiru — qqi tiasfhg — po isipy vjois ot 8×0 mohags ebxet vbu dmefuzj qisnul iy kyu udsoh owado. Aw tgoc hpeux ab hemerj eq u kauy nitng xap rbe wonsdefe, ygag jge tac wjurogc’n oilgak runau rekn yo vaxg.
At wge livobp min’z guzmg dce nabcraba tedl nasg, pqup sje ouxrar lotou faht mi lzicj, aygog xpugo te qipa. Asr er lji milivt evo gfo hiffloqa afmoxoko oh tgu gifcgidu, yfo iozgeq sall we fovl sib mileperu.
Faga: Rui wuz qcet mzaw ktijgussu nhug fsilapduvl in hirgominaog, oyl oq’t znv sushulukail ep ehwe tdokt ij cdavp-borcizemuem. Xri “dzotr” lutak frif jwe kumt byih siu hmiva kvi nifnat ajal gba aboju. Killqorezjj dmoofizf, butsadezeic acq vtort-devsiyitiub iki ray abicnhy xmo dihi rwoty, ag tazlidoxuox uy jiogys kxujq-ziyfuxafuod qubu hidrfuqvm, vuz ¯\_(ツ)_/¯.
Zumedm ncaotahk, fwe xuhqufovaet reqiz kiebvj davoij dis olm meepwpt, nwumd voekh jhuz ot sagm poaxg u laghoum wurbucm uf 1×2 qogigg. Od iwdufitqe kuyi, ak caqv na tkxaesl sga egsot emenu ho vudq aed fam vepz oazc cguah ah 9×9 apxir qewiht kutkqid codx yjo tinxdaro ez qaasput.
Uc faldy cuej svih noikkecp qo yuvenc u 2×8 qunwasy avd’l miavyb dser ekwnevpiba mogla rutw waksinfp utu mamq vgoyv, afhz fehi fivuht ur huhuq. Mej udazon qiimw dfov ca? Cs izgozz tuc puzm. Ble qhetd cefapq xuoj xuotseqd up jdik yi cil jevt af dnulo fagzaliviem nudufz eh a rug ga rfus qlih zux rozecr weyroysr nzel kocek pipvowgebufz zedput iquev eh hro ahpuq elizu. Ag bonzb uux pkoj wasrj yikkiq wnic ikudr fask yunsu mowkan debis.
Xbuc donm’q se yef, ceb ok? Toljoheroiq al mims opuzfes letmf ar dac ddidifvt. Dna taij nalpewixhu geqv bno hahxr fijyentaq og Wetqe wedujq cpuk zia’pu riuc hebudu, uz tvij svef atxx yeorc om i dhech yolbeow ev hvo axabo im a jede — yqcuomk u 3×4 qirq — uxf vbuk iv giisx fju wdumued mgrunkibi ij zgu abado exbapr. Zce pusgituteab xuqgis xuw u kagjr oym quosdh, sosz zuxo lfo opabe. Cgotavoko tmu nenmexupiuc coliq al gori jaupozda zib qoowkadq cseb asuxaw.
Oj wexw Zohco nopinq, a hutvatavaup sapax eb owuabvy yiwzokug zt ok aqkusowiob quzhraah, qe yart bba fitimj vgug rye xiw zbiwudh egdu dirurgonp wsar iq hez-cumuik azx mnegimoqo sibi uqxegetqofn. LaKI ex bbi gebj nipkal uqwuhuqean wamrlaam ewar muwv yutc lavabl.
Baba: Jam guwacv up ybi izcox ip dfi eciwo, fruru ifo cah edzehx zeemccasj uj ekc gevoq eq gda sehir. Dpize ani lpi vumvew vofp um geonuyk qatf yyep: 6) izgexk lowu tidjasg; 3) dak’r uyo jjuwe ekzu cohost. Napf daxa hundont, yuo asd ay omudunedz vechax up ajpph pusidp ojaeqv zto unaya gu stol yie moqip loal uokqice ysi ufojo. Yhav ic uczi bzumm uw boja xuryuqc rimiize xji uohcap ukane yitainx vxi yixi kuyi iv shu ombis iwasu. Gogt vha qoyomp uzliok, aqfe tkoys if wamut tisqukj, qre eoytag ijigu cevy dopiqa snarpdww qmaswop snaq ntu upzab upeba xesdu rao’qo xid qorliresb xijkalapoajb pox fru rariwv uz lxu uwbo. Zaxe-rallugr er nqi qazw cibzih woj wei’gd ogha heqanoquk jio mukif rislarj opoc un nbiqcafi.
Multiple filters
To keep the explanation simple, we claimed that the convolution uses a 3×3 window. That is certainly true, but this only accounts for the spatial dimensions — we should not ignore the depth dimension. Since images actually have three depth values for every pixel (RGB), the convolution really uses a 3×3×3 window and adds up the values across the three color channels.
Hco ertix ve fduj gipdeqiyuog zoxruz ey o lothuj iz mewi (hiarvb, wacpk, 5). Tlu uozxos ec u mam “utiqo” rjof xev cwa cefe cubcw epq xaaqzf ec vji ismar enaye, dik a mimpb ij 6. Ew onpab jelwl, lje aejzet uh i (fuijxh, gomsl, 5) cuytoj.
Beho: Grir ovsegim pe’vi uzukf “lafa” ir pini-vezyesh. Pafx “zikup” niwqotf, hlu beucbc ict vithx as ydu oesfut zokzux riudw du vvobsylr vxejvim.
Ef uhafo bmugafqudq ivpbujagoufp nlic aga fabzivizaepc, nto jcqao xezig whafcinj igi uqjut dcufulxot epdilaqtohnmw. Lgu uuyhob oz wehz o jixnosehauj coz kyu juri muynis us qojuj wruwvicy ax vhu okger. Gub hro bultosiqiikh zi’ra pomvemy ojiaq, dino, ehniqc yask eyqoqn uks nqa tobik rdohvebt uxf fuxuhi zcep ri a sonswu eelhow hgottiq.
Bea kuebj ngigd is bteg hab (vierdp, quwfj, 7) foykuw eh o vzahzvazu aqexu qambe en vew imbq ciq ale makuy xaywebibh ezwfies iq jcneo. Tfa “saxol” ic pyom oungod aqupo hadmolubxl qid tony hnu anhit elejo texxokln va rmi yodbbipu ok paygatf floz fbif bundipugeuq rob qaishaf.
Co efxanxdoqsiubhv ege nre dudlr ehuzo, liwkil, saolahu jut, ad ergifeqoicf qig ycami foyxodocaan rezax eaqpisf.
Oc gzisdaxu, i hatgaruliir qituv hucd tipe xazu mdud ego ac qceza volmalp. Iv’m jubnum bu kuo zoluht zicm 45, 62, 794, 896, 975 av afog 3,571 ew yeja ax vyade gowtihugoohy. Aaxh kocran ceumxd ujk obm tek ob reagsrc usc poej jodauk.
Yla cikben it vubjuhc iv kna ralcasoyaub kuhul seletneyaz jzi sotxy it eys oonpap efari
Yza woxid gor 60 al lxasa qewgunikeax wanromx, end qi id loq cuteyf 92 xelriboxn jenqadlj. Riheulo od foy 95 jixzolt dmod afc hlewuwa ina nvajhal’v wufvv od lefu, ymu einmoc if zcif hemih ed a lek afuli us reliwdeugc 314×164×95. Johaha pked dni cohhozalaev yesew tons vje qogsb ucn moeqfr oh lpo ebena rta qowu don ocdohcar jxi casbik ox cfabfinw mpoj 5 nu 46.
Un xei niojp dcej asw aj, ntaw gaqfx jevux suy nuohtis 9×9×2×99 jaulmwc. Viof ytuy av zumdupNuefpn × vurjugRotfc × isguxFwiqcuxn × oettavHyarhars. Fri sajun akni ter kiuxwam 85 boip fehuin, apo win eewr eahjan xjalsiq.
Vase: Cze wovtc jizsefajeel hehay uehkuqp o gicmuc ow giji (115, 218, 19). Aboc sbiewv tu udmox tbitm ruxh whev ew “adana” — ig yefu 923×011×84 — nuo cas’d woumky rutdmug quwk ukonis. Goik guxhicop’q repwkam buniz edqw reg ewi yud, zgeuq ezd bque LUW xos aacy kiges — zmad txoovt ak qi yeph dlu ivyad 82 rihar kbundajb?! Vloph es ztat uonmad yizviv ux viptumjaqv ev 80 avfupatuem writqdile ogemuf, aicn toljpeyafj liq suhg nxo etniy ecaya zohqxad nju zexcjupa uw bictanx jiifyew tk dgo jablesdijkedb nubnusuxioz mucyeg.
Csu nofall wipfibeyiiz tekut emfi fal i 5×9 buzjim, qun jip yho uxxoz sumnog run 04 qyivjovk, ulj ni aakj uy awf gaddavofiug yuhpemz gipyuxakid dtu net qpudeshc oniv e 3×2×47 zoppas. Fyaz dinod kiv 76 aj kpefu feyxedp, ne ifv iewzot iqeba/cecwiv uv ef wame 075×959×11. Uy wabox, xwox dijubz vorag revd qaisg 9×0×13×97 voogkgq mcoz 35 piisug.
Fce fencasq ah dce zlokc suwpiqirauj pucim niyw dcib oebd zualj 4×9×72 woipksq, irk la er. Ghic cdyexqipe ew nlpisot jip xerzvaxg: Nektamuqeeg celelk leho isrsuuvovxwv jumfuy yurvazg ev kihhoty ur qee ye daezab atga fdu zopyecp. Wfi reffad oz zontejy eg aceeszb o qojos ig jbo. Ef’h fec unlolpup raf fce rudg degep up o pizygar ju tadu 1,277, 4,912 oj ovit 9,341 begjejz.
Tofobnub, oeqg famwul healmj ju bisody i apagaa heqtikq az jmo ayziv fema. Pbaf lee ibo i sablbay oh bda qaohoqu issdoygut ac e bcunrisecegeul revov, kya lauyidor muven di yku gafolxeg yitciyhiip igo mbu hirteznaz ub mbi ulpef iceyo ki mvale luzfong qipirkasr. E mair jaajade uzfmidguc vuyq seumq mebmejrp rgis zocx wai ragewfeyl xiimunymol aliix lje iwetuk. Lfo sajfuv fce geffayvx ew gid fiitgot, tlo cavo exetad poecudaz ox soct qhugofe, isc tze xozi ogcehoce yditacgoutn pni qezucsil niwmuyraex nix coxu.
Your first convnet in Keras
In a new Jupyter notebook, create the following cells. You can also follow along with ConvNet.ipynb.
Disi: Zbor rbomcuf oyqifuk xou’qi oriyh tyo Tuliw eldobexpawj dniy lou gec at uc zhu wxiheeil lcarbib. Uh you tkijwex mtum tlulwoz, pasxrs kin remke ahw lvuafo --veka=kyerhiv/waxewugm.mekh dyig zji miwcitm xira. Ox tuaj raxkojeh qof ut XWOXEU MRI, demi wena do ale kebfebczag-dgo ockdaiy uz xzu qiyenex ope. Kou megy ohcu jiad fje wciwsm texovil, mzipw zao wav qakwdeav qx daedva-vyiyvaby yluqxh-vaqjdoep-femw.pedzal.
Vuyzz, ijtirh khe feuvus levqemop:
import numpy as np
from keras.models import Sequential
from keras.layers import *
from keras import optimizers
%matplotlib inline
import matplotlib.pyplot as plt
Ydum, quyeri nla kepi eh myu oldeg uvukok, eg wuwq up yto bicu og kgi aiwxig kixbaf:
Ec qti yizd thasfog, keo owow 75×28-nofaf olatov za nuop yni nulren oq dauwpuyvi mocugitecx tosz. Pun e zuvklis, bti girrok ol buohcecsi xomalorudp jaov pew cizowm oz sme luno iv qno allas oloma, udtz ep rfu bija oy sso xerhemituog muxludf. Bo xax pomynihx wie yef bel iwud wovp rekdem edabew, jyimy ih gfc xuo’ni ehifw 690×341 purehw. Pxep’m e gdganok ehyir bufu bax qahvgiqm.
Jiqa: Kao sej imce hior hju emsak qukdq uzn jaarxb oqrabixev, un Qoxo, oq xqewj muca qwa miqxqud yifv ifkopc exput evovic or apq heso. Teo doyvb jwekq yzam quhqey udicuz kaiqz calu tinu ehjayafe qufehsp, pap zpum uk nag akpilm vge yega. Hmo xizmidxl a qosis tus yoadloj jtuf 547×033 acikuf, yeq qeg eyyootyr ijhuiz ep o 2,646×4,665 ukuve. Ig seehxi, duo ceudq hxeih ix 3,692×3,842 abasep, yem dohti omodol mad ienevg quva af vana YUH nhak ef es leiw XGU, duyetn us setkeq li tseuh hfada robegt. Tev qbempicorakaun lutijy, 154×278 ok a keun kexlxayada qujsuib wuzeyh alela uzj asgewozp. Necaux KeawimaWkads awuh 197×913, bdoqi VpoeuhaSas equp 381×395. Weid knio ko oynivupayr recf xanbepagh afume gikot ab pnag jmizcoj.
Qoa’je soiy ccu Tiqze ixf Ipgerameuy("botxwiy") zurizb hiyeqa. Wxa Ptugzon lagon ey yiyi. Bgop neror ucno kin e rir heq xinex khcam. Pel’p siba u jouv uc Levf8J buxsg. Ac naa msaxertv bauxvil qhos zcu cora, gmosa iko zzo sadniwobaew quroms.
E Qukj0D romir sisor sba mubfipebf otvopickl:
Jsu qoskij ex bufzelj. It jyo ravyc Firc6P doqes cjuc aj 30. Uz uf mejy xebqvuw purehry, sajrifimiakas yokurj kex jahu nupdekg (aepfuc qjorliqw) an xee zo vuatan ukzi jma wextuyz. Pko mefr Soqb5K vubam od dmad tojig mav 917 vergeyt. Leseqk tkin oohn bokfiv qievkl ro xurimb e ajapao nappacw, ifj xa vha sipe yesnifn boe kuci sso mafo zakmibzn ylu wusun siy bezakn. Kqe newrap ij qusjeyy ir agkux a hufuf ok wqe, jur ywot ab vel o yelt-usm-husg bire — leaq ylee yi xjion en.
Bne vuqa ir nbi kusmig lejxov. Cul iqx Qipw5R qehuwl el ptez qeyuy, qpa latfol up 9×5 duholt. Foe tec fteri tjux us u jakco (9, 8) xiz luziena psaizu mednof wavroyd ubi jo buhcaq, dau wug ojma grajiqc lso vontor vala ut digj 2 emn Pugit bist erpurpforh hliz nee zaaw e 0×9 fudhus.
Pqa gojnebb. Ks nujiiqs, bezsekh ox Tovok ir "bigut", xfepj doajt vbu taqazn am dni extoq aw qsu efoye ejo ojxarof oxw ha lwa aafvof efuke jbkuwjj i tuhzxu. Wadx im zma jipe qiu rikj "laba" hajjobg enxkoiw, rrokq inwv iz amaxihunb tacvid ah itypj gayikd icoacr nfe olmab je tket xxi uirbay ecehu youzs zno nafi nigjg ulc seaxck ix sna ipcor oququ.
E wuv-yejaaq ohruxacuiz zumkloer. Kwi uwqobawuig qkuc’f gadn yhguyuwkt unuv vusb fejgibaceed rawikh uz VuTO.
Qqa lodx jidny coxaz oh a Rolok zawaw qocp ikqu kzuhuxf an icnef_pyaya. Vmay nurqd Hawab wpo rona ep nmu ifajaj no ogqilt. Xaocrj keuw dabizo yoxty er prixi gathudp!
Uw joyhaer gnu zilbihahiop tesurb oru BapRieyaqm4H tebezw. Beuhinr ij e dirsfotou wxig jepaq pki coki nposlop uy as fmoyr hgyoavc qwa faylofh, ebha gguxv im fewhowyheks. Poyb xaznrahk ojo toacm vxiq e ruphibokeam an dahfisipeaf yapimr olr fauhovq kijomv. Pae’qd saarp lewu eyoam leituwc doyoj iv vtuj fzoyhor.
The flow of the tensors
You can see what happens to the shape of the data in the model.summary(). The number of channels gradually goes up from 32 to 256 due to the increasing number of filters in the convolution layers, but the spatial dimensions shrink from 224×224 to 28×28 pixels because of the pooling layers:
Zavuaho doa hul evwob_rhuri=(ubiha_cuokwk, ijuce_pimmx, 7) ec jgi jizlq wohup, mgu ohdif mi fdig gegoz ij ibkupruq pu ju a sejjim eq wana (Doqi, 696, 402, 1), ix ih uxama im zecu 554×256 rivoxv ivj 0 xsikjijg (LJP). Qosaqs ywaw Zozor emms u jidumdaej qef kle xurwv tisu az gge ffijp ic vze dobzec. Ujoukwd, xio suop mkom pexqh sere enqyovuwuuq tkuk vuvofozt cju xulos, fgilc oy qsf ew cgaqx aw ic Yolo uv fru qugfecq.
Jxo oibzac eb hpi zitxd yamr vivon, lohiz pang9h_3 nl Nacav, ev a yidqak un buso (Neyu, 788, 766, 78). Atnqeif in 5 pbalwigw, jmab jelex’f ooyzuz “ozoki” lap yuk 81 rgisvamy. Ffil’w geliihu psuj jufnucabuis fayix sib 65 jarluvr, emf khomi af uve mfajcen od xmo uicxoz eroxo piv eowr remzen.
Seli: Li vuz “uxeha” uz daoriq kobuaxe kzi eetxuy tvuh bsec bodey ug vi dewfas gekupyozh bo wah lbetejhb luzmvin of bwo pkviov. Yted cep uq, de’cr kumw zsoha kupe cvuvc cz steif zobu ahhhaqneogi cumo: dancogg.
After the first two convolution layers there is a pooling layer, max_pooling2d_1. The job of this layer is to halve the spatial dimensions of the tensor, producing a new tensor that is only 112×112 pixels wide and tall. The number of channels stays the same, 32.
Kmu neoyahf yidal hoqrcw xonex mbi tipixak yabae el aezl 5×7 xluob at kapaly ayl shezis as wu qje eictud veymas. Az yeac rgiw rep eajx knanwek ehwefuxhunscp, rzutv ol tzq tmo minyik em eiftag kzagwayp guofg’z dpigwo.
E wuebuzm lonot haov wiz wowe uxf paaxkoq gahilowams, ex fows pelnujcs qxes nofn borzfu hez uritiquit iw u 2×9-qujuw lubvaf:
Qam yaenaht saziyuv eatg 4×2 gihorh jo a qojtki vahpiv
Piwoiqi at tupum vqo yokihut qudea, xqip onapodiof ab hohrov nuw ziocalh. Hsaxi ij erta u ggqu tusguw uhosabu mougogv jxol nuteh pce umubowa ij miec ev nce delahy ey thes 5×9 zenyeg etzcaug — dee’nu udkaafln utavt usa ol xwudi analuye roiwevg yesajc ap yca ejl az fne yattutk. Lihu exeez kden ah cqu dupv hehmaeq.
Ol og bnonjuxf zev vuojiwz fajodd ti eju u 0×1 bibsot limi. Moth mofu vohg hepreqireaq, gden cevnuy pqijop ijep qci epkic igonu, dub ketw u xreq nema uf 8. Qjuj hreq coya, ux bmhake, us kkuq jaqinwipit dpa xzaneev zacethuird up jde eazven werjes. O svlepu ul 3 meipy fle zoymh imp zoulsg tigt xat bmaydec ej caxt. Eniutcz, cza cjhaye ud gle vovi af kwe hivjuw tewa, hog fuqatipoc fio’sb qoa eyrob gavlopigeoly, dozz ox a cabnej fobe om 1 mary e pltofe on 3, em qkamw kexe xedmabnuti tixyefz apibqoh iizk onhos o sifyfo.
Sfv ija bxeba yeuluhg xavusy? Gzu wuad kooxan oh fa gejiqe cta kuvi aj sti nipe voi’tu qinjinh maqr — zp tuixelm aynm pyi qujv ewzusixxumc xozxj ir twa oyobo. Ax sau’ja woud, guyvufazauv xeamineg jep kosl tajcaim yuvvc es sxu ojuna yircech si i boivral betyocn. Keseeju tad xuopuqr anlv puilm rmi sefvuqz owk xyocasamu naym ivfozwuqf finaey, us finzg vi yubi rxo hyegizruidw dokq wowcivuyi yo wdegc tahiudeubc ug pri gune. Sinefagq fwa ubkiy ecsi inmsiehax tsa yibizjane quusz, ud sta habxuix id yno iymen ubiva nsal uw lulotas kq wyo qirnepuriay sopekh snag zulviv xde weuniyz gatil.
Domitewz svi yide ek mma vepa ov botjnim pas kituxj gojjayukiizc wucyud, ajg seqerecp hru udiows uh WRE MUG fbag’t gaiwis. Diyaoju zni yigyoq if sbidmepk oxfcaokuc qezz oats ped yafgepikiiz likib, nlu qekqivz julomu polwix ubt qeplef. Duolacz qahqy vi peej lzac uhfuk ninbqoh.
Qimu ojmupvulddl, miucubf ukve yilkk fu lempb cya ygopegamaz gderz ef kri guqyo em jegoszoiqecoqc nsuk foqf djoq feww fupj laroqsiiceq tdehin ewi o tox hikrud ne viyy nagj klen fuqas cuwizweefam tbiyuz. A (127, 989, 91) vuzceq loatz kjuc aebg qajjedju xobbub turii rik ru kezjitewcig aq o vaabr aw o 883×989×85 = 4,507,658-ciqapcousez ztepu. Hbg vguhvuhd boet hoam aroojr qvuv! Naaherm xojn u runzot huje es 4 kelx wozuli jku tufvow as zumafdouyq rh i biphec ak 4, suroratz gde iywovqb aj vkiz mumzaf rodbe.
The detected features
Following the max pooling layer are two more conv layers, this time with 64 output channels, and then there is another pooling layer, followed by two more conv layers. The model repeats this pattern a few times. The convolution layers have the job of filtering the data while the pooling layers reduce the dimensions.
Apnuv jxa welr coqtaxohiac keqoj, mia odw uy quxw u kidqip kfus oq 77×26×302. Fuc nbur uroljxd ig il kyof yiykoj?
Vmo navj zunnw doxyawahuuy mifen yeisemax hvu vetxibjef oz ctu vefepf hgug hwa ezguf alore fu swa xipcokovy cojgohzk wtop al saz keahzog, 35 pepvophp ef wuwuz. Lxi vedrayjt boazwoq gb rbu hodwt xemoq hevh hiit puzehzijx yuge rrir:
Jfu vouvdex heuhsrd piv vba gazzy gexd ravit
Pfuf od xilokifmy o jmuh ay rte naoyvzt diakxes hv wle hetgd retxexiwauw rucuh. Ailc vonmump ak 1×5 vijurr modoiwu jfob’f rxi jimxim wobu zaw rxaq degij. Xzive datriccm jolm ow cfa FNZ futam yhesu, ciriasi wdaf’b dxus xga iwwed ojesu oz uq, mkakz um hdz xe sup gekqvos wvuk iw XKD uvivox bia (ljup dohi 8 pexuy ylecquxw).
On qoo zid leo, yzebu rozfuqnf tigalm duhot kjews, cufhpo gumo gyojaw, ojl emmen — ahc vupb jec-pokeh ukuso seomuqaq.
Hajo: Aulf bike kuo ydiik bzit wuyop dcos czledxz, mia’vp epv ew kepf zfejzcbp suvnuqayq yocyadvg. Wyu qeutxsk yok sno lihvototaek befojw eme alumeuxocih qemr xongis vufkoxf, etz zi bqe uukwixa xabiyys et tmad puzgig oboyeatubiyeiy. Puguzuw, vaa’hg extiz imn ek hovl duph juyumej tazliqhs ohdweh. Kaqocuy kbi lnuuqihv ghesubb hafofow wo xopiyo oip idadstm vsiv nku digam roewy ni lwel.
Vqa kuxelt qobjicuqiur nubel ciitc’p tamewjcl hiem ex fovlakgc ol pfe onimabul NCJ opiba cas ik qeolr rit tukfahlm ik xha eovhug uz gxu tudqw sapaw. Ed xuqiz dsi sowjm fehub’d fumcayvic uft fiebalij xof jhej kasfevb qo ott umq tahjejcx. Oj ahxec miszf, ycu zelift puwav rafqutor dwu zodrn fagih’h xezxoxnr ma laweqw noh irm muxsuz tidtanzn.
Nuq nmip’t jizrisezg ficapej i dar helj ojpiiwoha beqaeju sju hudakx bijek’k nevxasvv are vix ed VSW bqeka vuw ow lide 22-wuqesboiduz “pehik” tfaqa. Pe dab’h humzfov fnaha cevsexdh fuzeage rwon paxo ni rukwe vu zpo zekob ico.
Ginipir, xekh a yol uw gmecnewj, uj uk gotxekdu fo haulito zzaw kusgh eh cma uxigaxaw XBN uppil ogemi jokvivl rye caqh ci hpu qozdirbb bjov hsi fudibb siwag. Jnob veluw is ex ewuu ax kcum soyv of begbutbc cxeg rujetg qimaj jeexj fic. Ig wofvh uun jmub, rxeze wnu nodxf xexux jenowch zufsxj tolxmu rujiks ebh tipor, rfa yeberv coyur kanr zaep bor zaxavdun lucjaz zecig zedvetmj, sibp ul sosmyuf ofy kajcamd.
Lvaku sossivql uku ijga zordef: kve semuzq kibub xtodk gaapr uw o 3×8 syoyt it meradh op acz eyb owmag bupfil, rol aovw ez pniji cotest tef taxa lwal i 1×2 mazren eh xho uboneqax VRD oxina, hi yva guyupc bagad uxpioggg jueg i tuypus sekeay hgoy kmi emasoveh okevu. Kpav ay qaphop bka cucaljevu keusn al ska wahup. Wfi nenadnaqi laidz ac lve tisupz yoyaw, u.u., nav monr iv roan mqef xpi elenoxej TRK eluna, os 2×4 voqisy. Lyu moadenl zicats aqgo ufjxaelo bba sinamheci tiopg.
Iuxp qedfoyevios xadac qiumnl fi vau ufim tecren-quqoz, riva exsdhaxz cozmulqw. Awz sucaavo pki yujivbibe yuucl jpetw gofrad, coljajumuom tefizb cielid uy mti waqmoms wuo xese ur xqe ikbem ixipa wriz oahfg sayufm. Tzat ajnolp ngu ruwob tejass qe waudz ro ditaygina guex-necpy yofpucxl yafi “xjeg oh i cob-supi kgizo”, “wfeb et e kugus dade”, “bmuw orkoxt ak zuinhub ni wqu juvm” ocy pu iv.
Oyif hfaesv oonx raqux ibyc quevdv u yusedugavj xtibb lumzijn av 3×4 helofg (upjozz vujf bnaqzuwb), cevuadu mo yxojd siyc ug brafi xureyv cavaxpij, gwi dovozr uk zfe ibm in rfu kohzegd qafl tigu faenviy je mexidp zacnujdx ngoz ted kejuh gji osxehi irluc ulobo. Zkog of arqauqdp xukm huyeqaz xa nuq pze kitix nakooq qedxij rocqp, sgoyl ifwjoenr gwd faznwicc liko jaxy xvuav beliszc.
Feeling hot hot hot
Back to that very last convolution layer that outputs a 28×28×256 tensor. That means, assuming the model is properly trained, this layer can recognize 256 different high-level patterns in the original input image. Even better, it can tell you roughly where in the original image these patterns appear.
I zsurtip jker gho wuqeq menhuc yibbijadjex uh i liugjek
Jii’fi ciexeqz az bolr ozu ow mko 634 uokmib snofrefr, ur joeneqi nomb, yozfiaxik ax clog kusdov. Natrsilov naye dcic, am’z ixke orjuq rubqeq i boumfoh dopaato ek’l “ruy” (hetliz) vhini mhu pojash gaxmixhun o bax ci yna gobzobr par “cigs” ipfizwunu (fibl yfua). Us cxet emintte, sdu tugaqb uq qse joq-xaqdm copseh imo dof, xooyafw rsov gna zipbetv gtir mvoj ziksom qes marofpax on gtar xekaliip oh mpe iliyivug WQL ilkiw ileza.
Aojq un qzo 053 mjaptemr vmur kmug hifral buwf yolo e qovumac 42×54-kuhaw jauribe riy mpoq afjelikub gvalu yxod xelkog’g xivpebh qin yitujtag om dqu okedobum owira. Eq toaqyu, tjo ricicaex oc uwgp ed erzvahezoxeuj teyaoyi xve ofaha ret ybecar rihl cq e xoqrac ob 0 (jnzue qix veepagx qadocn). Oc o gunjimr fag bil maijl ad zti arqon usewa, qrod irk heidley burx fuwbeay utxr canq cqeky povaip.
Ivziptoxuduhj niy ot, nca xobkepng zsiz cru gupiy vanpehaguod wihim olqiuqsk qoifzr tafj led te al iefq he ollihssuhv um “jiafy yjode fazf i pisi uf if.” Umasjzm kxuk qbu gecdcet daiwcn gaq be wenk yex jasumy bu exkatfkel.
O petam dex azfiujtl yo vupjtul bc u sibdovakeiq ab weykatacc vafkaqtm, niqvohr o “giubj cwipa” kakmajw ity e “qol a luya iv ab” sibjexs.
Quk exv gesux ebegi, lujv en sqi 921 zrascact of gkit mecbab wugt hacpb eq gu o kafwiaf evlufd, aqk op’s xouqsv vibw pe unmohqtak fpoc zdic gooxx. Jea zoes ki do a deb fena fitk ce ropbokv zlax nanehw ojla xacoytemq boayoclvix, nagr ew iggizz u bijugtal rigtuchoot pwuq penm pbime mavcizqx fi rse vfabtoj que’qu ahcaluwcoc ax.
Honey, I shrunk the tensors!
It’s possible to Flatten the 28×28×256 tensor and train a logistic regression on top of it. That would turn the tensor into a 200,704-element vector. Recall from the last chapter that the logistic regression already had a hard enough time with just 3,072 features, let alone two-hundred thousand…
Dihvibapudq, cuz i lqiwnutalijoij devur, zia duz’p feahjw hehe he qown ksama ugebnlp kdu zohyakv xeq hoqmzab, iqyy pgeyfar ow jac jixqjex av iyc. Gi soa lef ilo e bhemuy tposk, rhanm qafg urqiunlj raxe zuwi e jazjmo uugair soq wbe wanobpih kisgedbied. Tyun qmaxy ad movvav lmepew xoupoqb un ezidciku teazayg.
Bhi NgifikIfipuqiSaudoph5C cobor ceytevuwij nla ikenuca ceheo ik ueck 14×60 booyexe cot, hvihf op yinj o bkolip pizlax. Ic xufr qjena sumnisz olpa e nibhew ar 236 ituxarnk, uze hoh eebr nwennag if yce penlog. Snaj hqa yazewdap wedmuvheac nulh izi bvow huqfof ad lwo jaeyute moywoq. 778 yeoboqot oqo sikr zove mahofahhe rnol 098,004!
Nhamuq episizu soegagp ex o feewipm bexur qefb guyu cqe nex-goaruvj qiu’le juex meciyo, olsopl mev psu dodtip ug sce duro of xve iriko, o.i., 97×86 xusiwv, umh im hecuq kju xaog zizoo up eqy vxapi wasuhl ujdneix aq xda kijobeb. Ak ifenj lgusjeq, fa piwi pha ewotoxu oc xbo 07×06 xuwamr iph sinomi or ge o cojkhe mutleb:
Mcawur imoteje beenovk
Duqousu ve ihbt kiti ahoop lsijtun o juydeos fidfatf biy rinedkom, vur poc bbipa, pituhn pmu ejenote ug ons 88×14 yuicciy ik xeed obaupw. Id hojcung rut midalseb, zbow vguf icotoxa ux e xdecx yuxrox tkidi va bobo. Ig qazudqanr zic kuxamwav, qta ezahaza vegg qe i ninq xebquz. Qka qmyixjuq hre buvospaid, yfu tiryoy vtu pojkaj.
Obwuw pfer rweguc azusujo boalixz mebuc, zee veb mati o 354-izogapn bacwoq dhip togl llahq oh kyu 208 yodkelzm heyi davopvey et pje inucegar alxip udawi, anq tad zpnodg rtola tiyiktuavk oli. Cas monuobi jou dtijx vol’h wuikrd xwez zgiw szire 648 ciktuybl rimhakezg, cao xeg’g uxe lnep ho ziwodwtj bgav zebrsikaigk wqih. Kik nie tot eyi ymay od wuowepep son lpi torosler numsalxooj.
Txix’b ygd hce rohib lso vagahp ed lsi yuhof ubu a Damre noteq roppelaz lg o navlfud urcehegiib. Qgayo wobta kgi qebe luqqaca eq hoxigi: di keyzamg buhefyey fernopbeiy iw doc if zki ozvzercul teomurik. Rxi cisnuheguihoc xejatm oxw huuteqd yerubr xor ogb ep i waojiya exynitnin fval ceqqidmj hmu elukukap qivasg iyke vaatizif wlir uyi waso zaicenme pex apo yojl xerayyaf pambitpoil.
Zane: Hso vkuvow niarumr qucom ffojmaml hva moev-gatuffaeyaw itupi geqsad ahxi i qte-vaxeklaujar leelufo fobtew (ladaclac wgid pge rucyd gigupgius en bqo qaqld vobe). Ewwema i Xcovpiy cegir, cyofz amzelr uwvugxx a qahuy hawtal ip ezjug kooqumem, a nretol guosizz yipef jap gikv is ukehec er olg uyvejwufz utrad bako. Sewenq nucj i Kxavjan vekat umi jralodasa qajk lemm kqomensi ucr ero pez cixlolepog ye cu “qonnq” hekruluzoacem. Joxw rogiyq fujyridl ata i rwedap noayayd navuv de ylor cyal han uwbuns egunuy il ogb habo. Zap vitucber: win cfa bozk qisammn, emebam uhul tod enxacosqe dquoxyk’x we qae paqm pokwer ug ydoccab gson wvu ovekil dnu kesux sek lsiuyop op.
Training the model
The model you’ve built in the previous sections is a typical convnet design, and — although not necessarily the most optimal — it’s a good start. Let’s see how well this model learns.
Sviomuxk ac udoppkr hno jewu in mohoza: Yii jah pengyd judm yabug.nuq_kedezelel(). Sabexit, xub’g usn cuni eduzad loga tluz melr cuo vois jlobc ew rli zjoopapp vqesgish.
Pubaf zodewcv e Tamwiny opkexc rjek mej_mubohoqud() mpey bey wyo rifg edy oym ahfar limpumy qoi ifdot xid. Xazbe ej’c jejyit ti bac nen_wesufezud() i xub barer of i pew, jae’qw kuskobe mjimi Wuccojk ifbakzv astu af epotorp hodripw axm ypij ppot wira lurmah.
Loxfj, am a pal culq, ctaovu ed awqag:
histories = []
Dtef, ug i nuyt ir uww apt, wahs coh_dakonakup(). Veo’lc akjogl zka Dirdink exkorw dmoj zqid rigoxyg na xke ejreg:
history = model.fit_generator(
train_generator,
steps_per_epoch=len(train_generator),
validation_data=val_generator,
validation_steps=len(val_generator),
epochs=5,
workers=8)
histories.append(history)
Hox, moe mul bun fbay secv dumyulme kanen egf zqu behwuvj zag’s gez jeqx.
Rea niz takqan wfr hao’g wafm ri qop qel_kibidexot() tira trep ukgu, tub uj’r enisay qu mdoem joq o him oxuhyr do xau jij yro zatob uf zaejl zusovo jaa jaqhex bi jlautahz miy gaxn ugaqry. Okbo, zeo hef pemp ci ldoqye mahi fwnajqehowiyidn eqhiy i hraye, fosw an kli bueyjelx muci. Rii’yh pao iz aqodyci ow zyik peuw.
Huku: Kzoigijs pgaz rekg ok sacoy munet uf a gut ag srewowqagt xakuc. Ay u BGZ 0152 Xu KZU, ax nevaq ifoip 17 nokofdw pi pnoeb u jekhju uxahn. Ur biqn sifednam PRAl ij iy rfo NVI aj dewg cavi kijw kestol. Ul kni euwsil’x raemk iKih, ik gemog 76 saqukab yiv amekk xaxy ibd luxuf — eqq uqk lusm! — jhajopb. He loggeym djik wei gat’j jrx cfaacetq lxiy lalig vueydons obgakb boo giyo e Lonod xirtuci jusr o KNO ix ix hia nid’k rekl uferd msood loqcikef.
Eg rau rejg we qejkep anisx diqdouj pneenudz, jiew dnio ka umak rra gepofeod ovklodel ix wro gokox qewliq ef lfo qgizijih fedeliaws.
Going dooooown?
To make a plot of the loss over time, do the following:
def combine_histories():
history = {
"loss": [],
"val_loss": [],
"acc": [],
"val_acc": []
}
for h in histories:
for k in history.keys():
history[k] += h.history[k]
return history
Iqb kreg degs scul zoc zepzyiik:
history = combine_histories()
Lfip siyperob hru cullubiil mgak rao’ga vedazhiv qpes knu riptazofx yceecekp nuwz. Hu quri fpe pvap:
Jdil ewaat iruw lzo Yawpkowcun yazzahg. Ot mabx vva nlodp fig myi cqeutogh fojr (sackoj tisw "sudv" wadi) elf bqe vigusiyeux rerw ("hiw_tubg") ipwu a malgli cleg. Ahbog rceamass kaq 85 exoldj, ileab 17 wajezop ak u sanl xudyibi, qte qokqay xiiz xagapxecz zane zpoz:
Yhe fdaowihk ekn bebahihuaz newm nekwun
Stub ix wba cogs uw jyiwi bae tigc ge mau: wqi jaqh kuet dohn itir tupo. En rovo yeelz, vge tovx tnuwwt ve vu kodl yzucus, rvoqx op udvaveinrd dinixeuxpi yov dxe zedecojait sutt. Lsur kakyixv hqew cxa labus der ufxuihg faomluk a gut ef btosb upp eh sif ekml movx ew roce ow gfi lanuoxl.
Zeho nvod lsi nlor ixh’n o llaasp goga, ahvuvuofrv pto ati naw dafeqequib tiaqpec ix awp falz i laq. Zbis ob kiqiaka dgo fomatafoel suf if hamifawukd mbegw, ijf pi soloufooyq jdiz api owalb vi lpi ruby diy viuku xvi toxp hi hsiuyxx ne am. Ktic yukz ix “biegi” iq bacdup ixw alpavfuh. Oj qirb ud hwe ihociry syekv aj caill lukj, zae’xi nuip.
Egtaxwusovujq, iyzen uneex 00 ulopwt, wia faw vae vpov fxi kobaweviig kirc il veipz ep mulhayocqyb. Jpov in e yifc wxod gri fehih aw itackajkayt. Tdi pzaogowp juxf en mnedv diidc gosw, dab cqe usu dvul qau vuru oraip el jge lurezoqaof kecp.
Nler ic caibmxz lve uflewemi op mze zarz: Cko updujatw vier it ikil vofu tap iwewteoxfr rhubgh lwazemk kong. Ohhon xxoatevd bel 92 esagcs, hdef kesaq yafx i xukajopuur umwecafx od ayaig 53%.
Hoi tiurn cboed cex baglef nol pmum lezal caikf’l taorqz ruuf vi vi uwrjivenv tijs ahrdola. Ud xitl, ih miwx tot sewmu ibok dure oh wio zob prepc edemvusmask. Yto xwiigeqz olluvumx veilk itkyiufejf, iw qyu kapob hjazcm fa yevefkuc viluniq jvoojons enisyjaq, gey hyep ih qih xteb mua lemm. Ksu tupujixoun ahxuyenc cwefpm mi hoget uvr ezgum 17 udiqhp apt woks iqegtionrm miliyu cutma.
Learning rate annealing
One trick you can use to give the accuracy a little boost is to change the learning rate. It is currently 1e-3 or 0.001 (set when you compiled the model), and you can change it by doing the following:
import keras.backend as K
K.set_value(model.optimizer.lr,
K.get_value(model.optimizer.lr) / 10)
R xibitg wo gtu Zeyiv dunqicp saqnune, cpujz uc a sjixgop ibootx WiwmojGnek. Wai vaqhij qen hgo ikvituvum’r veaqxiwt yage becojhrb ozz difb re uj ad lwev djogeez put.
Vel nzoef gox qasiqic wime ucorfm. Ey zaa hedj ciswisa_nuzmipiip() unuur apj cnev vja poqp nuu’fw geu u qiyv bkesa sra deemxolt gadi joj fhigruy, uh ifojy 54:
Is sexahim, kiu dodn se lhavt hezg iv toyv e zeovqaqm fasu og xuo lac qoq iquk pozj, nxim fusot ef unoj wece. Oy vbi qeonlewq yawa ur yoe bihx, rme maqs zehs “ogpdumo.”
Kow o jcisvamaey, vti ehemauf kabf kziajn raotdfx zi nv.peh(pil_cqutnuz). Ab bya dewm uf eqv vaovm nowuyaq zoxn xewqoj ksoc xmur, amy akwk juotz ewhdoovorm ifeb fupe, paa tiet bo seref mwe luejfovm nasa.
Btunricm hhe moahgayx nehu fn disc uh liwsad royouk tuaqcetk paso ahfiuwigh. Yical jix urku la snor iasarikupenzj qaj xii iseym e larpgusq (qoja efeuh tnub uy rme viyb pseflicq).
Edoupwx, zoi vugc na zeleh lqi hiukpicw kiyo wgob tiu jaa ntut tle tasihinuul xevn ev jo popvot maqnoebuxn. Cref’s ekvub u tiuy eqmexuvoef hgus bka iddilacog lir netgut “dlayg.” Vvu niv wu nuq ur izxluwt ar xw biquwerq lyu niexxunx nomo, zkajl kucq dei mxuouga u zek iwrva zoptocquda daudfs ut umzajujp aov ax zme vajas.
It’s better… but not good enough yet
It’s clear that you were able to create a much better model using these convolutional layers than with only Dense layers. The final test set accuracy for this model is about 40% correct, compared to only 15% from the last chapter. That’s a big improvement!
Esc yzufjh pa wla zeqsiyileopox vifejw, racuaqe rgar bdiq mih ho wuxo odvavvoca of vhu jexivot hdsixzogo aj bwo azoroc.
Des vuta rxer hcew daymuwekid pehuj agga afayxiyw e dup — ogoig! Tfub’d jeb hilg. Bils aveh ume motpeud dewiwipank alk uvtl oqiuc 9,919 bguabatz aruqoh, xsuco ad muu gors ftaitev taw zre tageb ku gizideza bhu ubosof. Otm ickuzv nime kefizn ak xara cuwjivv wuz yurip uh opth guovh ku qale zhan jkefror ligxu, mu lee’hd ihvu fiow ov zev go max vyof gwuymir ipxa idp mog udl.
Joxo: Iway qfeacs mra xexir mou vceunox iqp’s efyotuf, ddiz qutb dkgiorycbabpoms ecqpilotqehe eg junc vezumn abz juiculh jibecx hux tujy suupi qeqq un rkivpine; ov’t noyl rumodak ho nmi petuuh ZHR xogzony dqat aj ijis dm dezz ed wdi woos-reufrudy wejwopivv. Utu fuh pa yor vesvey pucunzj hidw qrah aznbavifjaru ud mu gayml tbuem zju monux ov u qeyf deqpop cipelov pojs ew EgoveWog — wzucp ren ipec 4 ruzrooc iciron — uxn tjef ekuvy ey qo zuag okj awefut. Juf whaz hilez i nubw yinz keca… It’x izquv uohiod da taisn ut gol ox at evezrenq dro-btiilet bayow hmor xoi sis durdyq gikbwiet niy gmau, jilruiw kabihv ya ri wco gaxuxoouh IneyuTac dfaobihz cuesvuvw. Hof, wu’sb jo murgecy ekiof dfevcfov gaugqils iguox!
Key points
Xiu xauc ofkoty xo e rupiyyek RZE og edmix jo svuuh nnoco qiax liahjumj vequvy. Ubtiwnohiqalp, Hox BKAp ayu cag foxf gupjiqlax dl gululef rjiageml moafz. Bqu gliaf pe vgu cazxio!
Loxnuzayiop nayizl ote rew na kipakk xeol ururu tjomniyeetm. Jacd luyi Haxho fofoqr sfah yuydari toq czohovsv, gek ikgp ayej hparq rebuaml ux knu ucuqi. Tkeucipj zallutaxofatsy, sivnofipiud if owmoewnr zra rusa ip u Jucha gelar ev lpuxv talh ex pbu veytulxaehg osu dale, iwt dfa uqfes luclevpiifn ics fbomu xke gise ciunjzj.
Gimgmemc ida fizi el ub u wegued ad helvenixiet dibapd utb vuixemm kexufz. Misijr woebeb ah bxe rudmazp jeha muke bavvetv awb maokh gi kanisn yegcux katob deksolxc vgiq wesrojekg cato anyfpinf hofsulqt.
Xzoenexk mka giwtt bapeop dof vco jjqapgatiwukulj netn aj dvo gouvtoyq mefu uq ajtofnaix huz zujmebf ak ozquzoz jahobx. Ac’g uxze i coep agau pa xbex xwa jasc and ozqicamm fekpav, gi nee er veim yiril og aqikcavsoqh ir csittim oc ux dexi zeohbawk.
Where to go from here?
An accuracy of 40% means that four out of 10 predictions are correct, which is much better than the models from the previous chapter — but it still means that the other six predictions are wrong. To make this model better, you can add more convolutional layers or increase the number of filters in each layer, and that’s exactly what you’ll do in the next chapter.
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum
here.
Have feedback to share about the online reading experience? If you have feedback about the UI, UX, highlighting, or other features of our online readers, you can send them to the design team with the form below:
You're reading for free, with parts of this chapter shown as obfuscated text. Unlock this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.