Note
Go to the end to download the full example code
YOLO/PASCAL-VOC detection tutorial
This tutorial demonstrates that Akida can perform object detection. This is illustrated using a subset of the PASCAL-VOC 2007 dataset with “car” and “person” classes only. The YOLOv2 architecture from Redmon et al (2016) has been chosen to tackle this object detection problem.
1. Introduction
1.1 Object detection
Object detection is a computer vision task that combines two elemental tasks:
object classification that consists in assigning a class label to an image like shown in the AkidaNet/ImageNet inference example
object localization that consists in drawing a bounding box around one or several objects in an image
One can learn more about the subject reading this introduction to object detection blog article.
1.2 YOLO key concepts
You Only Look Once (YOLO) is a deep neural network architecture dedicated to object detection.
As opposed to classic networks that handle object detection, YOLO predicts bounding boxes (localization task) and class probabilities (classification task) from a single neural network in a single evaluation. The object detection task is reduced to a regression problem to spatially separated boxes and associated class probabilities.
YOLO base concept is to divide an input image into regions, forming a grid, and to predict bounding boxes and probabilities for each region. The bounding boxes are weighted by the prediction probabilities.
YOLO also uses the concept of “anchors boxes” or “prior boxes”. The network does not actually predict the actual bounding boxes but offsets from anchors boxes which are templates (width/height ratio) computed by clustering the dimensions of the ground truth boxes from the training dataset. The anchors then represent the average shape and size of the objects to detect. More details on the anchors boxes concept are given in this blog article.
Additional information about YOLO can be found on the Darknet website and source code for the preprocessing and postprocessing functions that are included in akida_models package (see the processing section in the model zoo) is largely inspired from experiencor github.
2. Preprocessing tools
As this example focuses on car and person detection only, a subset of VOC has been prepared with test images from VOC2007 that contains at least one of the occurence of the two classes. Just like the VOC dataset, the subset contains an image folder, an annotation folder and a text file listing the file names of interest.
The YOLO toolkit offers several methods to prepare data for processing, see load_image, preprocess_image or parse_voc_annotations.
import os
from akida_models import fetch_file
from akida_models.detection.processing import parse_voc_annotations
# Download validation set from Brainchip data server
data_path = fetch_file(
fname="voc_test_car_person.tar.gz",
origin="https://data.brainchip.com/dataset-mirror/voc/voc_test_car_person.tar.gz",
cache_subdir='datasets/voc',
extract=True)
data_dir = os.path.dirname(data_path)
gt_folder = os.path.join(data_dir, 'voc_test_car_person', 'Annotations')
image_folder = os.path.join(data_dir, 'voc_test_car_person', 'JPEGImages')
file_path = os.path.join(
data_dir, 'voc_test_car_person', 'test_car_person.txt')
labels = ['car', 'person']
val_data = parse_voc_annotations(gt_folder, image_folder, file_path, labels)
print("Loaded VOC2007 test data for car and person classes: "
f"{len(val_data)} images.")
Downloading data from https://data.brainchip.com/dataset-mirror/voc/voc_test_car_person.tar.gz.
0/221551911 [..............................] - ETA: 0s
122880/221551911 [..............................] - ETA: 1:32
630784/221551911 [..............................] - ETA: 35s
1212416/221551911 [..............................] - ETA: 27s
1794048/221551911 [..............................] - ETA: 24s
2375680/221551911 [..............................] - ETA: 23s
2957312/221551911 [..............................] - ETA: 22s
3547136/221551911 [..............................] - ETA: 21s
4128768/221551911 [..............................] - ETA: 21s
4718592/221551911 [..............................] - ETA: 20s
5308416/221551911 [..............................] - ETA: 20s
5881856/221551911 [..............................] - ETA: 20s
6291456/221551911 [..............................] - ETA: 20s
6881280/221551911 [..............................] - ETA: 20s
7471104/221551911 [>.............................] - ETA: 20s
8060928/221551911 [>.............................] - ETA: 20s
8650752/221551911 [>.............................] - ETA: 19s
9240576/221551911 [>.............................] - ETA: 19s
9322496/221551911 [>.............................] - ETA: 21s
10731520/221551911 [>.............................] - ETA: 19s
11321344/221551911 [>.............................] - ETA: 19s
11911168/221551911 [>.............................] - ETA: 19s
12500992/221551911 [>.............................] - ETA: 19s
13090816/221551911 [>.............................] - ETA: 18s
13680640/221551911 [>.............................] - ETA: 18s
14270464/221551911 [>.............................] - ETA: 18s
14860288/221551911 [=>............................] - ETA: 18s
15450112/221551911 [=>............................] - ETA: 18s
15663104/221551911 [=>............................] - ETA: 19s
16859136/221551911 [=>............................] - ETA: 18s
17448960/221551911 [=>............................] - ETA: 18s
18038784/221551911 [=>............................] - ETA: 18s
18628608/221551911 [=>............................] - ETA: 18s
19218432/221551911 [=>............................] - ETA: 18s
19808256/221551911 [=>............................] - ETA: 17s
20398080/221551911 [=>............................] - ETA: 17s
20987904/221551911 [=>............................] - ETA: 17s
21577728/221551911 [=>............................] - ETA: 17s
22167552/221551911 [==>...........................] - ETA: 17s
22757376/221551911 [==>...........................] - ETA: 17s
23347200/221551911 [==>...........................] - ETA: 17s
23937024/221551911 [==>...........................] - ETA: 17s
24526848/221551911 [==>...........................] - ETA: 17s
25116672/221551911 [==>...........................] - ETA: 17s
25706496/221551911 [==>...........................] - ETA: 17s
26296320/221551911 [==>...........................] - ETA: 17s
26886144/221551911 [==>...........................] - ETA: 17s
27475968/221551911 [==>...........................] - ETA: 17s
28065792/221551911 [==>...........................] - ETA: 17s
28655616/221551911 [==>...........................] - ETA: 16s
29245440/221551911 [==>...........................] - ETA: 16s
29835264/221551911 [===>..........................] - ETA: 16s
30425088/221551911 [===>..........................] - ETA: 16s
31014912/221551911 [===>..........................] - ETA: 16s
31604736/221551911 [===>..........................] - ETA: 16s
32194560/221551911 [===>..........................] - ETA: 16s
32784384/221551911 [===>..........................] - ETA: 16s
33374208/221551911 [===>..........................] - ETA: 16s
33964032/221551911 [===>..........................] - ETA: 16s
34553856/221551911 [===>..........................] - ETA: 16s
35143680/221551911 [===>..........................] - ETA: 16s
35733504/221551911 [===>..........................] - ETA: 16s
36290560/221551911 [===>..........................] - ETA: 16s
36814848/221551911 [===>..........................] - ETA: 16s
37322752/221551911 [====>.........................] - ETA: 16s
37847040/221551911 [====>.........................] - ETA: 16s
38371328/221551911 [====>.........................] - ETA: 16s
38895616/221551911 [====>.........................] - ETA: 16s
39419904/221551911 [====>.........................] - ETA: 16s
39960576/221551911 [====>.........................] - ETA: 16s
40370176/221551911 [====>.........................] - ETA: 16s
40812544/221551911 [====>.........................] - ETA: 16s
41205760/221551911 [====>.........................] - ETA: 16s
41598976/221551911 [====>.........................] - ETA: 16s
41992192/221551911 [====>.........................] - ETA: 16s
42385408/221551911 [====>.........................] - ETA: 16s
42811392/221551911 [====>.........................] - ETA: 16s
43253760/221551911 [====>.........................] - ETA: 16s
43679744/221551911 [====>.........................] - ETA: 16s
44105728/221551911 [====>.........................] - ETA: 16s
44564480/221551911 [=====>........................] - ETA: 16s
45006848/221551911 [=====>........................] - ETA: 16s
45465600/221551911 [=====>........................] - ETA: 16s
45924352/221551911 [=====>........................] - ETA: 16s
46383104/221551911 [=====>........................] - ETA: 16s
46858240/221551911 [=====>........................] - ETA: 16s
47333376/221551911 [=====>........................] - ETA: 16s
47808512/221551911 [=====>........................] - ETA: 16s
48250880/221551911 [=====>........................] - ETA: 16s
48594944/221551911 [=====>........................] - ETA: 16s
48955392/221551911 [=====>........................] - ETA: 16s
49315840/221551911 [=====>........................] - ETA: 16s
49676288/221551911 [=====>........................] - ETA: 16s
50053120/221551911 [=====>........................] - ETA: 16s
50429952/221551911 [=====>........................] - ETA: 16s
50823168/221551911 [=====>........................] - ETA: 16s
51216384/221551911 [=====>........................] - ETA: 16s
51609600/221551911 [=====>........................] - ETA: 16s
52002816/221551911 [======>.......................] - ETA: 16s
52330496/221551911 [======>.......................] - ETA: 16s
52756480/221551911 [======>.......................] - ETA: 16s
53182464/221551911 [======>.......................] - ETA: 16s
53608448/221551911 [======>.......................] - ETA: 16s
54042624/221551911 [======>.......................] - ETA: 16s
54476800/221551911 [======>.......................] - ETA: 16s
54919168/221551911 [======>.......................] - ETA: 16s
55377920/221551911 [======>.......................] - ETA: 16s
55836672/221551911 [======>.......................] - ETA: 16s
56295424/221551911 [======>.......................] - ETA: 16s
56754176/221551911 [======>.......................] - ETA: 16s
57229312/221551911 [======>.......................] - ETA: 16s
57704448/221551911 [======>.......................] - ETA: 16s
58195968/221551911 [======>.......................] - ETA: 16s
58703872/221551911 [======>.......................] - ETA: 16s
59195392/221551911 [=======>......................] - ETA: 16s
59703296/221551911 [=======>......................] - ETA: 15s
60211200/221551911 [=======>......................] - ETA: 15s
60719104/221551911 [=======>......................] - ETA: 15s
61243392/221551911 [=======>......................] - ETA: 15s
61767680/221551911 [=======>......................] - ETA: 15s
62308352/221551911 [=======>......................] - ETA: 15s
62849024/221551911 [=======>......................] - ETA: 15s
63389696/221551911 [=======>......................] - ETA: 15s
63897600/221551911 [=======>......................] - ETA: 15s
64454656/221551911 [=======>......................] - ETA: 15s
65011712/221551911 [=======>......................] - ETA: 15s
65568768/221551911 [=======>......................] - ETA: 15s
66125824/221551911 [=======>......................] - ETA: 15s
66715648/221551911 [========>.....................] - ETA: 15s
67076096/221551911 [========>.....................] - ETA: 15s
67469312/221551911 [========>.....................] - ETA: 15s
67895296/221551911 [========>.....................] - ETA: 15s
68337664/221551911 [========>.....................] - ETA: 15s
68780032/221551911 [========>.....................] - ETA: 15s
69222400/221551911 [========>.....................] - ETA: 15s
69664768/221551911 [========>.....................] - ETA: 15s
70123520/221551911 [========>.....................] - ETA: 15s
70574080/221551911 [========>.....................] - ETA: 15s
70893568/221551911 [========>.....................] - ETA: 15s
71204864/221551911 [========>.....................] - ETA: 15s
71499776/221551911 [========>.....................] - ETA: 15s
71745536/221551911 [========>.....................] - ETA: 15s
71991296/221551911 [========>.....................] - ETA: 15s
72228864/221551911 [========>.....................] - ETA: 15s
72450048/221551911 [========>.....................] - ETA: 15s
72712192/221551911 [========>.....................] - ETA: 15s
72974336/221551911 [========>.....................] - ETA: 15s
73236480/221551911 [========>.....................] - ETA: 15s
73465856/221551911 [========>.....................] - ETA: 15s
73744384/221551911 [========>.....................] - ETA: 15s
74039296/221551911 [=========>....................] - ETA: 15s
74326016/221551911 [=========>....................] - ETA: 15s
74629120/221551911 [=========>....................] - ETA: 15s
74940416/221551911 [=========>....................] - ETA: 15s
75268096/221551911 [=========>....................] - ETA: 15s
75595776/221551911 [=========>....................] - ETA: 15s
75923456/221551911 [=========>....................] - ETA: 15s
76251136/221551911 [=========>....................] - ETA: 15s
76595200/221551911 [=========>....................] - ETA: 15s
76939264/221551911 [=========>....................] - ETA: 15s
77266944/221551911 [=========>....................] - ETA: 15s
77529088/221551911 [=========>....................] - ETA: 15s
77791232/221551911 [=========>....................] - ETA: 15s
78036992/221551911 [=========>....................] - ETA: 15s
78299136/221551911 [=========>....................] - ETA: 15s
78577664/221551911 [=========>....................] - ETA: 15s
78856192/221551911 [=========>....................] - ETA: 15s
79134720/221551911 [=========>....................] - ETA: 15s
79429632/221551911 [=========>....................] - ETA: 15s
79740928/221551911 [=========>....................] - ETA: 15s
80044032/221551911 [=========>....................] - ETA: 15s
80347136/221551911 [=========>....................] - ETA: 15s
80658432/221551911 [=========>....................] - ETA: 15s
80986112/221551911 [=========>....................] - ETA: 15s
81313792/221551911 [==========>...................] - ETA: 15s
81657856/221551911 [==========>...................] - ETA: 15s
82001920/221551911 [==========>...................] - ETA: 15s
82362368/221551911 [==========>...................] - ETA: 15s
82722816/221551911 [==========>...................] - ETA: 15s
83083264/221551911 [==========>...................] - ETA: 15s
83460096/221551911 [==========>...................] - ETA: 15s
83836928/221551911 [==========>...................] - ETA: 15s
84230144/221551911 [==========>...................] - ETA: 15s
84606976/221551911 [==========>...................] - ETA: 15s
85000192/221551911 [==========>...................] - ETA: 15s
85393408/221551911 [==========>...................] - ETA: 15s
85786624/221551911 [==========>...................] - ETA: 15s
86196224/221551911 [==========>...................] - ETA: 15s
86540288/221551911 [==========>...................] - ETA: 15s
86982656/221551911 [==========>...................] - ETA: 15s
87425024/221551911 [==========>...................] - ETA: 15s
87867392/221551911 [==========>...................] - ETA: 14s
88309760/221551911 [==========>...................] - ETA: 14s
88752128/221551911 [===========>..................] - ETA: 14s
89210880/221551911 [===========>..................] - ETA: 14s
89669632/221551911 [===========>..................] - ETA: 14s
90128384/221551911 [===========>..................] - ETA: 14s
90603520/221551911 [===========>..................] - ETA: 14s
91078656/221551911 [===========>..................] - ETA: 14s
91553792/221551911 [===========>..................] - ETA: 14s
92045312/221551911 [===========>..................] - ETA: 14s
92536832/221551911 [===========>..................] - ETA: 14s
93028352/221551911 [===========>..................] - ETA: 14s
93536256/221551911 [===========>..................] - ETA: 14s
94044160/221551911 [===========>..................] - ETA: 14s
94552064/221551911 [===========>..................] - ETA: 14s
95027200/221551911 [===========>..................] - ETA: 14s
95387648/221551911 [===========>..................] - ETA: 14s
95764480/221551911 [===========>..................] - ETA: 14s
96141312/221551911 [============>.................] - ETA: 14s
96518144/221551911 [============>.................] - ETA: 14s
96911360/221551911 [============>.................] - ETA: 13s
97304576/221551911 [============>.................] - ETA: 13s
97697792/221551911 [============>.................] - ETA: 13s
98107392/221551911 [============>.................] - ETA: 13s
98533376/221551911 [============>.................] - ETA: 13s
98959360/221551911 [============>.................] - ETA: 13s
99385344/221551911 [============>.................] - ETA: 13s
99794944/221551911 [============>.................] - ETA: 13s
100204544/221551911 [============>.................] - ETA: 13s
100597760/221551911 [============>.................] - ETA: 13s
100909056/221551911 [============>.................] - ETA: 13s
101220352/221551911 [============>.................] - ETA: 13s
101531648/221551911 [============>.................] - ETA: 13s
101859328/221551911 [============>.................] - ETA: 13s
102203392/221551911 [============>.................] - ETA: 13s
102547456/221551911 [============>.................] - ETA: 13s
102793216/221551911 [============>.................] - ETA: 13s
103006208/221551911 [============>.................] - ETA: 13s
103251968/221551911 [============>.................] - ETA: 13s
103514112/221551911 [=============>................] - ETA: 13s
103776256/221551911 [=============>................] - ETA: 13s
104038400/221551911 [=============>................] - ETA: 13s
104316928/221551911 [=============>................] - ETA: 13s
104595456/221551911 [=============>................] - ETA: 13s
104890368/221551911 [=============>................] - ETA: 13s
105168896/221551911 [=============>................] - ETA: 13s
105480192/221551911 [=============>................] - ETA: 13s
105791488/221551911 [=============>................] - ETA: 13s
106102784/221551911 [=============>................] - ETA: 13s
106414080/221551911 [=============>................] - ETA: 13s
106659840/221551911 [=============>................] - ETA: 13s
106987520/221551911 [=============>................] - ETA: 13s
107233280/221551911 [=============>................] - ETA: 13s
107479040/221551911 [=============>................] - ETA: 13s
107724800/221551911 [=============>................] - ETA: 13s
107970560/221551911 [=============>................] - ETA: 13s
108216320/221551911 [=============>................] - ETA: 13s
108494848/221551911 [=============>................] - ETA: 13s
108756992/221551911 [=============>................] - ETA: 13s
109019136/221551911 [=============>................] - ETA: 13s
109297664/221551911 [=============>................] - ETA: 13s
109592576/221551911 [=============>................] - ETA: 13s
109838336/221551911 [=============>................] - ETA: 13s
110149632/221551911 [=============>................] - ETA: 13s
110460928/221551911 [=============>................] - ETA: 13s
110788608/221551911 [==============>...............] - ETA: 13s
111116288/221551911 [==============>...............] - ETA: 13s
111443968/221551911 [==============>...............] - ETA: 13s
111788032/221551911 [==============>...............] - ETA: 13s
112132096/221551911 [==============>...............] - ETA: 13s
112476160/221551911 [==============>...............] - ETA: 13s
112803840/221551911 [==============>...............] - ETA: 13s
113147904/221551911 [==============>...............] - ETA: 12s
113508352/221551911 [==============>...............] - ETA: 12s
113868800/221551911 [==============>...............] - ETA: 12s
114245632/221551911 [==============>...............] - ETA: 12s
114638848/221551911 [==============>...............] - ETA: 12s
115032064/221551911 [==============>...............] - ETA: 12s
115425280/221551911 [==============>...............] - ETA: 12s
115818496/221551911 [==============>...............] - ETA: 12s
116211712/221551911 [==============>...............] - ETA: 12s
116604928/221551911 [==============>...............] - ETA: 12s
116998144/221551911 [==============>...............] - ETA: 12s
117309440/221551911 [==============>...............] - ETA: 12s
117653504/221551911 [==============>...............] - ETA: 12s
118079488/221551911 [==============>...............] - ETA: 12s
118505472/221551911 [===============>..............] - ETA: 12s
118964224/221551911 [===============>..............] - ETA: 12s
119406592/221551911 [===============>..............] - ETA: 12s
119848960/221551911 [===============>..............] - ETA: 12s
120307712/221551911 [===============>..............] - ETA: 12s
120766464/221551911 [===============>..............] - ETA: 12s
121257984/221551911 [===============>..............] - ETA: 12s
121749504/221551911 [===============>..............] - ETA: 11s
122241024/221551911 [===============>..............] - ETA: 11s
122732544/221551911 [===============>..............] - ETA: 11s
123240448/221551911 [===============>..............] - ETA: 11s
123748352/221551911 [===============>..............] - ETA: 11s
124256256/221551911 [===============>..............] - ETA: 11s
124780544/221551911 [===============>..............] - ETA: 11s
125304832/221551911 [===============>..............] - ETA: 11s
125829120/221551911 [================>.............] - ETA: 11s
126337024/221551911 [================>.............] - ETA: 11s
126894080/221551911 [================>.............] - ETA: 11s
127434752/221551911 [================>.............] - ETA: 11s
127975424/221551911 [================>.............] - ETA: 11s
128532480/221551911 [================>.............] - ETA: 11s
129024000/221551911 [================>.............] - ETA: 11s
129417216/221551911 [================>.............] - ETA: 10s
129826816/221551911 [================>.............] - ETA: 10s
130236416/221551911 [================>.............] - ETA: 10s
130662400/221551911 [================>.............] - ETA: 10s
131104768/221551911 [================>.............] - ETA: 10s
131547136/221551911 [================>.............] - ETA: 10s
131940352/221551911 [================>.............] - ETA: 10s
132349952/221551911 [================>.............] - ETA: 10s
132792320/221551911 [================>.............] - ETA: 10s
133251072/221551911 [=================>............] - ETA: 10s
133693440/221551911 [=================>............] - ETA: 10s
134135808/221551911 [=================>............] - ETA: 10s
134610944/221551911 [=================>............] - ETA: 10s
135086080/221551911 [=================>............] - ETA: 10s
135561216/221551911 [=================>............] - ETA: 10s
136052736/221551911 [=================>............] - ETA: 10s
136560640/221551911 [=================>............] - ETA: 10s
137068544/221551911 [=================>............] - ETA: 10s
137576448/221551911 [=================>............] - ETA: 9s
138084352/221551911 [=================>............] - ETA: 9s
138592256/221551911 [=================>............] - ETA: 9s
139116544/221551911 [=================>............] - ETA: 9s
139640832/221551911 [=================>............] - ETA: 9s
140165120/221551911 [=================>............] - ETA: 9s
140705792/221551911 [==================>...........] - ETA: 9s
141246464/221551911 [==================>...........] - ETA: 9s
141803520/221551911 [==================>...........] - ETA: 9s
142360576/221551911 [==================>...........] - ETA: 9s
142917632/221551911 [==================>...........] - ETA: 9s
143474688/221551911 [==================>...........] - ETA: 9s
144031744/221551911 [==================>...........] - ETA: 9s
144605184/221551911 [==================>...........] - ETA: 9s
145096704/221551911 [==================>...........] - ETA: 8s
145522688/221551911 [==================>...........] - ETA: 8s
145948672/221551911 [==================>...........] - ETA: 8s
146374656/221551911 [==================>...........] - ETA: 8s
146817024/221551911 [==================>...........] - ETA: 8s
147259392/221551911 [==================>...........] - ETA: 8s
147718144/221551911 [===================>..........] - ETA: 8s
148176896/221551911 [===================>..........] - ETA: 8s
148635648/221551911 [===================>..........] - ETA: 8s
149110784/221551911 [===================>..........] - ETA: 8s
149585920/221551911 [===================>..........] - ETA: 8s
150061056/221551911 [===================>..........] - ETA: 8s
150536192/221551911 [===================>..........] - ETA: 8s
151027712/221551911 [===================>..........] - ETA: 8s
151502848/221551911 [===================>..........] - ETA: 8s
152010752/221551911 [===================>..........] - ETA: 8s
152518656/221551911 [===================>..........] - ETA: 8s
152944640/221551911 [===================>..........] - ETA: 8s
153124864/221551911 [===================>..........] - ETA: 8s
153632768/221551911 [===================>..........] - ETA: 7s
154140672/221551911 [===================>..........] - ETA: 7s
154664960/221551911 [===================>..........] - ETA: 7s
155172864/221551911 [====================>.........] - ETA: 7s
155385856/221551911 [====================>.........] - ETA: 7s
155582464/221551911 [====================>.........] - ETA: 7s
156123136/221551911 [====================>.........] - ETA: 7s
156663808/221551911 [====================>.........] - ETA: 7s
157220864/221551911 [====================>.........] - ETA: 7s
157532160/221551911 [====================>.........] - ETA: 7s
157679616/221551911 [====================>.........] - ETA: 7s
158236672/221551911 [====================>.........] - ETA: 7s
158810112/221551911 [====================>.........] - ETA: 7s
159383552/221551911 [====================>.........] - ETA: 7s
159678464/221551911 [====================>.........] - ETA: 7s
159817728/221551911 [====================>.........] - ETA: 7s
160235520/221551911 [====================>.........] - ETA: 7s
160825344/221551911 [====================>.........] - ETA: 7s
161415168/221551911 [====================>.........] - ETA: 7s
161890304/221551911 [====================>.........] - ETA: 6s
162103296/221551911 [====================>.........] - ETA: 6s
162463744/221551911 [====================>.........] - ETA: 6s
163053568/221551911 [=====================>........] - ETA: 6s
163643392/221551911 [=====================>........] - ETA: 6s
164184064/221551911 [=====================>........] - ETA: 6s
164397056/221551911 [=====================>........] - ETA: 6s
164593664/221551911 [=====================>........] - ETA: 6s
165183488/221551911 [=====================>........] - ETA: 6s
165773312/221551911 [=====================>........] - ETA: 6s
166363136/221551911 [=====================>........] - ETA: 6s
166756352/221551911 [=====================>........] - ETA: 6s
166952960/221551911 [=====================>........] - ETA: 6s
167378944/221551911 [=====================>........] - ETA: 6s
167968768/221551911 [=====================>........] - ETA: 6s
168558592/221551911 [=====================>........] - ETA: 6s
169148416/221551911 [=====================>........] - ETA: 6s
169738240/221551911 [=====================>........] - ETA: 6s
170328064/221551911 [======================>.......] - ETA: 5s
170917888/221551911 [======================>.......] - ETA: 5s
171507712/221551911 [======================>.......] - ETA: 5s
172097536/221551911 [======================>.......] - ETA: 5s
172687360/221551911 [======================>.......] - ETA: 5s
173277184/221551911 [======================>.......] - ETA: 5s
173867008/221551911 [======================>.......] - ETA: 5s
174456832/221551911 [======================>.......] - ETA: 5s
175046656/221551911 [======================>.......] - ETA: 5s
175636480/221551911 [======================>.......] - ETA: 5s
176226304/221551911 [======================>.......] - ETA: 5s
176816128/221551911 [======================>.......] - ETA: 5s
177405952/221551911 [=======================>......] - ETA: 5s
177995776/221551911 [=======================>......] - ETA: 5s
178585600/221551911 [=======================>......] - ETA: 4s
179175424/221551911 [=======================>......] - ETA: 4s
179765248/221551911 [=======================>......] - ETA: 4s
180355072/221551911 [=======================>......] - ETA: 4s
180944896/221551911 [=======================>......] - ETA: 4s
181501952/221551911 [=======================>......] - ETA: 4s
182140928/221551911 [=======================>......] - ETA: 4s
182583296/221551911 [=======================>......] - ETA: 4s
183386112/221551911 [=======================>......] - ETA: 4s
183943168/221551911 [=======================>......] - ETA: 4s
184500224/221551911 [=======================>......] - ETA: 4s
185073664/221551911 [========================>.....] - ETA: 4s
185630720/221551911 [========================>.....] - ETA: 4s
186187776/221551911 [========================>.....] - ETA: 4s
186777600/221551911 [========================>.....] - ETA: 3s
187367424/221551911 [========================>.....] - ETA: 3s
187940864/221551911 [========================>.....] - ETA: 3s
188530688/221551911 [========================>.....] - ETA: 3s
189120512/221551911 [========================>.....] - ETA: 3s
189710336/221551911 [========================>.....] - ETA: 3s
190300160/221551911 [========================>.....] - ETA: 3s
190889984/221551911 [========================>.....] - ETA: 3s
191447040/221551911 [========================>.....] - ETA: 3s
191840256/221551911 [========================>.....] - ETA: 3s
192331776/221551911 [=========================>....] - ETA: 3s
192790528/221551911 [=========================>....] - ETA: 3s
193249280/221551911 [=========================>....] - ETA: 3s
193708032/221551911 [=========================>....] - ETA: 3s
194183168/221551911 [=========================>....] - ETA: 3s
194674688/221551911 [=========================>....] - ETA: 3s
195166208/221551911 [=========================>....] - ETA: 2s
195657728/221551911 [=========================>....] - ETA: 2s
196149248/221551911 [=========================>....] - ETA: 2s
196657152/221551911 [=========================>....] - ETA: 2s
197165056/221551911 [=========================>....] - ETA: 2s
197689344/221551911 [=========================>....] - ETA: 2s
198213632/221551911 [=========================>....] - ETA: 2s
198737920/221551911 [=========================>....] - ETA: 2s
199278592/221551911 [=========================>....] - ETA: 2s
199819264/221551911 [==========================>...] - ETA: 2s
200359936/221551911 [==========================>...] - ETA: 2s
200916992/221551911 [==========================>...] - ETA: 2s
201457664/221551911 [==========================>...] - ETA: 2s
202014720/221551911 [==========================>...] - ETA: 2s
202588160/221551911 [==========================>...] - ETA: 2s
203161600/221551911 [==========================>...] - ETA: 2s
203718656/221551911 [==========================>...] - ETA: 2s
204308480/221551911 [==========================>...] - ETA: 1s
204832768/221551911 [==========================>...] - ETA: 1s
205258752/221551911 [==========================>...] - ETA: 1s
205684736/221551911 [==========================>...] - ETA: 1s
206127104/221551911 [==========================>...] - ETA: 1s
206585856/221551911 [==========================>...] - ETA: 1s
207028224/221551911 [===========================>..] - ETA: 1s
207470592/221551911 [===========================>..] - ETA: 1s
207929344/221551911 [===========================>..] - ETA: 1s
208388096/221551911 [===========================>..] - ETA: 1s
208846848/221551911 [===========================>..] - ETA: 1s
209305600/221551911 [===========================>..] - ETA: 1s
209780736/221551911 [===========================>..] - ETA: 1s
210272256/221551911 [===========================>..] - ETA: 1s
210747392/221551911 [===========================>..] - ETA: 1s
211238912/221551911 [===========================>..] - ETA: 1s
211599360/221551911 [===========================>..] - ETA: 1s
211959808/221551911 [===========================>..] - ETA: 1s
212336640/221551911 [===========================>..] - ETA: 1s
212713472/221551911 [===========================>..] - ETA: 0s
213090304/221551911 [===========================>..] - ETA: 0s
213483520/221551911 [===========================>..] - ETA: 0s
213876736/221551911 [===========================>..] - ETA: 0s
214269952/221551911 [============================>.] - ETA: 0s
214663168/221551911 [============================>.] - ETA: 0s
215056384/221551911 [============================>.] - ETA: 0s
215351296/221551911 [============================>.] - ETA: 0s
215646208/221551911 [============================>.] - ETA: 0s
215941120/221551911 [============================>.] - ETA: 0s
216137728/221551911 [============================>.] - ETA: 0s
216449024/221551911 [============================>.] - ETA: 0s
216678400/221551911 [============================>.] - ETA: 0s
216907776/221551911 [============================>.] - ETA: 0s
217137152/221551911 [============================>.] - ETA: 0s
217366528/221551911 [============================>.] - ETA: 0s
217612288/221551911 [============================>.] - ETA: 0s
217874432/221551911 [============================>.] - ETA: 0s
218120192/221551911 [============================>.] - ETA: 0s
218382336/221551911 [============================>.] - ETA: 0s
218652672/221551911 [============================>.] - ETA: 0s
218923008/221551911 [============================>.] - ETA: 0s
219201536/221551911 [============================>.] - ETA: 0s
219480064/221551911 [============================>.] - ETA: 0s
219774976/221551911 [============================>.] - ETA: 0s
220086272/221551911 [============================>.] - ETA: 0s
220397568/221551911 [============================>.] - ETA: 0s
220708864/221551911 [============================>.] - ETA: 0s
221020160/221551911 [============================>.] - ETA: 0s
221347840/221551911 [============================>.] - ETA: 0s
221551911/221551911 [==============================] - 26s 0us/step
Loaded VOC2007 test data for car and person classes: 2500 images.
Anchors can also be computed easily using YOLO toolkit.
Note
The following code is given as an example. In a real use case scenario, anchors are computed on the training dataset.
from akida_models.detection.generate_anchors import generate_anchors
num_anchors = 5
grid_size = (7, 7)
anchors_example = generate_anchors(val_data, num_anchors, grid_size)
Average IOU for 5 anchors: 0.61
Anchors: [[0.63263, 1.13864], [1.29467, 2.90717], [2.26527, 2.97757], [3.80627, 5.03516], [5.21984, 5.79988]]
3. Model architecture
The model zoo contains a YOLO model that is built upon the AkidaNet architecture and 3 separable convolutional layers at the top for bounding box and class estimation followed by a final separable convolutional which is the detection layer. Note that for efficiency, the alpha parameter in AkidaNet (network width or number of filter in each layer) is set to 0.5.
from akida_models import yolo_base
# Create a yolo model for 2 classes with 5 anchors and grid size of 7
classes = 2
model = yolo_base(input_shape=(224, 224, 3),
classes=classes,
nb_box=num_anchors,
alpha=0.5)
model.summary()
Model: "yolo_base"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input (InputLayer) [(None, 224, 224, 3)] 0
rescaling (Rescaling) (None, 224, 224, 3) 0
conv_0 (Conv2D) (None, 112, 112, 16) 432
conv_0/BN (BatchNormalizati (None, 112, 112, 16) 64
on)
conv_0/relu (ReLU) (None, 112, 112, 16) 0
conv_1 (Conv2D) (None, 112, 112, 32) 4608
conv_1/BN (BatchNormalizati (None, 112, 112, 32) 128
on)
conv_1/relu (ReLU) (None, 112, 112, 32) 0
conv_2 (Conv2D) (None, 56, 56, 64) 18432
conv_2/BN (BatchNormalizati (None, 56, 56, 64) 256
on)
conv_2/relu (ReLU) (None, 56, 56, 64) 0
conv_3 (Conv2D) (None, 56, 56, 64) 36864
conv_3/BN (BatchNormalizati (None, 56, 56, 64) 256
on)
conv_3/relu (ReLU) (None, 56, 56, 64) 0
dw_separable_4 (DepthwiseCo (None, 28, 28, 64) 576
nv2D)
pw_separable_4 (Conv2D) (None, 28, 28, 128) 8192
pw_separable_4/BN (BatchNor (None, 28, 28, 128) 512
malization)
pw_separable_4/relu (ReLU) (None, 28, 28, 128) 0
dw_separable_5 (DepthwiseCo (None, 28, 28, 128) 1152
nv2D)
pw_separable_5 (Conv2D) (None, 28, 28, 128) 16384
pw_separable_5/BN (BatchNor (None, 28, 28, 128) 512
malization)
pw_separable_5/relu (ReLU) (None, 28, 28, 128) 0
dw_separable_6 (DepthwiseCo (None, 14, 14, 128) 1152
nv2D)
pw_separable_6 (Conv2D) (None, 14, 14, 256) 32768
pw_separable_6/BN (BatchNor (None, 14, 14, 256) 1024
malization)
pw_separable_6/relu (ReLU) (None, 14, 14, 256) 0
dw_separable_7 (DepthwiseCo (None, 14, 14, 256) 2304
nv2D)
pw_separable_7 (Conv2D) (None, 14, 14, 256) 65536
pw_separable_7/BN (BatchNor (None, 14, 14, 256) 1024
malization)
pw_separable_7/relu (ReLU) (None, 14, 14, 256) 0
dw_separable_8 (DepthwiseCo (None, 14, 14, 256) 2304
nv2D)
pw_separable_8 (Conv2D) (None, 14, 14, 256) 65536
pw_separable_8/BN (BatchNor (None, 14, 14, 256) 1024
malization)
pw_separable_8/relu (ReLU) (None, 14, 14, 256) 0
dw_separable_9 (DepthwiseCo (None, 14, 14, 256) 2304
nv2D)
pw_separable_9 (Conv2D) (None, 14, 14, 256) 65536
pw_separable_9/BN (BatchNor (None, 14, 14, 256) 1024
malization)
pw_separable_9/relu (ReLU) (None, 14, 14, 256) 0
dw_separable_10 (DepthwiseC (None, 14, 14, 256) 2304
onv2D)
pw_separable_10 (Conv2D) (None, 14, 14, 256) 65536
pw_separable_10/BN (BatchNo (None, 14, 14, 256) 1024
rmalization)
pw_separable_10/relu (ReLU) (None, 14, 14, 256) 0
dw_separable_11 (DepthwiseC (None, 14, 14, 256) 2304
onv2D)
pw_separable_11 (Conv2D) (None, 14, 14, 256) 65536
pw_separable_11/BN (BatchNo (None, 14, 14, 256) 1024
rmalization)
pw_separable_11/relu (ReLU) (None, 14, 14, 256) 0
dw_separable_12 (DepthwiseC (None, 7, 7, 256) 2304
onv2D)
pw_separable_12 (Conv2D) (None, 7, 7, 512) 131072
pw_separable_12/BN (BatchNo (None, 7, 7, 512) 2048
rmalization)
pw_separable_12/relu (ReLU) (None, 7, 7, 512) 0
dw_separable_13 (DepthwiseC (None, 7, 7, 512) 4608
onv2D)
pw_separable_13 (Conv2D) (None, 7, 7, 512) 262144
pw_separable_13/BN (BatchNo (None, 7, 7, 512) 2048
rmalization)
pw_separable_13/relu (ReLU) (None, 7, 7, 512) 0
dw_1conv (DepthwiseConv2D) (None, 7, 7, 512) 4608
pw_1conv (Conv2D) (None, 7, 7, 1024) 524288
pw_1conv/BN (BatchNormaliza (None, 7, 7, 1024) 4096
tion)
pw_1conv/relu (ReLU) (None, 7, 7, 1024) 0
dw_2conv (DepthwiseConv2D) (None, 7, 7, 1024) 9216
pw_2conv (Conv2D) (None, 7, 7, 1024) 1048576
pw_2conv/BN (BatchNormaliza (None, 7, 7, 1024) 4096
tion)
pw_2conv/relu (ReLU) (None, 7, 7, 1024) 0
dw_3conv (DepthwiseConv2D) (None, 7, 7, 1024) 9216
pw_3conv (Conv2D) (None, 7, 7, 1024) 1048576
pw_3conv/BN (BatchNormaliza (None, 7, 7, 1024) 4096
tion)
pw_3conv/relu (ReLU) (None, 7, 7, 1024) 0
dw_detection_layer (Depthwi (None, 7, 7, 1024) 9216
seConv2D)
pw_detection_layer (Conv2D) (None, 7, 7, 35) 35875
=================================================================
Total params: 3,573,715
Trainable params: 3,561,587
Non-trainable params: 12,128
_________________________________________________________________
The model output can be reshaped to a more natural shape of:
(grid_height, grid_width, anchors_box, 4 + 1 + num_classes)
where the “4 + 1” term represents the coordinates of the estimated bounding boxes (top left x, top left y, width and height) and a confidence score. In other words, the output channels are actually grouped by anchor boxes, and in each group one channel provides either a coordinate, a global confidence score or a class confidence score. This process is done automatically in the decode_output function.
from tensorflow.keras import Model
from tensorflow.keras.layers import Reshape
# Define a reshape output to be added to the YOLO model
output = Reshape((grid_size[1], grid_size[0], num_anchors, 4 + 1 + classes),
name="YOLO_output")(model.output)
# Build the complete model
full_model = Model(model.input, output)
full_model.output
<KerasTensor: shape=(None, 7, 7, 5, 7) dtype=float32 (created by layer 'YOLO_output')>
4. Training
As the YOLO model relies on Brainchip AkidaNet/ImageNet network, it is possible to perform transfer learning from ImageNet pretrained weights when training a YOLO model. See the PlantVillage transfer learning example for a detail explanation on transfer learning principles.
5. Performance
The model zoo also contains an helper method that allows to create a YOLO model for VOC and load pretrained weights for the car and person detection task and the corresponding anchors. The anchors are used to interpret the model outputs.
The metric used to evaluate YOLO is the mean average precision (mAP) which is the percentage of correct prediction and is given for an intersection over union (IoU) ratio. Scores in this example are given for the standard IoU of 0.5 meaning that a detection is considered valid if the intersection over union ratio with its ground truth equivalent is above 0.5.
Note
A call to evaluate_map will preprocess the images, make the call to
Model.predictand use decode_output before computing precision for all classes.
from timeit import default_timer as timer
from akida_models import yolo_voc_pretrained
from akida_models.detection.map_evaluation import MapEvaluation
# Load the pretrained model along with anchors
model_keras, anchors = yolo_voc_pretrained()
model_keras.summary()
Downloading data from https://data.brainchip.com/dataset-mirror/voc/voc_anchors.pkl.
0/126 [..............................] - ETA: 0s
126/126 [==============================] - 0s 3us/step
Downloading data from https://data.brainchip.com/models/AkidaV2/yolo/yolo_akidanet_voc_i8_w4_a4.h5.
0/14557688 [..............................] - ETA: 0s
204800/14557688 [..............................] - ETA: 3s
770048/14557688 [>.............................] - ETA: 1s
1351680/14557688 [=>............................] - ETA: 1s
1933312/14557688 [==>...........................] - ETA: 1s
2523136/14557688 [====>.........................] - ETA: 1s
3104768/14557688 [=====>........................] - ETA: 1s
3686400/14557688 [======>.......................] - ETA: 1s
4268032/14557688 [=======>......................] - ETA: 0s
4857856/14557688 [=========>....................] - ETA: 0s
5439488/14557688 [==========>...................] - ETA: 0s
6029312/14557688 [===========>..................] - ETA: 0s
6602752/14557688 [============>.................] - ETA: 0s
7192576/14557688 [=============>................] - ETA: 0s
7798784/14557688 [===============>..............] - ETA: 0s
8380416/14557688 [================>.............] - ETA: 0s
8953856/14557688 [=================>............] - ETA: 0s
9543680/14557688 [==================>...........] - ETA: 0s
10133504/14557688 [===================>..........] - ETA: 0s
10723328/14557688 [=====================>........] - ETA: 0s
11329536/14557688 [======================>.......] - ETA: 0s
11894784/14557688 [=======================>......] - ETA: 0s
12484608/14557688 [========================>.....] - ETA: 0s
13074432/14557688 [=========================>....] - ETA: 0s
13656064/14557688 [===========================>..] - ETA: 0s
14245888/14557688 [============================>.] - ETA: 0s
14557688/14557688 [==============================] - 1s 0us/step
Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input (InputLayer) [(None, 224, 224, 3)] 0
rescaling (QuantizedRescali (None, 224, 224, 3) 0
ng)
conv_0 (QuantizedConv2D) (None, 112, 112, 16) 448
conv_0/relu (QuantizedReLU) (None, 112, 112, 16) 32
conv_1 (QuantizedConv2D) (None, 112, 112, 32) 4640
conv_1/relu (QuantizedReLU) (None, 112, 112, 32) 64
conv_2 (QuantizedConv2D) (None, 56, 56, 64) 18496
conv_2/relu (QuantizedReLU) (None, 56, 56, 64) 128
conv_3 (QuantizedConv2D) (None, 56, 56, 64) 36928
conv_3/relu (QuantizedReLU) (None, 56, 56, 64) 128
dw_separable_4 (QuantizedDe (None, 28, 28, 64) 704
pthwiseConv2D)
pw_separable_4 (QuantizedCo (None, 28, 28, 128) 8320
nv2D)
pw_separable_4/relu (Quanti (None, 28, 28, 128) 256
zedReLU)
dw_separable_5 (QuantizedDe (None, 28, 28, 128) 1408
pthwiseConv2D)
pw_separable_5 (QuantizedCo (None, 28, 28, 128) 16512
nv2D)
pw_separable_5/relu (Quanti (None, 28, 28, 128) 256
zedReLU)
dw_separable_6 (QuantizedDe (None, 14, 14, 128) 1408
pthwiseConv2D)
pw_separable_6 (QuantizedCo (None, 14, 14, 256) 33024
nv2D)
pw_separable_6/relu (Quanti (None, 14, 14, 256) 512
zedReLU)
dw_separable_7 (QuantizedDe (None, 14, 14, 256) 2816
pthwiseConv2D)
pw_separable_7 (QuantizedCo (None, 14, 14, 256) 65792
nv2D)
pw_separable_7/relu (Quanti (None, 14, 14, 256) 512
zedReLU)
dw_separable_8 (QuantizedDe (None, 14, 14, 256) 2816
pthwiseConv2D)
pw_separable_8 (QuantizedCo (None, 14, 14, 256) 65792
nv2D)
pw_separable_8/relu (Quanti (None, 14, 14, 256) 512
zedReLU)
dw_separable_9 (QuantizedDe (None, 14, 14, 256) 2816
pthwiseConv2D)
pw_separable_9 (QuantizedCo (None, 14, 14, 256) 65792
nv2D)
pw_separable_9/relu (Quanti (None, 14, 14, 256) 512
zedReLU)
dw_separable_10 (QuantizedD (None, 14, 14, 256) 2816
epthwiseConv2D)
pw_separable_10 (QuantizedC (None, 14, 14, 256) 65792
onv2D)
pw_separable_10/relu (Quant (None, 14, 14, 256) 512
izedReLU)
dw_separable_11 (QuantizedD (None, 14, 14, 256) 2816
epthwiseConv2D)
pw_separable_11 (QuantizedC (None, 14, 14, 256) 65792
onv2D)
pw_separable_11/relu (Quant (None, 14, 14, 256) 512
izedReLU)
dw_separable_12 (QuantizedD (None, 7, 7, 256) 2816
epthwiseConv2D)
pw_separable_12 (QuantizedC (None, 7, 7, 512) 131584
onv2D)
pw_separable_12/relu (Quant (None, 7, 7, 512) 1024
izedReLU)
dw_separable_13 (QuantizedD (None, 7, 7, 512) 5632
epthwiseConv2D)
pw_separable_13 (QuantizedC (None, 7, 7, 512) 262656
onv2D)
pw_separable_13/relu (Quant (None, 7, 7, 512) 1024
izedReLU)
dw_1conv (QuantizedDepthwis (None, 7, 7, 512) 5632
eConv2D)
pw_1conv (QuantizedConv2D) (None, 7, 7, 1024) 525312
pw_1conv/relu (QuantizedReL (None, 7, 7, 1024) 2048
U)
dw_2conv (QuantizedDepthwis (None, 7, 7, 1024) 11264
eConv2D)
pw_2conv (QuantizedConv2D) (None, 7, 7, 1024) 1049600
pw_2conv/relu (QuantizedReL (None, 7, 7, 1024) 2048
U)
dw_3conv (QuantizedDepthwis (None, 7, 7, 1024) 11264
eConv2D)
pw_3conv (QuantizedConv2D) (None, 7, 7, 1024) 1049600
pw_3conv/relu (QuantizedReL (None, 7, 7, 1024) 2048
U)
dw_detection_layer (Quantiz (None, 7, 7, 1024) 11264
edDepthwiseConv2D)
pw_detection_layer (Quantiz (None, 7, 7, 35) 35875
edConv2D)
dequantizer (Dequantizer) (None, 7, 7, 35) 0
=================================================================
Total params: 3,579,555
Trainable params: 3,555,523
Non-trainable params: 24,032
_________________________________________________________________
# Define the final reshape and build the model
output = Reshape((grid_size[1], grid_size[0], num_anchors, 4 + 1 + classes),
name="YOLO_output")(model_keras.output)
model_keras = Model(model_keras.input, output)
# Create the mAP evaluator object
num_images = 100
map_evaluator = MapEvaluation(model_keras, val_data[:num_images], labels,
anchors)
# Compute the scores for all validation images
start = timer()
mAP, average_precisions = map_evaluator.evaluate_map()
end = timer()
for label, average_precision in average_precisions.items():
print(labels[label], '{:.4f}'.format(average_precision))
print('mAP: {:.4f}'.format(mAP))
print(f'Keras inference on {num_images} images took {end-start:.2f} s.\n')
car 0.4726
person 0.4384
mAP: 0.4555
Keras inference on 100 images took 13.07 s.
6. Conversion to Akida
6.1 Convert to Akida model
The last YOLO_output layer that was added for splitting channels into values for each box must be removed before Akida conversion.
# Rebuild a model without the last layer
compatible_model = Model(model_keras.input, model_keras.layers[-2].output)
When converting to an Akida model, we just need to pass the Keras model to cnn2snn.convert.
from cnn2snn import convert
model_akida = convert(compatible_model)
model_akida.summary()
Model Summary
________________________________________________
Input shape Output shape Sequences Layers
================================================
[224, 224, 3] [7, 7, 35] 1 33
________________________________________________
__________________________________________________________________________
Layer (type) Output shape Kernel shape
==================== SW/conv_0-dequantizer (Software) ====================
conv_0 (InputConv2D) [112, 112, 16] (3, 3, 3, 16)
__________________________________________________________________________
conv_1 (Conv2D) [112, 112, 32] (3, 3, 16, 32)
__________________________________________________________________________
conv_2 (Conv2D) [56, 56, 64] (3, 3, 32, 64)
__________________________________________________________________________
conv_3 (Conv2D) [56, 56, 64] (3, 3, 64, 64)
__________________________________________________________________________
dw_separable_4 (DepthwiseConv2D) [28, 28, 64] (3, 3, 64, 1)
__________________________________________________________________________
pw_separable_4 (Conv2D) [28, 28, 128] (1, 1, 64, 128)
__________________________________________________________________________
dw_separable_5 (DepthwiseConv2D) [28, 28, 128] (3, 3, 128, 1)
__________________________________________________________________________
pw_separable_5 (Conv2D) [28, 28, 128] (1, 1, 128, 128)
__________________________________________________________________________
dw_separable_6 (DepthwiseConv2D) [14, 14, 128] (3, 3, 128, 1)
__________________________________________________________________________
pw_separable_6 (Conv2D) [14, 14, 256] (1, 1, 128, 256)
__________________________________________________________________________
dw_separable_7 (DepthwiseConv2D) [14, 14, 256] (3, 3, 256, 1)
__________________________________________________________________________
pw_separable_7 (Conv2D) [14, 14, 256] (1, 1, 256, 256)
__________________________________________________________________________
dw_separable_8 (DepthwiseConv2D) [14, 14, 256] (3, 3, 256, 1)
__________________________________________________________________________
pw_separable_8 (Conv2D) [14, 14, 256] (1, 1, 256, 256)
__________________________________________________________________________
dw_separable_9 (DepthwiseConv2D) [14, 14, 256] (3, 3, 256, 1)
__________________________________________________________________________
pw_separable_9 (Conv2D) [14, 14, 256] (1, 1, 256, 256)
__________________________________________________________________________
dw_separable_10 (DepthwiseConv2D) [14, 14, 256] (3, 3, 256, 1)
__________________________________________________________________________
pw_separable_10 (Conv2D) [14, 14, 256] (1, 1, 256, 256)
__________________________________________________________________________
dw_separable_11 (DepthwiseConv2D) [14, 14, 256] (3, 3, 256, 1)
__________________________________________________________________________
pw_separable_11 (Conv2D) [14, 14, 256] (1, 1, 256, 256)
__________________________________________________________________________
dw_separable_12 (DepthwiseConv2D) [7, 7, 256] (3, 3, 256, 1)
__________________________________________________________________________
pw_separable_12 (Conv2D) [7, 7, 512] (1, 1, 256, 512)
__________________________________________________________________________
dw_separable_13 (DepthwiseConv2D) [7, 7, 512] (3, 3, 512, 1)
__________________________________________________________________________
pw_separable_13 (Conv2D) [7, 7, 512] (1, 1, 512, 512)
__________________________________________________________________________
dw_1conv (DepthwiseConv2D) [7, 7, 512] (3, 3, 512, 1)
__________________________________________________________________________
pw_1conv (Conv2D) [7, 7, 1024] (1, 1, 512, 1024)
__________________________________________________________________________
dw_2conv (DepthwiseConv2D) [7, 7, 1024] (3, 3, 1024, 1)
__________________________________________________________________________
pw_2conv (Conv2D) [7, 7, 1024] (1, 1, 1024, 1024)
__________________________________________________________________________
dw_3conv (DepthwiseConv2D) [7, 7, 1024] (3, 3, 1024, 1)
__________________________________________________________________________
pw_3conv (Conv2D) [7, 7, 1024] (1, 1, 1024, 1024)
__________________________________________________________________________
dw_detection_layer (DepthwiseConv2D) [7, 7, 1024] (3, 3, 1024, 1)
__________________________________________________________________________
pw_detection_layer (Conv2D) [7, 7, 35] (1, 1, 1024, 35)
__________________________________________________________________________
dequantizer (Dequantizer) [7, 7, 35] N/A
__________________________________________________________________________
6.1 Check performance
Akida model accuracy is tested on the first n images of the validation set.
# Create the mAP evaluator object
map_evaluator_ak = MapEvaluation(model_akida,
val_data[:num_images],
labels,
anchors,
is_keras_model=False)
# Compute the scores for all validation images
start = timer()
mAP_ak, average_precisions_ak = map_evaluator_ak.evaluate_map()
end = timer()
for label, average_precision in average_precisions_ak.items():
print(labels[label], '{:.4f}'.format(average_precision))
print('mAP: {:.4f}'.format(mAP_ak))
print(f'Akida inference on {num_images} images took {end-start:.2f} s.\n')
car 0.4997
person 0.4164
mAP: 0.4580
Akida inference on 100 images took 12.39 s.
6.2 Show predictions for a random image
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as patches
from akida_models.detection.processing import load_image, preprocess_image, decode_output
# Take a random test image
i = np.random.randint(len(val_data))
input_shape = model_akida.layers[0].input_dims
# Load the image
raw_image = load_image(val_data[i]['image_path'])
# Keep the original image size for later bounding boxes rescaling
raw_height, raw_width, _ = raw_image.shape
# Pre-process the image
image = preprocess_image(raw_image, input_shape)
input_image = image[np.newaxis, :].astype(np.uint8)
# Call evaluate on the image
pots = model_akida.predict(input_image)[0]
# Reshape the potentials to prepare for decoding
h, w, c = pots.shape
pots = pots.reshape((h, w, len(anchors), 4 + 1 + len(labels)))
# Decode potentials into bounding boxes
raw_boxes = decode_output(pots, anchors, len(labels))
# Rescale boxes to the original image size
pred_boxes = np.array([[
box.x1 * raw_width, box.y1 * raw_height, box.x2 * raw_width,
box.y2 * raw_height,
box.get_label(),
box.get_score()
] for box in raw_boxes])
fig = plt.figure(num='VOC2012 car and person detection by Akida')
ax = fig.subplots(1)
img_plot = ax.imshow(np.zeros(raw_image.shape, dtype=np.uint8))
img_plot.set_data(raw_image)
for box in pred_boxes:
rect = patches.Rectangle((box[0], box[1]),
box[2] - box[0],
box[3] - box[1],
linewidth=1,
edgecolor='r',
facecolor='none')
ax.add_patch(rect)
class_score = ax.text(box[0],
box[1] - 5,
f"{labels[int(box[4])]} - {box[5]:.2f}",
color='red')
plt.axis('off')
plt.show()
Total running time of the script: (1 minutes 3.495 seconds)