Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf.keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, width, height, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
16384/11490434 [..............................] - ETA: 1:39
32768/11490434 [..............................] - ETA: 1:11
49152/11490434 [..............................] - ETA: 1:08
73728/11490434 [..............................] - ETA: 56s
98304/11490434 [..............................] - ETA: 49s
131072/11490434 [..............................] - ETA: 42s
172032/11490434 [..............................] - ETA: 35s
212992/11490434 [..............................] - ETA: 31s
270336/11490434 [..............................] - ETA: 27s
344064/11490434 [..............................] - ETA: 22s
434176/11490434 [>.............................] - ETA: 19s
540672/11490434 [>.............................] - ETA: 16s
671744/11490434 [>.............................] - ETA: 13s
843776/11490434 [=>............................] - ETA: 11s
1056768/11490434 [=>............................] - ETA: 9s
1310720/11490434 [==>...........................] - ETA: 7s
1646592/11490434 [===>..........................] - ETA: 6s
2064384/11490434 [====>.........................] - ETA: 5s
2596864/11490434 [=====>........................] - ETA: 3s
3244032/11490434 [=======>......................] - ETA: 3s
4055040/11490434 [=========>....................] - ETA: 2s
5070848/11490434 [============>.................] - ETA: 1s
6283264/11490434 [===============>..............] - ETA: 1s
7151616/11490434 [=================>............] - ETA: 0s
8142848/11490434 [====================>.........] - ETA: 0s
9232384/11490434 [=======================>......] - ETA: 0s
10240000/11490434 [=========================>....] - ETA: 0s
11198464/11490434 [============================>.] - ETA: 0s
11490434/11490434 [==============================] - 2s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard Keras CNN model.
import keras
model_keras = keras.models.Sequential([
keras.layers.Rescaling(1. / 255, input_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:00:37 - loss: 2.9346 - accuracy: 0.1250
23/1688 [..............................] - ETA: 3s - loss: 1.5046 - accuracy: 0.5408
46/1688 [..............................] - ETA: 3s - loss: 1.0670 - accuracy: 0.6800
69/1688 [>.............................] - ETA: 3s - loss: 0.8674 - accuracy: 0.7355
92/1688 [>.............................] - ETA: 3s - loss: 0.7429 - accuracy: 0.7738
115/1688 [=>............................] - ETA: 3s - loss: 0.6503 - accuracy: 0.8027
138/1688 [=>............................] - ETA: 3s - loss: 0.5988 - accuracy: 0.8191
161/1688 [=>............................] - ETA: 3s - loss: 0.5556 - accuracy: 0.8319
184/1688 [==>...........................] - ETA: 3s - loss: 0.5222 - accuracy: 0.8417
207/1688 [==>...........................] - ETA: 3s - loss: 0.4962 - accuracy: 0.8502
229/1688 [===>..........................] - ETA: 3s - loss: 0.4705 - accuracy: 0.8570
252/1688 [===>..........................] - ETA: 3s - loss: 0.4528 - accuracy: 0.8624
275/1688 [===>..........................] - ETA: 3s - loss: 0.4342 - accuracy: 0.8686
298/1688 [====>.........................] - ETA: 3s - loss: 0.4172 - accuracy: 0.8727
321/1688 [====>.........................] - ETA: 3s - loss: 0.4044 - accuracy: 0.8766
344/1688 [=====>........................] - ETA: 3s - loss: 0.3902 - accuracy: 0.8808
367/1688 [=====>........................] - ETA: 2s - loss: 0.3771 - accuracy: 0.8845
390/1688 [=====>........................] - ETA: 2s - loss: 0.3651 - accuracy: 0.8889
413/1688 [======>.......................] - ETA: 2s - loss: 0.3532 - accuracy: 0.8923
436/1688 [======>.......................] - ETA: 2s - loss: 0.3421 - accuracy: 0.8954
459/1688 [=======>......................] - ETA: 2s - loss: 0.3318 - accuracy: 0.8991
482/1688 [=======>......................] - ETA: 2s - loss: 0.3254 - accuracy: 0.9013
505/1688 [=======>......................] - ETA: 2s - loss: 0.3173 - accuracy: 0.9039
528/1688 [========>.....................] - ETA: 2s - loss: 0.3101 - accuracy: 0.9057
551/1688 [========>.....................] - ETA: 2s - loss: 0.3030 - accuracy: 0.9080
575/1688 [=========>....................] - ETA: 2s - loss: 0.2955 - accuracy: 0.9099
597/1688 [=========>....................] - ETA: 2s - loss: 0.2895 - accuracy: 0.9119
620/1688 [==========>...................] - ETA: 2s - loss: 0.2837 - accuracy: 0.9137
643/1688 [==========>...................] - ETA: 2s - loss: 0.2783 - accuracy: 0.9152
666/1688 [==========>...................] - ETA: 2s - loss: 0.2734 - accuracy: 0.9168
688/1688 [===========>..................] - ETA: 2s - loss: 0.2675 - accuracy: 0.9185
711/1688 [===========>..................] - ETA: 2s - loss: 0.2633 - accuracy: 0.9201
734/1688 [============>.................] - ETA: 2s - loss: 0.2591 - accuracy: 0.9215
756/1688 [============>.................] - ETA: 2s - loss: 0.2556 - accuracy: 0.9226
779/1688 [============>.................] - ETA: 2s - loss: 0.2514 - accuracy: 0.9237
802/1688 [=============>................] - ETA: 1s - loss: 0.2475 - accuracy: 0.9250
825/1688 [=============>................] - ETA: 1s - loss: 0.2435 - accuracy: 0.9263
846/1688 [==============>...............] - ETA: 1s - loss: 0.2406 - accuracy: 0.9273
868/1688 [==============>...............] - ETA: 1s - loss: 0.2379 - accuracy: 0.9282
889/1688 [==============>...............] - ETA: 1s - loss: 0.2354 - accuracy: 0.9288
911/1688 [===============>..............] - ETA: 1s - loss: 0.2333 - accuracy: 0.9296
932/1688 [===============>..............] - ETA: 1s - loss: 0.2318 - accuracy: 0.9301
953/1688 [===============>..............] - ETA: 1s - loss: 0.2297 - accuracy: 0.9308
974/1688 [================>.............] - ETA: 1s - loss: 0.2273 - accuracy: 0.9313
995/1688 [================>.............] - ETA: 1s - loss: 0.2251 - accuracy: 0.9319
1016/1688 [=================>............] - ETA: 1s - loss: 0.2229 - accuracy: 0.9325
1038/1688 [=================>............] - ETA: 1s - loss: 0.2208 - accuracy: 0.9333
1059/1688 [=================>............] - ETA: 1s - loss: 0.2183 - accuracy: 0.9339
1080/1688 [==================>...........] - ETA: 1s - loss: 0.2160 - accuracy: 0.9347
1102/1688 [==================>...........] - ETA: 1s - loss: 0.2138 - accuracy: 0.9353
1124/1688 [==================>...........] - ETA: 1s - loss: 0.2114 - accuracy: 0.9361
1145/1688 [===================>..........] - ETA: 1s - loss: 0.2088 - accuracy: 0.9370
1166/1688 [===================>..........] - ETA: 1s - loss: 0.2064 - accuracy: 0.9377
1188/1688 [====================>.........] - ETA: 1s - loss: 0.2046 - accuracy: 0.9382
1209/1688 [====================>.........] - ETA: 1s - loss: 0.2032 - accuracy: 0.9385
1230/1688 [====================>.........] - ETA: 1s - loss: 0.2018 - accuracy: 0.9388
1252/1688 [=====================>........] - ETA: 1s - loss: 0.2000 - accuracy: 0.9391
1273/1688 [=====================>........] - ETA: 0s - loss: 0.1983 - accuracy: 0.9398
1295/1688 [======================>.......] - ETA: 0s - loss: 0.1964 - accuracy: 0.9404
1316/1688 [======================>.......] - ETA: 0s - loss: 0.1947 - accuracy: 0.9409
1338/1688 [======================>.......] - ETA: 0s - loss: 0.1932 - accuracy: 0.9414
1359/1688 [=======================>......] - ETA: 0s - loss: 0.1921 - accuracy: 0.9418
1380/1688 [=======================>......] - ETA: 0s - loss: 0.1911 - accuracy: 0.9420
1401/1688 [=======================>......] - ETA: 0s - loss: 0.1895 - accuracy: 0.9425
1423/1688 [========================>.....] - ETA: 0s - loss: 0.1878 - accuracy: 0.9430
1445/1688 [========================>.....] - ETA: 0s - loss: 0.1863 - accuracy: 0.9434
1466/1688 [=========================>....] - ETA: 0s - loss: 0.1853 - accuracy: 0.9438
1487/1688 [=========================>....] - ETA: 0s - loss: 0.1841 - accuracy: 0.9442
1508/1688 [=========================>....] - ETA: 0s - loss: 0.1823 - accuracy: 0.9447
1529/1688 [==========================>...] - ETA: 0s - loss: 0.1808 - accuracy: 0.9450
1550/1688 [==========================>...] - ETA: 0s - loss: 0.1800 - accuracy: 0.9453
1571/1688 [==========================>...] - ETA: 0s - loss: 0.1790 - accuracy: 0.9456
1592/1688 [===========================>..] - ETA: 0s - loss: 0.1780 - accuracy: 0.9459
1613/1688 [===========================>..] - ETA: 0s - loss: 0.1767 - accuracy: 0.9463
1635/1688 [============================>.] - ETA: 0s - loss: 0.1758 - accuracy: 0.9465
1656/1688 [============================>.] - ETA: 0s - loss: 0.1748 - accuracy: 0.9469
1678/1688 [============================>.] - ETA: 0s - loss: 0.1737 - accuracy: 0.9473
1688/1688 [==============================] - ETA: 0s - loss: 0.1734 - accuracy: 0.9474
1688/1688 [==============================] - 7s 3ms/step - loss: 0.1734 - accuracy: 0.9474 - val_loss: 0.1160 - val_accuracy: 0.9675
Epoch 2/10
1/1688 [..............................] - ETA: 3s - loss: 0.0117 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0932 - accuracy: 0.9701
47/1688 [..............................] - ETA: 3s - loss: 0.0873 - accuracy: 0.9727
70/1688 [>.............................] - ETA: 3s - loss: 0.0904 - accuracy: 0.9732
93/1688 [>.............................] - ETA: 3s - loss: 0.0867 - accuracy: 0.9735
116/1688 [=>............................] - ETA: 3s - loss: 0.0834 - accuracy: 0.9739
140/1688 [=>............................] - ETA: 3s - loss: 0.0836 - accuracy: 0.9732
163/1688 [=>............................] - ETA: 3s - loss: 0.0792 - accuracy: 0.9743
186/1688 [==>...........................] - ETA: 3s - loss: 0.0775 - accuracy: 0.9756
209/1688 [==>...........................] - ETA: 3s - loss: 0.0764 - accuracy: 0.9767
231/1688 [===>..........................] - ETA: 3s - loss: 0.0775 - accuracy: 0.9770
254/1688 [===>..........................] - ETA: 3s - loss: 0.0792 - accuracy: 0.9766
276/1688 [===>..........................] - ETA: 3s - loss: 0.0784 - accuracy: 0.9768
299/1688 [====>.........................] - ETA: 3s - loss: 0.0766 - accuracy: 0.9772
322/1688 [====>.........................] - ETA: 3s - loss: 0.0746 - accuracy: 0.9773
345/1688 [=====>........................] - ETA: 2s - loss: 0.0742 - accuracy: 0.9775
368/1688 [=====>........................] - ETA: 2s - loss: 0.0722 - accuracy: 0.9779
391/1688 [=====>........................] - ETA: 2s - loss: 0.0725 - accuracy: 0.9779
413/1688 [======>.......................] - ETA: 2s - loss: 0.0716 - accuracy: 0.9778
436/1688 [======>.......................] - ETA: 2s - loss: 0.0710 - accuracy: 0.9779
458/1688 [=======>......................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9774
481/1688 [=======>......................] - ETA: 2s - loss: 0.0726 - accuracy: 0.9772
503/1688 [=======>......................] - ETA: 2s - loss: 0.0728 - accuracy: 0.9771
526/1688 [========>.....................] - ETA: 2s - loss: 0.0710 - accuracy: 0.9778
549/1688 [========>.....................] - ETA: 2s - loss: 0.0704 - accuracy: 0.9780
572/1688 [=========>....................] - ETA: 2s - loss: 0.0697 - accuracy: 0.9780
595/1688 [=========>....................] - ETA: 2s - loss: 0.0689 - accuracy: 0.9783
618/1688 [=========>....................] - ETA: 2s - loss: 0.0684 - accuracy: 0.9786
641/1688 [==========>...................] - ETA: 2s - loss: 0.0697 - accuracy: 0.9783
664/1688 [==========>...................] - ETA: 2s - loss: 0.0698 - accuracy: 0.9781
687/1688 [===========>..................] - ETA: 2s - loss: 0.0702 - accuracy: 0.9778
710/1688 [===========>..................] - ETA: 2s - loss: 0.0706 - accuracy: 0.9775
733/1688 [============>.................] - ETA: 2s - loss: 0.0705 - accuracy: 0.9776
756/1688 [============>.................] - ETA: 2s - loss: 0.0697 - accuracy: 0.9778
778/1688 [============>.................] - ETA: 2s - loss: 0.0703 - accuracy: 0.9776
800/1688 [=============>................] - ETA: 1s - loss: 0.0707 - accuracy: 0.9775
821/1688 [=============>................] - ETA: 1s - loss: 0.0717 - accuracy: 0.9774
842/1688 [=============>................] - ETA: 1s - loss: 0.0721 - accuracy: 0.9772
864/1688 [==============>...............] - ETA: 1s - loss: 0.0718 - accuracy: 0.9772
885/1688 [==============>...............] - ETA: 1s - loss: 0.0719 - accuracy: 0.9772
906/1688 [===============>..............] - ETA: 1s - loss: 0.0718 - accuracy: 0.9773
926/1688 [===============>..............] - ETA: 1s - loss: 0.0717 - accuracy: 0.9773
946/1688 [===============>..............] - ETA: 1s - loss: 0.0715 - accuracy: 0.9773
966/1688 [================>.............] - ETA: 1s - loss: 0.0716 - accuracy: 0.9774
986/1688 [================>.............] - ETA: 1s - loss: 0.0712 - accuracy: 0.9776
1006/1688 [================>.............] - ETA: 1s - loss: 0.0711 - accuracy: 0.9776
1026/1688 [=================>............] - ETA: 1s - loss: 0.0711 - accuracy: 0.9776
1046/1688 [=================>............] - ETA: 1s - loss: 0.0724 - accuracy: 0.9773
1066/1688 [=================>............] - ETA: 1s - loss: 0.0722 - accuracy: 0.9774
1086/1688 [==================>...........] - ETA: 1s - loss: 0.0723 - accuracy: 0.9774
1105/1688 [==================>...........] - ETA: 1s - loss: 0.0728 - accuracy: 0.9774
1125/1688 [==================>...........] - ETA: 1s - loss: 0.0728 - accuracy: 0.9775
1145/1688 [===================>..........] - ETA: 1s - loss: 0.0726 - accuracy: 0.9775
1165/1688 [===================>..........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9775
1185/1688 [====================>.........] - ETA: 1s - loss: 0.0730 - accuracy: 0.9776
1205/1688 [====================>.........] - ETA: 1s - loss: 0.0731 - accuracy: 0.9776
1225/1688 [====================>.........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9777
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0731 - accuracy: 0.9775
1264/1688 [=====================>........] - ETA: 1s - loss: 0.0731 - accuracy: 0.9776
1284/1688 [=====================>........] - ETA: 0s - loss: 0.0730 - accuracy: 0.9776
1303/1688 [======================>.......] - ETA: 0s - loss: 0.0733 - accuracy: 0.9775
1323/1688 [======================>.......] - ETA: 0s - loss: 0.0733 - accuracy: 0.9775
1343/1688 [======================>.......] - ETA: 0s - loss: 0.0728 - accuracy: 0.9777
1363/1688 [=======================>......] - ETA: 0s - loss: 0.0727 - accuracy: 0.9776
1383/1688 [=======================>......] - ETA: 0s - loss: 0.0726 - accuracy: 0.9775
1403/1688 [=======================>......] - ETA: 0s - loss: 0.0726 - accuracy: 0.9776
1423/1688 [========================>.....] - ETA: 0s - loss: 0.0721 - accuracy: 0.9777
1444/1688 [========================>.....] - ETA: 0s - loss: 0.0720 - accuracy: 0.9778
1464/1688 [=========================>....] - ETA: 0s - loss: 0.0717 - accuracy: 0.9778
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0716 - accuracy: 0.9779
1504/1688 [=========================>....] - ETA: 0s - loss: 0.0713 - accuracy: 0.9781
1524/1688 [==========================>...] - ETA: 0s - loss: 0.0709 - accuracy: 0.9782
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0710 - accuracy: 0.9781
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0708 - accuracy: 0.9782
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0709 - accuracy: 0.9781
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0705 - accuracy: 0.9783
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0708 - accuracy: 0.9782
1644/1688 [============================>.] - ETA: 0s - loss: 0.0708 - accuracy: 0.9781
1664/1688 [============================>.] - ETA: 0s - loss: 0.0708 - accuracy: 0.9782
1684/1688 [============================>.] - ETA: 0s - loss: 0.0709 - accuracy: 0.9781
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0710 - accuracy: 0.9781 - val_loss: 0.0721 - val_accuracy: 0.9790
Epoch 3/10
1/1688 [..............................] - ETA: 4s - loss: 0.0072 - accuracy: 1.0000
21/1688 [..............................] - ETA: 4s - loss: 0.0523 - accuracy: 0.9792
41/1688 [..............................] - ETA: 4s - loss: 0.0615 - accuracy: 0.9787
61/1688 [>.............................] - ETA: 4s - loss: 0.0593 - accuracy: 0.9805
81/1688 [>.............................] - ETA: 4s - loss: 0.0579 - accuracy: 0.9819
101/1688 [>.............................] - ETA: 4s - loss: 0.0580 - accuracy: 0.9827
121/1688 [=>............................] - ETA: 4s - loss: 0.0567 - accuracy: 0.9827
141/1688 [=>............................] - ETA: 3s - loss: 0.0549 - accuracy: 0.9823
161/1688 [=>............................] - ETA: 3s - loss: 0.0536 - accuracy: 0.9827
181/1688 [==>...........................] - ETA: 3s - loss: 0.0552 - accuracy: 0.9826
201/1688 [==>...........................] - ETA: 3s - loss: 0.0526 - accuracy: 0.9829
220/1688 [==>...........................] - ETA: 3s - loss: 0.0509 - accuracy: 0.9830
240/1688 [===>..........................] - ETA: 3s - loss: 0.0499 - accuracy: 0.9828
260/1688 [===>..........................] - ETA: 3s - loss: 0.0491 - accuracy: 0.9832
280/1688 [===>..........................] - ETA: 3s - loss: 0.0485 - accuracy: 0.9835
300/1688 [====>.........................] - ETA: 3s - loss: 0.0480 - accuracy: 0.9836
320/1688 [====>.........................] - ETA: 3s - loss: 0.0472 - accuracy: 0.9839
340/1688 [=====>........................] - ETA: 3s - loss: 0.0480 - accuracy: 0.9838
360/1688 [=====>........................] - ETA: 3s - loss: 0.0479 - accuracy: 0.9840
380/1688 [=====>........................] - ETA: 3s - loss: 0.0488 - accuracy: 0.9838
400/1688 [======>.......................] - ETA: 3s - loss: 0.0507 - accuracy: 0.9833
420/1688 [======>.......................] - ETA: 3s - loss: 0.0513 - accuracy: 0.9830
440/1688 [======>.......................] - ETA: 3s - loss: 0.0515 - accuracy: 0.9828
460/1688 [=======>......................] - ETA: 3s - loss: 0.0530 - accuracy: 0.9820
480/1688 [=======>......................] - ETA: 3s - loss: 0.0521 - accuracy: 0.9823
500/1688 [=======>......................] - ETA: 3s - loss: 0.0523 - accuracy: 0.9823
520/1688 [========>.....................] - ETA: 3s - loss: 0.0526 - accuracy: 0.9823
540/1688 [========>.....................] - ETA: 2s - loss: 0.0522 - accuracy: 0.9822
560/1688 [========>.....................] - ETA: 2s - loss: 0.0523 - accuracy: 0.9824
580/1688 [=========>....................] - ETA: 2s - loss: 0.0535 - accuracy: 0.9819
600/1688 [=========>....................] - ETA: 2s - loss: 0.0531 - accuracy: 0.9821
620/1688 [==========>...................] - ETA: 2s - loss: 0.0527 - accuracy: 0.9823
639/1688 [==========>...................] - ETA: 2s - loss: 0.0528 - accuracy: 0.9821
659/1688 [==========>...................] - ETA: 2s - loss: 0.0539 - accuracy: 0.9819
680/1688 [===========>..................] - ETA: 2s - loss: 0.0548 - accuracy: 0.9818
699/1688 [===========>..................] - ETA: 2s - loss: 0.0544 - accuracy: 0.9818
718/1688 [===========>..................] - ETA: 2s - loss: 0.0547 - accuracy: 0.9818
737/1688 [============>.................] - ETA: 2s - loss: 0.0547 - accuracy: 0.9820
756/1688 [============>.................] - ETA: 2s - loss: 0.0544 - accuracy: 0.9821
775/1688 [============>.................] - ETA: 2s - loss: 0.0542 - accuracy: 0.9822
793/1688 [=============>................] - ETA: 2s - loss: 0.0546 - accuracy: 0.9822
811/1688 [=============>................] - ETA: 2s - loss: 0.0548 - accuracy: 0.9823
829/1688 [=============>................] - ETA: 2s - loss: 0.0546 - accuracy: 0.9824
848/1688 [==============>...............] - ETA: 2s - loss: 0.0543 - accuracy: 0.9825
866/1688 [==============>...............] - ETA: 2s - loss: 0.0542 - accuracy: 0.9826
884/1688 [==============>...............] - ETA: 2s - loss: 0.0543 - accuracy: 0.9826
902/1688 [===============>..............] - ETA: 2s - loss: 0.0543 - accuracy: 0.9826
920/1688 [===============>..............] - ETA: 2s - loss: 0.0540 - accuracy: 0.9827
938/1688 [===============>..............] - ETA: 1s - loss: 0.0542 - accuracy: 0.9828
956/1688 [===============>..............] - ETA: 1s - loss: 0.0539 - accuracy: 0.9829
975/1688 [================>.............] - ETA: 1s - loss: 0.0536 - accuracy: 0.9830
993/1688 [================>.............] - ETA: 1s - loss: 0.0538 - accuracy: 0.9830
1011/1688 [================>.............] - ETA: 1s - loss: 0.0535 - accuracy: 0.9830
1029/1688 [=================>............] - ETA: 1s - loss: 0.0533 - accuracy: 0.9831
1047/1688 [=================>............] - ETA: 1s - loss: 0.0529 - accuracy: 0.9833
1065/1688 [=================>............] - ETA: 1s - loss: 0.0527 - accuracy: 0.9834
1083/1688 [==================>...........] - ETA: 1s - loss: 0.0527 - accuracy: 0.9834
1101/1688 [==================>...........] - ETA: 1s - loss: 0.0534 - accuracy: 0.9833
1119/1688 [==================>...........] - ETA: 1s - loss: 0.0534 - accuracy: 0.9833
1137/1688 [===================>..........] - ETA: 1s - loss: 0.0529 - accuracy: 0.9835
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0527 - accuracy: 0.9835
1174/1688 [===================>..........] - ETA: 1s - loss: 0.0523 - accuracy: 0.9836
1192/1688 [====================>.........] - ETA: 1s - loss: 0.0530 - accuracy: 0.9835
1210/1688 [====================>.........] - ETA: 1s - loss: 0.0534 - accuracy: 0.9833
1229/1688 [====================>.........] - ETA: 1s - loss: 0.0535 - accuracy: 0.9832
1247/1688 [=====================>........] - ETA: 1s - loss: 0.0535 - accuracy: 0.9832
1265/1688 [=====================>........] - ETA: 1s - loss: 0.0535 - accuracy: 0.9832
1283/1688 [=====================>........] - ETA: 1s - loss: 0.0536 - accuracy: 0.9832
1301/1688 [======================>.......] - ETA: 1s - loss: 0.0534 - accuracy: 0.9833
1319/1688 [======================>.......] - ETA: 0s - loss: 0.0534 - accuracy: 0.9832
1338/1688 [======================>.......] - ETA: 0s - loss: 0.0534 - accuracy: 0.9832
1356/1688 [=======================>......] - ETA: 0s - loss: 0.0533 - accuracy: 0.9833
1375/1688 [=======================>......] - ETA: 0s - loss: 0.0530 - accuracy: 0.9833
1394/1688 [=======================>......] - ETA: 0s - loss: 0.0532 - accuracy: 0.9832
1412/1688 [========================>.....] - ETA: 0s - loss: 0.0535 - accuracy: 0.9831
1430/1688 [========================>.....] - ETA: 0s - loss: 0.0541 - accuracy: 0.9830
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0539 - accuracy: 0.9831
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0537 - accuracy: 0.9832
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0537 - accuracy: 0.9832
1502/1688 [=========================>....] - ETA: 0s - loss: 0.0540 - accuracy: 0.9831
1520/1688 [==========================>...] - ETA: 0s - loss: 0.0542 - accuracy: 0.9830
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0540 - accuracy: 0.9830
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0540 - accuracy: 0.9830
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0539 - accuracy: 0.9831
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0543 - accuracy: 0.9830
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0544 - accuracy: 0.9830
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0543 - accuracy: 0.9829
1648/1688 [============================>.] - ETA: 0s - loss: 0.0543 - accuracy: 0.9829
1666/1688 [============================>.] - ETA: 0s - loss: 0.0544 - accuracy: 0.9828
1684/1688 [============================>.] - ETA: 0s - loss: 0.0549 - accuracy: 0.9828
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0549 - accuracy: 0.9828 - val_loss: 0.0659 - val_accuracy: 0.9807
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0087 - accuracy: 1.0000
20/1688 [..............................] - ETA: 4s - loss: 0.0300 - accuracy: 0.9906
38/1688 [..............................] - ETA: 4s - loss: 0.0276 - accuracy: 0.9893
56/1688 [..............................] - ETA: 4s - loss: 0.0288 - accuracy: 0.9883
75/1688 [>.............................] - ETA: 4s - loss: 0.0292 - accuracy: 0.9892
93/1688 [>.............................] - ETA: 4s - loss: 0.0328 - accuracy: 0.9889
111/1688 [>.............................] - ETA: 4s - loss: 0.0322 - accuracy: 0.9890
129/1688 [=>............................] - ETA: 4s - loss: 0.0364 - accuracy: 0.9881
148/1688 [=>............................] - ETA: 4s - loss: 0.0343 - accuracy: 0.9886
166/1688 [=>............................] - ETA: 4s - loss: 0.0360 - accuracy: 0.9887
184/1688 [==>...........................] - ETA: 4s - loss: 0.0368 - accuracy: 0.9890
202/1688 [==>...........................] - ETA: 4s - loss: 0.0364 - accuracy: 0.9889
220/1688 [==>...........................] - ETA: 4s - loss: 0.0365 - accuracy: 0.9886
239/1688 [===>..........................] - ETA: 4s - loss: 0.0378 - accuracy: 0.9885
257/1688 [===>..........................] - ETA: 4s - loss: 0.0378 - accuracy: 0.9882
275/1688 [===>..........................] - ETA: 3s - loss: 0.0382 - accuracy: 0.9876
293/1688 [====>.........................] - ETA: 3s - loss: 0.0375 - accuracy: 0.9876
311/1688 [====>.........................] - ETA: 3s - loss: 0.0371 - accuracy: 0.9877
329/1688 [====>.........................] - ETA: 3s - loss: 0.0368 - accuracy: 0.9877
347/1688 [=====>........................] - ETA: 3s - loss: 0.0371 - accuracy: 0.9876
365/1688 [=====>........................] - ETA: 3s - loss: 0.0372 - accuracy: 0.9874
384/1688 [=====>........................] - ETA: 3s - loss: 0.0377 - accuracy: 0.9873
402/1688 [======>.......................] - ETA: 3s - loss: 0.0373 - accuracy: 0.9873
420/1688 [======>.......................] - ETA: 3s - loss: 0.0373 - accuracy: 0.9875
439/1688 [======>.......................] - ETA: 3s - loss: 0.0368 - accuracy: 0.9878
457/1688 [=======>......................] - ETA: 3s - loss: 0.0364 - accuracy: 0.9878
476/1688 [=======>......................] - ETA: 3s - loss: 0.0363 - accuracy: 0.9878
494/1688 [=======>......................] - ETA: 3s - loss: 0.0366 - accuracy: 0.9877
512/1688 [========>.....................] - ETA: 3s - loss: 0.0374 - accuracy: 0.9877
530/1688 [========>.....................] - ETA: 3s - loss: 0.0373 - accuracy: 0.9877
548/1688 [========>.....................] - ETA: 3s - loss: 0.0372 - accuracy: 0.9877
567/1688 [=========>....................] - ETA: 3s - loss: 0.0368 - accuracy: 0.9879
586/1688 [=========>....................] - ETA: 3s - loss: 0.0365 - accuracy: 0.9881
604/1688 [=========>....................] - ETA: 3s - loss: 0.0365 - accuracy: 0.9882
622/1688 [==========>...................] - ETA: 2s - loss: 0.0364 - accuracy: 0.9882
641/1688 [==========>...................] - ETA: 2s - loss: 0.0367 - accuracy: 0.9882
660/1688 [==========>...................] - ETA: 2s - loss: 0.0367 - accuracy: 0.9881
678/1688 [===========>..................] - ETA: 2s - loss: 0.0375 - accuracy: 0.9880
696/1688 [===========>..................] - ETA: 2s - loss: 0.0381 - accuracy: 0.9877
714/1688 [===========>..................] - ETA: 2s - loss: 0.0387 - accuracy: 0.9877
732/1688 [============>.................] - ETA: 2s - loss: 0.0394 - accuracy: 0.9877
750/1688 [============>.................] - ETA: 2s - loss: 0.0394 - accuracy: 0.9877
768/1688 [============>.................] - ETA: 2s - loss: 0.0395 - accuracy: 0.9876
787/1688 [============>.................] - ETA: 2s - loss: 0.0400 - accuracy: 0.9875
805/1688 [=============>................] - ETA: 2s - loss: 0.0399 - accuracy: 0.9875
824/1688 [=============>................] - ETA: 2s - loss: 0.0395 - accuracy: 0.9876
842/1688 [=============>................] - ETA: 2s - loss: 0.0398 - accuracy: 0.9875
861/1688 [==============>...............] - ETA: 2s - loss: 0.0399 - accuracy: 0.9875
879/1688 [==============>...............] - ETA: 2s - loss: 0.0396 - accuracy: 0.9875
897/1688 [==============>...............] - ETA: 2s - loss: 0.0397 - accuracy: 0.9875
916/1688 [===============>..............] - ETA: 2s - loss: 0.0396 - accuracy: 0.9875
935/1688 [===============>..............] - ETA: 2s - loss: 0.0398 - accuracy: 0.9875
953/1688 [===============>..............] - ETA: 2s - loss: 0.0396 - accuracy: 0.9875
971/1688 [================>.............] - ETA: 2s - loss: 0.0401 - accuracy: 0.9872
989/1688 [================>.............] - ETA: 1s - loss: 0.0402 - accuracy: 0.9872
1007/1688 [================>.............] - ETA: 1s - loss: 0.0403 - accuracy: 0.9872
1026/1688 [=================>............] - ETA: 1s - loss: 0.0400 - accuracy: 0.9872
1045/1688 [=================>............] - ETA: 1s - loss: 0.0398 - accuracy: 0.9874
1063/1688 [=================>............] - ETA: 1s - loss: 0.0399 - accuracy: 0.9874
1081/1688 [==================>...........] - ETA: 1s - loss: 0.0402 - accuracy: 0.9873
1099/1688 [==================>...........] - ETA: 1s - loss: 0.0401 - accuracy: 0.9873
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0402 - accuracy: 0.9873
1135/1688 [===================>..........] - ETA: 1s - loss: 0.0402 - accuracy: 0.9873
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0405 - accuracy: 0.9872
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0407 - accuracy: 0.9872
1190/1688 [====================>.........] - ETA: 1s - loss: 0.0410 - accuracy: 0.9871
1208/1688 [====================>.........] - ETA: 1s - loss: 0.0413 - accuracy: 0.9870
1227/1688 [====================>.........] - ETA: 1s - loss: 0.0412 - accuracy: 0.9871
1245/1688 [=====================>........] - ETA: 1s - loss: 0.0415 - accuracy: 0.9872
1264/1688 [=====================>........] - ETA: 1s - loss: 0.0414 - accuracy: 0.9872
1283/1688 [=====================>........] - ETA: 1s - loss: 0.0418 - accuracy: 0.9871
1301/1688 [======================>.......] - ETA: 1s - loss: 0.0419 - accuracy: 0.9871
1320/1688 [======================>.......] - ETA: 1s - loss: 0.0416 - accuracy: 0.9872
1338/1688 [======================>.......] - ETA: 0s - loss: 0.0418 - accuracy: 0.9871
1357/1688 [=======================>......] - ETA: 0s - loss: 0.0415 - accuracy: 0.9871
1375/1688 [=======================>......] - ETA: 0s - loss: 0.0414 - accuracy: 0.9872
1393/1688 [=======================>......] - ETA: 0s - loss: 0.0413 - accuracy: 0.9872
1411/1688 [========================>.....] - ETA: 0s - loss: 0.0415 - accuracy: 0.9872
1429/1688 [========================>.....] - ETA: 0s - loss: 0.0416 - accuracy: 0.9871
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0417 - accuracy: 0.9870
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0416 - accuracy: 0.9871
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0413 - accuracy: 0.9871
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0412 - accuracy: 0.9871
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0413 - accuracy: 0.9870
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0413 - accuracy: 0.9870
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0415 - accuracy: 0.9869
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0416 - accuracy: 0.9869
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0421 - accuracy: 0.9868
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0419 - accuracy: 0.9869
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0416 - accuracy: 0.9870
1648/1688 [============================>.] - ETA: 0s - loss: 0.0417 - accuracy: 0.9870
1667/1688 [============================>.] - ETA: 0s - loss: 0.0422 - accuracy: 0.9868
1685/1688 [============================>.] - ETA: 0s - loss: 0.0421 - accuracy: 0.9868
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0420 - accuracy: 0.9868 - val_loss: 0.0729 - val_accuracy: 0.9800
Epoch 5/10
1/1688 [..............................] - ETA: 4s - loss: 0.1456 - accuracy: 0.9688
22/1688 [..............................] - ETA: 4s - loss: 0.0215 - accuracy: 0.9957
42/1688 [..............................] - ETA: 4s - loss: 0.0250 - accuracy: 0.9918
62/1688 [>.............................] - ETA: 4s - loss: 0.0235 - accuracy: 0.9909
83/1688 [>.............................] - ETA: 3s - loss: 0.0254 - accuracy: 0.9891
103/1688 [>.............................] - ETA: 3s - loss: 0.0245 - accuracy: 0.9903
123/1688 [=>............................] - ETA: 3s - loss: 0.0238 - accuracy: 0.9911
144/1688 [=>............................] - ETA: 3s - loss: 0.0244 - accuracy: 0.9913
165/1688 [=>............................] - ETA: 3s - loss: 0.0250 - accuracy: 0.9917
185/1688 [==>...........................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9924
205/1688 [==>...........................] - ETA: 3s - loss: 0.0231 - accuracy: 0.9928
226/1688 [===>..........................] - ETA: 3s - loss: 0.0236 - accuracy: 0.9924
247/1688 [===>..........................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9919
267/1688 [===>..........................] - ETA: 3s - loss: 0.0281 - accuracy: 0.9913
287/1688 [====>.........................] - ETA: 3s - loss: 0.0284 - accuracy: 0.9915
308/1688 [====>.........................] - ETA: 3s - loss: 0.0278 - accuracy: 0.9918
328/1688 [====>.........................] - ETA: 3s - loss: 0.0273 - accuracy: 0.9917
349/1688 [=====>........................] - ETA: 3s - loss: 0.0270 - accuracy: 0.9919
369/1688 [=====>........................] - ETA: 3s - loss: 0.0267 - accuracy: 0.9916
390/1688 [=====>........................] - ETA: 3s - loss: 0.0274 - accuracy: 0.9913
411/1688 [======>.......................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9913
431/1688 [======>.......................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9913
451/1688 [=======>......................] - ETA: 3s - loss: 0.0281 - accuracy: 0.9911
471/1688 [=======>......................] - ETA: 3s - loss: 0.0289 - accuracy: 0.9907
491/1688 [=======>......................] - ETA: 3s - loss: 0.0294 - accuracy: 0.9907
511/1688 [========>.....................] - ETA: 2s - loss: 0.0296 - accuracy: 0.9907
531/1688 [========>.....................] - ETA: 2s - loss: 0.0290 - accuracy: 0.9909
551/1688 [========>.....................] - ETA: 2s - loss: 0.0291 - accuracy: 0.9909
571/1688 [=========>....................] - ETA: 2s - loss: 0.0289 - accuracy: 0.9910
591/1688 [=========>....................] - ETA: 2s - loss: 0.0291 - accuracy: 0.9910
611/1688 [=========>....................] - ETA: 2s - loss: 0.0292 - accuracy: 0.9909
632/1688 [==========>...................] - ETA: 2s - loss: 0.0293 - accuracy: 0.9909
653/1688 [==========>...................] - ETA: 2s - loss: 0.0289 - accuracy: 0.9909
674/1688 [==========>...................] - ETA: 2s - loss: 0.0293 - accuracy: 0.9909
695/1688 [===========>..................] - ETA: 2s - loss: 0.0295 - accuracy: 0.9908
716/1688 [===========>..................] - ETA: 2s - loss: 0.0291 - accuracy: 0.9909
736/1688 [============>.................] - ETA: 2s - loss: 0.0293 - accuracy: 0.9907
756/1688 [============>.................] - ETA: 2s - loss: 0.0298 - accuracy: 0.9906
776/1688 [============>.................] - ETA: 2s - loss: 0.0297 - accuracy: 0.9906
797/1688 [=============>................] - ETA: 2s - loss: 0.0294 - accuracy: 0.9906
817/1688 [=============>................] - ETA: 2s - loss: 0.0294 - accuracy: 0.9907
837/1688 [=============>................] - ETA: 2s - loss: 0.0299 - accuracy: 0.9904
857/1688 [==============>...............] - ETA: 2s - loss: 0.0303 - accuracy: 0.9904
878/1688 [==============>...............] - ETA: 2s - loss: 0.0303 - accuracy: 0.9904
899/1688 [==============>...............] - ETA: 1s - loss: 0.0303 - accuracy: 0.9903
919/1688 [===============>..............] - ETA: 1s - loss: 0.0306 - accuracy: 0.9902
940/1688 [===============>..............] - ETA: 1s - loss: 0.0304 - accuracy: 0.9903
960/1688 [================>.............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9902
980/1688 [================>.............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9901
1001/1688 [================>.............] - ETA: 1s - loss: 0.0312 - accuracy: 0.9901
1022/1688 [=================>............] - ETA: 1s - loss: 0.0312 - accuracy: 0.9901
1043/1688 [=================>............] - ETA: 1s - loss: 0.0312 - accuracy: 0.9901
1063/1688 [=================>............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9901
1083/1688 [==================>...........] - ETA: 1s - loss: 0.0315 - accuracy: 0.9900
1103/1688 [==================>...........] - ETA: 1s - loss: 0.0315 - accuracy: 0.9900
1123/1688 [==================>...........] - ETA: 1s - loss: 0.0318 - accuracy: 0.9899
1144/1688 [===================>..........] - ETA: 1s - loss: 0.0322 - accuracy: 0.9898
1164/1688 [===================>..........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9897
1184/1688 [====================>.........] - ETA: 1s - loss: 0.0329 - accuracy: 0.9896
1204/1688 [====================>.........] - ETA: 1s - loss: 0.0328 - accuracy: 0.9896
1224/1688 [====================>.........] - ETA: 1s - loss: 0.0327 - accuracy: 0.9897
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0328 - accuracy: 0.9896
1264/1688 [=====================>........] - ETA: 1s - loss: 0.0328 - accuracy: 0.9895
1284/1688 [=====================>........] - ETA: 1s - loss: 0.0330 - accuracy: 0.9895
1304/1688 [======================>.......] - ETA: 0s - loss: 0.0331 - accuracy: 0.9895
1325/1688 [======================>.......] - ETA: 0s - loss: 0.0332 - accuracy: 0.9895
1345/1688 [======================>.......] - ETA: 0s - loss: 0.0332 - accuracy: 0.9894
1365/1688 [=======================>......] - ETA: 0s - loss: 0.0335 - accuracy: 0.9893
1385/1688 [=======================>......] - ETA: 0s - loss: 0.0334 - accuracy: 0.9893
1406/1688 [=======================>......] - ETA: 0s - loss: 0.0335 - accuracy: 0.9893
1427/1688 [========================>.....] - ETA: 0s - loss: 0.0334 - accuracy: 0.9894
1447/1688 [========================>.....] - ETA: 0s - loss: 0.0332 - accuracy: 0.9894
1468/1688 [=========================>....] - ETA: 0s - loss: 0.0331 - accuracy: 0.9895
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0335 - accuracy: 0.9895
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0336 - accuracy: 0.9894
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0335 - accuracy: 0.9894
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0333 - accuracy: 0.9895
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0333 - accuracy: 0.9894
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0336 - accuracy: 0.9893
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0338 - accuracy: 0.9893
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0340 - accuracy: 0.9892
1651/1688 [============================>.] - ETA: 0s - loss: 0.0340 - accuracy: 0.9891
1671/1688 [============================>.] - ETA: 0s - loss: 0.0339 - accuracy: 0.9892
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0341 - accuracy: 0.9890 - val_loss: 0.0696 - val_accuracy: 0.9833
Epoch 6/10
1/1688 [..............................] - ETA: 3s - loss: 0.0010 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0234 - accuracy: 0.9922
47/1688 [..............................] - ETA: 3s - loss: 0.0253 - accuracy: 0.9894
70/1688 [>.............................] - ETA: 3s - loss: 0.0224 - accuracy: 0.9906
93/1688 [>.............................] - ETA: 3s - loss: 0.0197 - accuracy: 0.9923
116/1688 [=>............................] - ETA: 3s - loss: 0.0213 - accuracy: 0.9925
138/1688 [=>............................] - ETA: 3s - loss: 0.0228 - accuracy: 0.9923
161/1688 [=>............................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9928
184/1688 [==>...........................] - ETA: 3s - loss: 0.0196 - accuracy: 0.9937
207/1688 [==>...........................] - ETA: 3s - loss: 0.0189 - accuracy: 0.9940
230/1688 [===>..........................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9932
252/1688 [===>..........................] - ETA: 3s - loss: 0.0205 - accuracy: 0.9932
274/1688 [===>..........................] - ETA: 3s - loss: 0.0224 - accuracy: 0.9928
296/1688 [====>.........................] - ETA: 3s - loss: 0.0238 - accuracy: 0.9921
319/1688 [====>.........................] - ETA: 3s - loss: 0.0230 - accuracy: 0.9925
342/1688 [=====>........................] - ETA: 3s - loss: 0.0237 - accuracy: 0.9923
365/1688 [=====>........................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9923
387/1688 [=====>........................] - ETA: 2s - loss: 0.0240 - accuracy: 0.9920
410/1688 [======>.......................] - ETA: 2s - loss: 0.0244 - accuracy: 0.9921
433/1688 [======>.......................] - ETA: 2s - loss: 0.0242 - accuracy: 0.9921
456/1688 [=======>......................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9920
479/1688 [=======>......................] - ETA: 2s - loss: 0.0244 - accuracy: 0.9920
502/1688 [=======>......................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9918
525/1688 [========>.....................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9915
548/1688 [========>.....................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9916
571/1688 [=========>....................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9912
594/1688 [=========>....................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9912
616/1688 [=========>....................] - ETA: 2s - loss: 0.0265 - accuracy: 0.9907
639/1688 [==========>...................] - ETA: 2s - loss: 0.0268 - accuracy: 0.9907
661/1688 [==========>...................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9907
683/1688 [===========>..................] - ETA: 2s - loss: 0.0265 - accuracy: 0.9907
706/1688 [===========>..................] - ETA: 2s - loss: 0.0271 - accuracy: 0.9906
729/1688 [===========>..................] - ETA: 2s - loss: 0.0273 - accuracy: 0.9904
751/1688 [============>.................] - ETA: 2s - loss: 0.0272 - accuracy: 0.9906
773/1688 [============>.................] - ETA: 2s - loss: 0.0273 - accuracy: 0.9905
796/1688 [=============>................] - ETA: 2s - loss: 0.0278 - accuracy: 0.9903
818/1688 [=============>................] - ETA: 1s - loss: 0.0277 - accuracy: 0.9903
841/1688 [=============>................] - ETA: 1s - loss: 0.0274 - accuracy: 0.9904
864/1688 [==============>...............] - ETA: 1s - loss: 0.0275 - accuracy: 0.9904
886/1688 [==============>...............] - ETA: 1s - loss: 0.0276 - accuracy: 0.9904
908/1688 [===============>..............] - ETA: 1s - loss: 0.0274 - accuracy: 0.9905
931/1688 [===============>..............] - ETA: 1s - loss: 0.0276 - accuracy: 0.9904
953/1688 [===============>..............] - ETA: 1s - loss: 0.0275 - accuracy: 0.9904
975/1688 [================>.............] - ETA: 1s - loss: 0.0274 - accuracy: 0.9904
998/1688 [================>.............] - ETA: 1s - loss: 0.0277 - accuracy: 0.9902
1021/1688 [=================>............] - ETA: 1s - loss: 0.0278 - accuracy: 0.9903
1044/1688 [=================>............] - ETA: 1s - loss: 0.0280 - accuracy: 0.9902
1066/1688 [=================>............] - ETA: 1s - loss: 0.0280 - accuracy: 0.9903
1088/1688 [==================>...........] - ETA: 1s - loss: 0.0277 - accuracy: 0.9904
1110/1688 [==================>...........] - ETA: 1s - loss: 0.0277 - accuracy: 0.9904
1133/1688 [===================>..........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9903
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0280 - accuracy: 0.9903
1179/1688 [===================>..........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9902
1202/1688 [====================>.........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9902
1225/1688 [====================>.........] - ETA: 1s - loss: 0.0281 - accuracy: 0.9902
1248/1688 [=====================>........] - ETA: 0s - loss: 0.0278 - accuracy: 0.9903
1270/1688 [=====================>........] - ETA: 0s - loss: 0.0281 - accuracy: 0.9902
1292/1688 [=====================>........] - ETA: 0s - loss: 0.0282 - accuracy: 0.9902
1315/1688 [======================>.......] - ETA: 0s - loss: 0.0284 - accuracy: 0.9902
1338/1688 [======================>.......] - ETA: 0s - loss: 0.0285 - accuracy: 0.9902
1360/1688 [=======================>......] - ETA: 0s - loss: 0.0286 - accuracy: 0.9902
1383/1688 [=======================>......] - ETA: 0s - loss: 0.0287 - accuracy: 0.9902
1406/1688 [=======================>......] - ETA: 0s - loss: 0.0295 - accuracy: 0.9901
1429/1688 [========================>.....] - ETA: 0s - loss: 0.0293 - accuracy: 0.9901
1452/1688 [========================>.....] - ETA: 0s - loss: 0.0291 - accuracy: 0.9902
1475/1688 [=========================>....] - ETA: 0s - loss: 0.0291 - accuracy: 0.9902
1498/1688 [=========================>....] - ETA: 0s - loss: 0.0290 - accuracy: 0.9902
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0289 - accuracy: 0.9903
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0291 - accuracy: 0.9902
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0289 - accuracy: 0.9902
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0290 - accuracy: 0.9902
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0291 - accuracy: 0.9902
1635/1688 [============================>.] - ETA: 0s - loss: 0.0293 - accuracy: 0.9902
1657/1688 [============================>.] - ETA: 0s - loss: 0.0292 - accuracy: 0.9902
1678/1688 [============================>.] - ETA: 0s - loss: 0.0293 - accuracy: 0.9901
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0295 - accuracy: 0.9900 - val_loss: 0.0596 - val_accuracy: 0.9858
Epoch 7/10
1/1688 [..............................] - ETA: 3s - loss: 0.0013 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0137 - accuracy: 0.9948
47/1688 [..............................] - ETA: 3s - loss: 0.0167 - accuracy: 0.9927
71/1688 [>.............................] - ETA: 3s - loss: 0.0166 - accuracy: 0.9934
94/1688 [>.............................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9937
117/1688 [=>............................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9939
140/1688 [=>............................] - ETA: 3s - loss: 0.0176 - accuracy: 0.9933
162/1688 [=>............................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9929
185/1688 [==>...........................] - ETA: 3s - loss: 0.0197 - accuracy: 0.9921
207/1688 [==>...........................] - ETA: 3s - loss: 0.0201 - accuracy: 0.9921
230/1688 [===>..........................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9923
253/1688 [===>..........................] - ETA: 3s - loss: 0.0214 - accuracy: 0.9921
276/1688 [===>..........................] - ETA: 3s - loss: 0.0211 - accuracy: 0.9922
299/1688 [====>.........................] - ETA: 3s - loss: 0.0214 - accuracy: 0.9921
322/1688 [====>.........................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9920
345/1688 [=====>........................] - ETA: 3s - loss: 0.0220 - accuracy: 0.9917
368/1688 [=====>........................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9916
391/1688 [=====>........................] - ETA: 2s - loss: 0.0214 - accuracy: 0.9918
414/1688 [======>.......................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9921
437/1688 [======>.......................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9922
460/1688 [=======>......................] - ETA: 2s - loss: 0.0210 - accuracy: 0.9923
483/1688 [=======>......................] - ETA: 2s - loss: 0.0214 - accuracy: 0.9922
506/1688 [=======>......................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9922
529/1688 [========>.....................] - ETA: 2s - loss: 0.0210 - accuracy: 0.9923
552/1688 [========>.....................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9925
575/1688 [=========>....................] - ETA: 2s - loss: 0.0204 - accuracy: 0.9925
598/1688 [=========>....................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9927
621/1688 [==========>...................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9929
643/1688 [==========>...................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9929
664/1688 [==========>...................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9929
686/1688 [===========>..................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9928
707/1688 [===========>..................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9927
729/1688 [===========>..................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9928
750/1688 [============>.................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9926
771/1688 [============>.................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9926
792/1688 [=============>................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9925
813/1688 [=============>................] - ETA: 1s - loss: 0.0213 - accuracy: 0.9924
834/1688 [=============>................] - ETA: 1s - loss: 0.0215 - accuracy: 0.9924
855/1688 [==============>...............] - ETA: 1s - loss: 0.0216 - accuracy: 0.9923
877/1688 [==============>...............] - ETA: 1s - loss: 0.0220 - accuracy: 0.9922
898/1688 [==============>...............] - ETA: 1s - loss: 0.0222 - accuracy: 0.9921
919/1688 [===============>..............] - ETA: 1s - loss: 0.0220 - accuracy: 0.9922
940/1688 [===============>..............] - ETA: 1s - loss: 0.0219 - accuracy: 0.9923
961/1688 [================>.............] - ETA: 1s - loss: 0.0219 - accuracy: 0.9922
982/1688 [================>.............] - ETA: 1s - loss: 0.0220 - accuracy: 0.9922
1003/1688 [================>.............] - ETA: 1s - loss: 0.0223 - accuracy: 0.9921
1024/1688 [=================>............] - ETA: 1s - loss: 0.0225 - accuracy: 0.9921
1045/1688 [=================>............] - ETA: 1s - loss: 0.0226 - accuracy: 0.9921
1066/1688 [=================>............] - ETA: 1s - loss: 0.0227 - accuracy: 0.9921
1087/1688 [==================>...........] - ETA: 1s - loss: 0.0226 - accuracy: 0.9922
1109/1688 [==================>...........] - ETA: 1s - loss: 0.0225 - accuracy: 0.9922
1130/1688 [===================>..........] - ETA: 1s - loss: 0.0223 - accuracy: 0.9923
1151/1688 [===================>..........] - ETA: 1s - loss: 0.0226 - accuracy: 0.9922
1172/1688 [===================>..........] - ETA: 1s - loss: 0.0227 - accuracy: 0.9922
1193/1688 [====================>.........] - ETA: 1s - loss: 0.0227 - accuracy: 0.9922
1215/1688 [====================>.........] - ETA: 1s - loss: 0.0226 - accuracy: 0.9923
1236/1688 [====================>.........] - ETA: 1s - loss: 0.0227 - accuracy: 0.9923
1257/1688 [=====================>........] - ETA: 1s - loss: 0.0231 - accuracy: 0.9922
1279/1688 [=====================>........] - ETA: 0s - loss: 0.0232 - accuracy: 0.9921
1300/1688 [======================>.......] - ETA: 0s - loss: 0.0233 - accuracy: 0.9921
1321/1688 [======================>.......] - ETA: 0s - loss: 0.0237 - accuracy: 0.9920
1343/1688 [======================>.......] - ETA: 0s - loss: 0.0236 - accuracy: 0.9920
1364/1688 [=======================>......] - ETA: 0s - loss: 0.0236 - accuracy: 0.9920
1385/1688 [=======================>......] - ETA: 0s - loss: 0.0236 - accuracy: 0.9919
1406/1688 [=======================>......] - ETA: 0s - loss: 0.0239 - accuracy: 0.9919
1425/1688 [========================>.....] - ETA: 0s - loss: 0.0240 - accuracy: 0.9919
1445/1688 [========================>.....] - ETA: 0s - loss: 0.0243 - accuracy: 0.9917
1465/1688 [=========================>....] - ETA: 0s - loss: 0.0243 - accuracy: 0.9917
1485/1688 [=========================>....] - ETA: 0s - loss: 0.0247 - accuracy: 0.9916
1505/1688 [=========================>....] - ETA: 0s - loss: 0.0246 - accuracy: 0.9916
1525/1688 [==========================>...] - ETA: 0s - loss: 0.0246 - accuracy: 0.9916
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0248 - accuracy: 0.9916
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0252 - accuracy: 0.9915
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0254 - accuracy: 0.9913
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0254 - accuracy: 0.9914
1625/1688 [===========================>..] - ETA: 0s - loss: 0.0253 - accuracy: 0.9913
1645/1688 [============================>.] - ETA: 0s - loss: 0.0253 - accuracy: 0.9913
1665/1688 [============================>.] - ETA: 0s - loss: 0.0251 - accuracy: 0.9914
1685/1688 [============================>.] - ETA: 0s - loss: 0.0250 - accuracy: 0.9914
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0251 - accuracy: 0.9914 - val_loss: 0.0630 - val_accuracy: 0.9853
Epoch 8/10
1/1688 [..............................] - ETA: 3s - loss: 0.0116 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9961
47/1688 [..............................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9947
69/1688 [>.............................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9946
91/1688 [>.............................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9952
114/1688 [=>............................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9956
137/1688 [=>............................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9945
160/1688 [=>............................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9947
183/1688 [==>...........................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9949
206/1688 [==>...........................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9944
229/1688 [===>..........................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9947
252/1688 [===>..........................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9945
275/1688 [===>..........................] - ETA: 3s - loss: 0.0158 - accuracy: 0.9947
297/1688 [====>.........................] - ETA: 3s - loss: 0.0163 - accuracy: 0.9943
320/1688 [====>.........................] - ETA: 3s - loss: 0.0165 - accuracy: 0.9944
343/1688 [=====>........................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9944
366/1688 [=====>........................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9945
389/1688 [=====>........................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9945
412/1688 [======>.......................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9944
433/1688 [======>.......................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9942
454/1688 [=======>......................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9939
475/1688 [=======>......................] - ETA: 2s - loss: 0.0175 - accuracy: 0.9940
496/1688 [=======>......................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9943
517/1688 [========>.....................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9943
539/1688 [========>.....................] - ETA: 2s - loss: 0.0173 - accuracy: 0.9943
560/1688 [========>.....................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9943
581/1688 [=========>....................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9942
602/1688 [=========>....................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9941
623/1688 [==========>...................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9942
644/1688 [==========>...................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9944
665/1688 [==========>...................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9943
686/1688 [===========>..................] - ETA: 2s - loss: 0.0173 - accuracy: 0.9944
708/1688 [===========>..................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9945
729/1688 [===========>..................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9945
750/1688 [============>.................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9944
771/1688 [============>.................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9944
792/1688 [=============>................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9940
814/1688 [=============>................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9939
836/1688 [=============>................] - ETA: 1s - loss: 0.0181 - accuracy: 0.9939
857/1688 [==============>...............] - ETA: 1s - loss: 0.0182 - accuracy: 0.9938
879/1688 [==============>...............] - ETA: 1s - loss: 0.0180 - accuracy: 0.9939
901/1688 [===============>..............] - ETA: 1s - loss: 0.0179 - accuracy: 0.9939
922/1688 [===============>..............] - ETA: 1s - loss: 0.0180 - accuracy: 0.9939
943/1688 [===============>..............] - ETA: 1s - loss: 0.0180 - accuracy: 0.9939
964/1688 [================>.............] - ETA: 1s - loss: 0.0178 - accuracy: 0.9940
985/1688 [================>.............] - ETA: 1s - loss: 0.0176 - accuracy: 0.9940
1006/1688 [================>.............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9940
1027/1688 [=================>............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9940
1048/1688 [=================>............] - ETA: 1s - loss: 0.0176 - accuracy: 0.9940
1069/1688 [=================>............] - ETA: 1s - loss: 0.0178 - accuracy: 0.9939
1091/1688 [==================>...........] - ETA: 1s - loss: 0.0179 - accuracy: 0.9938
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0179 - accuracy: 0.9937
1133/1688 [===================>..........] - ETA: 1s - loss: 0.0184 - accuracy: 0.9936
1154/1688 [===================>..........] - ETA: 1s - loss: 0.0187 - accuracy: 0.9935
1175/1688 [===================>..........] - ETA: 1s - loss: 0.0187 - accuracy: 0.9935
1196/1688 [====================>.........] - ETA: 1s - loss: 0.0188 - accuracy: 0.9935
1218/1688 [====================>.........] - ETA: 1s - loss: 0.0188 - accuracy: 0.9935
1240/1688 [=====================>........] - ETA: 1s - loss: 0.0188 - accuracy: 0.9936
1261/1688 [=====================>........] - ETA: 1s - loss: 0.0187 - accuracy: 0.9936
1282/1688 [=====================>........] - ETA: 0s - loss: 0.0185 - accuracy: 0.9937
1303/1688 [======================>.......] - ETA: 0s - loss: 0.0187 - accuracy: 0.9937
1324/1688 [======================>.......] - ETA: 0s - loss: 0.0188 - accuracy: 0.9936
1345/1688 [======================>.......] - ETA: 0s - loss: 0.0191 - accuracy: 0.9935
1366/1688 [=======================>......] - ETA: 0s - loss: 0.0192 - accuracy: 0.9935
1387/1688 [=======================>......] - ETA: 0s - loss: 0.0194 - accuracy: 0.9934
1408/1688 [========================>.....] - ETA: 0s - loss: 0.0195 - accuracy: 0.9934
1430/1688 [========================>.....] - ETA: 0s - loss: 0.0195 - accuracy: 0.9934
1451/1688 [========================>.....] - ETA: 0s - loss: 0.0196 - accuracy: 0.9934
1472/1688 [=========================>....] - ETA: 0s - loss: 0.0198 - accuracy: 0.9934
1494/1688 [=========================>....] - ETA: 0s - loss: 0.0197 - accuracy: 0.9934
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0198 - accuracy: 0.9934
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0198 - accuracy: 0.9934
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0199 - accuracy: 0.9934
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0199 - accuracy: 0.9934
1601/1688 [===========================>..] - ETA: 0s - loss: 0.0200 - accuracy: 0.9933
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0200 - accuracy: 0.9933
1643/1688 [============================>.] - ETA: 0s - loss: 0.0203 - accuracy: 0.9932
1664/1688 [============================>.] - ETA: 0s - loss: 0.0205 - accuracy: 0.9931
1685/1688 [============================>.] - ETA: 0s - loss: 0.0203 - accuracy: 0.9931
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0204 - accuracy: 0.9931 - val_loss: 0.0676 - val_accuracy: 0.9855
Epoch 9/10
1/1688 [..............................] - ETA: 3s - loss: 0.0055 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9961
46/1688 [..............................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9952
69/1688 [>.............................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9941
91/1688 [>.............................] - ETA: 3s - loss: 0.0203 - accuracy: 0.9938
114/1688 [=>............................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9945
136/1688 [=>............................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9945
157/1688 [=>............................] - ETA: 3s - loss: 0.0167 - accuracy: 0.9946
178/1688 [==>...........................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9946
198/1688 [==>...........................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9951
218/1688 [==>...........................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9954
238/1688 [===>..........................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9955
258/1688 [===>..........................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9958
278/1688 [===>..........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9958
298/1688 [====>.........................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9960
318/1688 [====>.........................] - ETA: 3s - loss: 0.0149 - accuracy: 0.9957
338/1688 [=====>........................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9957
358/1688 [=====>........................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9958
378/1688 [=====>........................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9959
398/1688 [======>.......................] - ETA: 3s - loss: 0.0142 - accuracy: 0.9960
418/1688 [======>.......................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9958
438/1688 [======>.......................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9957
458/1688 [=======>......................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9957
478/1688 [=======>......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9958
498/1688 [=======>......................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9957
518/1688 [========>.....................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9957
538/1688 [========>.....................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9957
558/1688 [========>.....................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9957
578/1688 [=========>....................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9957
598/1688 [=========>....................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9956
618/1688 [=========>....................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9957
638/1688 [==========>...................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9955
658/1688 [==========>...................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9954
678/1688 [===========>..................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9955
698/1688 [===========>..................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9954
718/1688 [===========>..................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9954
738/1688 [============>.................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9954
758/1688 [============>.................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9954
778/1688 [============>.................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9953
798/1688 [=============>................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9953
818/1688 [=============>................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9951
837/1688 [=============>................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9950
857/1688 [==============>...............] - ETA: 2s - loss: 0.0162 - accuracy: 0.9950
877/1688 [==============>...............] - ETA: 2s - loss: 0.0162 - accuracy: 0.9950
897/1688 [==============>...............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9950
917/1688 [===============>..............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9950
937/1688 [===============>..............] - ETA: 1s - loss: 0.0162 - accuracy: 0.9950
957/1688 [================>.............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9949
977/1688 [================>.............] - ETA: 1s - loss: 0.0166 - accuracy: 0.9948
997/1688 [================>.............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9948
1017/1688 [=================>............] - ETA: 1s - loss: 0.0169 - accuracy: 0.9947
1037/1688 [=================>............] - ETA: 1s - loss: 0.0170 - accuracy: 0.9946
1057/1688 [=================>............] - ETA: 1s - loss: 0.0169 - accuracy: 0.9946
1077/1688 [==================>...........] - ETA: 1s - loss: 0.0168 - accuracy: 0.9947
1097/1688 [==================>...........] - ETA: 1s - loss: 0.0168 - accuracy: 0.9947
1116/1688 [==================>...........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9945
1136/1688 [===================>..........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9946
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0170 - accuracy: 0.9945
1176/1688 [===================>..........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9945
1196/1688 [====================>.........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9944
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9944
1236/1688 [====================>.........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9945
1256/1688 [=====================>........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9945
1276/1688 [=====================>........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9945
1296/1688 [======================>.......] - ETA: 0s - loss: 0.0173 - accuracy: 0.9945
1316/1688 [======================>.......] - ETA: 0s - loss: 0.0174 - accuracy: 0.9944
1336/1688 [======================>.......] - ETA: 0s - loss: 0.0173 - accuracy: 0.9945
1356/1688 [=======================>......] - ETA: 0s - loss: 0.0172 - accuracy: 0.9945
1376/1688 [=======================>......] - ETA: 0s - loss: 0.0172 - accuracy: 0.9945
1396/1688 [=======================>......] - ETA: 0s - loss: 0.0175 - accuracy: 0.9944
1416/1688 [========================>.....] - ETA: 0s - loss: 0.0175 - accuracy: 0.9944
1435/1688 [========================>.....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9944
1455/1688 [========================>.....] - ETA: 0s - loss: 0.0175 - accuracy: 0.9944
1475/1688 [=========================>....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9943
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9943
1514/1688 [=========================>....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9943
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0176 - accuracy: 0.9943
1554/1688 [==========================>...] - ETA: 0s - loss: 0.0176 - accuracy: 0.9942
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0179 - accuracy: 0.9942
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0180 - accuracy: 0.9941
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0180 - accuracy: 0.9941
1634/1688 [============================>.] - ETA: 0s - loss: 0.0181 - accuracy: 0.9940
1654/1688 [============================>.] - ETA: 0s - loss: 0.0185 - accuracy: 0.9939
1674/1688 [============================>.] - ETA: 0s - loss: 0.0186 - accuracy: 0.9938
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0186 - accuracy: 0.9937 - val_loss: 0.0773 - val_accuracy: 0.9810
Epoch 10/10
1/1688 [..............................] - ETA: 4s - loss: 0.0023 - accuracy: 1.0000
21/1688 [..............................] - ETA: 4s - loss: 0.0099 - accuracy: 0.9970
40/1688 [..............................] - ETA: 4s - loss: 0.0115 - accuracy: 0.9945
60/1688 [>.............................] - ETA: 4s - loss: 0.0176 - accuracy: 0.9937
80/1688 [>.............................] - ETA: 4s - loss: 0.0155 - accuracy: 0.9949
100/1688 [>.............................] - ETA: 4s - loss: 0.0133 - accuracy: 0.9956
120/1688 [=>............................] - ETA: 4s - loss: 0.0147 - accuracy: 0.9953
140/1688 [=>............................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9955
160/1688 [=>............................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9953
180/1688 [==>...........................] - ETA: 3s - loss: 0.0149 - accuracy: 0.9948
200/1688 [==>...........................] - ETA: 3s - loss: 0.0142 - accuracy: 0.9952
220/1688 [==>...........................] - ETA: 3s - loss: 0.0144 - accuracy: 0.9952
240/1688 [===>..........................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9953
260/1688 [===>..........................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9956
279/1688 [===>..........................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9956
299/1688 [====>.........................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9955
318/1688 [====>.........................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9954
338/1688 [=====>........................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9956
358/1688 [=====>........................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9955
378/1688 [=====>........................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9957
398/1688 [======>.......................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9956
417/1688 [======>.......................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9958
437/1688 [======>.......................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9957
457/1688 [=======>......................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9957
477/1688 [=======>......................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9955
497/1688 [=======>......................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9957
517/1688 [========>.....................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9958
537/1688 [========>.....................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9959
557/1688 [========>.....................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9958
577/1688 [=========>....................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9957
597/1688 [=========>....................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9956
617/1688 [=========>....................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9955
637/1688 [==========>...................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9955
657/1688 [==========>...................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9955
677/1688 [===========>..................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9956
697/1688 [===========>..................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9956
717/1688 [===========>..................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9955
737/1688 [============>.................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9955
758/1688 [============>.................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9956
778/1688 [============>.................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9956
798/1688 [=============>................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9955
818/1688 [=============>................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9954
838/1688 [=============>................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9955
858/1688 [==============>...............] - ETA: 2s - loss: 0.0126 - accuracy: 0.9955
878/1688 [==============>...............] - ETA: 2s - loss: 0.0127 - accuracy: 0.9955
897/1688 [==============>...............] - ETA: 2s - loss: 0.0130 - accuracy: 0.9954
917/1688 [===============>..............] - ETA: 1s - loss: 0.0129 - accuracy: 0.9954
937/1688 [===============>..............] - ETA: 1s - loss: 0.0129 - accuracy: 0.9954
957/1688 [================>.............] - ETA: 1s - loss: 0.0129 - accuracy: 0.9954
977/1688 [================>.............] - ETA: 1s - loss: 0.0128 - accuracy: 0.9955
996/1688 [================>.............] - ETA: 1s - loss: 0.0128 - accuracy: 0.9955
1016/1688 [=================>............] - ETA: 1s - loss: 0.0127 - accuracy: 0.9955
1036/1688 [=================>............] - ETA: 1s - loss: 0.0129 - accuracy: 0.9955
1056/1688 [=================>............] - ETA: 1s - loss: 0.0129 - accuracy: 0.9955
1076/1688 [==================>...........] - ETA: 1s - loss: 0.0129 - accuracy: 0.9955
1096/1688 [==================>...........] - ETA: 1s - loss: 0.0129 - accuracy: 0.9956
1116/1688 [==================>...........] - ETA: 1s - loss: 0.0133 - accuracy: 0.9954
1136/1688 [===================>..........] - ETA: 1s - loss: 0.0137 - accuracy: 0.9953
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0137 - accuracy: 0.9953
1176/1688 [===================>..........] - ETA: 1s - loss: 0.0138 - accuracy: 0.9953
1196/1688 [====================>.........] - ETA: 1s - loss: 0.0140 - accuracy: 0.9952
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0141 - accuracy: 0.9951
1236/1688 [====================>.........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9950
1256/1688 [=====================>........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9948
1276/1688 [=====================>........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9948
1296/1688 [======================>.......] - ETA: 1s - loss: 0.0148 - accuracy: 0.9949
1315/1688 [======================>.......] - ETA: 0s - loss: 0.0148 - accuracy: 0.9949
1333/1688 [======================>.......] - ETA: 0s - loss: 0.0148 - accuracy: 0.9949
1351/1688 [=======================>......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9948
1369/1688 [=======================>......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9947
1388/1688 [=======================>......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9947
1406/1688 [=======================>......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9947
1424/1688 [========================>.....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9946
1443/1688 [========================>.....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9947
1461/1688 [========================>.....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9946
1479/1688 [=========================>....] - ETA: 0s - loss: 0.0150 - accuracy: 0.9947
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0150 - accuracy: 0.9947
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9947
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0152 - accuracy: 0.9946
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0152 - accuracy: 0.9946
1569/1688 [==========================>...] - ETA: 0s - loss: 0.0151 - accuracy: 0.9947
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0151 - accuracy: 0.9946
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0150 - accuracy: 0.9947
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0152 - accuracy: 0.9947
1642/1688 [============================>.] - ETA: 0s - loss: 0.0152 - accuracy: 0.9947
1661/1688 [============================>.] - ETA: 0s - loss: 0.0152 - accuracy: 0.9947
1679/1688 [============================>.] - ETA: 0s - loss: 0.0152 - accuracy: 0.9947
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0151 - accuracy: 0.9947 - val_loss: 0.0654 - val_accuracy: 0.9860
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9843999743461609
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bits integer activations and weights. Therefore, the floating point Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a Keras model with custom QuantizeML quantized layers substituted for the original Keras layers.
All Keras API functions can be applied on this new model: summary(), compile(), fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
model_quantized.summary()
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:461: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate.\
1/1024 [..............................] - ETA: 3:21
56/1024 [>.............................] - ETA: 0s
114/1024 [==>...........................] - ETA: 0s
169/1024 [===>..........................] - ETA: 0s
226/1024 [=====>........................] - ETA: 0s
283/1024 [=======>......................] - ETA: 0s
340/1024 [========>.....................] - ETA: 0s
397/1024 [==========>...................] - ETA: 0s
453/1024 [============>.................] - ETA: 0s
510/1024 [=============>................] - ETA: 0s
568/1024 [===============>..............] - ETA: 0s
625/1024 [=================>............] - ETA: 0s
683/1024 [===================>..........] - ETA: 0s
739/1024 [====================>.........] - ETA: 0s
796/1024 [======================>.......] - ETA: 0s
852/1024 [=======================>......] - ETA: 0s
910/1024 [=========================>....] - ETA: 0s
967/1024 [===========================>..] - ETA: 0s
1022/1024 [============================>.] - ETA: 0s
1024/1024 [==============================] - 1s 889us/step
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_input (InputLaye [(None, 28, 28, 1)] 0
r)
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9796000123023987
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.98089998960495
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 981us/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9785000085830688
2.4. Model fine tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 2:02:26 - loss: 0.0122 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0207 - accuracy: 0.9948
23/1688 [..............................] - ETA: 8s - loss: 0.0203 - accuracy: 0.9946
34/1688 [..............................] - ETA: 8s - loss: 0.0245 - accuracy: 0.9926
45/1688 [..............................] - ETA: 8s - loss: 0.0303 - accuracy: 0.9889
55/1688 [..............................] - ETA: 8s - loss: 0.0279 - accuracy: 0.9898
66/1688 [>.............................] - ETA: 8s - loss: 0.0272 - accuracy: 0.9896
77/1688 [>.............................] - ETA: 8s - loss: 0.0270 - accuracy: 0.9903
88/1688 [>.............................] - ETA: 7s - loss: 0.0276 - accuracy: 0.9901
99/1688 [>.............................] - ETA: 7s - loss: 0.0277 - accuracy: 0.9896
110/1688 [>.............................] - ETA: 7s - loss: 0.0272 - accuracy: 0.9901
121/1688 [=>............................] - ETA: 7s - loss: 0.0290 - accuracy: 0.9892
132/1688 [=>............................] - ETA: 7s - loss: 0.0294 - accuracy: 0.9896
143/1688 [=>............................] - ETA: 7s - loss: 0.0286 - accuracy: 0.9902
154/1688 [=>............................] - ETA: 7s - loss: 0.0277 - accuracy: 0.9907
165/1688 [=>............................] - ETA: 7s - loss: 0.0265 - accuracy: 0.9913
175/1688 [==>...........................] - ETA: 7s - loss: 0.0269 - accuracy: 0.9911
185/1688 [==>...........................] - ETA: 7s - loss: 0.0267 - accuracy: 0.9912
196/1688 [==>...........................] - ETA: 7s - loss: 0.0267 - accuracy: 0.9912
207/1688 [==>...........................] - ETA: 7s - loss: 0.0260 - accuracy: 0.9914
218/1688 [==>...........................] - ETA: 7s - loss: 0.0266 - accuracy: 0.9913
229/1688 [===>..........................] - ETA: 7s - loss: 0.0275 - accuracy: 0.9909
240/1688 [===>..........................] - ETA: 7s - loss: 0.0281 - accuracy: 0.9902
251/1688 [===>..........................] - ETA: 7s - loss: 0.0279 - accuracy: 0.9903
262/1688 [===>..........................] - ETA: 7s - loss: 0.0276 - accuracy: 0.9905
273/1688 [===>..........................] - ETA: 6s - loss: 0.0269 - accuracy: 0.9908
284/1688 [====>.........................] - ETA: 6s - loss: 0.0271 - accuracy: 0.9908
295/1688 [====>.........................] - ETA: 6s - loss: 0.0269 - accuracy: 0.9909
306/1688 [====>.........................] - ETA: 6s - loss: 0.0265 - accuracy: 0.9910
317/1688 [====>.........................] - ETA: 6s - loss: 0.0265 - accuracy: 0.9911
328/1688 [====>.........................] - ETA: 6s - loss: 0.0259 - accuracy: 0.9914
339/1688 [=====>........................] - ETA: 6s - loss: 0.0257 - accuracy: 0.9914
350/1688 [=====>........................] - ETA: 6s - loss: 0.0257 - accuracy: 0.9913
361/1688 [=====>........................] - ETA: 6s - loss: 0.0255 - accuracy: 0.9914
371/1688 [=====>........................] - ETA: 6s - loss: 0.0254 - accuracy: 0.9914
382/1688 [=====>........................] - ETA: 6s - loss: 0.0253 - accuracy: 0.9914
393/1688 [=====>........................] - ETA: 6s - loss: 0.0255 - accuracy: 0.9915
403/1688 [======>.......................] - ETA: 6s - loss: 0.0252 - accuracy: 0.9916
414/1688 [======>.......................] - ETA: 6s - loss: 0.0247 - accuracy: 0.9918
424/1688 [======>.......................] - ETA: 6s - loss: 0.0244 - accuracy: 0.9919
435/1688 [======>.......................] - ETA: 6s - loss: 0.0244 - accuracy: 0.9919
445/1688 [======>.......................] - ETA: 6s - loss: 0.0243 - accuracy: 0.9919
456/1688 [=======>......................] - ETA: 6s - loss: 0.0242 - accuracy: 0.9920
467/1688 [=======>......................] - ETA: 6s - loss: 0.0243 - accuracy: 0.9920
478/1688 [=======>......................] - ETA: 5s - loss: 0.0241 - accuracy: 0.9920
489/1688 [=======>......................] - ETA: 5s - loss: 0.0241 - accuracy: 0.9920
500/1688 [=======>......................] - ETA: 5s - loss: 0.0239 - accuracy: 0.9921
511/1688 [========>.....................] - ETA: 5s - loss: 0.0238 - accuracy: 0.9922
522/1688 [========>.....................] - ETA: 5s - loss: 0.0236 - accuracy: 0.9923
533/1688 [========>.....................] - ETA: 5s - loss: 0.0235 - accuracy: 0.9923
544/1688 [========>.....................] - ETA: 5s - loss: 0.0236 - accuracy: 0.9924
554/1688 [========>.....................] - ETA: 5s - loss: 0.0238 - accuracy: 0.9923
565/1688 [=========>....................] - ETA: 5s - loss: 0.0238 - accuracy: 0.9923
576/1688 [=========>....................] - ETA: 5s - loss: 0.0237 - accuracy: 0.9923
587/1688 [=========>....................] - ETA: 5s - loss: 0.0238 - accuracy: 0.9922
598/1688 [=========>....................] - ETA: 5s - loss: 0.0238 - accuracy: 0.9922
609/1688 [=========>....................] - ETA: 5s - loss: 0.0235 - accuracy: 0.9924
620/1688 [==========>...................] - ETA: 5s - loss: 0.0237 - accuracy: 0.9924
630/1688 [==========>...................] - ETA: 5s - loss: 0.0236 - accuracy: 0.9925
641/1688 [==========>...................] - ETA: 5s - loss: 0.0234 - accuracy: 0.9925
652/1688 [==========>...................] - ETA: 5s - loss: 0.0235 - accuracy: 0.9925
663/1688 [==========>...................] - ETA: 5s - loss: 0.0235 - accuracy: 0.9925
674/1688 [==========>...................] - ETA: 5s - loss: 0.0235 - accuracy: 0.9925
685/1688 [===========>..................] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
695/1688 [===========>..................] - ETA: 4s - loss: 0.0234 - accuracy: 0.9925
706/1688 [===========>..................] - ETA: 4s - loss: 0.0233 - accuracy: 0.9926
717/1688 [===========>..................] - ETA: 4s - loss: 0.0232 - accuracy: 0.9927
727/1688 [===========>..................] - ETA: 4s - loss: 0.0233 - accuracy: 0.9927
738/1688 [============>.................] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
749/1688 [============>.................] - ETA: 4s - loss: 0.0233 - accuracy: 0.9926
760/1688 [============>.................] - ETA: 4s - loss: 0.0236 - accuracy: 0.9925
770/1688 [============>.................] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
781/1688 [============>.................] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
792/1688 [=============>................] - ETA: 4s - loss: 0.0232 - accuracy: 0.9927
803/1688 [=============>................] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
813/1688 [=============>................] - ETA: 4s - loss: 0.0234 - accuracy: 0.9925
824/1688 [=============>................] - ETA: 4s - loss: 0.0234 - accuracy: 0.9925
835/1688 [=============>................] - ETA: 4s - loss: 0.0232 - accuracy: 0.9926
846/1688 [==============>...............] - ETA: 4s - loss: 0.0232 - accuracy: 0.9926
857/1688 [==============>...............] - ETA: 4s - loss: 0.0231 - accuracy: 0.9926
867/1688 [==============>...............] - ETA: 4s - loss: 0.0230 - accuracy: 0.9927
877/1688 [==============>...............] - ETA: 4s - loss: 0.0229 - accuracy: 0.9927
887/1688 [==============>...............] - ETA: 3s - loss: 0.0227 - accuracy: 0.9928
898/1688 [==============>...............] - ETA: 3s - loss: 0.0226 - accuracy: 0.9929
909/1688 [===============>..............] - ETA: 3s - loss: 0.0226 - accuracy: 0.9929
919/1688 [===============>..............] - ETA: 3s - loss: 0.0226 - accuracy: 0.9929
930/1688 [===============>..............] - ETA: 3s - loss: 0.0226 - accuracy: 0.9929
941/1688 [===============>..............] - ETA: 3s - loss: 0.0224 - accuracy: 0.9930
952/1688 [===============>..............] - ETA: 3s - loss: 0.0225 - accuracy: 0.9930
963/1688 [================>.............] - ETA: 3s - loss: 0.0224 - accuracy: 0.9930
974/1688 [================>.............] - ETA: 3s - loss: 0.0223 - accuracy: 0.9931
985/1688 [================>.............] - ETA: 3s - loss: 0.0221 - accuracy: 0.9932
996/1688 [================>.............] - ETA: 3s - loss: 0.0220 - accuracy: 0.9932
1007/1688 [================>.............] - ETA: 3s - loss: 0.0220 - accuracy: 0.9932
1018/1688 [=================>............] - ETA: 3s - loss: 0.0218 - accuracy: 0.9933
1029/1688 [=================>............] - ETA: 3s - loss: 0.0217 - accuracy: 0.9933
1039/1688 [=================>............] - ETA: 3s - loss: 0.0217 - accuracy: 0.9934
1049/1688 [=================>............] - ETA: 3s - loss: 0.0216 - accuracy: 0.9934
1060/1688 [=================>............] - ETA: 3s - loss: 0.0215 - accuracy: 0.9934
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0215 - accuracy: 0.9935
1082/1688 [==================>...........] - ETA: 3s - loss: 0.0214 - accuracy: 0.9935
1093/1688 [==================>...........] - ETA: 2s - loss: 0.0216 - accuracy: 0.9935
1104/1688 [==================>...........] - ETA: 2s - loss: 0.0216 - accuracy: 0.9934
1115/1688 [==================>...........] - ETA: 2s - loss: 0.0215 - accuracy: 0.9934
1126/1688 [===================>..........] - ETA: 2s - loss: 0.0215 - accuracy: 0.9934
1137/1688 [===================>..........] - ETA: 2s - loss: 0.0215 - accuracy: 0.9934
1148/1688 [===================>..........] - ETA: 2s - loss: 0.0214 - accuracy: 0.9935
1159/1688 [===================>..........] - ETA: 2s - loss: 0.0213 - accuracy: 0.9936
1169/1688 [===================>..........] - ETA: 2s - loss: 0.0213 - accuracy: 0.9935
1180/1688 [===================>..........] - ETA: 2s - loss: 0.0213 - accuracy: 0.9935
1191/1688 [====================>.........] - ETA: 2s - loss: 0.0212 - accuracy: 0.9936
1202/1688 [====================>.........] - ETA: 2s - loss: 0.0212 - accuracy: 0.9936
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0212 - accuracy: 0.9937
1223/1688 [====================>.........] - ETA: 2s - loss: 0.0211 - accuracy: 0.9937
1234/1688 [====================>.........] - ETA: 2s - loss: 0.0211 - accuracy: 0.9937
1244/1688 [=====================>........] - ETA: 2s - loss: 0.0211 - accuracy: 0.9937
1254/1688 [=====================>........] - ETA: 2s - loss: 0.0211 - accuracy: 0.9937
1265/1688 [=====================>........] - ETA: 2s - loss: 0.0211 - accuracy: 0.9937
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0210 - accuracy: 0.9937
1287/1688 [=====================>........] - ETA: 1s - loss: 0.0211 - accuracy: 0.9937
1298/1688 [======================>.......] - ETA: 1s - loss: 0.0210 - accuracy: 0.9937
1309/1688 [======================>.......] - ETA: 1s - loss: 0.0210 - accuracy: 0.9937
1320/1688 [======================>.......] - ETA: 1s - loss: 0.0210 - accuracy: 0.9937
1331/1688 [======================>.......] - ETA: 1s - loss: 0.0210 - accuracy: 0.9937
1342/1688 [======================>.......] - ETA: 1s - loss: 0.0210 - accuracy: 0.9937
1353/1688 [=======================>......] - ETA: 1s - loss: 0.0210 - accuracy: 0.9937
1364/1688 [=======================>......] - ETA: 1s - loss: 0.0210 - accuracy: 0.9937
1375/1688 [=======================>......] - ETA: 1s - loss: 0.0211 - accuracy: 0.9937
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0210 - accuracy: 0.9937
1396/1688 [=======================>......] - ETA: 1s - loss: 0.0209 - accuracy: 0.9937
1407/1688 [========================>.....] - ETA: 1s - loss: 0.0209 - accuracy: 0.9937
1418/1688 [========================>.....] - ETA: 1s - loss: 0.0209 - accuracy: 0.9937
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0209 - accuracy: 0.9937
1440/1688 [========================>.....] - ETA: 1s - loss: 0.0209 - accuracy: 0.9937
1451/1688 [========================>.....] - ETA: 1s - loss: 0.0208 - accuracy: 0.9937
1462/1688 [========================>.....] - ETA: 1s - loss: 0.0207 - accuracy: 0.9937
1473/1688 [=========================>....] - ETA: 1s - loss: 0.0206 - accuracy: 0.9938
1484/1688 [=========================>....] - ETA: 1s - loss: 0.0206 - accuracy: 0.9938
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9938
1505/1688 [=========================>....] - ETA: 0s - loss: 0.0205 - accuracy: 0.9938
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9938
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0205 - accuracy: 0.9938
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0204 - accuracy: 0.9938
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0205 - accuracy: 0.9938
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0204 - accuracy: 0.9939
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0203 - accuracy: 0.9939
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0203 - accuracy: 0.9939
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0203 - accuracy: 0.9939
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0203 - accuracy: 0.9939
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0203 - accuracy: 0.9939
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0202 - accuracy: 0.9939
1637/1688 [============================>.] - ETA: 0s - loss: 0.0203 - accuracy: 0.9939
1648/1688 [============================>.] - ETA: 0s - loss: 0.0202 - accuracy: 0.9939
1659/1688 [============================>.] - ETA: 0s - loss: 0.0202 - accuracy: 0.9940
1669/1688 [============================>.] - ETA: 0s - loss: 0.0201 - accuracy: 0.9940
1680/1688 [============================>.] - ETA: 0s - loss: 0.0201 - accuracy: 0.9940
1688/1688 [==============================] - 14s 6ms/step - loss: 0.0200 - accuracy: 0.9940 - val_loss: 0.0581 - val_accuracy: 0.9832
Epoch 2/5
1/1688 [..............................] - ETA: 8s - loss: 4.3815e-04 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0074 - accuracy: 1.0000
23/1688 [..............................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9986
34/1688 [..............................] - ETA: 8s - loss: 0.0110 - accuracy: 0.9982
45/1688 [..............................] - ETA: 8s - loss: 0.0140 - accuracy: 0.9972
55/1688 [..............................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9972
66/1688 [>.............................] - ETA: 8s - loss: 0.0135 - accuracy: 0.9972
76/1688 [>.............................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9971
87/1688 [>.............................] - ETA: 7s - loss: 0.0134 - accuracy: 0.9968
98/1688 [>.............................] - ETA: 7s - loss: 0.0133 - accuracy: 0.9968
108/1688 [>.............................] - ETA: 7s - loss: 0.0132 - accuracy: 0.9968
119/1688 [=>............................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9966
130/1688 [=>............................] - ETA: 7s - loss: 0.0132 - accuracy: 0.9969
141/1688 [=>............................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9971
152/1688 [=>............................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9971
163/1688 [=>............................] - ETA: 7s - loss: 0.0133 - accuracy: 0.9967
174/1688 [==>...........................] - ETA: 7s - loss: 0.0133 - accuracy: 0.9968
185/1688 [==>...........................] - ETA: 7s - loss: 0.0131 - accuracy: 0.9968
196/1688 [==>...........................] - ETA: 7s - loss: 0.0135 - accuracy: 0.9968
207/1688 [==>...........................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9965
218/1688 [==>...........................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9964
228/1688 [===>..........................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9964
238/1688 [===>..........................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9965
249/1688 [===>..........................] - ETA: 7s - loss: 0.0135 - accuracy: 0.9966
260/1688 [===>..........................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9964
271/1688 [===>..........................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9964
282/1688 [====>.........................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9963
293/1688 [====>.........................] - ETA: 6s - loss: 0.0145 - accuracy: 0.9962
303/1688 [====>.........................] - ETA: 6s - loss: 0.0145 - accuracy: 0.9960
314/1688 [====>.........................] - ETA: 6s - loss: 0.0147 - accuracy: 0.9957
325/1688 [====>.........................] - ETA: 6s - loss: 0.0148 - accuracy: 0.9958
336/1688 [====>.........................] - ETA: 6s - loss: 0.0148 - accuracy: 0.9957
347/1688 [=====>........................] - ETA: 6s - loss: 0.0146 - accuracy: 0.9959
357/1688 [=====>........................] - ETA: 6s - loss: 0.0145 - accuracy: 0.9960
368/1688 [=====>........................] - ETA: 6s - loss: 0.0143 - accuracy: 0.9961
379/1688 [=====>........................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9961
390/1688 [=====>........................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9961
400/1688 [======>.......................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9961
411/1688 [======>.......................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9961
421/1688 [======>.......................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9961
432/1688 [======>.......................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9961
442/1688 [======>.......................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9962
453/1688 [=======>......................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9961
464/1688 [=======>......................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9962
475/1688 [=======>......................] - ETA: 6s - loss: 0.0139 - accuracy: 0.9962
486/1688 [=======>......................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9962
497/1688 [=======>......................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9962
508/1688 [========>.....................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9963
519/1688 [========>.....................] - ETA: 5s - loss: 0.0136 - accuracy: 0.9964
530/1688 [========>.....................] - ETA: 5s - loss: 0.0136 - accuracy: 0.9964
541/1688 [========>.....................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9963
552/1688 [========>.....................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9962
563/1688 [=========>....................] - ETA: 5s - loss: 0.0137 - accuracy: 0.9963
574/1688 [=========>....................] - ETA: 5s - loss: 0.0137 - accuracy: 0.9964
584/1688 [=========>....................] - ETA: 5s - loss: 0.0140 - accuracy: 0.9963
595/1688 [=========>....................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9964
606/1688 [=========>....................] - ETA: 5s - loss: 0.0140 - accuracy: 0.9963
616/1688 [=========>....................] - ETA: 5s - loss: 0.0140 - accuracy: 0.9964
627/1688 [==========>...................] - ETA: 5s - loss: 0.0140 - accuracy: 0.9964
638/1688 [==========>...................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9965
649/1688 [==========>...................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9965
659/1688 [==========>...................] - ETA: 5s - loss: 0.0137 - accuracy: 0.9965
669/1688 [==========>...................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9965
680/1688 [===========>..................] - ETA: 4s - loss: 0.0138 - accuracy: 0.9965
691/1688 [===========>..................] - ETA: 4s - loss: 0.0136 - accuracy: 0.9965
702/1688 [===========>..................] - ETA: 4s - loss: 0.0135 - accuracy: 0.9966
713/1688 [===========>..................] - ETA: 4s - loss: 0.0135 - accuracy: 0.9965
724/1688 [===========>..................] - ETA: 4s - loss: 0.0134 - accuracy: 0.9965
735/1688 [============>.................] - ETA: 4s - loss: 0.0136 - accuracy: 0.9965
746/1688 [============>.................] - ETA: 4s - loss: 0.0137 - accuracy: 0.9965
757/1688 [============>.................] - ETA: 4s - loss: 0.0139 - accuracy: 0.9964
768/1688 [============>.................] - ETA: 4s - loss: 0.0138 - accuracy: 0.9964
779/1688 [============>.................] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
790/1688 [=============>................] - ETA: 4s - loss: 0.0138 - accuracy: 0.9964
801/1688 [=============>................] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
812/1688 [=============>................] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
823/1688 [=============>................] - ETA: 4s - loss: 0.0139 - accuracy: 0.9963
833/1688 [=============>................] - ETA: 4s - loss: 0.0139 - accuracy: 0.9962
844/1688 [==============>...............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9963
854/1688 [==============>...............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9962
865/1688 [==============>...............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9962
876/1688 [==============>...............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
887/1688 [==============>...............] - ETA: 3s - loss: 0.0137 - accuracy: 0.9963
897/1688 [==============>...............] - ETA: 3s - loss: 0.0137 - accuracy: 0.9963
908/1688 [===============>..............] - ETA: 3s - loss: 0.0137 - accuracy: 0.9963
919/1688 [===============>..............] - ETA: 3s - loss: 0.0137 - accuracy: 0.9963
930/1688 [===============>..............] - ETA: 3s - loss: 0.0137 - accuracy: 0.9963
941/1688 [===============>..............] - ETA: 3s - loss: 0.0138 - accuracy: 0.9962
952/1688 [===============>..............] - ETA: 3s - loss: 0.0138 - accuracy: 0.9962
963/1688 [================>.............] - ETA: 3s - loss: 0.0137 - accuracy: 0.9962
974/1688 [================>.............] - ETA: 3s - loss: 0.0141 - accuracy: 0.9961
984/1688 [================>.............] - ETA: 3s - loss: 0.0140 - accuracy: 0.9961
994/1688 [================>.............] - ETA: 3s - loss: 0.0140 - accuracy: 0.9961
1005/1688 [================>.............] - ETA: 3s - loss: 0.0142 - accuracy: 0.9961
1015/1688 [=================>............] - ETA: 3s - loss: 0.0142 - accuracy: 0.9961
1026/1688 [=================>............] - ETA: 3s - loss: 0.0141 - accuracy: 0.9961
1037/1688 [=================>............] - ETA: 3s - loss: 0.0142 - accuracy: 0.9961
1048/1688 [=================>............] - ETA: 3s - loss: 0.0142 - accuracy: 0.9962
1058/1688 [=================>............] - ETA: 3s - loss: 0.0141 - accuracy: 0.9962
1069/1688 [=================>............] - ETA: 3s - loss: 0.0143 - accuracy: 0.9961
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0144 - accuracy: 0.9960
1090/1688 [==================>...........] - ETA: 2s - loss: 0.0144 - accuracy: 0.9961
1101/1688 [==================>...........] - ETA: 2s - loss: 0.0144 - accuracy: 0.9961
1112/1688 [==================>...........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9960
1123/1688 [==================>...........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9960
1134/1688 [===================>..........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9961
1145/1688 [===================>..........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9960
1156/1688 [===================>..........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9960
1166/1688 [===================>..........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9960
1177/1688 [===================>..........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9960
1188/1688 [====================>.........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9960
1198/1688 [====================>.........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9960
1208/1688 [====================>.........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9960
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9961
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9961
1241/1688 [=====================>........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9961
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9961
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9961
1274/1688 [=====================>........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9961
1285/1688 [=====================>........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9961
1296/1688 [======================>.......] - ETA: 1s - loss: 0.0144 - accuracy: 0.9961
1307/1688 [======================>.......] - ETA: 1s - loss: 0.0144 - accuracy: 0.9961
1317/1688 [======================>.......] - ETA: 1s - loss: 0.0144 - accuracy: 0.9961
1328/1688 [======================>.......] - ETA: 1s - loss: 0.0143 - accuracy: 0.9961
1338/1688 [======================>.......] - ETA: 1s - loss: 0.0143 - accuracy: 0.9962
1349/1688 [======================>.......] - ETA: 1s - loss: 0.0142 - accuracy: 0.9962
1360/1688 [=======================>......] - ETA: 1s - loss: 0.0143 - accuracy: 0.9962
1371/1688 [=======================>......] - ETA: 1s - loss: 0.0143 - accuracy: 0.9962
1382/1688 [=======================>......] - ETA: 1s - loss: 0.0144 - accuracy: 0.9961
1393/1688 [=======================>......] - ETA: 1s - loss: 0.0144 - accuracy: 0.9961
1404/1688 [=======================>......] - ETA: 1s - loss: 0.0144 - accuracy: 0.9960
1415/1688 [========================>.....] - ETA: 1s - loss: 0.0144 - accuracy: 0.9960
1425/1688 [========================>.....] - ETA: 1s - loss: 0.0143 - accuracy: 0.9961
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0143 - accuracy: 0.9961
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0143 - accuracy: 0.9961
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0143 - accuracy: 0.9961
1468/1688 [=========================>....] - ETA: 1s - loss: 0.0144 - accuracy: 0.9961
1478/1688 [=========================>....] - ETA: 1s - loss: 0.0143 - accuracy: 0.9961
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0144 - accuracy: 0.9961
1500/1688 [=========================>....] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1554/1688 [==========================>...] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0142 - accuracy: 0.9962
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0142 - accuracy: 0.9962
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0142 - accuracy: 0.9962
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0143 - accuracy: 0.9962
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0142 - accuracy: 0.9962
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1640/1688 [============================>.] - ETA: 0s - loss: 0.0142 - accuracy: 0.9962
1651/1688 [============================>.] - ETA: 0s - loss: 0.0143 - accuracy: 0.9962
1662/1688 [============================>.] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1673/1688 [============================>.] - ETA: 0s - loss: 0.0143 - accuracy: 0.9961
1684/1688 [============================>.] - ETA: 0s - loss: 0.0143 - accuracy: 0.9962
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0143 - accuracy: 0.9961 - val_loss: 0.0556 - val_accuracy: 0.9862
Epoch 3/5
1/1688 [..............................] - ETA: 8s - loss: 0.0400 - accuracy: 0.9688
12/1688 [..............................] - ETA: 8s - loss: 0.0170 - accuracy: 0.9922
23/1688 [..............................] - ETA: 8s - loss: 0.0128 - accuracy: 0.9946
34/1688 [..............................] - ETA: 8s - loss: 0.0123 - accuracy: 0.9945
45/1688 [..............................] - ETA: 8s - loss: 0.0106 - accuracy: 0.9958
56/1688 [..............................] - ETA: 8s - loss: 0.0100 - accuracy: 0.9967
66/1688 [>.............................] - ETA: 8s - loss: 0.0094 - accuracy: 0.9972
77/1688 [>.............................] - ETA: 7s - loss: 0.0105 - accuracy: 0.9968
88/1688 [>.............................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9968
98/1688 [>.............................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9962
109/1688 [>.............................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9966
120/1688 [=>............................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9966
130/1688 [=>............................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9966
141/1688 [=>............................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9965
152/1688 [=>............................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9963
163/1688 [=>............................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9962
174/1688 [==>...........................] - ETA: 7s - loss: 0.0133 - accuracy: 0.9960
185/1688 [==>...........................] - ETA: 7s - loss: 0.0134 - accuracy: 0.9958
195/1688 [==>...........................] - ETA: 7s - loss: 0.0133 - accuracy: 0.9960
206/1688 [==>...........................] - ETA: 7s - loss: 0.0130 - accuracy: 0.9962
216/1688 [==>...........................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9964
227/1688 [===>..........................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9964
238/1688 [===>..........................] - ETA: 7s - loss: 0.0124 - accuracy: 0.9966
249/1688 [===>..........................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9967
260/1688 [===>..........................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9968
270/1688 [===>..........................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9968
281/1688 [===>..........................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9969
291/1688 [====>.........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9969
301/1688 [====>.........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9969
311/1688 [====>.........................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9969
322/1688 [====>.........................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9970
333/1688 [====>.........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9969
344/1688 [=====>........................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9970
355/1688 [=====>........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9969
366/1688 [=====>........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9970
377/1688 [=====>........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9970
388/1688 [=====>........................] - ETA: 6s - loss: 0.0116 - accuracy: 0.9971
399/1688 [======>.......................] - ETA: 6s - loss: 0.0115 - accuracy: 0.9972
409/1688 [======>.......................] - ETA: 6s - loss: 0.0114 - accuracy: 0.9972
420/1688 [======>.......................] - ETA: 6s - loss: 0.0115 - accuracy: 0.9971
431/1688 [======>.......................] - ETA: 6s - loss: 0.0115 - accuracy: 0.9971
441/1688 [======>.......................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9969
452/1688 [=======>......................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9969
463/1688 [=======>......................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9968
474/1688 [=======>......................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9968
485/1688 [=======>......................] - ETA: 5s - loss: 0.0121 - accuracy: 0.9968
496/1688 [=======>......................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9968
507/1688 [========>.....................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9967
518/1688 [========>.....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9968
529/1688 [========>.....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9968
539/1688 [========>.....................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9968
550/1688 [========>.....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9966
561/1688 [========>.....................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9964
572/1688 [=========>....................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9964
583/1688 [=========>....................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9964
594/1688 [=========>....................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9964
604/1688 [=========>....................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9964
615/1688 [=========>....................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
626/1688 [==========>...................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
637/1688 [==========>...................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
648/1688 [==========>...................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
659/1688 [==========>...................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9964
670/1688 [==========>...................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9965
681/1688 [===========>..................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9965
691/1688 [===========>..................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9964
702/1688 [===========>..................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9965
713/1688 [===========>..................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9964
724/1688 [===========>..................] - ETA: 4s - loss: 0.0123 - accuracy: 0.9965
735/1688 [============>.................] - ETA: 4s - loss: 0.0123 - accuracy: 0.9965
746/1688 [============>.................] - ETA: 4s - loss: 0.0123 - accuracy: 0.9965
756/1688 [============>.................] - ETA: 4s - loss: 0.0122 - accuracy: 0.9965
767/1688 [============>.................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9965
777/1688 [============>.................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9965
788/1688 [=============>................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9965
799/1688 [=============>................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9965
810/1688 [=============>................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9965
820/1688 [=============>................] - ETA: 4s - loss: 0.0123 - accuracy: 0.9966
831/1688 [=============>................] - ETA: 4s - loss: 0.0123 - accuracy: 0.9965
842/1688 [=============>................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9964
852/1688 [==============>...............] - ETA: 4s - loss: 0.0124 - accuracy: 0.9964
862/1688 [==============>...............] - ETA: 4s - loss: 0.0126 - accuracy: 0.9964
873/1688 [==============>...............] - ETA: 4s - loss: 0.0126 - accuracy: 0.9963
884/1688 [==============>...............] - ETA: 3s - loss: 0.0126 - accuracy: 0.9964
895/1688 [==============>...............] - ETA: 3s - loss: 0.0126 - accuracy: 0.9964
906/1688 [===============>..............] - ETA: 3s - loss: 0.0127 - accuracy: 0.9964
917/1688 [===============>..............] - ETA: 3s - loss: 0.0126 - accuracy: 0.9965
928/1688 [===============>..............] - ETA: 3s - loss: 0.0126 - accuracy: 0.9965
939/1688 [===============>..............] - ETA: 3s - loss: 0.0125 - accuracy: 0.9965
950/1688 [===============>..............] - ETA: 3s - loss: 0.0126 - accuracy: 0.9965
961/1688 [================>.............] - ETA: 3s - loss: 0.0125 - accuracy: 0.9965
972/1688 [================>.............] - ETA: 3s - loss: 0.0125 - accuracy: 0.9965
983/1688 [================>.............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9965
994/1688 [================>.............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9966
1005/1688 [================>.............] - ETA: 3s - loss: 0.0123 - accuracy: 0.9966
1015/1688 [=================>............] - ETA: 3s - loss: 0.0123 - accuracy: 0.9966
1026/1688 [=================>............] - ETA: 3s - loss: 0.0123 - accuracy: 0.9966
1036/1688 [=================>............] - ETA: 3s - loss: 0.0123 - accuracy: 0.9966
1046/1688 [=================>............] - ETA: 3s - loss: 0.0123 - accuracy: 0.9966
1057/1688 [=================>............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9966
1068/1688 [=================>............] - ETA: 3s - loss: 0.0123 - accuracy: 0.9966
1079/1688 [==================>...........] - ETA: 3s - loss: 0.0124 - accuracy: 0.9966
1089/1688 [==================>...........] - ETA: 2s - loss: 0.0123 - accuracy: 0.9966
1100/1688 [==================>...........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9966
1111/1688 [==================>...........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9966
1121/1688 [==================>...........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9966
1132/1688 [===================>..........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9966
1143/1688 [===================>..........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9966
1153/1688 [===================>..........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9966
1164/1688 [===================>..........] - ETA: 2s - loss: 0.0124 - accuracy: 0.9966
1175/1688 [===================>..........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9966
1185/1688 [====================>.........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9966
1196/1688 [====================>.........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9966
1207/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9966
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9966
1229/1688 [====================>.........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9966
1240/1688 [=====================>........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9966
1251/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9967
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9967
1272/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9966
1283/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9967
1293/1688 [=====================>........] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1304/1688 [======================>.......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1314/1688 [======================>.......] - ETA: 1s - loss: 0.0125 - accuracy: 0.9967
1325/1688 [======================>.......] - ETA: 1s - loss: 0.0125 - accuracy: 0.9967
1336/1688 [======================>.......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1347/1688 [======================>.......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1358/1688 [=======================>......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1369/1688 [=======================>......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1391/1688 [=======================>......] - ETA: 1s - loss: 0.0123 - accuracy: 0.9967
1401/1688 [=======================>......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1423/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1434/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1444/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9967
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9967
1487/1688 [=========================>....] - ETA: 0s - loss: 0.0123 - accuracy: 0.9967
1498/1688 [=========================>....] - ETA: 0s - loss: 0.0123 - accuracy: 0.9967
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0123 - accuracy: 0.9967
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0124 - accuracy: 0.9966
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9966
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9966
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9966
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9966
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9966
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9966
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9966
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9966
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9966
1625/1688 [===========================>..] - ETA: 0s - loss: 0.0123 - accuracy: 0.9967
1636/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9967
1647/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9967
1658/1688 [============================>.] - ETA: 0s - loss: 0.0124 - accuracy: 0.9966
1668/1688 [============================>.] - ETA: 0s - loss: 0.0124 - accuracy: 0.9966
1679/1688 [============================>.] - ETA: 0s - loss: 0.0125 - accuracy: 0.9966
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0124 - accuracy: 0.9966 - val_loss: 0.0558 - val_accuracy: 0.9862
Epoch 4/5
1/1688 [..............................] - ETA: 8s - loss: 0.0133 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0213 - accuracy: 0.9974
23/1688 [..............................] - ETA: 8s - loss: 0.0147 - accuracy: 0.9973
34/1688 [..............................] - ETA: 8s - loss: 0.0141 - accuracy: 0.9972
45/1688 [..............................] - ETA: 8s - loss: 0.0124 - accuracy: 0.9979
56/1688 [..............................] - ETA: 8s - loss: 0.0108 - accuracy: 0.9983
67/1688 [>.............................] - ETA: 8s - loss: 0.0106 - accuracy: 0.9986
77/1688 [>.............................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9976
87/1688 [>.............................] - ETA: 7s - loss: 0.0116 - accuracy: 0.9978
97/1688 [>.............................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9974
108/1688 [>.............................] - ETA: 7s - loss: 0.0115 - accuracy: 0.9977
119/1688 [=>............................] - ETA: 7s - loss: 0.0115 - accuracy: 0.9976
130/1688 [=>............................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9978
141/1688 [=>............................] - ETA: 7s - loss: 0.0105 - accuracy: 0.9980
152/1688 [=>............................] - ETA: 7s - loss: 0.0109 - accuracy: 0.9975
163/1688 [=>............................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9971
174/1688 [==>...........................] - ETA: 7s - loss: 0.0115 - accuracy: 0.9973
184/1688 [==>...........................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9975
195/1688 [==>...........................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9973
206/1688 [==>...........................] - ETA: 7s - loss: 0.0109 - accuracy: 0.9973
216/1688 [==>...........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9973
227/1688 [===>..........................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9972
238/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9974
249/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9972
260/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9972
271/1688 [===>..........................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9973
281/1688 [===>..........................] - ETA: 6s - loss: 0.0108 - accuracy: 0.9973
292/1688 [====>.........................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9973
303/1688 [====>.........................] - ETA: 6s - loss: 0.0109 - accuracy: 0.9973
314/1688 [====>.........................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9973
325/1688 [====>.........................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9973
336/1688 [====>.........................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9973
346/1688 [=====>........................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9973
357/1688 [=====>........................] - ETA: 6s - loss: 0.0109 - accuracy: 0.9974
368/1688 [=====>........................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9973
379/1688 [=====>........................] - ETA: 6s - loss: 0.0113 - accuracy: 0.9972
389/1688 [=====>........................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9970
400/1688 [======>.......................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9970
410/1688 [======>.......................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9970
421/1688 [======>.......................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9970
431/1688 [======>.......................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9970
442/1688 [======>.......................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9971
453/1688 [=======>......................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9970
463/1688 [=======>......................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9971
473/1688 [=======>......................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9971
483/1688 [=======>......................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9972
494/1688 [=======>......................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9972
505/1688 [=======>......................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9972
516/1688 [========>.....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9971
527/1688 [========>.....................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9972
538/1688 [========>.....................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9971
549/1688 [========>.....................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9970
560/1688 [========>.....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9970
570/1688 [=========>....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9971
580/1688 [=========>....................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9971
591/1688 [=========>....................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9971
602/1688 [=========>....................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9972
612/1688 [=========>....................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9971
622/1688 [==========>...................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9972
633/1688 [==========>...................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9972
644/1688 [==========>...................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9970
655/1688 [==========>...................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9971
666/1688 [==========>...................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9971
676/1688 [===========>..................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9971
687/1688 [===========>..................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
698/1688 [===========>..................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
709/1688 [===========>..................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9972
720/1688 [===========>..................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9972
731/1688 [===========>..................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
741/1688 [============>.................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9970
752/1688 [============>.................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9970
763/1688 [============>.................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9970
773/1688 [============>.................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9970
784/1688 [============>.................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
795/1688 [=============>................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9970
805/1688 [=============>................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9970
816/1688 [=============>................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9970
827/1688 [=============>................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
838/1688 [=============>................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9970
849/1688 [==============>...............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9969
860/1688 [==============>...............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9969
870/1688 [==============>...............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9969
881/1688 [==============>...............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9969
891/1688 [==============>...............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9969
902/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
913/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9969
924/1688 [===============>..............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9969
935/1688 [===============>..............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9969
946/1688 [===============>..............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9969
956/1688 [===============>..............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9969
967/1688 [================>.............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9969
978/1688 [================>.............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9970
988/1688 [================>.............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9970
999/1688 [================>.............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9970
1010/1688 [================>.............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9970
1021/1688 [=================>............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9969
1032/1688 [=================>............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9969
1043/1688 [=================>............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
1054/1688 [=================>............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9968
1065/1688 [=================>............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9968
1076/1688 [==================>...........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9969
1087/1688 [==================>...........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9969
1098/1688 [==================>...........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9969
1108/1688 [==================>...........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9969
1119/1688 [==================>...........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9969
1130/1688 [===================>..........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9969
1141/1688 [===================>..........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9969
1152/1688 [===================>..........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9969
1163/1688 [===================>..........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9968
1174/1688 [===================>..........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9969
1185/1688 [====================>.........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9969
1196/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9969
1207/1688 [====================>.........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9968
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9968
1229/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9968
1240/1688 [=====================>........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9968
1251/1688 [=====================>........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9968
1262/1688 [=====================>........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9968
1273/1688 [=====================>........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9968
1284/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9968
1294/1688 [=====================>........] - ETA: 1s - loss: 0.0119 - accuracy: 0.9968
1304/1688 [======================>.......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9968
1314/1688 [======================>.......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9968
1324/1688 [======================>.......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9968
1335/1688 [======================>.......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9968
1346/1688 [======================>.......] - ETA: 1s - loss: 0.0118 - accuracy: 0.9969
1357/1688 [=======================>......] - ETA: 1s - loss: 0.0118 - accuracy: 0.9968
1368/1688 [=======================>......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9968
1379/1688 [=======================>......] - ETA: 1s - loss: 0.0118 - accuracy: 0.9968
1389/1688 [=======================>......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9968
1400/1688 [=======================>......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9968
1411/1688 [========================>.....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9968
1421/1688 [========================>.....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9969
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9969
1443/1688 [========================>.....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9969
1454/1688 [========================>.....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9969
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9969
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9969
1487/1688 [=========================>....] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1498/1688 [=========================>....] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0119 - accuracy: 0.9969
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9968
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1616/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1627/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9969
1638/1688 [============================>.] - ETA: 0s - loss: 0.0118 - accuracy: 0.9968
1649/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9969
1660/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9968
1671/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9969
1682/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9969
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0117 - accuracy: 0.9968 - val_loss: 0.0560 - val_accuracy: 0.9853
Epoch 5/5
1/1688 [..............................] - ETA: 8s - loss: 8.8950e-04 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0088 - accuracy: 0.9974
23/1688 [..............................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9986
34/1688 [..............................] - ETA: 8s - loss: 0.0100 - accuracy: 0.9982
45/1688 [..............................] - ETA: 8s - loss: 0.0093 - accuracy: 0.9979
56/1688 [..............................] - ETA: 8s - loss: 0.0096 - accuracy: 0.9972
67/1688 [>.............................] - ETA: 8s - loss: 0.0095 - accuracy: 0.9977
77/1688 [>.............................] - ETA: 7s - loss: 0.0095 - accuracy: 0.9976
87/1688 [>.............................] - ETA: 7s - loss: 0.0094 - accuracy: 0.9978
98/1688 [>.............................] - ETA: 7s - loss: 0.0092 - accuracy: 0.9978
109/1688 [>.............................] - ETA: 7s - loss: 0.0094 - accuracy: 0.9980
120/1688 [=>............................] - ETA: 7s - loss: 0.0088 - accuracy: 0.9982
130/1688 [=>............................] - ETA: 7s - loss: 0.0085 - accuracy: 0.9983
141/1688 [=>............................] - ETA: 7s - loss: 0.0090 - accuracy: 0.9980
152/1688 [=>............................] - ETA: 7s - loss: 0.0090 - accuracy: 0.9981
163/1688 [=>............................] - ETA: 7s - loss: 0.0099 - accuracy: 0.9979
174/1688 [==>...........................] - ETA: 7s - loss: 0.0098 - accuracy: 0.9978
184/1688 [==>...........................] - ETA: 7s - loss: 0.0098 - accuracy: 0.9978
194/1688 [==>...........................] - ETA: 7s - loss: 0.0099 - accuracy: 0.9977
204/1688 [==>...........................] - ETA: 7s - loss: 0.0099 - accuracy: 0.9977
215/1688 [==>...........................] - ETA: 7s - loss: 0.0098 - accuracy: 0.9978
226/1688 [===>..........................] - ETA: 7s - loss: 0.0096 - accuracy: 0.9979
237/1688 [===>..........................] - ETA: 7s - loss: 0.0097 - accuracy: 0.9980
248/1688 [===>..........................] - ETA: 7s - loss: 0.0096 - accuracy: 0.9981
259/1688 [===>..........................] - ETA: 7s - loss: 0.0095 - accuracy: 0.9982
270/1688 [===>..........................] - ETA: 7s - loss: 0.0094 - accuracy: 0.9981
280/1688 [===>..........................] - ETA: 6s - loss: 0.0097 - accuracy: 0.9981
291/1688 [====>.........................] - ETA: 6s - loss: 0.0096 - accuracy: 0.9981
302/1688 [====>.........................] - ETA: 6s - loss: 0.0098 - accuracy: 0.9980
313/1688 [====>.........................] - ETA: 6s - loss: 0.0100 - accuracy: 0.9977
323/1688 [====>.........................] - ETA: 6s - loss: 0.0099 - accuracy: 0.9978
334/1688 [====>.........................] - ETA: 6s - loss: 0.0099 - accuracy: 0.9978
345/1688 [=====>........................] - ETA: 6s - loss: 0.0098 - accuracy: 0.9977
355/1688 [=====>........................] - ETA: 6s - loss: 0.0098 - accuracy: 0.9978
366/1688 [=====>........................] - ETA: 6s - loss: 0.0097 - accuracy: 0.9979
376/1688 [=====>........................] - ETA: 6s - loss: 0.0096 - accuracy: 0.9979
386/1688 [=====>........................] - ETA: 6s - loss: 0.0096 - accuracy: 0.9980
397/1688 [======>.......................] - ETA: 6s - loss: 0.0095 - accuracy: 0.9980
408/1688 [======>.......................] - ETA: 6s - loss: 0.0096 - accuracy: 0.9979
418/1688 [======>.......................] - ETA: 6s - loss: 0.0096 - accuracy: 0.9979
429/1688 [======>.......................] - ETA: 6s - loss: 0.0094 - accuracy: 0.9980
440/1688 [======>.......................] - ETA: 6s - loss: 0.0095 - accuracy: 0.9980
450/1688 [======>.......................] - ETA: 6s - loss: 0.0094 - accuracy: 0.9981
460/1688 [=======>......................] - ETA: 6s - loss: 0.0094 - accuracy: 0.9980
471/1688 [=======>......................] - ETA: 6s - loss: 0.0097 - accuracy: 0.9978
482/1688 [=======>......................] - ETA: 5s - loss: 0.0097 - accuracy: 0.9978
493/1688 [=======>......................] - ETA: 5s - loss: 0.0101 - accuracy: 0.9977
503/1688 [=======>......................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9976
513/1688 [========>.....................] - ETA: 5s - loss: 0.0101 - accuracy: 0.9976
524/1688 [========>.....................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9976
535/1688 [========>.....................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9974
545/1688 [========>.....................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9974
556/1688 [========>.....................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9974
567/1688 [=========>....................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9974
578/1688 [=========>....................] - ETA: 5s - loss: 0.0101 - accuracy: 0.9975
589/1688 [=========>....................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9975
600/1688 [=========>....................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9973
610/1688 [=========>....................] - ETA: 5s - loss: 0.0105 - accuracy: 0.9973
621/1688 [==========>...................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9973
632/1688 [==========>...................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9974
643/1688 [==========>...................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9974
654/1688 [==========>...................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9974
665/1688 [==========>...................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9974
676/1688 [===========>..................] - ETA: 5s - loss: 0.0105 - accuracy: 0.9973
687/1688 [===========>..................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9973
698/1688 [===========>..................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9973
708/1688 [===========>..................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9973
718/1688 [===========>..................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9973
729/1688 [===========>..................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9973
739/1688 [============>.................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9973
749/1688 [============>.................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9973
760/1688 [============>.................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9974
771/1688 [============>.................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
782/1688 [============>.................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
792/1688 [=============>................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9975
803/1688 [=============>................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9975
814/1688 [=============>................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9975
824/1688 [=============>................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9975
835/1688 [=============>................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9975
845/1688 [==============>...............] - ETA: 4s - loss: 0.0103 - accuracy: 0.9975
855/1688 [==============>...............] - ETA: 4s - loss: 0.0104 - accuracy: 0.9975
866/1688 [==============>...............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9975
876/1688 [==============>...............] - ETA: 4s - loss: 0.0108 - accuracy: 0.9973
886/1688 [==============>...............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
897/1688 [==============>...............] - ETA: 3s - loss: 0.0109 - accuracy: 0.9973
908/1688 [===============>..............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
919/1688 [===============>..............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
930/1688 [===============>..............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
941/1688 [===============>..............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
952/1688 [===============>..............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
963/1688 [================>.............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
974/1688 [================>.............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9974
985/1688 [================>.............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9974
996/1688 [================>.............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9974
1007/1688 [================>.............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
1018/1688 [=================>............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
1029/1688 [=================>............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
1040/1688 [=================>............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1051/1688 [=================>............] - ETA: 3s - loss: 0.0108 - accuracy: 0.9973
1062/1688 [=================>............] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1073/1688 [==================>...........] - ETA: 3s - loss: 0.0108 - accuracy: 0.9972
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0109 - accuracy: 0.9972
1094/1688 [==================>...........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1104/1688 [==================>...........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1115/1688 [==================>...........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1125/1688 [==================>...........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1136/1688 [===================>..........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1146/1688 [===================>..........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1157/1688 [===================>..........] - ETA: 2s - loss: 0.0110 - accuracy: 0.9972
1167/1688 [===================>..........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1178/1688 [===================>..........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1189/1688 [====================>.........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9973
1200/1688 [====================>.........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9973
1211/1688 [====================>.........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9973
1222/1688 [====================>.........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9973
1233/1688 [====================>.........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9973
1244/1688 [=====================>........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9973
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9972
1265/1688 [=====================>........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9972
1287/1688 [=====================>........] - ETA: 1s - loss: 0.0109 - accuracy: 0.9972
1298/1688 [======================>.......] - ETA: 1s - loss: 0.0109 - accuracy: 0.9972
1309/1688 [======================>.......] - ETA: 1s - loss: 0.0109 - accuracy: 0.9971
1320/1688 [======================>.......] - ETA: 1s - loss: 0.0110 - accuracy: 0.9971
1330/1688 [======================>.......] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1341/1688 [======================>.......] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1352/1688 [=======================>......] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1363/1688 [=======================>......] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1373/1688 [=======================>......] - ETA: 1s - loss: 0.0112 - accuracy: 0.9971
1384/1688 [=======================>......] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1405/1688 [=======================>......] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1416/1688 [========================>.....] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1427/1688 [========================>.....] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1438/1688 [========================>.....] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1448/1688 [========================>.....] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1459/1688 [========================>.....] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1480/1688 [=========================>....] - ETA: 1s - loss: 0.0111 - accuracy: 0.9971
1490/1688 [=========================>....] - ETA: 0s - loss: 0.0112 - accuracy: 0.9971
1501/1688 [=========================>....] - ETA: 0s - loss: 0.0112 - accuracy: 0.9971
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0112 - accuracy: 0.9971
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0112 - accuracy: 0.9971
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0112 - accuracy: 0.9971
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0112 - accuracy: 0.9971
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0112 - accuracy: 0.9971
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0112 - accuracy: 0.9971
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0112 - accuracy: 0.9970
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0112 - accuracy: 0.9971
1597/1688 [===========================>..] - ETA: 0s - loss: 0.0113 - accuracy: 0.9970
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0114 - accuracy: 0.9970
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0114 - accuracy: 0.9970
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0114 - accuracy: 0.9970
1640/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9970
1650/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9970
1661/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9970
1672/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9970
1682/1688 [============================>.] - ETA: 0s - loss: 0.0115 - accuracy: 0.9970
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0115 - accuracy: 0.9969 - val_loss: 0.0570 - val_accuracy: 0.9862
<keras.src.callbacks.History object at 0x7f48686a34d0>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine tuning:', score)
Test accuracy after fine tuning: 0.9858999848365784
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
WARNING:tensorflow:5 out of the last 5 calls to <function NonTrackVariable.set_var at 0x7f486a95a700> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
WARNING:tensorflow:6 out of the last 6 calls to <function NonTrackVariable.set_var at 0x7f486a984220> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test)
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9869999885559082
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-11.964804 -6.436502 -3.433234 -0.16535743 -11.083805
-6.0291085 -17.113314 6.5153823 -4.036972 -0.7724161 ]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 7.091 seconds)