German Traffic Sign Classifier

Here is a visual summary of the number of training images with respect to each label:

There are in total of 34799 training images, 4410 validation images and 12630 testing images. Each image has a dimension of 32 x 32 x 3. In total, there are 43 labels that uniquely determine 43 German Traffic Signs.

Here is a visualization of each unique label and corresponding traffic sign:

I have tried to train the neural network with different image pre-processing techniques and I notice the normalization plays a huge important role. Since I have a powerful GPU on my local machine (GTX980Ti), I decide to use image input with three-color channels (RGB) directly without applying gray-scale. First, I tried to apply normalization with (pixel - 128)/ 128 and the validation accuracy can barely reach 85%. Next, I applied the standard statistical normalization technique for the training data using pixel - pixel.mean()/pixel.std() and the result improved the accuracy dramatically.

Here is an example of an original image and the normalized image:

Here is an example of normalized image and brightness adjusted image (brighter in this case):

This particular CNN structure is inspired from VGG16. The idea is to multiple convolutional layers with decreasing kernel size as the layer gets deeper. However, the number of CNN filters doubles consecutively (32, 64, 128 …). The itution is that the first few CNN filter learns the general features, such as edges etc. As the layer gets deeper, the smaller filters are used to learn the more detailed features. To reduce the possibility of overfitting, I added two dropout layers that remove 50% of the input from previous layers.

To train the model, I initialize the weights of the CNN with a mean of 0 and variance of 0.01. I break the entire training data set into 256 images for each batch and perform training with 20 epochs.

The loss function I used here is multi-class cross-entropy function:

The design process is performed iteratively. Starting with Lenet from Yann LeCun, the validation accuracy can only reach roughly 85% even with image argumentation. From my experience, we need more complex neural network structure i.e., more CNN filters and deeper Neural Net layers, to train the model better. But there is no free lunch, as the more complicated Neural Net requires more data to train. I decided to not using complex networks such as Resnet-50 for this particular project. Therefore, the VGG-16 draws my attention during the design process. The idea is to design a CNN model that can capture enough features, while minimizing the number of layers and parameters. In the end, we do not want an over-complicated model that is difficult to train. After a few trial and errors, I ended up with a CNN that has 10 layers.

My final model has a validation accuracy around 96%.

To test out if my CNN can correctly classify new images that it has never seen before. Here are eight German traffic signs that I found on the web:

Before feeding testing images into the pipeline, I perform a quick resizing so it matches the input requirement of the CNN (In this case, 32x32x3). Next, we must also normalize the input using the previously introduced technique. The code is shown as the following:

My model has successfully identified all testing images from the web. Here are the results of the prediction:

The code for making predictions on my final model is the following:

It reads the previously saved model and then use evaluate() function to check the accuracy of each prediction.

For the testing image No Entry, the model believes it has a probability 1.0 to be No Entry.

The top-5 softmax probability is the following:

Note, mathematically the sum of all probabilities has to equal to 1.0. If the first entry is 1.0, that automatically implies all the other classes has a probability of 0. Therefore, any extremely small number is simply a numerical error and can be treated as 0. Here are the probability for the other 7 images:

The top-5 softmax probability is the following:

The top-5 softmax probability is the following:

The top-5 softmax probability is the following:

The top-5 softmax probability is the following:

The top-5 softmax probability is the following:

The top-5 softmax probability is the following:

The top-5 softmax probability is the following:

Related posts:

The current technology we hold in our pockets is constantly mapping and tracking. This isn’t necessarily a bad thing, but it can be depending on how our data is treated. Niantic’s Pokemon Go now…

I am a senior here at Marquette, and while my time as a student is coming to an end, I cannot express how blessed I feel to have spent the past four years here. I attended Catholic schools throughout…

There are few things in this world that everyone seems to agree on. One of those things is just how impactful can a solid morning routine be on creating a successful and a significant life. I have…