machine learning

Decision Tree from scratch

Alexey Novakov published on

8 min, 1538 words

Cropped view of one the region in the middle of the tree we will build further

Decision Tree classifier is one the simplest algorithm to implement from scratch. One of the benefit of this algorithm is it can be trained without spending too much efforst on data preparation and it is fast comparing to more complex algorithms like Neural Networks. In this blog post we are going to implement CART algorithm, which stands for Classification and Regression trees. There are many other algorithms in decision trees space, but we will not describe them in this blog post.

Data science practitioners often use decision tree algorithms to compare their performance with more advanced algorithms. Although decision tree is fast to train, its accuracy metric usually lower than accuracy on the other algorithms like deep feed forward networks or something more advanced using the same dataset. However, you do not always need high accuracy value, so using CART and other decision tree ensemble algorithms may be enough for solving particular problem.

Read More

Face Identification with VGGFace and OpenCV

Alexey Novakov published on

10 min, 1853 words

Face detection and recognition is one the area where Deep Learning is incredibly useful. There are many studies and datasets related to human faces and their detection/recognition. In this article we will implement Machine Learning pipeline for face detection and recognition using few libraries and CNN model.

Read More

Convolutional Neural Network in Scala

Alexey Novakov published on

9 min, 1737 words

Last time we used ANN to train a Deep Learning model for image recognition using MNIST dataset. This time we are going to look at more advanced network called Convolutional Neural Network or CNN in short.

CNN is designed to tackle image recognition problem. However, it can be used not only for image recognition. As we have seen last time, ANN using just hidden layers can learn quite well on MNIST. However, for real life use cases we need higher accuracy. The main idea of CNN is to learn how to recognise object in their different shapes and positions using specific features of the image data. The goal of CNN is better model regularisation by using convolution and pooling operations.

Read More

MNIST image recognition using Deep Feed Forward Network

Alexey Novakov published on

16 min, 3170 words

Deep Feed Forward Neural Network is one of the type of Artificial Neural Networks, which is also able to classify computer images. In order to feed pixel data into the neural net in RBG/Greyscale/other format one can map every pixel to network inputs. That means every pixel becomes a feature. It may sound scary and highly inefficient to feed, let's say, 28 hieght on 28 width image size, which is 784 features to learn from. However, neural networks can learn from the pixel data successfully and classify unseen data. We are going to prove this.

Please note, there are additional type of networks which are more efficient in image classification such as Convolutional Neural Network, but we are going to talk about that next time.


Wikipedia MnistExamples

Read More

Linear Regression with Adam Optimizer

Alexey Novakov published on

8 min, 1488 words

Adam is one more optimization algorithm used in neural networks. It is based on adaptive estimates of lower-order moments. It has more hyper-parameters than classic Gradient Descent to tune externally

Good default settings for the tested machine learning problems are:

  • α = 0.001, // learning rate. We have already seen this one in classic Gradient Descent.
  • β1 = 0.9,
  • β2 = 0.999
  • eps = 10−8.
Read More

Linear Regression with Gradient Descent

Alexey Novakov published on

9 min, 1670 words

In this article we are going to use Scala mini-library for Deep Learning that we developed earlier in order to study basic linear regression task. We will learn model weights using perceptron model, which will be our single unit network layer that emits target value. This model will predict a target value yHat based on two trained parameters: weight and bias. Both are scalar numbers. Weights optimization is going to be based on implemented Gradient descent algorithm:

Model equation:

y = bias + weight * x
Read More

Artificial Neural Network in Scala - part 1

Alexey Novakov published on

15 min, 2863 words

Deep Learning is a group of machine learning methods which are based on artificial neural networks. Some of the deep learning architectures are deep neural networks. Deep neural network is an artificial neural network (ANN further) with multiple layers between the input and output layers. There are different types of neural networks, but they always have neurons, synapses, weights, biases and functions.

Read More