What is a Neural Network?

A Neural Network, or Artificial Neural Network is an algorithm used to find important parameters from the input features in order to calculate a prediction. Put in practice the Neural Network Algorithms ca be useful to solve problems and make decisions.

Neural networks are within the field of Machine Learning, specifically they are supervised algorithms, that is, they need values ​​that will be used as input to the network and labels, which are the targets that the network aims to predict.

Type of models

Regression Model

With a regression model the Neural Network will work with a function that describes the relation between the input features and the output prediction.

The main formula of one regression model is:

$\hat{y} = \theta_0 + \theta_1 \cdot x_1 + \theta_2 \cdot x_2 + \theta_n \cdot x_n + b$

The aim of the Network will be to find the optimal parameters $\theta_s$ that satisfy the above equation. The number of optimal parameters will be the number of features plus one. Also de $b$, called bias term, it is considered an error that occurs in the machine learning model due to incorrect assumptions while the model is training. The model, during the training adjust this term in order to get the prediction closest to reality.

The output value of one Regression Model is always a continuos variable.

Classification Model

Those kind of models are used to categorize the data according to the input features, where, each sample will be categorized into one class.

We can have two main types of classification model. The binary classification where the data is categorized into two different categories and the multi-class classification model, where the data can be categorized into three or more categories.

At the difference to the Regression models, in the classification model we will use probabilities, the aim Neural Network will be to compute the probability that an example falls into a certain category denoted by $y=1$. Thus:

$\hat{y} = P(y=1|X)$

Due to the probabilities are numbers between 0 and 1, a threshold has to be set. Normally is 0.5, so we have:

Neural Network, main components

One Neural Network is made up of different layers. The simplest Neutal Network will be made up with one Input Layer and one Output layer. In the practice, this Net is not used, always the analistic use hidden layers to make complex the algorith and hepls to the Neural Network to learn the parameters more quickly.

Here we can visualize a common Neural network with one input layer (red dashed square), one hidden layer (blue dashed square) and one output layer (green dashed square).

Input layer.

The input layer of one Neural Network refers to the data that will be used to train the model, each $x_1,\ x_2,\ \dots,\ x_n$ represent the input features of the data.

For example, if we want to predict the price of houses, the inputs features could be the constructed area, the number of rooms, how many bathrooms are there, etc.

Hidden layers.

Between the input and output layer we can add the number os layers as the user wants.

Each hidden layer contains nodes, the number of nodes on each hidden layer also can be tuned by the user.

Output layer

The output layer of one Neural Networs refers to the prediction made by the algorithm.

The number of nodes will be selected according with the problem that is beeing analyzed.

• Neural Network Regression: On this kind of models, the commun number of nodes is just one, releated to the predicted value.

• Neural Network Classification (Binary Classification): The numbers of nodes needed is just one. But why not use two? This is not necessary since if the node is activated it will inform us that the prediction is one of the two possible classes, if it is not activated it will tell us that the prediction is the other class.

• Neural Network Classification (Multi-Class Classification): The number of nodes will be the number of classes which the sample can be categorized.