tensornn.loss

This file contains the loss functions used in TensorNN. Loss functions are ways your neural network calculates how off its calculations are. Then this information is used to improve/train it.

Classes

`BCE`
`BinaryCrossEntropy`	Sigmoid is the only activation function compatible with BinaryCrossEntropy loss.
`CCE`
`CategoricalCrossEntropy`	It is recommended to use the Softmax activation function with this loss.
`Loss`	Base loss class.
`MAE`
`MSE`
`MSLE`
`MeanAbsoluteError`	Mean absolute error is MSE but instead of squaring the values, you absolute value them.
`MeanSquaredError`	Mean squared error is calculated extremely simply.
`MeanSquaredLogarithmicError`	Mean squared logarithmic error is MSE but taking the log of our values before subtraction.
`Poisson`	Poisson loss is calculated with this formula: average of (pred-desired*logₑ(pred))
`RMSE`
`RSS`
`ResidualSumOfSquares`	Residual sum of squares loss is MSE but instead of doing the mean, you do the sum.
`RootMeanSquaredError`	Root mean squared error is just MSE, but it includes a square root after taking the average.
`SHL`
`SquaredHinge`	Square hinge loss is calculated with this formula: max(0, 1-pred*desired)^2

class tensornn.loss.CategoricalCrossEntropy

Bases: Loss

It is recommended to use the Softmax activation function with this loss. Despite its long name, the way that categorical cross entropy loss is calculated is simple.

Let’s say our prediction (after softmax) is [0.7, 0.2, 0.1], and the desired values are [1, 0, 0]. We can simply get the prediction number at the index of the 1 in the desired values. 1 is at index 0 so we look at index 0 of our prediction which would be 0.7. Now we just take the negative log of 0.7 and we are done!

Note: log in programming is usually logₑ or natural log or ln in math.

calculate(pred: Tensor, desired: Tensor) → Tensor

The loss function is used to calculate how off the predictions of the network are.

Parameters:

pred – the prediction of the network
desired – the desired values which the network should have gotten close to

Returns:

the average of calculated loss for one whole pass of the network

derivative(pred: Tensor, desired: Tensor) → Tensor

Used in backpropagation which helps calculates how much each neuron impacts the loss.

Parameters:

pred – the prediction of the network
desired – the desired values which the network should have gotten close to

Returns:

the derivative of the loss function wrt the last layer of the network

class tensornn.loss.Loss

Bases: ABC, TensorNNObject

Base loss class. All loss classes should inherit from this.

abstractmethod calculate(pred: Tensor, desired: Tensor) → Tensor

The loss function is used to calculate how off the predictions of the network are.

Parameters:

pred – the prediction of the network
desired – the desired values which the network should have gotten close to

Returns:

the average of calculated loss for one whole pass of the network

abstractmethod derivative(pred: Tensor, desired: Tensor) → Tensor

Used in backpropagation which helps calculates how much each neuron impacts the loss.

Parameters:

pred – the prediction of the network
desired – the desired values which the network should have gotten close to

Returns:

the derivative of the loss function wrt the last layer of the network

tensornn.loss.MAE: alias of MeanAbsoluteError

tensornn.loss.MSE: alias of MeanSquaredError

tensornn.loss.MSLE: alias of MeanSquaredLogarithmicError

class tensornn.loss.Poisson

Bases: Loss

Poisson loss is calculated with this formula: average of (pred-desired*logₑ(pred))

calculate(pred: Tensor, desired: Tensor) → Tensor

The loss function is used to calculate how off the predictions of the network are.

Parameters:

pred – the prediction of the network
desired – the desired values which the network should have gotten close to

Returns:

the average of calculated loss for one whole pass of the network

tensornn.loss.RMSE: alias of RootMeanSquaredError

tensornn.loss.RSS: alias of ResidualSumOfSquares

class tensornn.loss.SquaredHinge

Bases: Loss

Square hinge loss is calculated with this formula: max(0, 1-pred*desired)^2

calculate(pred: Tensor, desired: Tensor) → Tensor

The loss function is used to calculate how off the predictions of the network are.

Parameters:

pred – the prediction of the network
desired – the desired values which the network should have gotten close to

Returns:

the average of calculated loss for one whole pass of the network