tensornn.layers

This file contains different types of layers used in neural networks. Layers need to be able to propagate their inputs forward.

Classes

`Convolutional`	Convolutional layer.
`Dense`	Each neuron is connected to all neurons in the previous layer.
`Input`	Input layer.
`Layer`	Abstract base layer class.

class tensornn.layers.Dense(num_neurons: int, activation: Activation = TensorNN.NoActivation, parameter_init: tuple[str, str] = ('default', 'default'))

Bases: Layer

Each neuron is connected to all neurons in the previous layer. Output is calculated by: (output of previous layer * weights) + biases.

b_init: str

backward(accumulated_gradient: Tensor) → Tensor

Backpropagation step. This is called when the loss is calculated and the gradients are propagated back to this layer.

Parameters:: accumulated_gradient – the gradient from the next layer
Returns:: the gradient for the previous layer

calculated: Tensor

forward(inputs: Tensor) → Tuple[Tensor, Tensor]

Calculates and returns a forward pass of this layer, before and after activation.

Parameters:: inputs – outputs from the previous layer
Returns:: the output calculated after this layer before and after activation

grad_biases: Tensor

grad_weights: Tensor

inputs: Tensor

register(prev: int) → None

Number of inputs in the previous layer. This is called whenever the NeuralNetwork is registered(NeuralNetwork.register()) with the optimizer and loss, it calls this method for all layers giving information to it. If your layer doesn’t need this, you don’t need to implement this.

Parameters:: prev – number of neurons in previous layer
Returns:: Nothing

step(adjust_w: Tensor, adjust_b: Tensor) → None

Update the weights and biases of this layer.

Parameters:

adjust_w – the adjustment to be made to the weights
adjust_b – the adjustment to be made to the biases

w_init: str

class tensornn.layers.Input(num_inputs: int, activation: Activation = TensorNN.NoActivation)

Bases: Layer

Input layer. This is a dummy layer that just passes the input to the next layer. This layer should be the first layer in every network to describe the input shape. It is possible to give this layer an activation function if you want an activation applied to the inputs every time the network is run. It could be useful in situations where you want to normalize the inputs before passing them to the next layer.

backward(accumulated_gradient: Tensor) → Tensor

Backpropagation step. This is called when the loss is calculated and the gradients are propagated back to this layer.

Parameters:: accumulated_gradient – the gradient from the next layer
Returns:: the gradient for the previous layer

forward(inputs: Tensor) → Tuple[Tensor, Tensor]

Calculates and returns a forward pass of this layer, before and after activation.

Parameters:: inputs – outputs from the previous layer
Returns:: the output calculated after this layer before and after activation

register(prev: int) → None

Number of inputs in the previous layer. This is called whenever the NeuralNetwork is registered(NeuralNetwork.register()) with the optimizer and loss, it calls this method for all layers giving information to it. If your layer doesn’t need this, you don’t need to implement this.

Parameters:: prev – number of neurons in previous layer
Returns:: Nothing

step(adjust_w: Tensor, adjust_b: Tensor) → None

Update the weights and biases of this layer.

Parameters:

adjust_w – the adjustment to be made to the weights
adjust_b – the adjustment to be made to the biases

class tensornn.layers.Layer(num_neurons: int, activation: Activation = TensorNN.NoActivation)

Bases: ABC, TensorNNObject

Abstract base layer class. All layer classes should inherit from this.

A neural network is composed of layers. A set of inputs are moved from one layer to another. Each layer has its own way of calculating the output of its own inputs(outputs of previous layer). Some layers also have a few tweakable parameters, tweaking these parameters will allow the network to learn and adapt to the inputs to produce the correct outputs.

activation: Activation

abstractmethod backward(accumulated_gradient: Tensor) → Tensor

Backpropagation step. This is called when the loss is calculated and the gradients are propagated back to this layer.

Parameters:: accumulated_gradient – the gradient from the next layer
Returns:: the gradient for the previous layer

biases: Tensor

abstractmethod forward(inputs: Tensor) → Tuple[Tensor, Tensor]

Calculates and returns a forward pass of this layer, before and after activation.

Parameters:: inputs – outputs from the previous layer
Returns:: the output calculated after this layer before and after activation

grad_biases: Tensor

grad_weights: Tensor

gradients: Tensor

neurons: int

abstractmethod register(prev: int) → None

Number of inputs in the previous layer. This is called whenever the NeuralNetwork is registered(NeuralNetwork.register()) with the optimizer and loss, it calls this method for all layers giving information to it. If your layer doesn’t need this, you don’t need to implement this.

Parameters:: prev – number of neurons in previous layer
Returns:: Nothing

reset_gradients() → None: Reset the gradients of the layer. This is called at the start of each epoch.

abstractmethod step(adjust_w: Tensor, adjust_b: Tensor) → None

Update the weights and biases of this layer.

Parameters:

adjust_w – the adjustment to be made to the weights
adjust_b – the adjustment to be made to the biases

weights: Tensor