tensornn.layers.Dense

class tensornn.layers.Dense(num_neurons: int, activation: Activation = TensorNN.NoActivation, parameter_init: tuple[str, str] = ('default', 'default'))

Bases: Layer

Each neuron is connected to all neurons in the previous layer. Output is calculated by: (output of previous layer * weights) + biases.

Methods

backward	Backpropagation step.
forward	Calculates and returns a forward pass of this layer, before and after activation.
register	Number of inputs in the previous layer.
reset_gradients	Reset the gradients of the layer.
source
step	Update the weights and biases of this layer.

Attributes

neurons
weights
biases
activation
gradients

__init__(num_neurons: int, activation: Activation = TensorNN.NoActivation, parameter_init: tuple[str, str] = ('default', 'default')) → None

Initialize dense layer.

Parameters:

• num_neurons – the number of neurons in this layer/number of outputs of this layer

• activation – the activation function applied before the layer output is calculated

• parameter_init – the initialization method for the weights and biases.

#TODO have stuff like zero_biases go in a dictionary like config or options

__repr__() → str

Return repr(self).

backward(accumulated_gradient: Tensor) → Tensor

Backpropagation step. This is called when the loss is calculated and the gradients are propagated back to this layer.

Parameters:

accumulated_gradient – the gradient from the next layer

Returns:

the gradient for the previous layer

forward(inputs: Tensor) → Tuple[Tensor, Tensor]

Calculates and returns a forward pass of this layer, before and after activation.

Parameters:

inputs – outputs from the previous layer

Returns:

the output calculated after this layer before and after activation

register(prev: int) → None

Number of inputs in the previous layer. This is called whenever the NeuralNetwork is registered(NeuralNetwork.register()) with the optimizer and loss, it calls this method for all layers giving information to it. If your layer doesn’t need this, you don’t need to implement this.

Parameters:

prev – number of neurons in previous layer

Returns:

Nothing

reset_gradients() → None

Reset the gradients of the layer. This is called at the start of each epoch.

step(adjust_w: Tensor, adjust_b: Tensor) → None

Update the weights and biases of this layer.

Parameters:

• adjust_w – the adjustment to be made to the weights

• adjust_b – the adjustment to be made to the biases