Implements a dense (fully-connected) layer. More...

Inheritance diagram for fnn_layer_dense::denselayer:

Collaboration diagram for fnn_layer_dense::denselayer:

Public Member Functions
procedure, pass, public	tofile => dense_tofile
	Saves the layer. Implemented by dense_tofile. More...

procedure, pass, public	apply_forward => dense_apply_forward
	Applies and linearises the layer. Implemented by dense_apply_forward. More...

procedure, pass, public	apply_tangent_linear => dense_apply_tangent_linear
	Applies the TL of the layer. Implemented by dense_apply_tangent_linear. More...

procedure, pass, public	apply_adjoint => dense_apply_adjoint
	Applies the adjoint of the layer. Implemented by dense_apply_adjoint. More...

Public Member Functions inherited from fnn_layer::layer
procedure, pass, public	get_num_parameters => layer_get_num_parameters
	Returns the number of parameters. Implemented by layer_get_num_parameters. More...

procedure, pass, public	set_parameters => layer_set_parameters
	Setter for layer::parameters. Implemented by layer_set_parameters. More...

procedure, pass, public	get_parameters => layer_get_parameters
	Getter for layer::parameters. Implemented by layer_get_parameters. More...

procedure, pass, public	tofile => layer_tofile
	Saves the layer. Implemented by layer_tofile. More...

procedure, pass, public	apply_forward => layer_apply_forward
	Applies and linearises the layer. Implemented by layer_apply_forward. More...

procedure, pass, public	apply_tangent_linear => layer_apply_tangent_linear
	Applies the TL of the layer. Implemented by layer_apply_tangent_linear. More...

procedure, pass, public	apply_adjoint => layer_apply_adjoint
	Applies the adjoint of the layer. Implemented by layer_apply_adjoint. More...

Additional Inherited Members
Public Attributes inherited from fnn_layer::layer
integer(ik), public	input_size
	The dimension of the input of the layer. More...

integer(ik), public	output_size
	The dimension of the output of the layer. More...

integer(ik), public	batch_size
	The batch size. More...

integer(ik), public	num_parameters
	The number of (trainable) parameters. More...

class(linearactivation), allocatable, public	activation
	The activation function. More...

real(rk), dimension(:), allocatable, public	parameters
	The vector of (trainable) parameters. More...

real(rk), dimension(:, :), allocatable, public	forward_input
	The storage for the layer::apply_forward method. More...

real(rk), dimension(:, :), allocatable, public	tangent_linear_input
	The storage for the layer::apply_tangent_linear method. More...

real(rk), dimension(:, :), allocatable, public	adjoint_input
	The storage for the layer::apply_adjoint method. More...

Detailed Description

Implements a dense (fully-connected) layer.

This layer has two sets of (trainable) parameters:

the kernel $\mathbf{W}$ , a matrix of size (layer::output_size, layer::input_size);
the bias $\mathbf{b}$ , a vector of size (layer::output_size).

Using numpy syntax, $\mathbf{b}$ and $\mathbf{W}$ are obtained from $\mathbf{p}$ through

$\mathbf{b} = \mathbf{p}[:N_{\mathrm{out}}],$

$\mathbf{W} = \mathbf{p}[N_{\mathrm{out}}:].\mathrm{reshape}((N_{\mathrm{out}}, N_{\mathrm{in}}), \mathrm{order="F"}),$

where $N_{\mathrm{out}}$ is layer::output_size and $N_{\mathrm{in}}$ is layer::input_size.

A similar relationship holds between $d\mathbf{b}$ , $d\mathbf{W}$ and $d\mathbf{p}$ in the dense_apply_tangent_linear and dense_apply_adjoint methods.