Encoder

cnn_encoder

CNN encoder modules

class mindnlp.modules.encoder.cnn_encoder.CNNEncoder(embedding, convs, conv_layer_activation=Tanh<>, output_dim=None)

Bases: EncoderBase

CNN Encoder.

Convolutional encoder consisting of len(convolutions) layers.

Parameters:

• embedding (Cell) – The embedding layer.

• convs (list[Cell]) – The list of Conv Cell.

• conv_layer_activation (Module) – Activation to use after the convolution layers.

• output_dim (int) – The output vector of collected features after doing convolutions and pooling. If this value is None, return the result of the max pooling, an output of shape.

Raises:

• TypeError – if embedding is not a Cell.

• TypeErrpr – if convs is not a list[Cell].

• TypeError – if conv_layer_activation is not a Module.

• TypeError – if output_dim is not a int.

• RuntimeError – If embedding is None.

• RuntimeError – If convs is None.

Examples

>>> vocab_size = 1000
>>> embedding_size = 32
>>> num_filter = 128
>>> ngram_filter_sizes = (2, 3, 4, 5)
>>> output_dim = 16
>>> embedding = nn.Embedding(vocab_size, embedding_size)
>>> convs = [
...     nn.Conv2d(in_channels=1,
...               out_channels=num_filter,
...               kernel_size=(i, embedding_size),
...               pad_mode="pad") for i in ngram_filter_sizes
... ]
>>> cnn_encoder = CNNEncoder(embedding, convs, output_dim=output_dim)
>>> src_tokens = Tensor(np.ones([8, 16]), mindspore.int32)
>>> result = cnn_encoder(src_tokens)
>>> print(result.shape)
(8, 16)

construct(src_token, src_length=None, mask=None)

Construct function of CNNEncoder.

Parameters:

• src_token (Tensor) – Tokens in the source language with shape [batch, max_len].

• mask (Tensor) – Its elements identify whether the corresponding input token is padding or not. If the value is 1, not padding token. If the value is 0, padding token. Defaults to None.

Returns:

Tensor, If output_dim is None, the result shape is of (batch_size, len(convs) * num_filter) and dtype is float; If not, the result shape is of (batch_size, output_dim).

get_input_dim()

Returns the dimension of input vector

get_output_dim()

Returns the dimension of the output vector

rnn_encoder

RNN encoder modules

class mindnlp.modules.encoder.rnn_encoder.RNNEncoder(embedding, rnn)

Bases: EncoderBase

Stacked Elman RNN Encoder.

Parameters:

• embedding (Cell) – The embedding layer.

• rnn (Cell) – The RNN Layer.

Examples

>>> vocab_size = 1000
>>> embedding_size = 32
>>> hidden_size = 16
>>> num_layers = 2
>>> has_bias = True
>>> dropout = 0.1
>>> bidirectional = False
>>> embedding = nn.Embedding(vocab_size, embedding_size)
>>> rnn = nn.RNN(embedding_size, hidden_size, num_layers=num_layers, has_bias=has_bias,
...              batch_first=True, dropout=dropout, bidirectional=bidirectional)
>>> rnn_encoder = RNNEncoder(embedding, rnn)
>>> src_tokens = Tensor(np.ones([8, 16]), mindspore.int32)
>>> src_length = Tensor(np.ones([8]), mindspore.int32)
>>> mask = Tensor(np.ones([8, 16]), mindspore.int32)
>>> output, hiddens_n, mask = rnn_encoder(src_tokens, src_length, mask=mask)
>>> print(output.shape)
>>> print(hiddens_n.shape)
>>> print(mask.shape)
(8, 16, 16)
(2, 8, 16)
(8, 16)

construct(src_token, src_length=None, mask=None)

Construct method.

Parameters:

• src_token (Tensor) – Tokens in the source language with shape [batch, max_len].

• src_length (Tensor) – Lengths of each sentence with shape [batch].

• mask (Tensor) – Its elements identify whether the corresponding input token is padding or not. If the value is 1, not padding token. If the value is 0, padding token. Defaults to None.

Returns:

Tuple, a tuple contains (output, hiddens_n, mask).

• output (Tensor): Tensor of shape (seq_len, batch_size, num_directions * hidden_size).

• hiddens_n (Tensor): Tensor of shape (num_directions * num_layers, batch_size, hidden_size).

• mask (Tensor): Mask Tensor used in decoder.

reorder_encoder_out(encoder_out, new_order)

Reorder encoder output according to new_order.

Parameters:

• encoder_out (Union[Tensor, tuple]) – The encoder’s output.

• new_order (Tensor) – Desired order.

Returns:

Tuple, encoder_out rearranged according to new_order.