Loss

Losses

class mindnlp.modules.loss.CMRC2018Loss(reduction='mean')

Bases: Cell

used to compute CMRC2018 chinese Q&A task

Parameters:

reduction (str) – Indicate how to average the loss, the candicates are “mean” and “sum”. Default: “mean”.

construct(target_start, target_end, context_len, pred_start, pred_end)

compute CMRC2018Loss

Parameters:

• target_start (Tensor) – size: batch_size, dtype: int.

• target_end (Tensor) – size: batch_size, dtype: int.

• context_len (Tensor) – size: batch_size, dtype: float.

• pred_start (Tensor) – size: batch_size*max_len, dtype: float.

• pred_end (Tensor) – size: batch_size*max_len, dtype: float.

Returns:

Tensor, the CMRC2018 loss.

Raises:

ValueError – if ‘reduction’ is not ‘sum’ or ‘mean’.

Example

>>> cmrc_loss = CMRC2018Loss()
>>> tensor_a = mindspore.Tensor(np.array([1, 2, 1]), mindspore.int32)
>>> tensor_b = mindspore.Tensor(np.array([2, 1, 2]), mindspore.int32)
>>> my_context_len = mindspore.Tensor(np.array([2., 1., 2.]), mindspore.float32)
>>> tensor_c = mindspore.Tensor(np.array([
>>>     [0.1, 0.2, 0.1],
>>>     [0.1, 0.2, 0.1],
>>>     [0.1, 0.2, 0.1]
>>> ]), mindspore.float32)
>>> tensor_d = mindspore.Tensor(np.array([
>>>     [0.2, 0.1, 0.2],
>>>     [0.2, 0.1, 0.2],
>>>     [0.2, 0.1, 0.2]
>>> ]), mindspore.float32)
>>> my_loss = cmrc_loss(tensor_a, tensor_b, my_context_len, tensor_c, tensor_d)
>>> print(my_loss)

class mindnlp.modules.loss.RDropLoss(reduction='none')

Bases: Cell

R-Drop Loss implementation For more information about R-drop please refer to this paper: https://arxiv.org/abs/2106.14448

Original implementation please refer to this code: https://github.com/dropreg/R-Drop

Parameters:

reduction (str) –

Indicate how to average the loss, the candicates are “none”, “batchmean”,”mean”,”sum”. Default: “none”.

• ”mean”: The reduced mean loss is returned.

• ”batchmean”: The sum loss divided by batch size is returned.

• ”sum”: The reduced sum loss is returned.

• ”none”: No reduction will be applied.

construct(p, q, pad_mask=None)

Returns loss tensor, the rdrop loss of p and q.

Parameters:

• p (Tensor) – The first forward logits of training examples.

• q (Tensor) – The second forward logits of training examples.

• pad_mask (Tensor) – The Tensor containing the binary mask to index with, it’s data type is bool. Default: None.

Returns:

Tensor, the rdrop loss of p and q.

Raises:

ValueError – if ‘reduction’ in ‘RDropLoss’ is not ‘sum’, ‘mean’ ‘batchmean’, or ‘none’.

Examples

>>> r_drop_loss = RDropLoss()
>>> p = Tensor(np.array([1., 0. , 1.]), mindspore.float32)
>>> q = Tensor(np.array([0.2, 0.3 , 1.1]), mindspore.float32)
>>> loss = r_drop_loss(p, q)
>>> print(loss)
0.100136