crf

class LinearChainCrf(num_labels, crf_lr=0.1, with_start_stop_tag=True)

Bases: paddle.fluid.dygraph.layers.Layer

LinearChainCrf is a linear chain Conditional Random Field layer, it can implement sequential dependencies in the predictions. Therefore, it can take context into account whereas a classifier predicts a label for a single sample without considering “neighboring” samples. See https://repository.upenn.edu/cgi/viewcontent.cgi?article=1162&context=cis_papers for reference.

Parameters

• batch_size (int) – The batch size.

• num_labels (int) – The label number.

• crf_lr (float) – The crf layer learning rate.

• with_start_stop_tag (bool) – If set to True, the start tag and stop tag will be considered, the transitions params will be a tensor with shape [num_labels+2, num_labels+2]. Else, the transitions params will be a tensor with shape [num_labels, num_labels].

forward(inputs, lengths)

Computes the normalization in a linear-chain CRF. See http://www.cs.columbia.edu/~mcollins/fb.pdf for reference.

\[ \begin{align}\begin{aligned}F & = logZ(x) = log\sum_y exp(score(x,y))\\score(x,y) & = \sum_i Emit(x_i,y_i) + Trans(y_{i-1}, y_i)\\p(y_i) & = Emit(x_i,y_i), T(y_{i-1}, y_i) = Trans(y_{i-1}, y_i)\end{aligned}\end{align} \]

then we can get:

\[F(1) = log\sum_{y1} exp(p(y_1) + T([START], y1))\]

\[\begin{split}F(2) & = log\sum_{y1}\sum_{y2} exp(p(y_1) + T([START], y1) + p(y_2) + T(y_1,y_2)) \\ & = log\sum_{y2} exp(F(1) + p(y_2) + T(y_1,y_2))\end{split}\]

\[F(...) = ...\]

A recursive formula.

Parameters

• inputs (Tensor) – The input tensor with shape [batch_size, sequence_length, num_tags].

• lengths (Tensor) – The input length with shape [batch_size].

Returns

The normalizers tensor, with shape [batch_size].

Return type

Tensor

gold_score(inputs, labels, lengths)

Computes the unnormalized score for a tag sequence. $$ score(x,y) = sum_i Emit(x_i,y_i) + Trans(y_{i-1}, y_i) $$

Parameters

• inputs (Tensor) – The input tensor with shape [batch_size, sequence_length, num_tags].

• labels (Tensor) – The label tensor with shape [batch_size, sequence_length]

• lengths (Tensor) – The input length with shape [batch_size].

Returns

The unnormalized sequence scores tensor, with shape [batch_size].

Return type

Tensor

class LinearChainCrfLoss(crf)

Bases: paddle.fluid.dygraph.layers.Layer

The negative log-likelihood for linear chain Conditional Random Field (CRF).

let $$ Z(x) = sum_{y’}exp(score(x,y’)) $$, means the sum of all path scores, then we have $$ loss = -logp(y|x) = -log(exp(score(x,y))/Z(x)) = -score(x,y) + logZ(x) $$

Parameters

crf (LinearChainCrf) – The LinearChainCrf network.

forward(inputs, lengths, labels, old_version_labels=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class ViterbiDecoder(transitions, with_start_stop_tag=True)

Bases: paddle.fluid.dygraph.layers.Layer

ViterbiDecoder can decode the highest scoring sequence of tags, it should only be used at test time.

Parameters

• transitions (tensor) – The transition matrix with shape [num_tags, num_tags].

• with_start_stop_tag (bool) – If set to True, the last row and the last column of transitions will be considered as start tag, the the penultimate row and the penultimate column of transitions will be considered as stop tag. Else, all the rows and columns will be considered as the real tag.

forward(inputs, lengths)

Decode the highest scoring sequence of tags.

Parameters

• inputs – The unary emission tensor with shape [batch_size, sequence_length, num_tags].

• length – The input length tensor with shape [batch_size], storing real length of each sequence for correctness.

Returns

The scores tensor containing the score for the Viterbi sequence, with shape [batch_size]. paths: The paths tensor containing the highest scoring tag indices, with shape [batch_size, sequence_length].

Return type

scores