modeling

class ElectraModel(vocab_size, embedding_size, hidden_size, num_hidden_layers, num_attention_heads, intermediate_size, hidden_act, hidden_dropout_prob, attention_probs_dropout_prob, max_position_embeddings, type_vocab_size, initializer_range, pad_token_id)

Bases: paddlenlp.transformers.electra.modeling.ElectraPretrainedModel

forward(input_ids, token_type_ids=None, position_ids=None, attention_mask=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class ElectraPretrainedModel(name_scope=None, dtype='float32')

Bases: paddlenlp.transformers.model_utils.PretrainedModel

An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models.

init_weights()

Initializes and tie weights if needed.

tie_weights()

Tie the weights between the input embeddings and the output embeddings.

base_model_class

alias of paddlenlp.transformers.electra.modeling.ElectraModel

class ElectraForTotalPretraining(generator, discriminator)

Bases: paddlenlp.transformers.electra.modeling.ElectraPretrainedModel

get_discriminator_inputs(inputs, raw_inputs, gen_logits, gen_labels, use_softmax_sample)

Sample from the generator to create discriminator input.

forward(input_ids=None, token_type_ids=None, position_ids=None, attention_mask=None, raw_input_ids=None, gen_labels=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class ElectraDiscriminator(electra)

Bases: paddlenlp.transformers.electra.modeling.ElectraPretrainedModel

forward(input_ids, token_type_ids=None, position_ids=None, attention_mask=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class ElectraGenerator(electra)

Bases: paddlenlp.transformers.electra.modeling.ElectraPretrainedModel

forward(input_ids=None, token_type_ids=None, position_ids=None, attention_mask=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class ElectraClassificationHead(hidden_size, hidden_dropout_prob, num_classes)

Bases: paddle.fluid.dygraph.layers.Layer

Head for sentence-level classification tasks.

forward(features, **kwargs)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class ElectraForSequenceClassification(electra, num_classes)

Bases: paddlenlp.transformers.electra.modeling.ElectraPretrainedModel

forward(input_ids=None, token_type_ids=None, position_ids=None, attention_mask=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class ElectraForTokenClassification(electra, num_classes)

Bases: paddlenlp.transformers.electra.modeling.ElectraPretrainedModel

forward(input_ids=None, token_type_ids=None, position_ids=None, attention_mask=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class ElectraPretrainingCriterion(vocab_size, gen_weight, disc_weight)

Bases: paddle.fluid.dygraph.layers.Layer

forward(generator_prediction_scores, discriminator_prediction_scores, generator_labels, discriminator_labels, attention_mask)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments