modeling

class BertModel(vocab_size, hidden_size=768, num_hidden_layers=12, num_attention_heads=12, intermediate_size=3072, hidden_act='gelu', hidden_dropout_prob=0.1, attention_probs_dropout_prob=0.1, max_position_embeddings=512, type_vocab_size=16, initializer_range=0.02, pad_token_id=0)

Bases: paddlenlp.transformers.bert.modeling.BertPretrainedModel

forward(input_ids, token_type_ids=None, position_ids=None, attention_mask=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class BertPretrainedModel(name_scope=None, dtype='float32')

Bases: paddlenlp.transformers.model_utils.PretrainedModel

An abstract class for pretrained BERT models. It provides BERT related model_config_file, resource_files_names, pretrained_resource_files_map, pretrained_init_configuration, base_model_prefix for downloading and loading pretrained models. See PretrainedModel for more details.

init_weights(layer)

Initialization hook

base_model_class

alias of paddlenlp.transformers.bert.modeling.BertModel

class BertForPretraining(bert)

Bases: paddlenlp.transformers.bert.modeling.BertPretrainedModel

forward(input_ids, token_type_ids=None, position_ids=None, attention_mask=None, masked_positions=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class BertPretrainingCriterion(vocab_size)

Bases: paddle.fluid.dygraph.layers.Layer

forward(prediction_scores, seq_relationship_score, masked_lm_labels, next_sentence_labels, masked_lm_scale)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class BertPretrainingHeads(hidden_size, vocab_size, activation, embedding_weights=None)

Bases: paddle.fluid.dygraph.layers.Layer

forward(sequence_output, pooled_output, masked_positions=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class BertForSequenceClassification(bert, num_classes=2, dropout=None)

Bases: paddlenlp.transformers.bert.modeling.BertPretrainedModel

Model for sentence (pair) classification task with BERT. :param bert: An instance of BertModel. :type bert: BertModel :param num_classes: The number of classes. Default 2 :type num_classes: int, optional :param dropout: The dropout probability for output of BERT.

If None, use the same value as hidden_dropout_prob of BertModel instance bert. Default None

forward(input_ids, token_type_ids=None, position_ids=None, attention_mask=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class BertForTokenClassification(bert, num_classes=2, dropout=None)

Bases: paddlenlp.transformers.bert.modeling.BertPretrainedModel

forward(input_ids, token_type_ids=None, position_ids=None, attention_mask=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class BertForQuestionAnswering(bert, dropout=None)

Bases: paddlenlp.transformers.bert.modeling.BertPretrainedModel

forward(input_ids, token_type_ids=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments