modeling¶
-
class
UnifiedTransformerPretrainedModel
(name_scope=None, dtype='float32')[source]¶ Bases:
paddlenlp.transformers.model_utils.PretrainedModel
An abstract class for pretrained UnifiedTransformer models. It provides UnifiedTransformer related
model_config_file
,resource_files_names
,pretrained_resource_files_map
,pretrained_init_configuration
,base_model_prefix
for downloading and loading pretrained models. SeePretrainedModel
for more details.-
base_model_class
¶ alias of
paddlenlp.transformers.unified_transformer.modeling.UnifiedTransformerModel
-
-
class
UnifiedTransformerModel
(vocab_size, hidden_size=768, num_hidden_layers=12, num_attention_heads=12, intermediate_size=3072, hidden_act='gelu', hidden_dropout_prob=0.1, attention_probs_dropout_prob=0.1, normalize_before=True, max_position_embeddings=512, type_vocab_size=2, initializer_range=0.02, unk_token_id=0, pad_token_id=0, bos_token_id=1, eos_token_id=2, mask_token_id=30000)[source]¶ Bases:
paddlenlp.transformers.unified_transformer.modeling.UnifiedTransformerPretrainedModel
-
class
UnifiedTransformerLMHeadModel
(unified_transformer)[source]¶ Bases:
paddlenlp.transformers.unified_transformer.modeling.UnifiedTransformerPretrainedModel
-
forward
(input_ids, token_type_ids, position_ids, attention_mask, masked_positions=None, use_cache=False, cache=None)[source]¶ Defines the computation performed at every call. Should be overridden by all subclasses.
- Parameters
*inputs (tuple) – unpacked tuple arguments
**kwargs (dict) – unpacked dict arguments
-