collate

class Stack(axis=0, dtype=None)

Bases: object

Stack the input data samples to construct the batch. The N input samples must have the same shape/length and will be stacked to construct a batch. :param axis: The axis in the result data along which the input

data are stacked. Default: 0.

Parameters

dtype (str|numpy.dtype, optional) – The value type of the output. If it is set to None, the input data type is used. Default: None.

Example

class Pad(pad_val=0, axis=0, ret_length=None, dtype=None, pad_right=True)

Bases: object

Return a callable that pads and stacks data. :param pad_val: The padding value. Default: 0. :type pad_val: float|int, optional :param axis: The axis to pad the arrays. The arrays will be

padded to the largest dimension at axis. For example, assume the input arrays have shape (10, 8, 5), (6, 8, 5), (3, 8, 5) and the axis is 0. Each input will be padded into (10, 8, 5) and then stacked to form the final output, which has shape（3, 10, 8, 5). Default: 0.

Parameters

• ret_length (bool|numpy.dtype, optional) – If it is bool, indicate whether to return the valid length in the output, and the data type of returned length is int32 if True. If it is numpy.dtype, indicate the data type of returned length. Default: False.

• dtype (numpy.dtype, optional) – The value type of the output. If it is set to None, the input data type is used. Default: None.

• pad_right (bool, optional) – Boolean argument indicating whether the padding direction is right-side. If True, it indicates we pad to the right side, while False indicates we pad to the left side. Default: True.

Example

class Tuple(fn, *args)

Bases: object

Wrap multiple batchify functions together. The input functions will be applied to the corresponding input fields.

Each sample should be a list or tuple containing multiple fields. The i’th batchify function stored in Tuple will be applied on the i’th field.

For example, when data sample is (nd_data, label), you can wrap two batchify functions using Tuple(DataBatchify, LabelBatchify) to batchify nd_data and label correspondingly. :param fn: The batchify functions to wrap. :type fn: list|tuple|callable :param *args: The additional batchify functions to wrap. :type *args: tuple of callable

Example

class Dict(fn)

Bases: object

Wrap multiple batchify functions together. The input functions will be applied to the corresponding input fields.

Each sample should be a dictionary containing multiple fields. Each batchify function with key stored in Dict will be applied on the field which has the same key.

For example, when data sample is {‘tokens’: tokens, ‘labels’: labels), you can wrap two batchify functions using Dict({'tokens': DataBatchify, 'labels': LabelBatchify}) to batchify tokens and labels correspondingly. :param fn: The batchify functions to wrap. :type fn: dict of callable

Example