Task

BaseClasses

OFATask

class ofasys.task.base.OFATask(cfg: Optional[TaskConfig] = None, **kwargs)

A Task in OFA-Sys describes an execution logic specifying which parts of the model should be involved in dealing with certain input-output mapping. It contains a declarative multi-modal instruction and a logical plan that supplements model implementation details for a task for certain datasets. Task contains Metrics, Preprocessor, Criterion , and data_iterators.

Parameters

• cfg (TaskConfig) – configuration for Task, including dataset config, preprocess config, instruction config,

• config. (criterion config and evaluation) –

begin_epoch()

Hook function called before the start of each epoch.

begin_valid_epoch(epoch, model)

Hook function called before the start of each validation epoch.

build_criterion(cfg: CriterionConfigs)

Build criterion for the task. If not assigned , LabelSmoothedCrossEntropyCriterion will be use as default.

Note

NOT support criterion with parameters yet.

Parameters

cfg (CriterionConfigs) – config object for Criterion.

Returns

Criterion object.

build_instruction(data: Dict[str, Any], split: str) → Instruction

Initialize an Instruction using a sampled template and format with input data.

Parameters

• data (Dict) – input data.

• split (str) – data split: train, valid, or test.

Returns

formatted instruction.

build_metrics(cfg: MetricConfigs) → List[BaseMetric]

Build all metrics for the task.

Parameters

cfg (MetricConfig) – config object for Metrics

Returns

List of metrics.

build_preprocess(cfg: PreprocessConfig, global_dict)

Build GeneralPreprocess.

Parameters

cfg – config object for Preprocess.

Returns

GeneralPreprocess object.

build_sequence_generator(**gen_kwargs)

Build a SequenceGenerator instance for this task.

Parameters

• models (List[OFAModel]) – ensemble of models

• gen_kwargs (Dict[str, Any]) – extra options to pass through to SequenceGenerator

evaluate(model, sample, **kwargs)

Do inference, and use every metrics to evaluate the inference result.

Parameters

• model (OFAModel) – the model

• sample (dict) – the mini-batch from preprocessor.

Returns

A dict contains compute results from each Metric

inference(model, sample, **kwargs)

Generate result for given sample, and convert the gen_outputs to raw data format using preprocessor.decode().

Parameters

• model (OFAModel) – the model

• sample (dict) – the mini-batch from preprocessor.

Returns:

inference_step(generator, model, sample, **kwargs)

Generate result for given sample.

Parameters

• generator – object of decoding strategy.

• model (OFAModel) – the model

• sample (dict) – the mini-batch from preprocessor.

Returns:

static logging_outputs_can_be_summed(criterion) → bool

Whether the logging outputs returned by train_step and valid_step can be summed across workers prior to calling aggregate_logging_outputs. Setting this to True will improve distributed training speed.

max_positions()

Return the max sentence length allowed by the task.

preprocess(data: Dict[str, Any], split: str) → Dict[str, Any]

Preprocess raw input data for a certain dataset.

Parameters

• data (Dict) – input data.

• split (str) – data split: train, valid, or test.

Returns:

property source_dictionary

Return the source Dictionary.

property target_dictionary

Return the target Dictionary.

train_step(sample, model, optimizer, update_num, ignore_grad=False)

Do forward and backward, and return the loss as computed by criterion for the given model and sample.

Parameters

• sample (dict) – the mini-batch from preprocessor.

• model (OFAModel) – the model

• optimizer (FairseqOptimizer) – the optimizer

• update_num (int) – the current update

• ignore_grad (bool) – multiply loss by 0 if this is set to True

Returns

• the loss

• the sample size, which is used as the denominator for the gradient

• logging outputs to display while training

Return type

tuple

valid_step(sample, model)

Do forward and return the loss as computed by criterion for the given model and sample. If the task has any metrics, will also call evaluate().

Parameters

• sample (dict) – the mini-batch from preprocessor.

• model (OFAModel) – the model

Returns

• the loss

• the sample size, which is used as the denominator for the gradient

• logging outputs to display while training

Return type

tuple

TaskConfig

class ofasys.task.base.TaskConfig(_name: Union[str, NoneType] = None, dataset: ofasys.task.base.DatasetConfig = <factory>, preprocess: ofasys.preprocessor.general.PreprocessConfig = <factory>, instruction: ofasys.task.base.InstructionConfig = <factory>, criterion: ofasys.task.base.CriterionConfigs = <factory>, evaluation: ofasys.task.base.EvaluationConfig = <factory>, max_source_positions: int = 1024, max_target_positions: int = 1024, max_src_length: int = 128, max_tgt_length: int = 30, max_object_length: int = 30, constraint_range: Union[str, NoneType] = None, scst: bool = False, scst_args: str = '{}', diffuser_args: str = '{"scheduler": "DDIMScheduler", "num_inference_steps": 50}')

constraint_range: Optional[str] = None

criterion: CriterionConfigs

dataset: DatasetConfig

diffuser_args: str = '{"scheduler": "DDIMScheduler", "num_inference_steps": 50}'

evaluation: EvaluationConfig

instruction: InstructionConfig

max_object_length: int = 30

max_source_positions: int = 1024

max_src_length: int = 128

max_target_positions: int = 1024

max_tgt_length: int = 30

preprocess: PreprocessConfig

scst: bool = False

scst_args: str = '{}'

update(**kwargs)

DatasetConfig

class ofasys.task.base.DatasetConfig(_name: Union[str, NoneType] = None, train_data: str = '', valid_data: str = '', test_data: str = '', selected_cols: str = '', use_hf_datasets: bool = False, sample_ratios: Any = 1, update_freq: Union[int, List[int]] = <factory>, micro_batch_size: int = 32, micro_valid_batch_size: Union[int, NoneType] = None, fixed_validation_seed: Union[int, NoneType] = 7, num_workers: int = 2, prefetch_factor: int = 5, common_io_capacity: int = 1024, common_io_num_threads: int = 2, seperator: str = '\t', oss_buffer_capacity: int = 64, header: bool = False, cached: bool = False, shuffle: bool = True, text_bin_length: int = 1024, interleaved_multiple_reader: bool = False)

cached: bool = False

common_io_capacity: int = 1024

common_io_num_threads: int = 2

fixed_validation_seed: Optional[int] = 7

header: bool = False

interleaved_multiple_reader: bool = False

micro_batch_size: int = 32

micro_valid_batch_size: Optional[int] = None

num_workers: int = 2

oss_buffer_capacity: int = 64

prefetch_factor: int = 5

sample_ratios: Any = 1

selected_cols: str = ''

seperator: str = '\t'

shuffle: bool = True

test_data: str = ''

text_bin_length: int = 1024

train_data: str = ''

update_freq: Union[int, List[int]]

use_hf_datasets: bool = False

valid_data: str = ''

PreprocessConfig

class ofasys.task.base.PreprocessConfig(_name: Union[str, NoneType] = None, text: ofasys.preprocessor.default.text.TextPreprocessConfig = <factory>, category: ofasys.preprocessor.default.category.CategoryPreprocessConfig = <factory>, image: ofasys.preprocessor.default.image.ImagePreprocessConfig = <factory>, image_vqgan: ofasys.preprocessor.default.image_code.VQGANCodePreprocessConfig = <factory>, box: ofasys.preprocessor.default.box.BoxPreprocessConfig = <factory>, audio: ofasys.preprocessor.default.audio.AudioPreprocessConfig = <factory>, phone: ofasys.preprocessor.default.phone.PhonePreprocessConfig = <factory>, audio_embed: ofasys.preprocessor.default.audio.AudioEmbedPreprocessConfig = <factory>, database: ofasys.preprocessor.default.struct.StructPreprocessConfig = <factory>, imagenet: ofasys.preprocessor.default.image.ImagePreprocessConfig = <factory>, imagepretrain: ofasys.preprocessor.default.image.ImagePreprocessConfig = <factory>, motion_6d: ofasys.preprocessor.default.motion_6d.Motion6dPreprocessConfig = <factory>, table: ofasys.preprocessor.default.struct.StructPreprocessConfig = <factory>, text_phone: ofasys.preprocessor.default.text.TextForPhonePreprocessConfig = <factory>, video: ofasys.preprocessor.default.video.VideoPreprocessConfig = <factory>)

audio: AudioPreprocessConfig

audio_embed: AudioEmbedPreprocessConfig

box: BoxPreprocessConfig

category: CategoryPreprocessConfig

database: StructPreprocessConfig

image: ImagePreprocessConfig

image_vqgan: VQGANCodePreprocessConfig

imagenet: ImagePreprocessConfig

imagepretrain: ImagePreprocessConfig

motion_6d: Motion6dPreprocessConfig

phone: PhonePreprocessConfig

table: StructPreprocessConfig

text: TextPreprocessConfig

text_phone: TextForPhonePreprocessConfig

video: VideoPreprocessConfig

InstructionConfig

class ofasys.task.base.InstructionConfig(_name: Union[str, NoneType] = None, template: Union[str, NoneType] = None, mode: ofasys.configure.constants.Choices = 'auto', decoder_plain_with_loss: bool = False)

decoder_plain_with_loss: bool = False

mode: Choices = 'auto'

template: Optional[str] = None

CriterionConfigs

class ofasys.task.base.CriterionConfigs(_name: Union[str, NoneType] = None, cross_entropy: ofasys.engine.criterion.cross_entropy.CrossEntropyCriterionConfig = <factory>, diffusion_criterion: ofasys.engine.criterion.diffusion_loss.DiffusionCriterionConfig = <factory>, label_smoothed_cross_entropy: ofasys.engine.criterion.label_smoothed_cross_entropy.LabelSmoothedCrossEntropyCriterionConfig = <factory>, ofa_tacotron2: ofasys.engine.criterion.tacotron2_loss.Tacotron2CriterionConfig = <factory>, scst_reward_criterion: ofasys.engine.criterion.scst_loss.ScstRewardCriterionConfig = <factory>, speech_pretrain_loss: ofasys.engine.criterion.speech_pretrain_criterion.SpeechPretrainCriterionConfig = <factory>, speech_to_text_loss: ofasys.engine.criterion.speech_to_text_loss.SpeechtoTextLossConfig = <factory>)

cross_entropy: CrossEntropyCriterionConfig

diffusion_criterion: DiffusionCriterionConfig

label_smoothed_cross_entropy: LabelSmoothedCrossEntropyCriterionConfig

ofa_tacotron2: Tacotron2CriterionConfig

scst_reward_criterion: ScstRewardCriterionConfig

speech_pretrain_loss: SpeechPretrainCriterionConfig

speech_to_text_loss: SpeechtoTextLossConfig

EvaluationConfig

class ofasys.task.base.EvaluationConfig(_name: Union[str, NoneType] = None, metrics: ofasys.task.base.MetricConfigs = <factory>, generator_args: str = '{"beam":5, "max_len_b":32, "no_repeat_ngram_size":3}', eval_print_samples: bool = False, output_dir: str = '')

eval_print_samples: bool = False

generator_args: str = '{"beam":5,"max_len_b":32,"no_repeat_ngram_size":3}'

metrics: MetricConfigs

output_dir: str = ''