Hub Interface#

OFASys#

class ofasys.hub_interface.OFASys(tasks, cfg, model, task_name=None, seed=42)[source]#

OFASys provides an easy-to-use inferface that allows users to load ckpt and use different instructions for inference.

Note

We do not recommend calling the __init__ function directly. Call from_pretrained instead.

Parameters

tasks – the list of tasks.
cfg – configuration object.
model – model object.
task_name – if not None, use specified task, rather than OFATask.
seed – random seed.

build_instruction(instruction_or_template: Union[str, Instruction], data: Optional[Dict[str, Any]] = None, split: str = 'test')[source]#: Fill template with input data.

build_sample(instructions: Union[Instruction, List[Instruction]])[source]#: Convert instruction into batched input data by calling the Generalpreprocess.

cpu()[source]#

Moves all model parameters and buffers to the CPU.

Note

This method modifies the module in-place.

Returns: self
Return type: Module

cuda(device=None)[source]#

Moves all model parameters and buffers to the GPU.

This also makes associated parameters and buffers different objects. So it should be called before constructing optimizer if the module will live on GPU while being optimized.

Note

This method modifies the module in-place.

Parameters: device (int, optional) – if specified, all parameters will be copied to that device
Returns: self
Return type: Module

double()[source]#

Casts all floating point parameters and buffers to double datatype.

Note

This method modifies the module in-place.

Returns: self
Return type: Module

classmethod from_pretrained(model_path, task_name=None, initialize_all_tasks=False)[source]#

Load pretrained OFASys ckpt and config from the given path.

Parameters

model_path – pretrained ckpt path.
task_name – if not None, the specified task will be used, rather than OFATask.
initialize_all_tasks – if True, all pretraining tasks will be initialized.

Return type

OFASys

half()[source]#

Casts all floating point parameters and buffers to half datatype.

Note

This method modifies the module in-place.

Returns: self
Return type: Module

inference(instructions_or_tasks: Union[str, Instruction, List[Union[str, Instruction]]], data: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, closed_set: Optional[Dict[str, Any]] = None, batch_size: int = 1, beam_size: Optional[int] = None, max_len: Optional[int] = None, min_len: Optional[int] = None, len_penalty: Optional[float] = None, unk_penalty: Optional[float] = None, temperature: Optional[float] = None, sampling: Optional[bool] = None, sampling_topk: Optional[int] = None, sampling_topp: Optional[float] = None, no_repeat_ngram_size: Optional[int] = None, return_n_best: Optional[int] = None, max_iter: Optional[int] = None, **extra_gen_kwargs)[source]#

Perform free-style inference according to the instruction using the loaded ckpt. Generator parameters will be transparently passed in. Single sample or list of samples are both supported.

Parameters

instructions_or_tasks – formatted instruction object, or template string, or task name, or List of them.
data – data to fill in slots in instrcution.
closed_set – perform a constraint generation on the given candidates set (default: None).
batch_size – batch size of data (default: 1).
beam_size – beam width (default: 5).
max_len – the maximum length of the generated output (not including end-of-sentence) (default: 256)
min_len – the minimum length of the generated output (not including end-of-sentence) (default: 1)
len_penalty – length penalty, where <1.0 favors shorter, >1.0 favors longer sentences (default: 1.0)
unk_penalty – unknown word penalty, where <0 produces more unks, >0 produces fewer (default: 0.0)
temperature – temperature, where values>1.0 produce more uniform samples and values <1.0 produce sharper samples (default: 1.0)
sampling – whether use sampling instead of beam search (default: false)
sampling_topk – sample from the k most likely tokens at each step (default: -1).
sampling_topp – sample among the smallest set of tokens whose cumulative probability mass exceeds p at each step (default: -1.0)
no_repeat_ngram_size – prevent decoding of ngrams that have already appeared (default: 3).
return_n_best – return best n results (default: -1, which indicates beam_size)
max_iter – max iteration steps for SpeechGenerator (default: 1500).
output_shape – output shape for DiffusionGenerator (default: None).

prepare_for_generation(instruction: Instruction, closed_set: Optional[Dict[str, Any]] = None, **gen_kwargs)[source]#: Parse the instruction and init the generator object for the target slot.