​

inference

This is a method of the TrainModel class in the train module.

​

Signature

def inference(instruction: Any, input_text: Any) -> Any

​

Parameters

​

Returns

​

Uses

• FastLanguageModel.for_inference
• to
• apply_chat_template
• generate
• batch_decode

​

Source