tokenize_function

Function

This function is defined in the train module.

Tokenizes a batch of text prompts with padding and truncation enabled.

Signature

def tokenize_function(examples: Any, hf_tokenizer: Any, max_length: Any) -> Any

examples

Any

required

No description available.

hf_tokenizer

Any

required

No description available.

max_length

Any

required

No description available.

Any

The result of the operation.