autogen.retrieve_utils.split_text_to_chunks

split_text_to_chunks

split_text_to_chunks(
    text: str,
    max_tokens: int = 4000,
    chunk_mode: str = 'multi_lines',
    must_break_at_empty_line: bool = True,
    overlap: int = 0
) -> 

Split a long text into chunks of max_tokens.

Parameters:

Name	Description
text	Type: str
max_tokens	Type: int Default: 4000
chunk_mode	Type: str Default: ‘multi_lines’
must_break_at_empty_line	Type: bool Default: True
overlap	Type: int Default: 0