LLMLingua

LLMLingua(prompt_compressor_kwargs: dict = {'model_name': 'microsoft/llmlingua-2-bert-base-multilingual-cased-meetingbank', 'use_llmlingua2': True, 'device_map': 'cpu'}, structured_compression: bool = False)

Compresses text messages using LLMLingua for improved efficiency in processing and response generation.
NOTE: The effectiveness of compression and the resultant token savings can vary based on the content of the messages and the specific configurations used for the PromptCompressor.

Parameters:
NameDescription
prompt_compressor_kwargsType: dict

Default: {‘model_name’
structured_compressionType: bool

Default: False

Instance Methods

compress_text

compress_text(
    self,
    text: str,
    **compression_params
) -> dict[str, Any]
Parameters:
NameDescription
textType: str
**compression_params