autogen.Completion

Completion

Completion()

(openai<1) A class for OpenAI completion API.
It also supports: ChatCompletion, Azure OpenAI API.

Class Attributes

cache_path

cache_seed

chat_models

default_search_space

logged_history

max_retry_period

openai_completion_class

optimization_budget

price1K

request_timeout

retry_wait_time

Static Methods

clear_cache

clear_cache(seed: int | None = None, cache_path_root: str | None = '.cache') -> 

Clear cache.

Parameters:

Name	Description
`seed`	The integer identifier for the pseudo seed. If omitted, all caches under cache_path_root will be cleared. Type: int \| None Default: None
`cache_path_root`	Type: str \| None Default: ‘.cache’

cost

cost(response: dict) -> 

Compute the cost of an API call.

Parameters:

Name	Description
`response`	The response from OpenAI API. Type: dict

create

create(
    context: dict[str, Any] | None = None,
    use_cache: bool | None = True,
    config_list: list[dict[str, Any]] | None = None,
    filter_func: Callable[[dict[str, Any], dict[str, Any]], bool] | None = None,
    raise_on_ratelimit_or_timeout: bool | None = True,
    allow_format_str_template: bool | None = False,
    **config
) -> 

Make a completion for a given context.

Parameters:

Name	Description
`context`	The context to instantiate the prompt. It needs to contain keys that are used by the prompt template or the filter function. E.g., `prompt="Complete the following sentence: \{prefix}, context=\{"prefix": "Today I feel"}`. The actual prompt will be: “Complete the following sentence: Today I feel”. Type: dict[str, typing.Any] \| None Default: None
`use_cache`	Whether to use cached responses. Type: bool \| None Default: True
`config_list`	List of configurations for the completion to try. The first one that does not raise an error will be used. Only the differences from the default config need to be provided. E.g., `response = oai.Completion.create( config_list = [ \{ "model": "gpt-4", "api_key": os.environ.get("AZURE_OPENAI_API_KEY"), "api_type": "azure", "base_url": os.environ.get("AZURE_OPENAI_API_BASE"), "api_version": "2024-02-01", }, \{ "model": "gpt-3.5-turbo", "api_key": os.environ.get("OPENAI_API_KEY"), "base_url": "https://api.openai.com/v1", }, \{ "model": "llama-7B", "base_url": "http://127.0.0.1:8080", }, ], prompt="Hi", )` Type: list[dict[str, typing.Any]] \| None Default: None
`filter_func`	A function that takes in the context and the response and returns a boolean to indicate whether the response is valid. E.g., `def yes_or_no_filter(context, config, response): return context.get("yes_or_no_choice", False) is False or any( text in ["Yes.", "No."] for text in oai.Completion.extract_text(response) )` Type: Callable[[dict[str, Any], dict[str, Any]], bool] \| None Default: None
`raise_on_ratelimit_or_timeout`	Whether to raise RateLimitError or Timeout when all configs fail. When set to False, -1 will be returned when all configs fail. Type: bool \| None Default: True
`allow_format_str_template`	Whether to allow format string template in the config. Type: bool \| None Default: False
`**config`	Configuration for the openai API call. This is used as parameters for calling openai API. The “prompt” or “messages” parameter can contain a template (str or Callable) which will be instantiated with the context. Besides the parameters for the openai API call, it can also contain: - `max_retry_period` (int): the total time (in seconds) allowed for retrying failed requests. - `retry_wait_time` (int): the time interval to wait (in seconds) before retrying a failed request. - `cache_seed` (int) for the cache. This is useful when implementing “controlled randomness” for the completion.

extract_text

extract_text(response: dict) -> list[str]

Extract the text from a completion or chat response.

Parameters:

Name	Description
`response`	The response from OpenAI API. Type: dict

Returns:

Type	Description
list[str]	A list of text in the responses.

extract_text_or_function_call

extract_text_or_function_call(response: dict) -> list[str]

Extract the text or function calls from a completion or chat response.

Parameters:

Name	Description
`response`	The response from OpenAI API. Type: dict

Returns:

Type	Description
list[str]	A list of text or function calls in the responses.

instantiate

instantiate(
    template: str | None,
    context: dict[str, Any] | None = None,
    allow_format_str_template: bool | None = False
) -> 

Parameters:

Name	Description
`template`	Type: str \| None
`context`	Type: dict[str, typing.Any] \| None Default: None
`allow_format_str_template`	Type: bool \| None Default: False

print_usage_summary

print_usage_summary() -> dict

Return the usage summary.

set_cache

set_cache(seed: int | None = 41, cache_path_root: str | None = '.cache') -> 

Set cache path.

Parameters:

Name	Description
`seed`	The integer identifier for the pseudo seed. Results corresponding to different seeds will be cached in different places. Type: int \| None Default: 41
`cache_path_root`	Type: str \| None Default: ‘.cache’

start_logging

start_logging(
    history_dict: dict[str, Any] | None = None,
    compact: bool | None = True,
    reset_counter: bool | None = True
) -> 

Start book keeping.

Parameters:

Name	Description
`history_dict`	A dictionary for book keeping. If no provided, a new one will be created. Type: dict[str, typing.Any] \| None Default: None
`compact`	Whether to keep the history dictionary compact. Compact history contains one key per conversation, and the value is a dictionary like: Type: bool \| None Default: True
`reset_counter`	whether to reset the counter of the number of API calls. Type: bool \| None Default: True

stop_logging

stop_logging() -> 

End book keeping.

test

test(
    cls,
    data,
    eval_func=None,
    use_cache=True,
    agg_method='avg',
    return_responses_and_per_instance_result=False,
    logging_level=30,
    **config
) -> 

Evaluate the responses created with the config for the OpenAI API call.

Parameters:

Name	Description
`cls`
`data`	The list of test data points.
`eval_func=None`
`use_cache=True`
`agg_method='avg'`
`return_responses_and_per_instance_result=False`
`logging_level=30`
`**config`

tune

tune(
    cls,
    data: list[dict[str, Any]],
    metric: str,
    mode: str,
    eval_func: Callable,
    log_file_name: str | None = None,
    inference_budget: float | None = None,
    optimization_budget: float | None = None,
    num_samples: int | None = 1,
    logging_level: int | None = 30,
    **config
) -> 

Tune the parameters for the OpenAI API call.
TODO: support parallel tuning with ray or spark.
TODO: support agg_method as in test

Parameters:

Name	Description
`cls`
`data`	The list of data points. Type: list[dict[str, typing.Any]]
`metric`	The metric to optimize. Type: str
`mode`	The optimization mode, “min” or “max. Type: str
`eval_func`	The evaluation function for responses. The function should take a list of responses and a data point as input, and return a dict of metrics. For example, Type: Callable
`log_file_name`	Type: str \| None Default: None
`inference_budget`	Type: float \| None Default: None
`optimization_budget`	Type: float \| None Default: None
`num_samples`	Type: int \| None Default: 1
`logging_level`	Type: int \| None Default: 30
`**config`

API Reference

​Completion

​Class Attributes

​cache_path

​cache_seed

​chat_models

​default_search_space

​logged_history

​max_retry_period

​openai_completion_class

​optimization_budget

​price1K

​request_timeout

​retry_wait_time

​Static Methods

​clear_cache

​cost

​create

​extract_text

​extract_text_or_function_call

​instantiate

​print_usage_summary

​set_cache

​start_logging

​stop_logging

​test

​tune

Completion

Class Attributes

cache_path

cache_seed

chat_models

default_search_space

logged_history

max_retry_period

openai_completion_class

optimization_budget

price1K

request_timeout

retry_wait_time

Static Methods

clear_cache

cost

create

extract_text

extract_text_or_function_call

instantiate

print_usage_summary

set_cache

start_logging

stop_logging

test

tune