oai.groq
Create an OpenAI-compatible client using Groq’s API.
Example:
llm_config={
-
"config_list"
- [{ -
"api_type"
- “groq”, -
"model"
- “mixtral-8x7b-32768”, -
"api_key"
- os.environ.get(“GROQ_API_KEY”) } ]}agent = autogen.AssistantAgent(“my_agent”, llm_config=llm_config)
Install Groq’s python library using: pip install —upgrade groq
Resources:
GroqClient
Client for Groq’s API.
__init__
Requires api_key or environment variable to be set
Arguments:
api_key
str - The API key for using Groq (or environment variable GROQ_API_KEY needs to be set)
message_retrieval
Retrieve and return a list of strings or a list of Choice.Message from the response.
NOTE: if a list of Choice.Message is returned, it currently needs to contain the fields of OpenAI’s ChatCompletion Message object, since that is expected for function or tool calling in the rest of the codebase at the moment, unless a custom agent is being used.
get_usage
Return usage summary of the response using RESPONSE_USAGE_KEYS.
parse_params
Loads the parameters for Groq API from the passed in parameters and returns a validated set. Checks types, ranges, and sets defaults
oai_messages_to_groq_messages
Convert messages from OAI format to Groq’s format. We correct for any specific role orders and types.
calculate_groq_cost
Calculate the cost of the completion using the Groq pricing.