OpenAI released GPT-4o a conversational chat-bot last week.
It is integrated into the online chat-bot as well as an API for integration with 3rd party systems. Early findings illustrate it is twice as fast and half the cost, with higher rate limits for better performance.
Â
Â
The multi-group query decoder architecture performs well based on zero-shot prompts, which are plain prompts that do not provide context in the prompt to help the LLM improve its response.Â
Â
The API leverages the existing OpenAI chat completion API, which requires an API key to load and interact with the model. Many additional parameters, such as temperature, can be tuned.Â
Â
from openai import OpenAI
OPENAI_API_KEY = "<your-api-key>"
client = OpenAI(api_key=OPENAI_API_KEY)
response = client.chat.completions.create(
  model="gpt-4o-2024-05-13",
  messages=Â
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Who won the world series in 2020?"},
    {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
    {"role": "user", "content": "Where was it played?"}
  ]
)
Â
GPT4o conversational tools are cool tools for users with the ability to consume APIs and work with external systems. For those that have to work on-prem, OpenAI plans to release a desktop version. It is unclear if the desktop version will require APIs for embeddings and prompts or available for work on private data.