How to survive the post-GPT apocalypse
Or what do to if OpenAI really goes sideways
Like may people in tech, I came to work this morning wondering whether I’d made the right call building OpenAI’s APIs into so many of our applications. While anyone paying attention to HuggingFace could tell you there isn’t a shortage of alternative large language models out there, like everyone else, it felt easier to go with OpenAI’s price point than trying to set something else up.
So, waking up to the threat of the entire OpenAI team walking out, it seemed like a good time to start trying the alternatives.
So what’s out there?
My team all like the look of Mistral 7B, but given that we already use AWS for much of our infrastructure, choosing one of the models already available on AWS Bedrock seems like the simplest approach.
Out of the box, I get access to Amazon’s own Titan G3 model. Within an hour of requesting, I have access to all the AI21’s Jurrasic, Anthropic’s Claude, and Meta’s Llama models, as well as Stability AI’s image models.
These are all available on-demand from AWS, with pricing ranging from $0.001 to $0.02 per thousand tokens. Meta’s Llama is the cheapest per token and returns the longest responses. These are all cheaper or comparable to OpenAI pricing.
Setting them up in AWS Bedrock is very straightforward, and within a few minutes, I tested our prompts in their playground. The APIs are similar to the OpenAI ones and have the advantage of being authenticated through AWS, so no additional security measures are required.
How do they perform?
First, I’ve got to be clear that all of our prompts are currently optimized for OpenAI’s chat API, so it would be surprising if they performed well out of the box. One of our use cases is writing questions and answers based on the contents of a TV show. Here’s what we see, with no prompt changes:
GPT - In the TV show, Candy makes a second call to 911 after her house was broken into earlier that morning. The police arrive at her house, and the perpetrator returns.
Jurrasic - In the TV show, Candy called 911 for the second time after her house was broken into. The police had come, but the guy had returned.
Titan - Candy called 911 for the second time because her house had been broken into. The cops came, and the guy who broke in came back.
Claude - Candy called 911 for the second time that day. She reported that her house had been broken into earlier that morning, and the police had come. However, the intruder had since returned to her home.
Llama - I don't know if he's still here. I'm scared. The police arrived and searched the house but did not find anyone. Candy was visibly shaken and asked the officers to stay with her until her husband arrived. The officers agreed and waited with Candy until her husband came home. As the officers were waiting, they heard a strange noise coming from the living room. They went to investigate and found a man hiding under the coffee table. The man was arrested and taken away by the police. The show is about a woman named Candy who calls 911 twice after her house is broken into and a man is found hiding in her living room. The police arrive and search the house, finding the man hiding under the coffee table. The officers wait with Candy until her husband arrives.
So, besides Llama, that tends to write far more than the other models, they all seem pretty usable. We’ve also tested a wider variety of prompts, and different models do seem to be better at different things. For example, asking the bots to “write me a python script to sort a list of integers using bubble sort” got good answers from all but the Titan algorithm that refused to answer.
What next?
The events of this weekend have switched me from being an OpenAI faithful to someone who is, at the very least, interested in understanding how the other models work. What I’ve found is that although the OpenAI models do seem to be further ahead, for most of the everyday use cases we’ve already engineered into our workflows, there are other models we could switch to if OpenAI does have a mass walk-out.
Although the current events are clearly a tragedy for all involved in OpenAI, for those of us who have already become dependent on their technology, it seems there are many other companies out there ready to provide us comparable services.
At least the OpenAI apocalypse won’t be the end of GenerativeAI as we know it…