The Impact of AI Chatbot Downtime: A Reflection on Reliability and Hybrid Strategies

English translation of an italian post that was originally published on Levysoft.it

Yesterday, Tuesday, June 4, 2024, OpenAI’s ChatGPT chatbot went down for a few hours, but unlike the last time this happened, it wasn’t the only AI provider affected. On Tuesday morning, both Anthropic’s Claude, Perplexity, and Google’s Gemini started experiencing issues, although they were resolved more quickly than ChatGPT’s.

It was indeed unusual for the main cloud-based AI providers to all be inactive at the same time, but it is possible that, after the initial downtime of ChatGPT, Claude, Perplexity (which still uses GPT-4), and Gemini experienced issues due to the intense traffic received in a short period of time as a result of OpenAI’s chatbot outage. It seems that only Microsoft’s Copilot managed to withstand the widespread downtime, likely because it used a separate proprietary cloud infrastructure (and perhaps one more equipped to handle the spike in access) from the LLM cloud services that experienced the interruption.

Between Memes and Reality

If on one hand, Twitter and Reddit exploded with memes, such as:

If on one hand, Twitter and Reddit exploded with memes, such as:

on the other hand, it is also true that every meme has a grain of truth (it used to be said about proverbs… but times have changed) because I personally witnessed scenes of bewilderment as people wandered around the office because ChatGPT was down, and they took a forced break from their programming work while waiting for the service to be restored. I also saw people taking refuge in Claude, while it was still functioning, trying to get answers of the same quality as GPT-4o, or students eager to quickly find solutions to their assignments, who found themselves having to use the good old millennial method: our brains. In just a few months, I’ve noticed a regression in our society, clinging more than ever to generative AI tools, to the point of feeling lost without them.

Closed Source and Open Source Models

This situation should make us reflect on the fact that relying exclusively on online solutions, often based on high-quality closed source LLM models, can pose a risk for business applications that depend entirely on them. The instances of widespread LLM service downtimes should prompt us to consider adopting solutions based on local open source models, such as Llama, Phi, Mistral, or Gemma, which can offer greater stability and control, avoiding service interruptions due to issues with external providers.

While waiting for open source models to reach the quality of closed source ones, the best solution might be to adopt a hybrid strategy. This means combining the use of cloud solutions for certain functionalities (performance, scalability, and constant updates, which can be difficult to replicate locally) with the local implementation of open models for critical components.