The Training Costs of AI Models Over Time
English translation of an italian post that was originally published on Levysoft.it
Training the most advanced artificial intelligence models requires significant financial investments, with costs rising rapidly. As the necessary computational complexity increases, the expenses for the computational power required to train these systems are soaring at an alarming rate.
According to a report by Visual Capitalist, last year, OpenAI incurred a cost of approximately $78.4 million for training GPT-4, a significant increase compared to Google’s PaLM (540B) model, which cost $12.4 million to train just a year earlier. Even more expensive is Google’s AI Gemini Ultra model, whose training involved an extraordinary cost of $191 million.
In retrospect, the cost (adjusted for inflation) of training Transformer, one of the first AI models developed in 2017 and a milestone in the architecture of many large language models still in use today, amounted to only $930.
