The two Swiss institutes of technology and a partner launched the Apertus language model in September. The system was trained using words taken from 1,800 languages, including Swiss German and Romansh. Apertus has been criticised for its mistakes but experts believe it just needs time.
I make my way down the pedestrian avenues of the Swiss Federal Institute of Technology Lausanne (EPFL) campus, where I meet with Antoine Bosselut, specialist in artificial intelligence and multilingual issues for large language models (LLMs). These artificial intelligence systems, packed with billions of units of data, can answer any number of questions, in a similar manner to ChatGPT. The 34-year-old professor, born in France and educated in the United States, knows his fair share about creating machines that can master languages from Tibetan to Romansh. He is one of the fathers of the new Swiss AI model, Apertus.
In early September, the two Swiss institutes of technology and the Swiss National Supercomputing Centre (CSCS) announced the launch of the first multilingual open-source LLM developed in Switzerland. “Apertus represents a major step forward in transparency and diversity for generative artificial intelligence,” its creators say. What makes this LLM different from Llama 4 (developed by Meta), Grok (produced by Elon Musk) or even ChatGPT, which is an entire AI system? The elements making up the Swiss system – its algorithms and computation parameters – are freely accessible. Instructions are provided, whereas ChatGPT (for example) remains an opaque commercial model.
Another difference is that Apertus is not a general system. “Commercial models are not sufficiently specialised for certain specific purposes; however, the more specialised AI is, the stronger it becomes,” explains Bosselut.
Hospitals could use Apertus as a tool – its algorithms and its computation system – for training systems to analyse thousands of radiographs. The AI is capable of comparing data and detecting differences barely visible to the human eye.
The CSCS supercomputer trained Apertus using billions of data items found online. This data constitutes an LLM’s basic lexicon. For this model, data was only used when the owners did not expressly forbid the use of “crawlers”, robots that scrape the web for content, according to the EPFL. “If, for example, the ‘New York Times’ were to block access to its articles from certain crawlers, we would exclude it as a source for our data,” the professor says.
Apertus’s training was based on 15 billion words taken from 1,800 languages (there are approximately 50,000 billion words on the internet). In this case, the creators of the LLM guarantee future users – such as businesses – that the data is reliable in the ethical and legal sense of the term, in contrast to the commercial stakeholders in AI, who refuse to publish their training data.
Large language models tend to focus on the traditional internet languages – English, French, Chinese, Japanese, etc. They use their calculators and algorithms to decode the languages’ structures. This time, however, the Swiss LLM searched for data in languages not often found on the internet, such as Tibetan, Yoruba, Swiss German and Romansh.
Since these languages are not widely “spoken” online, it was necessary to create content from adjacent languages. The idea is that the model should be able to learn Romansh in spite of the scarcity of data, because it has also been trained in Italian and there are similarities between the two languages, explains Bosselut. What is the objective? Apertus has been adopted by a school in Nigeria, for example, which can now develop its lessons on the basis of a language that is largely absent from other models. This corresponds to the EPFL’s aim of “democratising AI”.
To refine it further, the Swiss LLM was subjected to attempts to crack it at hackathons, a type of competition used to test systems. Students have used the tool to create services. It can be an interface for learning Tibetan. Some bright sparks have created a system called “Mut zur Lücke” (daring to have gaps). It tells students which parts of their lessons they can ignore without the risk of failing.
Zurich City Council also uses Apertus. “I am ZüriCityGPT and I know (almost) everything that’s published on the city’s website,” the site announces. With certain limits. How many armed police does the city have? Apertus is “unfortunately unable to help you”, the robot replies. GPT is a little savvier. “Around 1,700 agents are authorised to carry a service weapon, but there are no public sources indicating how many actually carry a weapon on a permanent basis,” it says.
Surprisingly, Apertus is provided without an interface that would let its users write prompts. This was not the objective: the LLM is there to act as source material, according to its creators. However, anyone can try out Apertus via publicai.co, a platform developed by an American non-profit organisation.
In Switzerland, the first feedback on Apertus centred on some glaring errors. “I am learning that Chillon Castle was originally a small, fortified village built on a limestone rock in the middle of the lake,” François Pilet, a journalist from French-speaking Switzerland and one of the founders of the investigation website Gottham City, wrote mockingly on LinkedIn. He is astonished at the price of the program. “At a time when the Swiss federal institutes of technology are tripling fees for foreign students, they had no qualms about spending ten million Swiss francs to finance what has turned out to be a modern art performance!” he complains.
The attack brought reactions from internet users, including Maxime Derian, a French artificial intelligence expert. “American and Chinese open-source models had a head start. So what? The first models from those countries were also far from perfect. Your Swiss model is local. The next versions will be better and it will be useful within two to three years,” the expert predicts. The fact that Apertus makes mistakes is because the model has not yet been sufficiently trained and does not have enough data. Antoine Bosselut shares this view: “We have done the lion’s share of the work, which involved building and training the model. The model is now available free of charge to future users,” the EPFL professor says in the model’s defence.
Comments