Vertigo 3 is open: the model, the engine, the bench
Vertigo is the model behind ParadoxalAI. We train it ourselves, on our own code, with our own data. Today we put the third version on GitHub: the weights, the engine that runs it, and the 150 questions we grade it on. Apache 2.0, so you can use it, read it, and check what we say about it.
github.com/nohamouazine/vertigo
No install needed to try it: the demo on Hugging Face runs the model in your browser, 98 MB downloaded once, then everything happens on your machine. The weights are also on Hugging Face.
What it is
A small French model. It fits in 98 MB and runs on a laptop with Node.js, no GPU, nothing to install. It answers in French, writes short texts, does small calculations, and when it does not know something it asks to read: it writes a search call, the engine reads the opening of the matching French Wikipedia article, and the answer comes with its source. That is the idea behind every receipt in our apps.
$ npm run ask "Qui a écrit Les Misérables ?"
Il s'agit d'un roman de Victor Hugo publié en 1862.
Source : Les Misérables · fr.wikipedia.org/wiki/Les_MisérablesWhat it scores
We wrote 150 questions in six families: facts, tool calls, calculations, reading a short text, requests like a poem or a message, and small talk. A rule grades each answer. No judge model, no human in the loop, so anyone can rerun it with npm run bench and get the same number.
Vertigo 3 gets 54.7 out of 100. It is good at knowing when to use a tool, decent at reading, weak at poems and at facts it has to recall from memory. The previous version got 51.1. The number is public so that the next one has something to beat. Other small models on the same questions, answers included: the French bench.
What it gets wrong
It is a small model, and it shows. It sometimes asks to search Wikipedia for a question that needs no search. Its poems are four lines when you ask for four, but they rhyme by accident. It can confuse two people with the same first name. When it reads an article that does not answer the question, it may answer beside the point. We did not hide any of this in the bench: the answers it gave are in bench/results.json.
Why open it now
Because the next version is training. It is four times bigger and it will be judged in public, against a well-known small model, in a blind duel anyone can take part in. Opening Vertigo 3 first means the comparison has a starting point you can run yourself.
The duel page is written. Twelve questions, two answers each, you pick the better one without knowing who wrote it, and the tally is public. It opens with the next model.
Who
ParadoxalAI is two people in France. The models are trained on one gaming PC and on free cloud hours. If you use Vertigo for something, write to us: contact@weareparis.eu.