An AI agent that learns a company's expert methods from its own reports and raw data files.
Karim Belaid · LMU Munich
An expert needs days to answer it by hand. A general LLM knows the physics but not the team's method, because reports keep only the final numbers.
“What was the maximum chest acceleration of the dummy in this crash test?”
Old reports with final numbers in them, and the raw test files behind those numbers.
It reads a number in the report, writes the question that number answers, and attaches the raw file that holds the data.
A question, its raw file and the answer from the report. The method that gets from file to answer is what the agent must learn.
The same happens for every page and file. Nobody writes or labels a single exercise.
Each circle is an exercise: its answer is known, the method to reach it is not.
The Retriever compares exercises field by field (question, file, measured quantity) and links each one to its closest matches. A thicker link means more similar.
Its answer is unknown, so the agent practises on linked exercises, where the answer is known and every attempt can be checked.
The LLM agent writes and runs code on the exercise's raw file. Its result is more than 10 % off the known answer, so the attempt failed.
A neighbour two links from the question is tried from scratch and solved. Its working method is saved as a skill: a short written procedure.
Retried with that skill as a starting point, the exercise that failed is now solved too.
Practice stops. The agent adapts the new skill to the question's own file and answers it.
This question already has a solved neighbour, so no practice is needed: the stored skill is adapted directly.
With no close solved exercise, the agent works the question out from scratch, like a plain LLM agent. The base model is never retrained, so it keeps all its general ability.
Some exercises are solved, some failed before, most were never tried. Which one is worth practising first?
Value = chance the attempt succeeds × how strongly it links to questions not yet covered. A solved neighbour raises the chance.
Solved with its neighbour's skill, it covers two more questions. Exercises that only help those questions lose value; its failed neighbour gains value, because it can now reuse this skill.
The next-best exercise is tried from scratch and fails. The attempt is spent anyway, and the other values stay the same.
The exercise that failed before now has a solved neighbour. Retried with that skill, it is solved. The budget is used up.
For each question, the agent adapts the skill of its best solved neighbour. The budget went only to exercises linked to these questions.
Provenalways picking the highest value reaches at least 63 % (1 − 1/e) of the best possible plan, on AIMBE's planning score
Thank you. Questions?