We’ve written earlier about the first round of our Enterprise RAG Challenge. In this friendly challenge different AI Assistants competed in answering questions based on the annual reports of public companies. Teams from the different countries competed in building such assistants. There were even a few commercial solutions in the mix.

We are open sourcing code of four solutions from this leaderboard, including the winning one.

You can find code for the solutions marked with “TTA” (TIMETOACT) on Github. Solutions include description of the approach, code itself, sometimes even the log of failed experiments.

Code is real, without any cleanups and beautification.

In short:

1. Daniel - winning and surprisingly simple solution that used checklist pattern with structured outputs. First place.
   → Check out Daniel's solution
    

2. Felix - multi-agent solution with ChatGPT-4o. 12th place.
   → Check out Felix' solution
    

3. Maria - solution using OpenAI Assistants API. 13th place.
   → Check out Maria's solution
    

4. Pedro - locally-capable solution using openchat-3.5-0106. ninth place.
   → Check out Pedro's solution  
    

What's next?

Next round of Enterprise RAG Challenge will take place later this fall, with a bigger audience. The exact time depends on the organisation process within the TIMEOTACT GROUP Austria. Perhaps, around November.

In the next round, question generator will be rebalanced, so that:

1. There are less questions that don't have an answer (agents must respond N/A to these)

2. There is more variability in the questions, so that “brute force” approach with checklists+structured outputs will not be able to win the competition so easily.

Questionnaires for the participants will also be reworked, so that we all together could learn more about approaches for Enterprise AI that work well in practice.