Bar chart showing cumulative ABAP code generation success rates by model and feedback round.

Benchmarking LLMs for ABAP: Why ABAP-1 Isn't a Code Generator (Yet)

Live benchmark results: abap-llm-benchmark.marianzeis.de In a lot of SAP webcasts and webinars, especially around AI, the question comes up very early: which model are you using, and which one do you recommend? For CAP and UI5 the answer is usually pretty simple: use the current best model from Anthropic. If you add good context via MCP servers from the community or SAP, you are basically fine. There is just a lot of public knowledge available, and most of it is in JavaScript/TypeScript, which LLMs handle extremely well. ...

February 9, 2026 · Marian Zeis
Line chart of average monthly question quality scores (LLM-rated) over time.

Has the Quality of SAP Community Questions Gotten Worse? A Data-Driven Perspective

This post is a mirrored copy of my LinkedIn article, kept here so it remains searchable and independent from external platforms. You can still find the original on LinkedIn: LinkedIn Pulse article. The SAP Community is and will remain the central point of entry for all issues and discussions relating to SAP. Over the past few years (and since the migration at the beginning of the year) I have had a feeling that the quality of questions in the technology area (the area I follow most closely) has gradually declined. ...

December 18, 2024 · Marian Zeis