Projects

Previous post (code generation benchmark): Benchmarking LLMs for ABAP Live benchmark results (old + new): abap-llm-benchmark.marianzeis.de In my first evaluation (based on the TH Köln benchmark paper), I extended the original setup with additional models and focused on a very concrete question: how well can LLMs generate ABAP code that actually compiles and passes ABAP Unit tests? I also tested SAP’s model ABAP-1, and it performed very poorly for code generation. To be fair: SAP also states this in the documentation. ABAP-1 is primarily meant for explaining ABAP code not for reliably generating full working implementations. ...

Live benchmark results: abap-llm-benchmark.marianzeis.de In a lot of SAP webcasts and webinars, especially around AI, the question comes up very early: which model are you using, and which one do you recommend? For CAP and UI5 the answer is usually pretty simple: use the current best model from Anthropic. If you add good context via MCP servers from the community or SAP, you are basically fine. There is just a lot of public knowledge available, and most of it is in JavaScript/TypeScript, which LLMs handle extremely well. ...

AI assistants already help a lot, but they work best when you can give them good context. In web apps, the most important context is usually the real traffic between frontend and backend: request URLs, payloads, and responses. When debugging, I want to copy that information quickly and completely. And since I mostly work with UI5 and OData, I usually need the full URL plus request and response body. In Chrome DevTools that often means a lot of manuall copy pasting, and if you need more than one request, it gets annoying fast. ...

Finally, there is an MCP server for ABAP. You can use it directly in Eclipse via https://mcp-abap.marianzeis.de/mcp. Setup instructions are here: Eclipse configuration (GitHub Copilot) Why a community ABAP MCP server? SAP has announced an MCP server for ABAP, so why create a community one? The trigger for me was the release of Agent Mode in GitHub Copilot for ADT, which finally makes it possible to edit ABAP code by Copilot. LLM models are already really good, but ABAP knowledge is still a recurring problem. So I wanted an MCP server that is actually tailored for ABAP. ...

I found Vivian Rose’s little web game “List Animals Until Failure” through a Hacker News thread (HN discussion) and immediately got hooked. Then I typed my first few animals… and realized my English animal vocabulary is not exactly perfect. So I did the only sensible thing and made a German version. The UI text is mostly translated with AI help (as all the other code). The part that actually took time was getting the animal names. ...

This post is a mirrored copy of my LinkedIn article, kept here so it remains searchable and independent from external platforms. You can still find the original on LinkedIn: LinkedIn Pulse article. Joule for Consultants costs 250 € and mainly searches SAP Help, SAP Community and SAP for Me (Notes). That is useful, but it stops where real projects start: your own systems, codebase and sensitive data. Even though public assistants like ChatGPT and Claude can already reach SAP Help and SAP Community, the real gap is internal/authenticated data and code. ...