)]}' { "log": [ { "commit": "1cbb15574990e0fa134f2ed5644e9907e314295a", "tree": "855c1c63e848665091180a1415da3cc1650e243c", "parents": [ "18e49583246446996f653974c7bbbdc859dd6776" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Sep 05 15:30:52 2026 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Sep 08 20:56:00 2026 +0200" }, "message": "DocPrompting: Switch GLM-5.3-Flash from Synthetic to OpenRouter\n\nThe endpoint stays OpenAI compatible, so only the base url, the key and\nthe model id change, from hf:zai-org/GLM-5.3-Flash to z-ai/glm-5.3-flash.\nAn hf: id now fails with a hint rather than being sent somewhere it is\nnot known, since it survives in local .Renviron files.\n\nOpenRouter serves the model from two dozen hosts and would pick one by\navailability, which the tests cannot use: the hosts differ in\nquantization, and in how long they take for a documentation prompt by\nmore than two orders of magnitude - one second on Baseten against seven\nminutes on GMICloud. So the hosts are named and everything outside the\nlist ruled out, which keeps a change in the generated code about the\nReadme rather than about which machine answered.\n\nThe list has to hold more than one, and not because a host might be down:\nOpenRouter reaches them through pools it shares between all of its users,\nrather than through our key, and those pools are regularly exhausted. Of\nten prompts pinned to Baseten alone, two came back rate-limited. A\nBaseten key of one\u0027s own, entered under Settings -\u003e Integrations, takes\nthat away for the host at the front; Relace and Z.ai carry what is left,\nat some fifteen seconds and tens of seconds against Baseten\u0027s one. The\ntimeout is five minutes for their sake, and\nRKORAP_OPENROUTER_PROVIDER overrides the whole list.\n\nRelace is the cheapest of the hosts and the quickest of those on shared\ncapacity, but it serves the model in fp4 where the others use fp8, which\nis why it is not at the front: what the tests measure should not depend\non how coarsely a machine was quantized. It solved the collocation task\nin five of five runs all the same.\n\nNeeds OPENROUTER_API_KEY as a masked variable in the GitLab project.\n\nChange-Id: Iba289424970aa933d58798be4c4b07b74c02f2f6\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\n" }, { "commit": "e406514c455b08c3aad5b4961eb93c324b2dbaa4", "tree": "c3ebd3f9a1cfc74bfd8930725477d4b198948cfc", "parents": [ "10bb3bf034d7c3c77d9f37760ac75ec75e664aa7" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Sep 05 15:47:17 2026 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Sep 05 16:49:25 2026 +0200" }, "message": "CI: export the model list, which a project variable would override\n\nA variable defined for the project or the instance takes precedence over\none declared under variables: in this file, so whatever is configured\noutside decided which models the tests prompt - and, in the job that runs\neverything else, whether they run there at all. The comment warning\nagainst that is easy to miss and reaches nobody who sets the variable in\nthe GitLab UI. A shell assignment in the script cannot be overridden that\nway, so the file decides again.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: I475b35750598a916858c55d917d953d27c2ce264\n" }, { "commit": "6957ac9ceaf4b0de7ce1e350ed5f10b4bcfca2ce", "tree": "2f472382e2dbeb87241e9e6577edec0de530a4c1", "parents": [ "323a4d399c1b52943d8cd72d8a7c31fe2dcddc57" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Sep 03 12:18:27 2026 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Sep 03 13:49:12 2026 +0200" }, "message": "CI: run the doc prompting tests in their own, non-blocking job\n\nThe Readme documentation prompting tests query LLMs, so they are neither\ndeterministic nor free: a model can miss an expectation on one run and\nmeet it on the next, and an empty API account fails them regardless of\nthe state of the package. Letting them decide whether the pipeline is\ngreen means red builds for commits that changed nothing related.\n\nThey therefore move into their own job with allow_failure: true, where a\nfailure is visible as what it is, a hint that the documentation may have\nbecome unclear, without blocking anything. That job runs GLM-5.3-Flash,\nwhich solved all three documentation tasks in every trial run, while\ngemini-3.5-flash-lite still missed an expectation every few runs.\n\nKeeping them out of the main job needs more than a testthat filter,\nsince devtools::check() runs the tests as well, and blanking the API\nkeys in the job would not work either, because project level variables\nset in the GitLab UI take precedence over those in the yml. The model\nlist is used as the switch instead: RKORAP_LLM_MODELS unset means the\ndefault models, set but empty means none at all, so the main job\ngenerates no LLM tests in either of its two steps.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: Ia1f29f05a0b8401f61307859e97948f4be977565\n" }, { "commit": "5057f50fb67cdca38c23a04e7fc91f6e0ce2d637", "tree": "854afebbde0fa259b3279c14c304a0be9d3a304d", "parents": [ "3b307c240c828eb5025fdd4ddbadd4eaaaded45e" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Apr 06 16:55:57 2025 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Apr 06 17:58:49 2025 +0200" }, "message": "CI: fix tests\n\nChange-Id: I48db4254cc47bc781967c76d2527fbfb48d95848\n" }, { "commit": "3b307c240c828eb5025fdd4ddbadd4eaaaded45e", "tree": "ef7caff863a96ae7206d4e6c599b0aa2bba8f6ac", "parents": [ "b8766592b9e87930aac06b797ba686ad2dce47ed" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Apr 06 16:32:27 2025 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Apr 06 17:58:49 2025 +0200" }, "message": "CI: drop installation on rolling\n\nChange-Id: I7bb9a066043c308f41df17fa388c0bdc1066de26\n" }, { "commit": "03f2225f42a6b71ba0fd2ee3f936072282539bbd", "tree": "11fd67bd9c28eb4c622dab65df54687f0c862733", "parents": [ "89ce727518b7236841d59999a345ce9002ddcf73" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Apr 06 16:14:50 2025 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Apr 06 17:58:49 2025 +0200" }, "message": "CI: add tests\n\nChange-Id: Ib75bbb9ca5c7e911e96def6964c54f0fe124cb8f\n" }, { "commit": "8a3402d95fec2d99b81bad02de02d4594bc718cd", "tree": "f5fa359166d6680101e62641333375aeab8ada9b", "parents": [ "16e66c3b7d5bc28a90a07a1715fca4d26fee07d0" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Jan 28 14:14:37 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Jan 28 14:14:37 2024 +0100" }, "message": "CI: try to fix include path\n\nChange-Id: Ifd19641ebd17ea4c42b50dcf2e871643891e399e\n" }, { "commit": "16e66c3b7d5bc28a90a07a1715fca4d26fee07d0", "tree": "8731e639993dfd5c5c171abe865b694c873db5dc", "parents": [ "15b49bfc1c61027086c4b93ed3ee38c2362b2a20" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Jan 28 13:31:25 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Jan 28 13:31:25 2024 +0100" }, "message": "CI: Factor out .install-rkorapclient to allow remote use\n\nChange-Id: Iddbd787290529649824c2912a28a6f4ebf99a20e\n" }, { "commit": "810bc19fdaf51dede01526c30d1f5a3cfbc954ed", "tree": "654fc39aababcc84944463fefc16bb5f517046ef", "parents": [ "742719590e2175feaced5fa6b31a5e6a307a8624" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jan 25 15:07:51 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jan 25 15:07:51 2024 +0100" }, "message": "CI: build also on Ubuntu stable\n\nChange-Id: I67ec007b39d97c718c470931d68cf0fd600f2e1a\n" }, { "commit": "742719590e2175feaced5fa6b31a5e6a307a8624", "tree": "11afb8992939a1a1ea4db90c5e036ad46d796372", "parents": [ "9e6661970f756cca53db348be09976bb8656dfae" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jan 25 14:26:58 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jan 25 14:26:58 2024 +0100" }, "message": "CI: no artifacts\n\nChange-Id: I34be30adad2867ecb32cdb4a44bd1c99d3fefdac\n" }, { "commit": "383900793cb4ebf937a6d485b803cedddbcd446b", "tree": "bf4ea5a1335dc9c6fdfc48b6c8fc115c8399158c", "parents": [ "fab6eaf3cb3c752344270fae47b2b5bd17e4ac30" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jan 25 13:33:23 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jan 25 13:33:23 2024 +0100" }, "message": "Add gitlab ci workflow\n\nMainly to check if the installation on ubuntu rolling works.\n\nChange-Id: I19f7fd847fdeaf4c683a40b6f240cde211913779\n" } ] }