tree 72193c6a46d2ead305d9765f85019f6f137b6c07
parent 37f96075e1fdcab348c358be3ce2820eb5ac0a50
author Marc Kupietz <kupietz@ids-mannheim.de> 1788429875 +0200
committer Marc Kupietz <kupietz@ids-mannheim.de> 1788436111 +0200

Doc prompting tests: run against several current low cost models

The Readme documentation tests ran against a single hardcoded model,
gemini-2.5-pro. Since the tasks are simple, and since what is being
tested is the Readme rather than the model, cheap and fast models are
the better fit: if such a model cannot follow the Readme, that is a
finding about the Readme. They are also considerably cheaper to run,
in money and in energy.

The model is therefore no longer a constant but a list, run one test per
model and task, and overridable via RKORAP_LLM_MODELS. The defaults are
gemini-3.5-flash-lite, claude-sonnet-5 and GLM-5.3-Flash, the latter via
the OpenAI compatible Synthetic API at https://api.synthetic.new/openai/v1,
authorized with SYNTHETIC_API_KEY. tidyllm's openai provider takes no
custom base url, so that endpoint is queried directly with httr2, which
is a hard dependency anyway.

Each model is skipped individually if its API key is missing, so a
partial set of keys tests whatever it can. Conditions of the account
rather than of the documentation - rate limits, failed authorization
and, newly, exhausted credits - lead to a skip instead of a failure.

Also fixes a problem that was independent of all this: the file called
library(tidyllm) at top level, although tidyllm is only suggested. Where
it is not installed, as when checking with _R_CHECK_DEPENDS_ONLY_, that
made the whole file fail instead of skip. tidyllm is now qualified at
its call sites and skipped on when missing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Change-Id: I8be20b040f7549f39996af1d02afcefa69e63ce2