)]}'
{
  "commit": "ca7510fe2dc04e383beea46d86f644d7c9658b74",
  "tree": "1796b87796de05133bf29aaa1943d48c589471e6",
  "parents": [
    "28f05d1e8639b16845d54ec9379fd116d5e66be3"
  ],
  "author": {
    "name": "Marc Kupietz",
    "email": "kupietz@ids-mannheim.de",
    "time": "Fri Sep 04 17:53:57 2026 +0200"
  },
  "committer": {
    "name": "Marc Kupietz",
    "email": "kupietz@ids-mannheim.de",
    "time": "Fri Sep 04 19:42:38 2026 +0200"
  },
  "message": "Scale the whole log-likelihood table with the window size\n\nca_ll() multiplied only its row total by the window size, leaving cells\nthat do not add up to one sample: the row was counted in window\npositions, the column and the sample size in corpus tokens.\n\nFollowing Evert (2004), which the function cites, the sample consists of\nco-occurrence tokens rather than corpus tokens. With a window of\nwindow_size positions, every occurrence of either word takes part in\nthat many pairs, so the sample size and both marginals scale with it.\n\nThe expected co-occurrence frequency comes out as window_size * f1 * f2\n/ n either way, which is why pmi, md, lfmd and npmi never depended on\nthis, and why it stayed unnoticed: only llr needs the rest of the table.\nThere the values differ, by 0.1% for a frequent collocate up to well\nover 100% for a frequent node in a wide window. Results with a window of\none position are unchanged, the two being identical there.\n\nIt also removes a pathology instead of guarding against it: n -\nwindow_size * f1 could turn negative, at which point the score became\nNaN. window_size * (n - f1) cannot. In this database the case was out of\nreach anyway, needing a word covering more than a tenth of the corpus,\nbut the same table is used elsewhere with freely chosen window sizes.\n\nReported-by: Tim Feldmüller \u003chttps://github.com/feldmueller\u003e\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: Ic396d586da7f6a68b2c911d871ff58032b554d1f\n",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "fd4916ff65179c8e8183a3d6548bbf995ef8f6f1",
      "old_mode": 33188,
      "old_path": "README.md",
      "new_id": "43838e0dafc571479251a0b776f7bffe4490a1d4",
      "new_mode": 33188,
      "new_path": "README.md"
    },
    {
      "type": "modify",
      "old_id": "6eda410273acf5b0855333167be30884e1527f5b",
      "old_mode": 33188,
      "old_path": "src/collocatordb.cc",
      "new_id": "63029c40da6184dab06e173dc24cd5c3dedc0042",
      "new_mode": 33188,
      "new_path": "src/collocatordb.cc"
    },
    {
      "type": "modify",
      "old_id": "d98aac469a761c471ed7364ae1afe0b562808df2",
      "old_mode": 33188,
      "old_path": "tests/basic_test.c",
      "new_id": "f4a003fd39cc83b6a51975fa88c81907ef8b7af6",
      "new_mode": 33188,
      "new_path": "tests/basic_test.c"
    }
  ]
}