)]}'
{
  "commit": "b6416be4b6192a6f498acca1e4fc3b05127c7dd6",
  "tree": "38ea04c1560562af4a4a75ad8754dd9ecd36dcf6",
  "parents": [
    "bccdf70f626be8d0efb3b63701caf2e789fe8dbf"
  ],
  "author": {
    "name": "Marc Kupietz",
    "email": "kupietz@ids-mannheim.de",
    "time": "Thu Sep 03 14:37:37 2026 +0200"
  },
  "committer": {
    "name": "Marc Kupietz",
    "email": "kupietz@ids-mannheim.de",
    "time": "Thu Sep 03 14:37:37 2026 +0200"
  },
  "message": "Document that logDice measures salience, not surprise\n\nA collocate that co-occurs less often than expected can still reach a\nlogDice close to that of a strongly attracted one, which is puzzling\nuntil one sees that logDice never compares against an expected\nfrequency. Since it only puts O into the numerator of the Dice\ncoefficient, the difference between the score of a pair and the score it\nwould get if it co-occurred exactly as often as expected is precisely\nthe pointwise mutual information:\n\n  logDice(O) - logDice(E) \u003d log2(O / E) \u003d pmi\n\nThe level a pair starts from thus depends on the marginal frequencies\nalone. Among the collocates of Grund in a 5+5 window, the rare\n\"triftiger\" starts at -4.60 and reaches 3.96, exceeding what is expected\nby a pmi of 8.57, while the frequent \"Berlin\" starts at 5.58 and still\nreaches 3.84 while co-occurring 1.74 bits less often than expected.\n\nThis is a property of the measure rather than a defect: logDice was\ndesigned to be corpus size independent and comparable across corpora,\nnot to express surprise. But since collocationAnalysis() ranks and\nthresholds by it, users should know to drop pairs that are not attested\nmore often than expected, with dplyr::filter(O \u003e E) or a minimum pmi or\nll. This is now explained in a section of the association score\ndocumentation, referred to from the thresholdScore parameter where the\ndefault takes effect, and summarized in the Readme.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: I9059cd064e2a612d9327cc009da0b4856612e41d\n",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "fec7991937b1939625f91cd01ac453ef5b4e10a3",
      "old_mode": 33188,
      "old_path": "R/association-scores.R",
      "new_id": "7807c080b245649993abd7f53b95d545805fb4fc",
      "new_mode": 33188,
      "new_path": "R/association-scores.R"
    },
    {
      "type": "modify",
      "old_id": "2d686368c87ff09c1959ab3793cc9c835d9127da",
      "old_mode": 33188,
      "old_path": "R/collocationAnalysis.R",
      "new_id": "34dc550de667033ea99b50147ffc19edd7aeca3c",
      "new_mode": 33188,
      "new_path": "R/collocationAnalysis.R"
    },
    {
      "type": "modify",
      "old_id": "21de106968519300bb5a15791fa193e83d0683c2",
      "old_mode": 33188,
      "old_path": "Readme.md",
      "new_id": "d7d2e9717c8970f736535cbc801187feceb0be01",
      "new_mode": 33188,
      "new_path": "Readme.md"
    },
    {
      "type": "modify",
      "old_id": "78b5ec4d3e153519f5289e0a08611d16fd5a1ec6",
      "old_mode": 33188,
      "old_path": "man/association-score-functions.Rd",
      "new_id": "ce83ef9886073d97536ee9d61256239083550ae8",
      "new_mode": 33188,
      "new_path": "man/association-score-functions.Rd"
    },
    {
      "type": "modify",
      "old_id": "734a2f635c73f55c59b8246ab373fcc79a81f8b7",
      "old_mode": 33188,
      "old_path": "man/collocationAnalysis-KorAPConnection-method.Rd",
      "new_id": "5a0a43d780a1b55c2835bfdc2c859a2909be9c00",
      "new_mode": 33188,
      "new_path": "man/collocationAnalysis-KorAPConnection-method.Rd"
    }
  ]
}