)]}' { "log": [ { "commit": "2f65bcf4181630bd55ddc8123579df3ea0c0afd5", "tree": "51a901b5519cd2bb84d621b9fcf119dd56e8935b", "parents": [ "1d76ed9ffa72aaffb0608b31651d793ea56dd51c" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Aug 02 14:04:37 2026 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Aug 02 14:04:37 2026 +0200" }, "message": "Do not lose increments\n\nIndexing threw counts away in two ways, both without a word.\n\nrocksdb was asked to write with low_pri and no_slowdown, which means \"cancel\nthis write when compaction is behind\" and \"do not wait, report it instead\", and\nthe reported status was discarded in both inc() overloads. Measured with 3\nmillion increments against small write buffers: 2,253,066 arrived, 746,934 were\ngone, a quarter of them. The writer no longer deprioritises itself and waits\nwhen it has to, and merge_one() repeats a cancelled write in any case. A\ncancelled write was not applied, so repeating it cannot count twice.\n\nThe second one is the end of a run. The database has no write ahead log, and\nnothing ever closed it, so everything still in a write buffer was lost when the\nindexer ended: of 2 million increments in a process that just exited, none were\nreadable afterwards. close_collocatordb() writes them and closes the database,\nand the destructor does the same.\n\nWhile testing this, two more:\n\n- get_collocators() and get_collocation_scores() finish a collocate when the\n next one starts, so the last one was never returned. Every word was missing\n a collocate, in the API and in the JSON.\n- the array both of them return was allocated with c.size() + sizeof c[0]\n instead of c.size() * sizeof c[0], far too small for more than one entry.\n Callers reading beyond the first entry read whatever was behind it. It is\n terminated with an empty entry now, so that they can tell where it ends.\n\nThe settings are aimed at what indexing does, a long stream of merge operands\nfor the same keys: bigger write buffers, several of them combined before they\nare written, which collapses the operands early, and compaction that is allowed\nto keep up. All of it can be set from the environment, to be able to tune a run\nof several days without recompiling. It also explains why more indexing threads\nstop helping: rocksdb inserts merge operands one writer at a time.\n\nThe test that found all of this is part of the suite now. The expectations of\nthe writing test were corrected, they described the missing collocate.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: I283b073b7f3fd36aa3ce960978e82105b1e44355\n" }, { "commit": "023d4591630964633a5547c0f086109948e36927", "tree": "4b4905f6bc060a0661f7084188df34c49d4f3bb1", "parents": [ "18b230d3fbb5a2ff3350c85d8a9930a585cf5175" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Jul 31 09:46:34 2026 +0900" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Jul 31 09:46:34 2026 +0900" }, "message": "Check the version against the one of the build\n\nThe version test compared against a hard coded 1.4.0, so it fails whenever the\nversion is bumped, as it did for 1.5.0. It uses PROJECT_VERSION now, so that a\nbump no longer has to be repeated here.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: I6d2632f1b2dc6fe8c1052ab7a377f09e78b16a31\n" }, { "commit": "21b964cba850116ee4bfe99950dff28f5c51cf6d", "tree": "eb99ec170ac714c0e5c5ec4231f44834536c9f79", "parents": [ "d26b105331ccab37288bfafe80cbb67bc7c99a3d" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 10 17:10:50 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 10 17:10:50 2024 +0100" }, "message": "Add get_word_frequency function\n\nChange-Id: Idef55f5a8f84c4bcba0a1bfdbe563eb74f26857a\n" }, { "commit": "d26b105331ccab37288bfafe80cbb67bc7c99a3d", "tree": "c29b01622de564840770e65297502ace9e28d417", "parents": [ "fa73154586cd642ee019afd58e8a8eb67f9a444a" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 10 16:56:39 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 10 16:56:39 2024 +0100" }, "message": "Add get_corpus_size function\n\nChange-Id: Ie95b0f14bb8784006e9dc788d02042b6b5b9485a\n" }, { "commit": "fa73154586cd642ee019afd58e8a8eb67f9a444a", "tree": "d0eaa5097ab3539505a4effdd89a2883d1d528f7", "parents": [ "94ea77bb1bfa0b1a3cb98b0bc4652742640cb114" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 12:35:14 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 12:35:14 2024 +0100" }, "message": "bump version to v1.4.0\n\nChange-Id: I8af6c4701d3f3f7cb0d360742cb8b0f3d6bbcd19\n" }, { "commit": "94ea77bb1bfa0b1a3cb98b0bc4652742640cb114", "tree": "742f88d7f54b3b0e51abba66974e18fbc9e57340", "parents": [ "f242ee93649d5eba6fb93261ebd491db4eb1499d" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 12:32:19 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 12:32:19 2024 +0100" }, "message": "add test for command line tool\n\nChange-Id: I5ec7c3284ea2cf01b363089cbc93e063ac034dac\n" }, { "commit": "e889cec5dd3a1b39ede361ac1e7cdd73ccb258f3", "tree": "0a81f43e18838f99d9e9cf9c7ecea7b4b1739b76", "parents": [ "b4ade0f87d230ed6eaeff56afc9c2c0dd9fed033" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 12:08:42 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 12:08:42 2024 +0100" }, "message": "Add `md_nws` MI² score based on nominal window size (\u003d10)\n\nChange-Id: I5e431a8ba0f863191f378d447621f3a10039d122\n" }, { "commit": "b4ade0f87d230ed6eaeff56afc9c2c0dd9fed033", "tree": "b02c613cef65fa9a7a567d294ac7260e1e2824ee", "parents": [ "6e2dbd11065e5e7e0b55ddb249c77ebcfae1063f" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 10:12:22 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 10:12:48 2024 +0100" }, "message": "code clean up in basic tests\n\nChange-Id: If53516597bfe3d667b02c9a8d5806ee9658a1372\n" }, { "commit": "6e2dbd11065e5e7e0b55ddb249c77ebcfae1063f", "tree": "d6bd191a215eb482fdddb6649eff2bfd6aed9dcf", "parents": [ "b9912e386a47251760e84d40fd3ad5c03d42c08b" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 10:11:32 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sat Nov 23 10:11:32 2024 +0100" }, "message": "add test for get word id\n\nChange-Id: Ie908a11e5d8618b289d1a130e0090e59c0a70f99\n" }, { "commit": "6208fd77e556fdc9219868002589b79b616e476e", "tree": "5bb582ff22617cd4babee90dc374e7a274bd46a9", "parents": [ "5ffc474b237885bf81026c63a9a89683691fd5d9" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Nov 15 15:46:19 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Nov 15 15:53:31 2024 +0100" }, "message": "add get_version function\n\nChange-Id: I7f294d3a0ce1acb961c49a8a9bee0f3e8e2824eb\n" }, { "commit": "5ffc474b237885bf81026c63a9a89683691fd5d9", "tree": "9d3450127e825f5806ba4066d613e6681a3bf68f", "parents": [ "0a3a09ad479b9aa06a9ac09c57b8896d46ee8f97" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Nov 15 15:45:12 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Nov 15 15:52:11 2024 +0100" }, "message": "code clean up\n\nChange-Id: I6109c795a6734b040a034e4b5ee6a62cdb56310e\n" }, { "commit": "2441f7c09b781efd7393c7209103a2012796ffba", "tree": "17752dd136c76bad4bf55ebde97b7b838d117912", "parents": [ "442292368bedc801988a57d2c60d0369f7066e26" ], "author": { "name": "„feldmueller“", "email": "„feldmueller@posteo.de“", "time": "Thu Nov 14 16:31:30 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Nov 14 17:22:06 2024 +0100" }, "message": "Fix calculation of total token count\n\nChange-Id: Ib8c2960f276107f7fe5dbf1699ff5b20f1315166\n" }, { "commit": "673bd815dbc883b89fcc858e94c903c5f949eb06", "tree": "9ce490d5991729898610e0e7a49813fb8947299c", "parents": [ "1b09e4dec0b9edd0a04ef93f6d1d76aad1bb5261" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Mar 14 17:27:44 2021 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Mar 14 22:23:54 2021 +0100" }, "message": "Fix installation on CentOS 7\n\nChange-Id: I673e89d567a23376655320ce9edce0fefc0f2d8c\n" }, { "commit": "1b09e4dec0b9edd0a04ef93f6d1d76aad1bb5261", "tree": "aa9a7713bb1391c0f1bcdd6d3f13ac987ad8b7f8", "parents": [ "ed0bfcf605566e1374e446210cfce7b5f2cbe7b3" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Mar 14 15:20:19 2021 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Mar 14 15:20:19 2021 +0100" }, "message": "Add static library test\n\nChange-Id: Id5212457a331811808c5ecfa24ca08498183fd48\n" }, { "commit": "6663f11ebcbc2b676eba5cf6060af959cdebc0bb", "tree": "e63c7ec18e42d5a12b26718c3b446331c9054acc", "parents": [ "b4a683c0e64ca17c1a20cff91427b35a1d1ca533" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Mar 14 09:20:59 2021 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Mar 14 14:44:28 2021 +0100" }, "message": "Add first tests\n\nChange-Id: I7ef95cef912feb2b997e1c7da5eb5a369cbe56b1\n" } ] }