)]}' { "log": [ { "commit": "c48d39095bce6758a351c510055c0771fed33908", "tree": "245dd0dd2aa3fed1725055a73ccbcc8058ec4a78", "parents": [ "b3cd59f0a422e7fcdf78658d4480966f61800bd1" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Aug 21 12:13:45 2026 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Aug 21 12:13:45 2026 +0200" }, "message": "Stop writing a few hundred log lines per request\n\nEvery neighbourhood request printed one line per window position, one per\nvocabulary lookup, the whole JSON of a similar profile and the comings and\ngoings of the worker threads. Together with the crawler traffic that came\nto about a million lines a day on corpora.ids-mannheim.de, which is what\nfilled the log partition of the machine.\n\nThose diagnostics are behind DEREKOVECS_DEBUG now, and the per request\nnarration of the perl side, which repeated what the access log line of the\nsame request already says, is logged at debug level. What is left in the\ndefault configuration is one line per request, plus the user agent of a\nrequest that was turned away as a crawler.\n\nStartup messages, the model mapping and error messages are untouched.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: I4059a1db5ba59a59da6177225c6f66ff53ddcaec\n" }, { "commit": "6f317a90d71d61df065c7a02edf602c857cec26a", "tree": "4b4742756cd78b51c23fb015158b1e5d0fb5e7d4", "parents": [ "d6a163cbe4dbebef25202bbf71fd29b190215e60" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 15:09:54 2026 +0900" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 15:09:54 2026 +0900" }, "message": "Fix collocator lookups for merged models and guard word ranks\n\nw2v.merge was unusable: the server died at startup with \"Undefined subroutine\n\u0026main::mergeVectors\", because mergeVectors was not exported.\n\nWith that fixed, count based collocators and precomputed profiles were looked\nup with unshifted ranks. The collocator db and the profiles belong to the\nprimary model, which mergeVectors puts at [merged_end, words), so its ranks\nhave to be shifted back. The adjustment existed but was dead code, as it read\n$IDS::DeReKoVecs::Read::mergedEnd while the value was assigned to\n$main::mergedEnd. Instead of wiring a second copy of the value, the merge\noffset is now read from the C layer, which is the only place that knows it.\n\nOut of range ranks reach libcollocatordb straight from the request and crash\nthe process there, so a single GET /getClassicCollocators?w\u003d\u003cbeyond vocabulary\u003e\nwas enough to kill a worker. Ranks are validated before use now, both for the\ncollocator db and for the model matrix in getVecs, and out of range ids yield\nan empty result.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: I26ef508e3682cbe00df27d4de5588d1f1493c701\n" }, { "commit": "d6a163cbe4dbebef25202bbf71fd29b190215e60", "tree": "2f2d7ab689f51271152e9d379bb814bc1437be6b", "parents": [ "7662ce124b3dd48e49ddc9e83e932893bc7bb550" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 14:48:10 2026 +0900" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 14:48:10 2026 +0900" }, "message": "Give every request its own window sums array\n\nThe collocator threads accumulated their per window position activation sums\nin a global array. This is not a live bug, because a worker processes requests\none after the other, but it is shared mutable state that only works by\naccident, and the array was allocated uninitialised. Every request allocates\nand zeroes its own array now, so the collocator threads of one request cannot\ninfluence another.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: I9b44ff38871ae2f2729542b8d9f66e489ccc5dd9\n" }, { "commit": "7662ce124b3dd48e49ddc9e83e932893bc7bb550", "tree": "f6cb00dee988bdacb8a43e927f4dddcf2755ffef", "parents": [ "d19578342c63aca95e581b4fe42f1b3001d639b2" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 14:48:10 2026 +0900" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 14:48:10 2026 +0900" }, "message": "Fix getVecsByRanks, used by the collocator SOM\n\ngetVecs was never exported, so POST /derekovecs/getVecsByRanks answered every\nrequest with \"Undefined subroutine \u0026main::getVecs\" and the collocator SOM\nnever got its vectors.\n\nThe route also only existed with the /derekovecs prefix, while the frontend\nderives its base url from the current path, so it could not be reached at all\nfor instances served from the root. Added the prefix independent route like the\nother web service functions have.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: I983b2a34f75892c2302861a0219e4614ffb1d4c9\n" }, { "commit": "af436141ee83e08aeb2c8da5f1368ef51724245c", "tree": "3d276ce85ce0460e9f29036cbcc0c33d1c26cbcc", "parents": [ "0413530655ec966e235cd862daa09bca166e4514" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 14:40:10 2026 +0900" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 14:40:10 2026 +0900" }, "message": "Limit the size of the result caches\n\nThe neighbourhood, classic collocator and similar profile caches grew by one\nentry per queried word and were never evicted, so a long running worker\neventually exhausted the machine\u0027s memory. They are size limited Mojo::Cache\ninstances now, configurable in the new cache section of the configuration\nfile.\n\nThe neighbourhood cache never hit anyway: the lookup used a comma inside the\nhash subscript, i.e. a $; joined multidimensional key, the value was read with\na second key and stored under a third one that did not include\nsearchBaseVocabFirst at all.\n\nStarting the server with -C now really bypasses the caches. It did not before,\nbecause $opt_C was read in IDS::DeReKoVecs::Read but set in main.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: I17b801df54137ea81b660229bf789725de5a2b1b\n" }, { "commit": "0413530655ec966e235cd862daa09bca166e4514", "tree": "b01deedb0f2892ef42cf55eaabb17549ba662d34", "parents": [ "0ab973950366ba7853a894e21676b86fd6b53453" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 14:40:02 2026 +0900" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Thu Jul 30 14:40:02 2026 +0900" }, "message": "Fix memory leaks in the request path\n\nEvery request leaked the target sum array (cutoff * sizeof(float), i.e. 2 MB\nat the default cutoff of 500000), the thread handle array, the target word\nlist and the collocator results of all window threads. getWordNumber() leaked\na word list on every word lookup, too.\n\nOn top of that every string and array returned from C to perl leaked: for a\nchar* return value Inline::C generates sv_setpv(), which copies the buffer but\nnever frees it, and for an AV* return value it generates newRV(), which leaves\nthe array behind with a reference count of one. All functions returning data\nto perl therefore return an SV* now and release their buffers.\n\ngetClassicCollocators() and getCollocationScores() leaked twice, because\nlibcollocatordb hands out strdup()ed strings that were strdup()ed again.\n\nAlso fixes crashes in the same code paths:\n- get_neighbours() divided by zero when called with nosp\u003d1\n- getPosWiseW2VCollocators() wrote before the start of its result buffer for\n empty results, called malloc(0) for models without a syn layer and\n dereferenced a NULL collocator result for multi word input starting with an\n out of dictionary word\n- getBiggestMergedDifferences() sized its buffer for 50 bytes per record,\n which is not enough for long words\n- getSimilarProfiles() wrote to buffer[-1] for empty profiles\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\nChange-Id: Iff4215745f26156408a056de6d867753993082c2\n" }, { "commit": "0ab973950366ba7853a894e21676b86fd6b53453", "tree": "a4db99b5517139b60aa57cdab20754690f814cd7", "parents": [ "57cc90271fcfcb0a7938dc5379e524aca71c359f" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 10 16:16:32 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 10 16:18:46 2024 +0100" }, "message": "Add getPosWiseW2VCollocators -\u003e json to API\n\nResolves #8\n\nChange-Id: Id1eca2896fadc59bebb2786e45fd8a6b3efdca6c\n" }, { "commit": "57cc90271fcfcb0a7938dc5379e524aca71c359f", "tree": "d0f0826bc0b1e4e4f7a824ce336ee044efeacc87", "parents": [ "8a99d64ceb37e3aa33fe6207e088885239ddc8a3" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 10 15:33:36 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 10 15:33:36 2024 +0100" }, "message": "Fix missing export of getPosWiseW2VCollocatorsAsTsv\n\nChange-Id: Id4f7a98d6d35026cd81218aad1934daac306b1a7\n" }, { "commit": "f604691e0e126a14d2e9d6c8ddd3ef834d25fb2d", "tree": "fda43c12c05d13ae3f0500cf44bc3d2e0d025687", "parents": [ "3eeec658a42dfa23434a21d70c7bc3005283af00" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 03 17:21:57 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Dec 03 17:21:57 2024 +0100" }, "message": "disable avx2 for Inline::C for more compatibility\n\nChange-Id: I6e85a0daf024b696979143e54105efe86e529728\n" }, { "commit": "3eeec658a42dfa23434a21d70c7bc3005283af00", "tree": "ab53ee9e956fe5256d952177dee000ecf801a82b", "parents": [ "6835bf8be40067db96f6aeb7c7d327db00ebb17a" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Nov 18 18:30:04 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Nov 18 20:29:05 2024 +0100" }, "message": "Add w2v-\u003ecompare_to parameter and use for count-based delta calcs\n\nChange-Id: Ic53cbc5b3984659b3ebfa6c8ad30fd42850a9fc9\n" }, { "commit": "9f6a19df86491bed130d56789665ea64f10b60fe", "tree": "ad3e60c28dddaef69e82e788125375e17f410e26", "parents": [ "e288d8e5bb61cf6d223a90761d05baf4f73acc0c" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Nov 15 16:22:47 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Nov 15 16:22:47 2024 +0100" }, "message": "switch to collocatordb 1.3.2\n\nChange-Id: I77420c7ef3b9184fa78107c42575c3f162c1ce20\n" }, { "commit": "e288d8e5bb61cf6d223a90761d05baf4f73acc0c", "tree": "21920fb9a7d763a1524b9d01bca6caff61a371dc", "parents": [ "46b352816a6db0c6d521c350b63e9ebd3a664c1e" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Nov 15 16:18:50 2024 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Fri Nov 15 16:18:50 2024 +0100" }, "message": "add some C includes\n\nChange-Id: I692d26e909240538685c10c1b347270db7359174\n" }, { "commit": "41d61c0bc831d007d420fad15ca72bd0d36e3939", "tree": "444033fba4b990399f75701a67cc0572a15f82fa", "parents": [ "bdd779ad9348daef91be2eaaf1eb5af7b97a42ea" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Aug 05 22:11:30 2024 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Aug 05 22:11:30 2024 +0200" }, "message": "Fix build on MacOS\n\nChange-Id: I5fa3c9675aab518f3b4c691c83b7d9defe8228ac\n" }, { "commit": "bdd779ad9348daef91be2eaaf1eb5af7b97a42ea", "tree": "6e3801fbd16b871517563e832d91538dec16024c", "parents": [ "06578a1c822abcb8dbae9df9267fe1725fd4c795" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Aug 05 10:02:29 2024 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Aug 05 10:03:00 2024 +0200" }, "message": "Fix build with gcc 14.1\n\nChange-Id: I29278eb7743458c8f6266f96bc0ba6cdf3dd8d6a\n" }, { "commit": "2c84b5dddbe6e8d9b7c09a7ab5a81cb46eec75e0", "tree": "66df4b154dc463fb7d8bf25db51bbe0c5dc37b48", "parents": [ "ddba9086ca3597459e3bc64c3ee05aba276849ed" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Nov 06 12:56:52 2023 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Nov 06 12:56:52 2023 +0100" }, "message": "Hotfix escaping of double quotes in count-based collocates\n\nChange-Id: I57fa6227a9e20c00553c935bdd2de1bbfcece26a\n" }, { "commit": "043db1500f3e2677770e2701cb96cd996482062d", "tree": "725b1e98f5d845c50c7b9f7d69a6e72502186037", "parents": [ "b36bc7439838d1a0102e85d117b066b9583094eb" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Nov 05 17:47:53 2023 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Nov 05 17:47:53 2023 +0100" }, "message": "Add getVocabSize\n\nChange-Id: I642b39b1186752387da2b555b6e6cb20a0952be5\n" }, { "commit": "3576c62c5b5ef6998bf0799f70b3e419bfded41b", "tree": "4115c549a1a9f163f8fcc49777018823153c6c14", "parents": [ "68e29be001bb9f0e6228aef2233aff284a75e72b" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Nov 05 08:51:58 2023 +0100" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Sun Nov 05 08:51:58 2023 +0100" }, "message": "Fetch downtime calendar on server side\n\nChange-Id: I15fbe6b42489fa233dc0e17c31910e8a06444a3c\n" }, { "commit": "f8d0c609c86758903479bd3e4a376da2ba570fb0", "tree": "65f0d549902f0ba383c74dc7c2638510159d0143", "parents": [ "dd163a371c2a80397d4283d7074fef2083f50459" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Apr 18 19:41:30 2023 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Apr 18 19:41:30 2023 +0200" }, "message": "Drop non-standard kl include paths\n\nChange-Id: I5b86acc047053a87ee862354be59360a65ec3976\n" }, { "commit": "dd163a371c2a80397d4283d7074fef2083f50459", "tree": "88baffa53580ee3ced50eced6fe6b4ad42745628", "parents": [ "42e6d0965c67fe7e0641cb933d760e415f5c597a" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Apr 18 19:36:38 2023 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Apr 18 19:36:38 2023 +0200" }, "message": "export getSimilarity\n\nChange-Id: Ia0b4ad86b6ae1be5136d68d07956e05f6f09a75d\n" }, { "commit": "c82b15fd22b3aabe7a59c41a8ced2d0d66de502f", "tree": "c7536dc9137d2a9914c5913e244d5c66475b759c", "parents": [ "89cb6bc0ca2cbed8209da15d8bb8ce36aefc84c4" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Jul 19 17:36:27 2022 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Jul 19 17:36:27 2022 +0200" }, "message": "Revive -d option to dump vecs as ascii file\n\nChange-Id: Ib78b6ad00af74aaf410544f8c872ac6cf67e90e8\n" }, { "commit": "9ad564d01460ddc46abfcd840d1b33445e53fb0d", "tree": "00d543a473c1fa1f4326dc33e7f75dd74ac8ba25", "parents": [ "5b6a51e9ce1d08effb38f9647e64f58ad0ccbc36" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Apr 12 13:25:22 2022 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Tue Apr 12 13:25:22 2022 +0200" }, "message": "Add missing exports\n\nChange-Id: I2d4a076b69f158b0983c986740a3ff092c96702b\n" }, { "commit": "bf9bac048b1cd4ce56945b4c00a5065e270f9190", "tree": "e7f3dba31d93120f23be6d69ea00d292eb9cffb7", "parents": [ "09a493bb2ae6c7395b531641e2b8f64868647f26" ], "author": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Apr 11 21:16:47 2022 +0200" }, "committer": { "name": "Marc Kupietz", "email": "kupietz@ids-mannheim.de", "time": "Mon Apr 11 23:23:51 2022 +0200" }, "message": "Make project a Perl package\n\nChange-Id: I9cf3c99a1b849d939ace9c9a3b265267e45ab21c\n" } ] }