Increase compaction limits
Increases write throughput by factor 10 on compute2
Creating the full collocation db for DeReKo-2025-I only takes less then
one day, now.
Thanks @benhur1999
Change-Id: I68eb18eab2358432aceacc98acbf930a19e76b90
diff --git a/src/collocatordb.cc b/src/collocatordb.cc
index b46d536..eaf0296 100644
--- a/src/collocatordb.cc
+++ b/src/collocatordb.cc
@@ -526,6 +526,10 @@
// options.env->SetBackgroundThreads(16, Env::Priority::LOW); // Increase background threads for low priority
options.create_if_missing = true;
options.merge_operator = std::make_shared<CountMergeOperator>();
+ options.max_background_jobs = max_cores * 2;
+ options.soft_pending_compaction_bytes_limit = 64 * 1024 * 1024; // 64MB
+ options.hard_pending_compaction_bytes_limit = 128 * 1024 * 1024; // 128MB
+
//options.max_successive_merges = 0;
// options.IncreaseParallelism(max_cores); // Utilize all available cores
// options.OptimizeLevelStyleCompaction();