collocationAnalysis() now discards collocates that occur less often than expected by chance. logDice, by which it ranks and thresholds, expresses how salient a pair is rather than how surprising, so a frequent word could appear among the top collocates although the node does not attract it at all: for Grund in a 5+5 window, Berlin reaches a logDice of 3.84, close to triftiger at 3.96, while co-occurring 1.74 bits less often than expected. The new minObservedExpectedRatio parameter defaults to 1 and keeps such pairs out. Raise it to demand a stronger contrast, e.g. 2 for collocates occurring at least twice as often as expected, or set it to 0 for the unfiltered result of earlier versions, e.g. to study repulsion. collocationScoreQuery() is unaffected, as there the pairs to score are given explicitly
changed logDice values: logDice() is now computed as defined by Rychlý (2008), 14 + log2(2 * O / (O1 + O2)), so that its values are comparable to those of Sketch Engine and other tools. It previously multiplied the node frequency by the window size, 14 + log2(2 * O / (w * O1 + O2)), which added a count of window positions to a count of word tokens and made the coefficient asymmetric, so that swapping node and collocate changed the score. Because w * O1 dominated the denominator for a frequent node, rare collocates were penalised: for triftiger as a collocate of Grund in a 5+5 window, logDice was 0.64, below Berlin at 2.03, although Berlin co-occurs with Grund less often than chance predicts. The values are now 3.96 and 3.84. Scores rise by up to log2(w), that is by up to 3.32 for the default context of 5 left and 5 right, so collocationAnalysis() with the default thresholdScore = "logDice" and threshold = 2 is now somewhat more permissive when recursing. Results computed with a total window size of 1, as in the light verb construction example of the Readme, are unaffected. The other association scores are unchanged: they take the window size into account through the expected frequency E, which is correct
ll() now returns NA with a warning instead of a silent NaN where the windows around the node would cover more than the whole corpus (window_size * O1 >= N), which its contingency table cannot represent. This needs a node covering more than 1/w of the corpus, so in DeReKo it is only reachable for the most frequent words combined with a wide window, e.g. der from a context of 18 left and 18 right
collocationAnalysis() now stores the analysis parameters in its cacheAs file and compares them on the next call. If they differ, the cached result is not the one that was asked for, so it is recomputed and the file overwritten, with a warning naming the parameters that differ. This catches the case of a parameter being changed while an old cache file is still lying around. Cache files written by 1.3.0 do not contain the parameters yet and keep being used as they are
dropped the PTXQC dependency, which was imported for two small string functions (lcpCount() and lcsCount(), used by queryStringToLabel()) but pulled in rmzqc, jsonvalidate and V8, and with them the only dependency requiring a libv8 installation. The two functions are now implemented in the package itself, 10 to 65 times faster than the originals, and with unchanged results
cacheAs parameter to collocationAnalysis() for transparent result caching: if the specified RDS file exists, the cached result is returned immediately; otherwise the analysis runs and the result is saved to the file (.rds extension is added automatically if omitted)max_delta_<score>, winner<score>, loser_score<score> columns etc.), including explicit winner/loser webUIRequestUrl columns for association scores, ranks, and percentile ranks. Missing per-label concordance URLs are now derived by replacing the cq parameter of an available row URL with the target label's vc, and unsuffixed consensus winner_webUIRequestUrl / loser_webUIRequestUrl columns are populated when score-based URL choices agree. The names and semantics of these columns may still change without a deprecation cycleimputed, n_imputed and imputed_<label> columns, and reported in verbose mode, so that presence/absence artifacts can be told apart from measured contrasts (filter(!imputed)). See the new "Interpreting multi-VC comparisons" section in ?collocationAnalysis for how to read the comparison columnscollocationAnalysis() now warns when repeated node/collocate/label rows are reduced to their first occurrence for the comparison columns, instead of dropping them silentlycollocationScoreQuery() now accepts a vector of collocates and queries every combination of collocate and virtual corpusKorAPConnection() is now a regular constructor function instead of the S4 class generator, so that its manual page documents how it is actually called. new("KorAPConnection", ...) and all existing calls keep working unchangedfetchAnnotations() morphology so MarMoT and other foundries keep all features from nested spans (#30)xml2, retaining multiple lemma/POS values and improving robustness of token alignmentfetchAnnotations() method added to KorAPQuery class, to fetch annotations for all collected matchesKorAPConnection(verbose=) default via env var KORAP_VERBOSE or R option rkorap.verbose (explicit argument still wins)ci functionauth() function (see ?auth)httr to httr2for all API requestsggplot2 3.6.0 (thanks @teunbrand)corpusQuery resultsmergeDuplicateCollocates functioncitation("RKorAPClient")matchStart and matchEnd columns to corpusQuery results, that contain the start and end position of the match in the textmergeDuplicateCollocates to merge collocation analysis results for different context positionsquery to collocation analysis results\link{} targets missing packagecorpusStats methodspan parameter in collocation analysis functionstextMetadata method, if available, to retrieve also custom metadata for a text based on its siglecorpusQuery method, if availabletextMetadata KorAPConnection method to retrieve all metadata for a text based on its siglewebUiRequestUrl column also to corpusStats results, so that also virtual corpus definitions can be linked to / tested directly in the KorAP UImetadataOnly is set, also tokenized snippets are now retrieved in corpus queries (stored in collectedMatches$tokens)%>% (pipe function)hc_add_onclick_korap_search() now also works with bar, column and pie highchartscollocationAnalysis function:maxRecurse - apply collocation analysis recursively maxRecurse timesaddExamples - If TRUE, examples for instances of collocations will be added in a column example. This makes a difference in particular if 'node' is given as a lemma query.thresholdScore - association score function to use for computing the threshold that is applied for recursive collocation analysis callsthreshold- minimum value of thresholdScore function call to apply collocation analysis recursively## Not run ... ## End(Not run)This was necessary to meet the time limits of automatic CRAN checks under all circumstances and has otherwise no significance,
The change of the minor version from 6 to 7 automatically invalidates your cached query results. However, since the API has not changed, you can continue using the old cached results by copying or renaming your cache folder.
On linux, for example, this can look like this:
mv ~/.cache/R/R.cache/RKorAPClient_0.6 ~/.cache/R/R.cache/RKorAPClient_0.7
or like this:
mv ~/.Rcache/RKorAPClient_0.6 ~/.Rcache/RKorAPClient_0.7
To find the RKorAPClient cache directory for your environment, you can use:
R.cache::getCacheRootPath()
collocationAnalysis method (client-sided)randomizePageOrder to fetch result pages in randomized orderignoreCollocateCase in collocationScoreQuerywithinSpan (default: base/s=s) in collocationScoreQueryverbose=TRUEsmooth added to hc_freq_by_year_ciUnknown or uninitialised column: 'condition'.as consequences.
hc_freq_by_year_ci.demo("writtenVsSpoken")