man/collocationAnalysis-KorAPConnection-method.Rd

% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/collocationAnalysis.R
\name{collocationAnalysis,KorAPConnection-method}
\alias{collocationAnalysis,KorAPConnection-method}
\alias{collocationAnalysis}
\title{Collocation analysis}
\usage{
\S4method{collocationAnalysis}{KorAPConnection}(
  kco,
  node,
  vc = "",
  lemmatizeNodeQuery = FALSE,
  minOccur = 5,
  leftContextSize = 5,
  rightContextSize = 5,
  topCollocatesLimit = 200,
  searchHitsSampleLimit = 20000,
  ignoreCollocateCase = FALSE,
  withinSpan = ifelse(exactFrequencies, "base/s=s", ""),
  exactFrequencies = TRUE,
  stopwords = append(RKorAPClient::synsemanticStopwords(), node),
  seed = 7,
  expand = length(vc) != length(node),
  maxRecurse = 0,
  addExamples = FALSE,
  thresholdScore = "logDice",
  threshold = 2,
  localStopwords = c(),
  collocateFilterRegex = "^[:alnum:]+-?[:alnum:]*$",
  minObservedExpectedRatio = 1,
  queryMissingScores = FALSE,
  missingScoreQuantile = 0.05,
  vcLabel = NA_character_,
  cacheAs = NULL,
  ...
)
}
\arguments{
\item{kco}{\code{\link[=KorAPConnection]{KorAPConnection()}} object (obtained e.g. from \code{KorAPConnection()}}

\item{node}{target word or query as a single character string}

\item{vc}{character vector describing the virtual corpus or corpora in which the query should be performed. An empty string (default) means the whole corpus, as far as it is license-wise accessible.}

\item{lemmatizeNodeQuery}{if TRUE, node query will be lemmatized, i.e. \verb{x -> [tt/l=x]}}

\item{minOccur}{minimum absolute number of observed co-occurrences to consider a collocate candidate}

\item{leftContextSize}{size of the left context window}

\item{rightContextSize}{size of the right context window}

\item{topCollocatesLimit}{limit analysis to the n most frequent collocates in the search hits sample}

\item{searchHitsSampleLimit}{limit the size of the search hits sample}

\item{ignoreCollocateCase}{logical, set to TRUE if collocate case should be ignored}

\item{withinSpan}{KorAP span specification (see \url{https://korap.ids-mannheim.de/doc/ql/poliqarp-plus?embedded=true#spans}) for collocations to be searched within. Defaults to \code{base/s=s}.}

\item{exactFrequencies}{if FALSE, extrapolate observed co-occurrence frequencies from frequencies in search hits sample, otherwise retrieve exact co-occurrence frequencies}

\item{stopwords}{vector of stopwords not to be considered as collocates}

\item{seed}{seed for random page collecting order}

\item{expand}{if TRUE, \code{node} and \code{vc} parameters are expanded to all of their combinations}

\item{maxRecurse}{apply collocation analysis recursively \code{maxRecurse} times}

\item{addExamples}{If TRUE, examples for instances of collocations will be added in a column \code{example}. This makes a difference in particular if \code{node} is given as a lemma query.}

\item{thresholdScore}{association score function (see \code{\link{association-score-functions}}) to use for computing the threshold that is applied for recursive collocation analysis calls (only applied when \code{maxRecurse > 0})}

\item{threshold}{minimum value of \code{thresholdScore} function call to apply collocation analysis recursively (only applied when \code{maxRecurse > 0}).
Note that the default score, \code{logDice}, expresses how salient a pair is
rather than how surprising, so that a frequent collocate can pass it while
co-occurring less often than expected. \code{minObservedExpectedRatio} keeps
those out. See the "Salience versus surprise" section of
\code{\link{association-score-functions}}.}

\item{localStopwords}{vector of stopwords that will not be considered as collocates in the current function call, but that will not be passed to recursive calls}

\item{collocateFilterRegex}{allow only collocates matching the regular expression}

\item{minObservedExpectedRatio}{minimum ratio of observed to expected co-occurrence
frequency a collocate must reach. Defaults to 1, which keeps only collocates
that occur at least as often as expected by chance, corresponding to a
non-negative \code{pmi}. Without it, frequent words can end up among the top
collocates by \code{logDice} although the node does not attract them at all (see
the "Salience versus surprise" section of
\code{\link{association-score-functions}}). Raise it to demand a stronger
contrast, e.g. 2 for collocates occurring at least twice as often as
expected, or set it to 0 to switch the filter off and obtain the unfiltered
result of earlier versions, e.g. in order to study repulsion.}

\item{queryMissingScores}{if TRUE, attempt to retrieve corpus-based association scores for vc/collocate combinations that would otherwise be imputed, by re-querying the KorAP backend without applying the collocate frequency threshold}

\item{missingScoreQuantile}{lower quantile (evaluated per association measure over the pooled result set) that anchors the adaptive floor used for imputing missing scores between virtual corpora; a robust spread is subtracted from this anchor so the imputed values stay at or below the weakest observed scores. Imputed cells are marked in the \verb{imputed*} columns; see the section on interpreting multi-VC comparisons below}

\item{vcLabel}{optional label override for the current virtual corpus (used internally when named VC collections are expanded)}

\item{cacheAs}{path to an RDS file to keep the result in. If the file exists and records the same call, it is read back instead of contacting the server; otherwise the query is run and its result stored there. Unlike the connection's \code{cache}, this file belongs to the caller, which is what keeps an analysis reproducible once the corpus has grown or the scores have changed. Defaults to \code{NULL} (no file).

The analysis parameters are stored alongside the result. If they differ from
those of the current call, the cached result would not be the one that was
asked for, so it is recomputed and the file overwritten, with a warning
naming the parameters that differ. Pass a different \code{cacheAs} file name
to keep an existing analysis. Cache files written by RKorAPClient 1.3.0 do
not contain the parameters yet and are used as they are.}

\item{...}{more arguments will be passed to \code{\link[=collocationScoreQuery]{collocationScoreQuery()}}}
}
\value{
A tibble where each row represents a candidate collocate for the requested node.
Columns include (depending on the selected association measures):

\itemize{
\item \code{node}, \code{collocate}, \code{vc}, \code{label}: identifiers for the query node, collocate, virtual corpus, and optional label.
\item Frequency and contingency information such as \code{frequency}, \code{O}, \code{O1}, \code{O2}, \code{E}, \code{leftContextSize}, \code{rightContextSize}, and \code{w}.
\item Association measures (e.g. \code{logDice}, \code{ll}, \code{mi}, ...), one column per requested scorer.
\item Per-labelled association scores produced by multi-VC comparisons using the pattern \code{<measure>_<label>}.
\item Ranks per label/measure with the pattern \code{rank_<label>_<measure>} (1 is best) and the corresponding percentile ranks \code{percentile_rank_<label>_<measure>}.
\item Pairwise contrasts for two-label comparisons, e.g. \code{delta_<measure>}, \code{delta_rank_<measure>}, and \code{delta_percentile_rank_<measure>}.
\item Summary columns describing the strongest labels per measure (\code{winner_*}, \code{runner_up_*}, \code{loser_*}, and \code{max_delta_*}), including winner/loser \code{webUIRequestUrl} columns. In multi-VC comparisons, missing per-label concordance URLs are derived from another available row URL for the same \code{node}/\code{collocate} by replacing the \code{cq} parameter with the target label's virtual corpus. Unsuffixed \code{winner_webUIRequestUrl} and \code{loser_webUIRequestUrl} columns are populated only when the score-based URL choices agree.
\item \code{imputed_<label>}: whether the score for that label was imputed rather than observed (see \code{missingScoreQuantile}). \code{n_imputed} counts them, and \code{imputed} is \code{n_imputed > 0}: it describes the node/collocate pair across all labels, not the label of the row it stands in. A row can therefore carry \code{imputed = TRUE} while its own scores are perfectly attested, because the pair is missing from some other virtual corpus - use \code{imputed_<label>} for the row itself. Filter with \code{dplyr::filter(!imputed)} to keep only collocates attested in every compared virtual corpus.
\item Optional helper columns such as \code{query}, \code{example}, or \code{url} when example retrieval is requested.
}
}
\description{
Performs a collocation analysis for the given node (or query)
in the given virtual corpus.
}
\details{
The collocation analysis is currently implemented on the client side, as some of the
functionality is not yet provided by the KorAP backend. Mainly for this reason
it is very slow (several minutes, up to hours), but on the other hand very flexible.
You can, for example, perform the analysis in arbitrary virtual corpora, use complex node queries,
and look for expression-internal collocates using the focus function (see examples and demo).

To increase speed at the cost of accuracy and possible false negatives,
you can decrease searchHitsSampleLimit and/or topCollocatesLimit and/or set exactFrequencies to FALSE.

Note that some outdated non-DeReKo back-ends might not yet support returning tokenized matches (warning issued).
In this case, the client library will fall back to client-side tokenization which might be slightly less accurate.
This might lead to false negatives and to frequencies that differ from corresponding ones acquired via the web
user interface.
}
\section{Interpreting multi-VC comparisons}{


\ifelse{html}{\href{https://lifecycle.r-lib.org/articles/stages.html#experimental}{\figure{lifecycle-experimental.svg}{options: alt='[Experimental]'}}}{\strong{[Experimental]}}

The comparison columns produced when \code{vc} holds more than one virtual corpus
are experimental: their names and semantics may still change in a future
release without a deprecation cycle. Code that has to keep working across
versions should select the columns it needs explicitly.

They are an exploration aid, not a significance test. When reading them, keep
three properties in mind.

\strong{Imputed scores describe presence/absence, not contrast.} A collocate
that passes the \code{minOccur} and \code{topCollocatesLimit} thresholds in one virtual
corpus but not in another has no observed score for the latter. Such cells are
imputed from a floor derived from the pooled result set (see
\code{missingScoreQuantile}), so the corresponding \verb{delta_*} and \verb{max_delta_*}
values measure the distance to that floor rather than an attested difference.
The \code{imputed}, \code{n_imputed} and \verb{imputed_<label>} columns mark these rows;
\code{dplyr::filter(!imputed)} restricts the result to collocates attested
everywhere, and \code{queryMissingScores = TRUE} replaces most imputed cells with
scores actually retrieved from the backend. Mind what \code{imputed} is about: the
pair, not the row. It is \code{TRUE} as soon as one label lacks the collocate, and
stays \code{TRUE} on the rows of the labels where it is attested, which is what
makes \code{dplyr::filter(!imputed)} drop the pair as a whole. Whether the row at
hand rests on an imputed score is what \verb{imputed_<label>} says.

\strong{Per-label columns carry syntactic names.} The label in
\verb{<measure>_<label>}, \verb{rank_<label>_<measure>} and \verb{imputed_<label>} is the one
the caller gave, put through \code{\link[=make.names]{make.names()}}, so that the result stays a well
formed data frame: a virtual corpus named \code{1976-1980} appears as
\code{logDice_X1976.1980}. The \code{label} column and the \verb{winner_*} / \verb{loser_*}
columns keep the name as it was given, so mapping between the two means
applying the same transformation, e.g.
\code{stats::setNames(make.names(labels), labels)}.

\strong{Imputed values are relative to one analysis.} The floor is computed
from the scores present in the result at hand. Analysing a node on its own and
analysing it together with other nodes therefore yield different imputed
values, and deltas involving imputed cells are not comparable across separate
calls. Deltas between observed scores are unaffected.

\strong{Winners carry no uncertainty.} Unlike \code{\link[=ci]{ci()}}, which attaches
confidence intervals to relative frequencies, the \verb{winner_*} / \verb{loser_*}
columns simply order point estimates. A collocate wins by a hair on six
occurrences exactly as decisively as one that wins by a wide margin on
thousands. Consult the observed frequencies (\code{O}, \code{O1}, \code{O2}) and the
\code{webUIRequestUrl} concordance links before drawing conclusions from a
small difference.

Note also that \verb{rank_<label>_<measure>} and
\verb{percentile_rank_<label>_<measure>} are computed within each label, over that
label's own candidate set. Candidate sets usually differ in size between
virtual corpora, so rank-based deltas compare positions in populations of
different sizes.
}

\examples{
\dontrun{

# Find top collocates of "Packung" inside and outside the sports domain.
KorAPConnection(verbose = TRUE) |>
  collocationAnalysis("Packung",
    vc = c("textClass=sport", "textClass!=sport"),
    leftContextSize = 1, rightContextSize = 1, topCollocatesLimit = 20
  ) |>
  dplyr::filter(logDice >= 5)
}

\dontrun{

# Identify the most prominent light verb construction with "in ... setzen".
# Note that, currently, the use of focus function disallows exactFrequencies.
KorAPConnection(verbose = TRUE) |>
  collocationAnalysis("focus(in [tt/p=NN] {[tt/l=setzen]})",
    leftContextSize = 1, rightContextSize = 0, exactFrequencies = FALSE, topCollocatesLimit = 20
  )
}

}
\seealso{
Other collocation analysis functions:
\code{\link{association-score-functions}},
\code{\link{collocationScoreQuery,KorAPConnection-method}},
\code{\link[=synsemanticStopwords]{synsemanticStopwords()}}
}
\concept{collocation analysis functions}