API Reference¶

This page is generated automatically from the docstrings in the asymcat package, so it always matches the installed code.

Package interface¶

ASymCat: a library for symmetric and asymmetric analysis of categorical co-occurrences

ASymCat is a Python library for obtaining symmetric and asymmetric measures of association between categorical variables in data exploration and description.

CatScorer ¶

CatScorer(cooccs: list[tuple[Any, Any]], smoothing_method: str = 'mle', smoothing_alpha: float = 1.0)

Class for computing categorical co-occurrence scores.

Initialization function.

Parameters¶

cooccs : List[Tuple[Any, Any]] A list of co-occurrence tuples. smoothing_method : str, optional Smoothing method for probability estimation. Options: 'mle', 'laplace', 'ele'. Default is 'mle'. smoothing_alpha : float, optional Smoothing parameter (alpha for Laplace/ELE). Default is 1.0.

mle ¶

mle() -> dict[tuple[Any, Any], tuple[float, float]]

Return an MLE scorer, computing it if necessary. Uses freqprob for robust probability estimation with optional smoothing.

get_smoothed_probabilities ¶

get_smoothed_probabilities() -> dict[str, dict[tuple[Any, Any], float]]

Return smoothed probability estimates using the configured freqprob method.

Returns¶

Dict[str, Dict[Tuple[Any, Any], float]] Dictionary containing 'xy_given_y', 'yx_given_x', 'joint', 'marginal_x', 'marginal_y' probabilities.

pmi_smoothed ¶

pmi_smoothed(normalized: bool = False) -> dict[tuple[Any, Any], tuple[float, float]]

Return a PMI scorer using freqprob smoothing for better numerical stability.

Parameters¶

normalized : bool Whether to return normalized PMI (NPMI) or standard PMI (default: False)

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] PMI scores for each pair (X→Y, Y→X)

pmi ¶

pmi(normalized: bool = False) -> dict[tuple[Any, Any], tuple[float, float]]

Return a PMI scorer.

Parameters¶

normalized : bool Whether to return a normalized PMI or not (default: False)

chi2 ¶

chi2(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return a Chi2 scorer.

Parameters¶

square_ct : bool Whether to return the score over a squared or non squared contingency table (default: True)

chi2_pvalue ¶

chi2_pvalue(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return the p-values of the chi-square test of independence.

This is the significance counterpart to :meth:chi2: for each pair it returns the p-value from scipy.stats.chi2_contingency (a small value indicates a statistically significant association). As with the other symmetric statistical tests, both tuple entries hold the same value.

Parameters¶

square_ct : bool Whether to compute the p-value over a squared (2x2) or non-squared (3x2) contingency table (default: True).

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (p-value, p-value) tuples.

cramers_v ¶

cramers_v(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return a Cramér's V scorer.

Parameters¶

square_ct : bool Whether to return the score over a squared or non squared contingency table (default: True)

fisher ¶

fisher() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Fisher Exact Odds Ratio scorer.

Please note that in scipy's implementation the calculated odds ratio is different from the one found in R. While the latter returns the "conditional Maximum Likelihood Estimate", this implementation computes the more common "unconditional Maximum Likelihood Estimate", which is known to be very slow for contingency tables with large numbers. If the computation is too slow, the similar but inexact chi-square test the the chi2 contingency scorer can be used.

fisher_pvalue ¶

fisher_pvalue() -> dict[tuple[Any, Any], tuple[float, float]]

Return the p-values of Fisher's exact test.

This is the significance counterpart to :meth:fisher: for each pair it returns the (two-sided) p-value from scipy.stats.fisher_exact over the 2x2 contingency table. As with the other symmetric statistical tests, both tuple entries hold the same value.

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (p-value, p-value) tuples.

theil_u ¶

theil_u() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Theil's U uncertainty scorer.

For every pair (x, y) this computes Theil's U in both directions over the subset of co-occurrences where pair[0] == x or pair[1] == y, matching compute_theil_u(Y, X) and compute_theil_u(X, Y) on that subset. The underlying entropies are computed with a vectorized formulation (see :meth:_subset_entropy_components) that evaluates all pairs at once rather than re-filtering the co-occurrence list for each pair.

cond_entropy ¶

cond_entropy() -> dict[tuple[Any, Any], tuple[float, float]]

Return a corrected conditional entropy scorer.

For every pair (x, y) this computes the conditional entropy in both directions over the subset of co-occurrences where pair[0] == x or pair[1] == y, matching conditional_entropy(Y, X) and conditional_entropy(X, Y) on that subset. The entropies come from the same vectorized computation as :meth:theil_u (see :meth:_subset_entropy_components).

tresoldi ¶

tresoldi() -> dict[tuple[Any, Any], tuple[float, float]]

Return a tresoldi asymmetric uncertainty scorer.

This is our intended scorer for alignments.

mutual_information ¶

mutual_information() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Mutual Information scorer.

Computes MI(X;Y) and MI(Y;X) for each symbol pair, measuring the overall statistical dependence between the symbols.

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (MI(X;Y), MI(Y;X)) tuples.

normalized_mutual_information ¶

normalized_mutual_information() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Normalized Mutual Information scorer.

Computes NMI(X;Y) and NMI(Y;X) for each symbol pair, measuring the statistical dependence normalized by joint entropy (range [0,1]).

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (NMI(X;Y), NMI(Y;X)) tuples.

jaccard_index ¶

jaccard_index() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Jaccard Index scorer.

Computes the Jaccard similarity coefficient between the context sets of each symbol pair, measuring overlap in their distributions.

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (J(contexts_x, contexts_y), J(contexts_y, contexts_x)) tuples.

goodman_kruskal_lambda ¶

goodman_kruskal_lambda() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Goodman-Kruskal Lambda scorer.

Computes λ(Y|X) and λ(X|Y) for each symbol pair, measuring the proportional reduction in error when predicting one variable from another.

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (λ(Y|X), λ(X|Y)) tuples.

log_likelihood_ratio ¶

log_likelihood_ratio(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return a Log-Likelihood Ratio (G²) scorer.

Computes the G² statistic as an alternative to Chi-square that works better with small expected frequencies.

Parameters¶

square_ct : bool Whether to return the score over a squared or non squared contingency table (default: True)

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (G²(X,Y), G²(Y,X)) tuples.

log_likelihood_ratio_pvalue ¶

log_likelihood_ratio_pvalue(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return the p-values of the Log-Likelihood Ratio (G²) test.

This is the significance counterpart to :meth:log_likelihood_ratio: under the null hypothesis of independence, G² is asymptotically chi-square distributed, and this returns the corresponding p-value for each pair. As with the other symmetric statistical tests, both tuple entries hold the same value.

Parameters¶

square_ct : bool Whether to compute the p-value over a squared (2x2) or non-squared (3x2) contingency table (default: True).

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (p-value, p-value) tuples.

permutation_pvalue ¶

permutation_pvalue(measure: str, n_permutations: int = 1000, alternative: str = 'greater', seed: int | None = None) -> dict[tuple[Any, Any], tuple[float, float]]

Estimate permutation-based p-values for an association measure.

Unlike the parametric tests (:meth:chi2_pvalue, :meth:fisher_pvalue, :meth:log_likelihood_ratio_pvalue), this works for any per-pair association measure -- including the information-theoretic ones (PMI, mutual information, Theil's U, ...) that have no closed-form null distribution.

The null hypothesis is that the two variables are independent. A null distribution is built by repeatedly shuffling the pairing between the x and y elements of the co-occurrences (which preserves both marginal frequency distributions and the total count while destroying the joint association) and recomputing the measure. For each pair and direction, the p-value is the fraction of permutations whose score is at least as extreme as the observed one, using the (count + 1) / (n_permutations + 1) correction so the estimate is never exactly zero.

Because the co-occurrences are treated as exchangeable, sequences that expand into many correlated co-occurrences (e.g. n-gram collection) make this test slightly anti-conservative; interpret accordingly.

Parameters¶

measure : str Name of the association measure to test; must be one of :attr:RESAMPLABLE_MEASURES. It is computed with its default parameters. n_permutations : int Number of random permutations forming the null distribution (default: 1000). Larger values give finer-grained p-values (the smallest reportable p-value is 1 / (n_permutations + 1)). alternative : str Which tail to test, matching SciPy's convention:

* ``"greater"`` (default): significant when the observed score is
  unusually high -- appropriate for measures where a larger value
  means stronger association (PMI, mutual information, Theil's U,
  chi-square, ...).
* ``"less"``: significant when the observed score is unusually low
  -- appropriate for measures where a smaller value means stronger
  association (e.g. ``cond_entropy``).
* ``"two-sided"``: significant in either tail.

seed : Optional[int] Seed for the random number generator, for reproducible results (default: None, i.e. non-deterministic).

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (p_xy, p_yx) p-values, one per direction, aligned with the measure's own output.

Raises¶

ValueError If measure is not supported, alternative is invalid, or n_permutations is not a positive integer.

bootstrap_ci ¶

bootstrap_ci(measure: str, n_bootstrap: int = 1000, confidence_level: float = 0.95, seed: int | None = None) -> dict[tuple[Any, Any], tuple[tuple[float, float], tuple[float, float]]]

Estimate bootstrap confidence intervals for an association measure.

Whereas :meth:permutation_pvalue tests whether an association differs from independence, this quantifies the sampling uncertainty around the observed score. A bootstrap distribution is built by resampling the co-occurrences with replacement (n draws from the n observed pairs, preserving each pair's x/y link) and recomputing the measure; the confidence interval is taken as the percentile interval of that distribution, per pair and direction.

The point estimate itself is not returned -- obtain it from the measure method directly (e.g. scorer.theil_u()).

Because resampling can drop a rarely-seen symbol from a replicate, a pair may be absent from some replicates; such replicates simply do not contribute to that pair's interval. As with :meth:permutation_pvalue, co-occurrences are treated as exchangeable, so sequence data that expands into many correlated co-occurrences yields intervals that are somewhat too narrow.

Parameters¶

measure : str Name of the association measure; must be one of :attr:RESAMPLABLE_MEASURES. It is computed with its default parameters. n_bootstrap : int Number of bootstrap replicates (default: 1000). confidence_level : float Confidence level in the open interval (0, 1) (default: 0.95 for a 95% interval, i.e. the 2.5th and 97.5th percentiles). seed : Optional[int] Seed for the random number generator, for reproducible results (default: None, i.e. non-deterministic).

Returns¶

Dict[Tuple[Any, Any], Tuple[Tuple[float, float], Tuple[float, float]]] Dictionary mapping symbol pairs to ((xy_low, xy_high), (yx_low, yx_high)) -- a lower/upper bound for each direction, aligned with the measure's own output.

Raises¶

ValueError If measure is not supported, confidence_level is not in (0, 1), or n_bootstrap is not a positive integer.

build_ct ¶

build_ct(observ, square: bool = True) -> list

Build a contingency table from a dictionary of observations.

The contingency table can be either square (2x2) or not (3x2). Non-squared contingency tables include co-occurrences where neither the x or y under investigation occur.

Parameters¶

observ : dict A dictionary of observations, as provided by common.collect_observations(). square : bool, optional Whether to return a square (2x2) or non-square (3x2) contingency table.

Returns¶

cont_table : np.array A contingency table as a numpy array.

collect_alphabets ¶

collect_alphabets(cooccs: list[tuple]) -> tuple

Return the x and y alphabets from a list of co-occurrences.

Parameters¶

cooccs : List[tuple] The list of co-occurrence tuples.

Returns¶

alphabets : tuple A tuple of two elements, the sorted list of symbols in series x and the sorted list of symbols in series y.

Raises¶

ValueError If cooccs is empty or contains invalid tuples. TypeError If cooccs is not a list or contains non-tuple elements.

collect_cooccs ¶

collect_cooccs(seqs: list[list[list[Any] | str] | tuple], order: int | None = None, pad: str = '#') -> list[tuple]

Collects tuples of co-occurring elements in pairs of sequences.

This function is used for collecting co-occurring elements in order to train or improve scorers. Co-occurrences can be collected either considering sequences in their entirety, or by aligned ngrams.

When collecting ngrams, a padding symbol is used internally to guarantee that elements at the extremities of the sequences will be collected the same number of times that other elements. The padding symbol defaults to "#", but given that the tuples where it occurs are automatically removed before returning, it is important that such symbol is not a part of either of the sequence alphabets. It is the user responsability to set an appropriate, non-conflicting pad symbol in case the default is used in the data.

Parameters¶

seqs : List[Union[List[Union[List[Any], str]], tuple]] A list of sequence pairs (each sequence can be a list or string). order : Optional[int] The order of the ngrams to be collected, with None indicating the collection of the entire sequences (default: None). pad : str The symbol for internal padding, which must not conflict with symbols in the sequences (default: "#").

Returns¶

coocc : List[tuple] A list of tuples of co-occurring elements.

Raises¶

ValueError If seqs is empty, contains invalid sequence pairs, or sequences have mismatched lengths. TypeError If seqs is not a list or contains invalid data types.

collect_ngrams ¶

collect_ngrams(seq: list[Any] | str, order: int, pad: str) -> Generator[tuple, None, None]

Function for yielding the ngrams of a sequence.

The sequence is padded so that symbols at the extremities will be repeated the same number of times that other symbols. This operation also guarantees that sequences shorter than the order will be collected.

Parameters¶

seq : Union[List[Any], str] The list/string of elements for ngram collection. order : int The ngram order. pad : str The padding symbol.

Yields¶

ngram : tuple The ngrams of the sequence.

Raises¶

ValueError If order is less than 1 or seq is empty. TypeError If seq is not a list, tuple, or string, or order is not an integer.

collect_observations ¶

collect_observations(cooccs: list[tuple]) -> dict

Build a dictionary of observations for all possible correspondences.

The counts of observations are derived from typical organization in a non-squared contingency table, using indexes 0, 1, and 2, where 0 means that the identity is not considered (so that "00" would refer to all correspondence pairs), 1 that there is a match, and 2 that there is a mismatch. In more detail, given a pair of symbols x and y, the counts of co-occurrences will be:

obs["10"]: number of pairs matching x (with any y)
obs["20"]: number of pairs not matching x (with any y)
obs["01"]: number of pairs matching y (with any x)
obs["02"]: number of pairs not matching y (with any x)
obs["11"]: number of pairs matching x and matching y
obs["12"]: number of pairs matching x and not matching y
obs["21"]: number of pairs not matching x and matching y
obs["22"]: number of pairs not matching x and matching Y

Note that obs["22"] is not the number of pairs where either x or y are mismatching, but where both necessarily mismatch.

When doing x->y, a non-squared contingency table will be

|            |  pair[1]  | pair[1]==y | pair[1]!=y |
| pair[0]==x | obs["10"] | obs["11"]  | obs["12"]  |
| pair[0]!=x | obs["20"] | obs["21"]  | obs["22"]  |

Parameters¶

cooccs : List[tuple] A list of tuples of co-occurring elements.

Returns¶

obs : dict A dictionary of observations per co-occurrence type.

Raises¶

ValueError If cooccs is empty or contains invalid tuples. TypeError If cooccs is not a list.

read_pa_matrix ¶

read_pa_matrix(filename: str, delimiter: str = '\t') -> list[tuple]

Reads a presence-absence matrix, returning as equivalent sequences.

The location must be specified under column name ID.

Parameters¶

filename : str Path to the file to be read. delimiter : str String used as field delimiter (default: " ").

Returns¶

obs_combinations : List[tuple] A list of tuples representing observed combinations.

Raises¶

FileNotFoundError If the specified file does not exist. ValueError If file format is invalid or ID column is missing.

read_sequences ¶

read_sequences(filename: str, cols: list[str] | None = None, col_delim: str = '\t', elem_delim: str = ' ') -> list[list[list[str]]]

Reads parallel sequences, returning them as a list of lists.

Datafiles should have one sequence pair per row, with pairs separated by col_delim and elements delimited by elem_delim, as in:

Orthography                 Segments
E X C L A M A T I O N       ɛ k s k l ʌ m eɪ ʃ ʌ n
C L O S E Q U O T E         k l oʊ z k w oʊ t
D O U B L E Q U O T E       d ʌ b ʌ l k w oʊ t
E N D O F Q U O T E         ɛ n d ʌ v k w oʊ t
E N D Q U O T E             ɛ n d k w oʊ t
I N Q U O T E S             ɪ n k w oʊ t s
Q U O T E                   k w oʊ t
U N Q U O T E               ʌ n k w oʊ t
H A S H M A R K             h æ m ɑ ɹ k

Rows without complete data are not included. The function allows to collect data from multiple columns, but it is inteded to pairwise comparison; if no column names are provided, as per default, the reading will skip the first row (assumed to be header).

Parameters¶

filename : str Path to the file to be read. cols : Optional[List[str]] List of column names to be collected (default: None) col_delim : str String used as field delimiter (default: " "). elem_delim : str String used as element delimiter (default: " ").

Returns¶

data : List[List[List[str]]] A list of lists, where each list contains the data from a row.

Raises¶

FileNotFoundError If the specified file does not exist. ValueError If file is empty or has invalid format. PermissionError If file cannot be read due to permissions.

Scoring: `asymcat.scorer`¶

Defines the various scorers for categorical co-occurrence analysis.

CatScorer ¶

CatScorer(cooccs: list[tuple[Any, Any]], smoothing_method: str = 'mle', smoothing_alpha: float = 1.0)

Class for computing categorical co-occurrence scores.

Initialization function.

Parameters¶

cooccs : List[Tuple[Any, Any]] A list of co-occurrence tuples. smoothing_method : str, optional Smoothing method for probability estimation. Options: 'mle', 'laplace', 'ele'. Default is 'mle'. smoothing_alpha : float, optional Smoothing parameter (alpha for Laplace/ELE). Default is 1.0.

mle ¶

mle() -> dict[tuple[Any, Any], tuple[float, float]]

Return an MLE scorer, computing it if necessary. Uses freqprob for robust probability estimation with optional smoothing.

get_smoothed_probabilities ¶

get_smoothed_probabilities() -> dict[str, dict[tuple[Any, Any], float]]

Return smoothed probability estimates using the configured freqprob method.

Returns¶

Dict[str, Dict[Tuple[Any, Any], float]] Dictionary containing 'xy_given_y', 'yx_given_x', 'joint', 'marginal_x', 'marginal_y' probabilities.

pmi_smoothed ¶

pmi_smoothed(normalized: bool = False) -> dict[tuple[Any, Any], tuple[float, float]]

Return a PMI scorer using freqprob smoothing for better numerical stability.

Parameters¶

normalized : bool Whether to return normalized PMI (NPMI) or standard PMI (default: False)

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] PMI scores for each pair (X→Y, Y→X)

pmi ¶

pmi(normalized: bool = False) -> dict[tuple[Any, Any], tuple[float, float]]

Return a PMI scorer.

Parameters¶

normalized : bool Whether to return a normalized PMI or not (default: False)

chi2 ¶

chi2(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return a Chi2 scorer.

Parameters¶

square_ct : bool Whether to return the score over a squared or non squared contingency table (default: True)

chi2_pvalue ¶

chi2_pvalue(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return the p-values of the chi-square test of independence.

This is the significance counterpart to :meth:chi2: for each pair it returns the p-value from scipy.stats.chi2_contingency (a small value indicates a statistically significant association). As with the other symmetric statistical tests, both tuple entries hold the same value.

Parameters¶

square_ct : bool Whether to compute the p-value over a squared (2x2) or non-squared (3x2) contingency table (default: True).

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (p-value, p-value) tuples.

cramers_v ¶

cramers_v(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return a Cramér's V scorer.

Parameters¶

square_ct : bool Whether to return the score over a squared or non squared contingency table (default: True)

fisher ¶

fisher() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Fisher Exact Odds Ratio scorer.

Please note that in scipy's implementation the calculated odds ratio is different from the one found in R. While the latter returns the "conditional Maximum Likelihood Estimate", this implementation computes the more common "unconditional Maximum Likelihood Estimate", which is known to be very slow for contingency tables with large numbers. If the computation is too slow, the similar but inexact chi-square test the the chi2 contingency scorer can be used.

fisher_pvalue ¶

fisher_pvalue() -> dict[tuple[Any, Any], tuple[float, float]]

Return the p-values of Fisher's exact test.

This is the significance counterpart to :meth:fisher: for each pair it returns the (two-sided) p-value from scipy.stats.fisher_exact over the 2x2 contingency table. As with the other symmetric statistical tests, both tuple entries hold the same value.

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (p-value, p-value) tuples.

theil_u ¶

theil_u() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Theil's U uncertainty scorer.

For every pair (x, y) this computes Theil's U in both directions over the subset of co-occurrences where pair[0] == x or pair[1] == y, matching compute_theil_u(Y, X) and compute_theil_u(X, Y) on that subset. The underlying entropies are computed with a vectorized formulation (see :meth:_subset_entropy_components) that evaluates all pairs at once rather than re-filtering the co-occurrence list for each pair.

cond_entropy ¶

cond_entropy() -> dict[tuple[Any, Any], tuple[float, float]]

Return a corrected conditional entropy scorer.

For every pair (x, y) this computes the conditional entropy in both directions over the subset of co-occurrences where pair[0] == x or pair[1] == y, matching conditional_entropy(Y, X) and conditional_entropy(X, Y) on that subset. The entropies come from the same vectorized computation as :meth:theil_u (see :meth:_subset_entropy_components).

tresoldi ¶

tresoldi() -> dict[tuple[Any, Any], tuple[float, float]]

Return a tresoldi asymmetric uncertainty scorer.

This is our intended scorer for alignments.

mutual_information ¶

mutual_information() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Mutual Information scorer.

Computes MI(X;Y) and MI(Y;X) for each symbol pair, measuring the overall statistical dependence between the symbols.

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (MI(X;Y), MI(Y;X)) tuples.

normalized_mutual_information ¶

normalized_mutual_information() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Normalized Mutual Information scorer.

Computes NMI(X;Y) and NMI(Y;X) for each symbol pair, measuring the statistical dependence normalized by joint entropy (range [0,1]).

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (NMI(X;Y), NMI(Y;X)) tuples.

jaccard_index ¶

jaccard_index() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Jaccard Index scorer.

Computes the Jaccard similarity coefficient between the context sets of each symbol pair, measuring overlap in their distributions.

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (J(contexts_x, contexts_y), J(contexts_y, contexts_x)) tuples.

goodman_kruskal_lambda ¶

goodman_kruskal_lambda() -> dict[tuple[Any, Any], tuple[float, float]]

Return a Goodman-Kruskal Lambda scorer.

Computes λ(Y|X) and λ(X|Y) for each symbol pair, measuring the proportional reduction in error when predicting one variable from another.

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (λ(Y|X), λ(X|Y)) tuples.

log_likelihood_ratio ¶

log_likelihood_ratio(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return a Log-Likelihood Ratio (G²) scorer.

Computes the G² statistic as an alternative to Chi-square that works better with small expected frequencies.

Parameters¶

square_ct : bool Whether to return the score over a squared or non squared contingency table (default: True)

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (G²(X,Y), G²(Y,X)) tuples.

log_likelihood_ratio_pvalue ¶

log_likelihood_ratio_pvalue(square_ct: bool = True) -> dict[tuple[Any, Any], tuple[float, float]]

Return the p-values of the Log-Likelihood Ratio (G²) test.

This is the significance counterpart to :meth:log_likelihood_ratio: under the null hypothesis of independence, G² is asymptotically chi-square distributed, and this returns the corresponding p-value for each pair. As with the other symmetric statistical tests, both tuple entries hold the same value.

Parameters¶

square_ct : bool Whether to compute the p-value over a squared (2x2) or non-squared (3x2) contingency table (default: True).

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (p-value, p-value) tuples.

permutation_pvalue ¶

permutation_pvalue(measure: str, n_permutations: int = 1000, alternative: str = 'greater', seed: int | None = None) -> dict[tuple[Any, Any], tuple[float, float]]

Estimate permutation-based p-values for an association measure.

Unlike the parametric tests (:meth:chi2_pvalue, :meth:fisher_pvalue, :meth:log_likelihood_ratio_pvalue), this works for any per-pair association measure -- including the information-theoretic ones (PMI, mutual information, Theil's U, ...) that have no closed-form null distribution.

The null hypothesis is that the two variables are independent. A null distribution is built by repeatedly shuffling the pairing between the x and y elements of the co-occurrences (which preserves both marginal frequency distributions and the total count while destroying the joint association) and recomputing the measure. For each pair and direction, the p-value is the fraction of permutations whose score is at least as extreme as the observed one, using the (count + 1) / (n_permutations + 1) correction so the estimate is never exactly zero.

Because the co-occurrences are treated as exchangeable, sequences that expand into many correlated co-occurrences (e.g. n-gram collection) make this test slightly anti-conservative; interpret accordingly.

Parameters¶

measure : str Name of the association measure to test; must be one of :attr:RESAMPLABLE_MEASURES. It is computed with its default parameters. n_permutations : int Number of random permutations forming the null distribution (default: 1000). Larger values give finer-grained p-values (the smallest reportable p-value is 1 / (n_permutations + 1)). alternative : str Which tail to test, matching SciPy's convention:

* ``"greater"`` (default): significant when the observed score is
  unusually high -- appropriate for measures where a larger value
  means stronger association (PMI, mutual information, Theil's U,
  chi-square, ...).
* ``"less"``: significant when the observed score is unusually low
  -- appropriate for measures where a smaller value means stronger
  association (e.g. ``cond_entropy``).
* ``"two-sided"``: significant in either tail.

seed : Optional[int] Seed for the random number generator, for reproducible results (default: None, i.e. non-deterministic).

Returns¶

Dict[Tuple[Any, Any], Tuple[float, float]] Dictionary mapping symbol pairs to (p_xy, p_yx) p-values, one per direction, aligned with the measure's own output.

Raises¶

ValueError If measure is not supported, alternative is invalid, or n_permutations is not a positive integer.

bootstrap_ci ¶

bootstrap_ci(measure: str, n_bootstrap: int = 1000, confidence_level: float = 0.95, seed: int | None = None) -> dict[tuple[Any, Any], tuple[tuple[float, float], tuple[float, float]]]

Estimate bootstrap confidence intervals for an association measure.

Whereas :meth:permutation_pvalue tests whether an association differs from independence, this quantifies the sampling uncertainty around the observed score. A bootstrap distribution is built by resampling the co-occurrences with replacement (n draws from the n observed pairs, preserving each pair's x/y link) and recomputing the measure; the confidence interval is taken as the percentile interval of that distribution, per pair and direction.

The point estimate itself is not returned -- obtain it from the measure method directly (e.g. scorer.theil_u()).

Because resampling can drop a rarely-seen symbol from a replicate, a pair may be absent from some replicates; such replicates simply do not contribute to that pair's interval. As with :meth:permutation_pvalue, co-occurrences are treated as exchangeable, so sequence data that expands into many correlated co-occurrences yields intervals that are somewhat too narrow.

Parameters¶

measure : str Name of the association measure; must be one of :attr:RESAMPLABLE_MEASURES. It is computed with its default parameters. n_bootstrap : int Number of bootstrap replicates (default: 1000). confidence_level : float Confidence level in the open interval (0, 1) (default: 0.95 for a 95% interval, i.e. the 2.5th and 97.5th percentiles). seed : Optional[int] Seed for the random number generator, for reproducible results (default: None, i.e. non-deterministic).

Returns¶

Dict[Tuple[Any, Any], Tuple[Tuple[float, float], Tuple[float, float]]] Dictionary mapping symbol pairs to ((xy_low, xy_high), (yx_low, yx_high)) -- a lower/upper bound for each direction, aligned with the measure's own output.

Raises¶

ValueError If measure is not supported, confidence_level is not in (0, 1), or n_bootstrap is not a positive integer.

conditional_entropy ¶

conditional_entropy(x_symbols: list[Any], y_symbols: list[Any]) -> float

Computes the entropy of x given y.

Parameters¶

x_symbols : List[Any] A list of all observed x symbols. y_symbols : List[Any] A list of all observed y symbols.

Returns¶

float The conditional entropy of x given y.

compute_cramers_v ¶

compute_cramers_v(cont_table: list[list[float]]) -> float

Compute Cramer's V from a contingency table.

Parameters¶

cont_table : np.array The contingency table for computation.

Returns¶

float The Cramér's V measure for the given contingency table.

compute_pmi ¶

compute_pmi(p_x: float, p_y: float, p_xy: float, normalized: bool, limit: float = 1e-06) -> float

Compute the Pointwise Mutual Information.

Parameters¶

p_x : float The probability of x, p(x). p_y : float The probability of y, p(y). p_xy : float The probability of xy, p(xy). normalized : bool Whether to return the normalized Pointwise Mutual Information in range [-1, 1]. limit : float, optional The value to use for computation when p_xy is 0.0 (i.e., non observed), as the limit of PMI to zero (default: 1e-6).

Returns¶

float The Pointwise Mutual Information.

compute_theil_u ¶

compute_theil_u(x_symbols: list[Any], y_symbols: list[Any]) -> float

Compute the uncertainty coefficient Theil's U.

Parameters¶

x_symbols : List[Any] The list of observed symbols in series x. y_symbols : List[Any] The list of observed symbols in series y.

Returns¶

float The uncertainty coefficient given x and y.

compute_mutual_information ¶

compute_mutual_information(x_symbols: list[Any], y_symbols: list[Any]) -> float

Compute the mutual information between X and Y.

MI(X;Y) = Σ_x Σ_y p(x,y) * log(p(x,y) / (p(x) * p(y)))

Parameters¶

x_symbols : List[Any] A list of all observed x symbols. y_symbols : List[Any] A list of all observed y symbols.

Returns¶

float The mutual information between X and Y.

compute_normalized_mutual_information ¶

compute_normalized_mutual_information(x_symbols: list[Any], y_symbols: list[Any]) -> float

Compute the normalized mutual information between X and Y.

NMI(X;Y) = MI(X;Y) / H(X,Y) where H(X,Y) is the joint entropy.

Parameters¶

x_symbols : List[Any] A list of all observed x symbols. y_symbols : List[Any] A list of all observed y symbols.

Returns¶

float The normalized mutual information between X and Y, in range [0, 1].

compute_jaccard_index ¶

compute_jaccard_index(x_contexts: list[Any], y_contexts: list[Any]) -> float

Compute the Jaccard Index between two sets of contexts.

J(X,Y) = |X ∩ Y| / |X ∪ Y|

Parameters¶

x_contexts : List[Any] Contexts where symbol x appears. y_contexts : List[Any] Contexts where symbol y appears.

Returns¶

float The Jaccard Index between the two context sets, in range [0, 1].

compute_goodman_kruskal_lambda ¶

compute_goodman_kruskal_lambda(x_symbols: list[Any], y_symbols: list[Any], direction: str = 'y_given_x') -> float

Compute Goodman and Kruskal's Lambda for asymmetric association.

λ(Y|X) = (Σ_x max(n_x,y) - max(n_y)) / (n - max(n_y))

Parameters¶

x_symbols : List[Any] A list of all observed x symbols. y_symbols : List[Any] A list of all observed y symbols. direction : str Either "y_given_x" or "x_given_y" to specify direction.

Returns¶

float The Goodman-Kruskal Lambda coefficient, in range [0, 1].

compute_log_likelihood_ratio ¶

compute_log_likelihood_ratio(cont_table: list[list[float]]) -> float

Compute the Log-Likelihood Ratio (G²) from a contingency table.

G² = 2 * Σ O_ij * ln(O_ij / E_ij) where O_ij are observed frequencies and E_ij are expected frequencies.

Parameters¶

cont_table : List[List[float]] The contingency table for computation.

Returns¶

float The Log-Likelihood Ratio statistic.

compute_log_likelihood_ratio_pvalue ¶

compute_log_likelihood_ratio_pvalue(cont_table: list[list[float]]) -> float

Compute the p-value of the Log-Likelihood Ratio (G²) statistic.

Under the null hypothesis of independence, G² is asymptotically chi-square distributed with (rows - 1) * (cols - 1) degrees of freedom.

Parameters¶

cont_table : List[List[float]] The contingency table for computation.

Returns¶

float The p-value for the Log-Likelihood Ratio statistic, in range [0, 1].

scale_scorer ¶

scale_scorer(scorer: dict[tuple[Any, Any], tuple[float, float]], method: str = 'minmax', nrange: tuple[float, float] = (0, 1)) -> dict[tuple[Any, Any], tuple[float, float]]

Scale a scorer.

The function returns a scaled version of a scorer considering all the asymmetric scorers (i.e., both x given y and y given x). Implemented scoring methods are "minmax" (by default on range [0, 1], which can be modified by the nrange parameter) and "mean".

Parameters¶

scorer : dict A scoring dictionary. method : str, optional The scoring method to be used, either "minmax" or "mean" or "stdev" (default: "minmax"). nrange : tuple, optional A tuple with the scaling range, to be used when applicable (default: (0, 1)).

Returns¶

dict A scaled version of the scorer.

invert_scorer ¶

invert_scorer(scorer: dict[tuple[Any, Any], tuple[float, float]]) -> dict[tuple[Any, Any], tuple[float, float]]

Inverts a scorer, so that the higher the affinity, the higher the score.

It is recommended than only scorers in range [0..] are inverted.

Parameters¶

scorer : dict A scoring dictionary.

Returns¶

dict An inverted version of the scorer.

scorer2matrices ¶

scorer2matrices(scorer: dict[tuple[Any, Any], tuple[float, float]]) -> tuple[np.ndarray, np.ndarray, list[Any], list[Any]]

Return the asymmetric matrices implied by a scorer and their alphabets.

Parameters¶

scorer : dict A scoring dictionary.

Returns¶

tuple A tuple with the following elements: a scoring matrix of y given x, a scoring matrix of x given y, the alphabet for matrix x, and the alphabet for matrix y.

Data handling: `asymcat.common`¶

Utility functions for the asymcat library.

The module mostly includes functions for input/output, as well as pre-computations from raw data.

collect_alphabets ¶

collect_alphabets(cooccs: list[tuple]) -> tuple

Return the x and y alphabets from a list of co-occurrences.

Parameters¶

cooccs : List[tuple] The list of co-occurrence tuples.

Returns¶

alphabets : tuple A tuple of two elements, the sorted list of symbols in series x and the sorted list of symbols in series y.

Raises¶

ValueError If cooccs is empty or contains invalid tuples. TypeError If cooccs is not a list or contains non-tuple elements.

collect_ngrams ¶

collect_ngrams(seq: list[Any] | str, order: int, pad: str) -> Generator[tuple, None, None]

Function for yielding the ngrams of a sequence.

The sequence is padded so that symbols at the extremities will be repeated the same number of times that other symbols. This operation also guarantees that sequences shorter than the order will be collected.

Parameters¶

seq : Union[List[Any], str] The list/string of elements for ngram collection. order : int The ngram order. pad : str The padding symbol.

Yields¶

ngram : tuple The ngrams of the sequence.

Raises¶

ValueError If order is less than 1 or seq is empty. TypeError If seq is not a list, tuple, or string, or order is not an integer.

collect_cooccs ¶

collect_cooccs(seqs: list[list[list[Any] | str] | tuple], order: int | None = None, pad: str = '#') -> list[tuple]

Collects tuples of co-occurring elements in pairs of sequences.

This function is used for collecting co-occurring elements in order to train or improve scorers. Co-occurrences can be collected either considering sequences in their entirety, or by aligned ngrams.

When collecting ngrams, a padding symbol is used internally to guarantee that elements at the extremities of the sequences will be collected the same number of times that other elements. The padding symbol defaults to "#", but given that the tuples where it occurs are automatically removed before returning, it is important that such symbol is not a part of either of the sequence alphabets. It is the user responsability to set an appropriate, non-conflicting pad symbol in case the default is used in the data.

Parameters¶

seqs : List[Union[List[Union[List[Any], str]], tuple]] A list of sequence pairs (each sequence can be a list or string). order : Optional[int] The order of the ngrams to be collected, with None indicating the collection of the entire sequences (default: None). pad : str The symbol for internal padding, which must not conflict with symbols in the sequences (default: "#").

Returns¶

coocc : List[tuple] A list of tuples of co-occurring elements.

Raises¶

ValueError If seqs is empty, contains invalid sequence pairs, or sequences have mismatched lengths. TypeError If seqs is not a list or contains invalid data types.

collect_observations ¶

collect_observations(cooccs: list[tuple]) -> dict

Build a dictionary of observations for all possible correspondences.

The counts of observations are derived from typical organization in a non-squared contingency table, using indexes 0, 1, and 2, where 0 means that the identity is not considered (so that "00" would refer to all correspondence pairs), 1 that there is a match, and 2 that there is a mismatch. In more detail, given a pair of symbols x and y, the counts of co-occurrences will be:

obs["10"]: number of pairs matching x (with any y)
obs["20"]: number of pairs not matching x (with any y)
obs["01"]: number of pairs matching y (with any x)
obs["02"]: number of pairs not matching y (with any x)
obs["11"]: number of pairs matching x and matching y
obs["12"]: number of pairs matching x and not matching y
obs["21"]: number of pairs not matching x and matching y
obs["22"]: number of pairs not matching x and matching Y

Note that obs["22"] is not the number of pairs where either x or y are mismatching, but where both necessarily mismatch.

When doing x->y, a non-squared contingency table will be

|            |  pair[1]  | pair[1]==y | pair[1]!=y |
| pair[0]==x | obs["10"] | obs["11"]  | obs["12"]  |
| pair[0]!=x | obs["20"] | obs["21"]  | obs["22"]  |

Parameters¶

cooccs : List[tuple] A list of tuples of co-occurring elements.

Returns¶

obs : dict A dictionary of observations per co-occurrence type.

Raises¶

ValueError If cooccs is empty or contains invalid tuples. TypeError If cooccs is not a list.

build_ct ¶

build_ct(observ, square: bool = True) -> list

Build a contingency table from a dictionary of observations.

The contingency table can be either square (2x2) or not (3x2). Non-squared contingency tables include co-occurrences where neither the x or y under investigation occur.

Parameters¶

observ : dict A dictionary of observations, as provided by common.collect_observations(). square : bool, optional Whether to return a square (2x2) or non-square (3x2) contingency table.

Returns¶

cont_table : np.array A contingency table as a numpy array.

read_sequences ¶

read_sequences(filename: str, cols: list[str] | None = None, col_delim: str = '\t', elem_delim: str = ' ') -> list[list[list[str]]]

Reads parallel sequences, returning them as a list of lists.

Datafiles should have one sequence pair per row, with pairs separated by col_delim and elements delimited by elem_delim, as in:

Orthography                 Segments
E X C L A M A T I O N       ɛ k s k l ʌ m eɪ ʃ ʌ n
C L O S E Q U O T E         k l oʊ z k w oʊ t
D O U B L E Q U O T E       d ʌ b ʌ l k w oʊ t
E N D O F Q U O T E         ɛ n d ʌ v k w oʊ t
E N D Q U O T E             ɛ n d k w oʊ t
I N Q U O T E S             ɪ n k w oʊ t s
Q U O T E                   k w oʊ t
U N Q U O T E               ʌ n k w oʊ t
H A S H M A R K             h æ m ɑ ɹ k

Rows without complete data are not included. The function allows to collect data from multiple columns, but it is inteded to pairwise comparison; if no column names are provided, as per default, the reading will skip the first row (assumed to be header).

Parameters¶

filename : str Path to the file to be read. cols : Optional[List[str]] List of column names to be collected (default: None) col_delim : str String used as field delimiter (default: " "). elem_delim : str String used as element delimiter (default: " ").

Returns¶

data : List[List[List[str]]] A list of lists, where each list contains the data from a row.

Raises¶

FileNotFoundError If the specified file does not exist. ValueError If file is empty or has invalid format. PermissionError If file cannot be read due to permissions.

read_pa_matrix ¶

read_pa_matrix(filename: str, delimiter: str = '\t') -> list[tuple]

Reads a presence-absence matrix, returning as equivalent sequences.

The location must be specified under column name ID.

Parameters¶

filename : str Path to the file to be read. delimiter : str String used as field delimiter (default: " ").

Returns¶

obs_combinations : List[tuple] A list of tuples representing observed combinations.

Raises¶

FileNotFoundError If the specified file does not exist. ValueError If file format is invalid or ID column is missing.

Sequence correlation: `asymcat.correlation`¶

Convenience wrappers for computing association measures on parallel series.

Unlike :class:asymcat.scorer.CatScorer, which returns per-symbol-pair directional scores from a list of co-occurrences, the helpers in this module operate on two aligned series (series_x and series_y) and return a single summary statistic for the pair of variables.

Note that not every measure here is symmetric: :func:cramers_v is symmetric by definition, whereas :func:conditional_entropy and :func:theil_u are directional (x given y) and are therefore, in general, asymmetric.

cramers_v ¶

cramers_v(series_x, series_y)

Compute Cramér's V between two aligned series.

Cramér's V is a symmetric measure of association derived from Pearson's chi-square statistic, ranging from 0 (no association) to 1 (perfect association).

Parameters¶

series_x : list The first series of observed symbols. series_y : list The second series of observed symbols, aligned with series_x.

Returns¶

float Cramér's V for the two series.

conditional_entropy ¶

conditional_entropy(series_x, series_y)

Compute the conditional entropy of series_x given series_y.

This is a directional measure: in general, H(x|y) != H(y|x).

Parameters¶

series_x : list The series whose entropy (conditioned on series_y) is computed. series_y : list The conditioning series, aligned with series_x.

Returns¶

float The conditional entropy H(x|y).

theil_u ¶

theil_u(series_x, series_y)

Compute Theil's U (uncertainty coefficient) of series_x given series_y.

Theil's U normalizes the reduction in uncertainty of series_x obtained from knowing series_y by the entropy of series_x, yielding a value in the [0, 1] range. It is a directional measure and is, in general, asymmetric (U(x|y) != U(y|x)).

Parameters¶

series_x : list The series whose uncertainty is being reduced. series_y : list The conditioning series, aligned with series_x.

Returns¶

float Theil's U for series_x given series_y, in [0, 1].

API Reference¶

Package interface¶

CatScorer ¶

Parameters¶

mle ¶

get_smoothed_probabilities ¶

Returns¶

pmi_smoothed ¶

Parameters¶

Returns¶

pmi ¶

Parameters¶

chi2 ¶

Parameters¶

chi2_pvalue ¶

Parameters¶

Returns¶

cramers_v ¶

Parameters¶

fisher ¶

fisher_pvalue ¶

Returns¶

theil_u ¶

cond_entropy ¶

tresoldi ¶

mutual_information ¶

Returns¶

normalized_mutual_information ¶

Returns¶

jaccard_index ¶

Returns¶

goodman_kruskal_lambda ¶

Returns¶

log_likelihood_ratio ¶

Parameters¶

Returns¶

log_likelihood_ratio_pvalue ¶

Parameters¶

Returns¶

permutation_pvalue ¶

Parameters¶

Returns¶

Raises¶

bootstrap_ci ¶

Parameters¶

Returns¶

Raises¶

build_ct ¶

Parameters¶

Returns¶

collect_alphabets ¶

Parameters¶

Returns¶

Raises¶

collect_cooccs ¶

Parameters¶

Returns¶

Raises¶

collect_ngrams ¶

Parameters¶

Yields¶

Raises¶

collect_observations ¶

Parameters¶

Returns¶

Raises¶

read_pa_matrix ¶

Parameters¶

Returns¶

Raises¶

read_sequences ¶

Parameters¶

Returns¶

Raises¶

Scoring: asymcat.scorer¶

CatScorer ¶

Parameters¶

mle ¶

get_smoothed_probabilities ¶

Returns¶

Scoring: `asymcat.scorer`¶