Skip to content

overlap

lab.connectome.overlap

Overlap between sets, as the Jaccard index.

The Jaccard index of two sets is the size of their intersection over the size of their union: 1 for the same set and 0 for disjoint sets.

jaccard(a, b)

Compute the Jaccard index of two sets, 0 for two empty sets.

Parameters:

Name Type Description Default
a set

The sets to compare.

 required
b set

The sets to compare.

 required

Returns:

Type Description
float

|a & b| / |a | b|.

Examples:

>>> jaccard({"A", "B", "C"}, {"B", "C", "D"})
0.5
>>> jaccard(set(), set())
0.0
Source code in lab/connectome/overlap.py 
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
def jaccard(a: set, b: set) -> float:
    """Compute the Jaccard index of two sets, 0 for two empty sets.

    Parameters
    ----------
    a, b : set
        The sets to compare.

    Returns
    -------
    float
        ``|a & b| / |a | b|``.

    Examples
    --------
    >>> jaccard({"A", "B", "C"}, {"B", "C", "D"})
    0.5
    >>> jaccard(set(), set())
    0.0
    """
    union = a | b
    if not union:
        return 0.0
    return len(a & b) / len(union)

jaccard_matrix(pairs, types)

Arrange the Jaccard index of every pair as a symmetric matrix.

Parameters:

Name Type Description Default
pairs DataFrame

One row per pair, with columns a, b, and jaccard, as pairwise_jaccard returns them.

 required
types sequence of str

The names, in the order of the rows and columns.

 required

Returns:

Type Description
ndarray

Square, with 1 on the diagonal.
See Also

pairwise_jaccard : Computes the pairs.

Examples:

>>> pairs = pl.DataFrame(
...     {"a": ["P"], "b": ["Q"], "jaccard": [0.25]}
... )
>>> jaccard_matrix(pairs, ["P", "Q"])
array([[1.  , 0.25],
       [0.25, 1.  ]])
Source code in lab/connectome/overlap.py 
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
def jaccard_matrix(pairs: pl.DataFrame, types: Sequence[str]) -> np.ndarray:
    """Arrange the Jaccard index of every pair as a symmetric matrix.

    Parameters
    ----------
    pairs : polars.DataFrame
        One row per pair, with columns ``a``, ``b``, and ``jaccard``,
        as `pairwise_jaccard` returns them.
    types : sequence of str
        The names, in the order of the rows and columns.

    Returns
    -------
    numpy.ndarray
        Square, with 1 on the diagonal.

    See Also
    --------
    pairwise_jaccard : Computes the pairs.

    Examples
    --------
    >>> pairs = pl.DataFrame(
    ...     {"a": ["P"], "b": ["Q"], "jaccard": [0.25]}
    ... )
    >>> jaccard_matrix(pairs, ["P", "Q"])
    array([[1.  , 0.25],
           [0.25, 1.  ]])
    """
    position = {t: n for n, t in enumerate(types)}
    matrix = np.eye(len(types))
    for row in pairs.iter_rows(named=True):
        i, j = position[row["a"]], position[row["b"]]
        matrix[i, j] = matrix[j, i] = row["jaccard"]
    return matrix

pairwise_jaccard(sets)

Compute the Jaccard index of every pair of the sets.

Parameters:

Name Type Description Default
sets dict of str to set

The sets, by name.

 required

Returns:

Type Description
DataFrame

One row per pair of names, with columns a, b, jaccard, shared (the size of the intersection), and union (the size of the union).
See Also

jaccard : The index of one pair. sets_by_type : Builds the sets from a table.

Examples:

>>> table = pairwise_jaccard({"P": {1, 2}, "Q": {2, 3}})
>>> table.select("a", "b", "shared", "union").rows()
[('P', 'Q', 1, 3)]
>>> round(table["jaccard"].item(), 3)
0.333
Source code in lab/connectome/overlap.py 
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
def pairwise_jaccard(sets: dict[str, set]) -> pl.DataFrame:
    """Compute the Jaccard index of every pair of the sets.

    Parameters
    ----------
    sets : dict of str to set
        The sets, by name.

    Returns
    -------
    polars.DataFrame
        One row per pair of names, with columns ``a``, ``b``,
        ``jaccard``, ``shared`` (the size of the intersection), and
        ``union`` (the size of the union).

    See Also
    --------
    jaccard : The index of one pair.
    sets_by_type : Builds the sets from a table.

    Examples
    --------
    >>> table = pairwise_jaccard({"P": {1, 2}, "Q": {2, 3}})
    >>> table.select("a", "b", "shared", "union").rows()
    [('P', 'Q', 1, 3)]
    >>> round(table["jaccard"].item(), 3)
    0.333
    """
    return pl.DataFrame(
        [
            {
                "a": a,
                "b": b,
                "jaccard": jaccard(sets[a], sets[b]),
                "shared": len(sets[a] & sets[b]),
                "union": len(sets[a] | sets[b]),
            }
            for a, b in combinations(sets, 2)
        ],
        schema={
            "a": pl.String,
            "b": pl.String,
            "jaccard": pl.Float64,
            "shared": pl.Int64,
            "union": pl.Int64,
        },
    )

sets_by_type(strong, column, types)

Collect the values of a column for each type, as sets.

Parameters:

Name Type Description Default
strong DataFrame

Has a column type and the column named by column.

 required
column str

Column whose values are collected.

 required
types sequence of str

The types to collect, in order.

 required

Returns:

Type Description
dict of str to set

One set per type; a type with no row has an empty set.
See Also

pairwise_jaccard : The overlap of these sets.

Examples:

>>> import polars as pl
>>> strong = pl.DataFrame(
...     {"type": ["P", "P", "Q"], "partner": [1, 2, 2]}
... )
>>> sets_by_type(strong, "partner", ["P", "Q", "R"])
{'P': {1, 2}, 'Q': {2}, 'R': set()}
Source code in lab/connectome/overlap.py 
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
def sets_by_type(
    strong: pl.DataFrame, column: str, types: Sequence[str]
) -> dict[str, set]:
    """Collect the values of a column for each type, as sets.

    Parameters
    ----------
    strong : polars.DataFrame
        Has a column ``type`` and the column named by `column`.
    column : str
        Column whose values are collected.
    types : sequence of str
        The types to collect, in order.

    Returns
    -------
    dict of str to set
        One set per type; a type with no row has an empty set.

    See Also
    --------
    pairwise_jaccard : The overlap of these sets.

    Examples
    --------
    >>> import polars as pl
    >>> strong = pl.DataFrame(
    ...     {"type": ["P", "P", "Q"], "partner": [1, 2, 2]}
    ... )
    >>> sets_by_type(strong, "partner", ["P", "Q", "R"])
    {'P': {1, 2}, 'Q': {2}, 'R': set()}
    """
    return {
        t: set(values.to_list())
        for t, values in values_by_type(strong, column, types).items()
    }