Inspect & map identifiers

To make data queryable by an entity identifier, one needs to ensure that identifiers comply to a chosen standard. Bionty enables this by mapping metadata on the versioned ontologies using inspect().

from bionty import Gene, CellMarker
import pandas as pd

Genes

To illustrate it, let us generate a DataFrame that stores a number of gene identifiers, some of which corrupted.

data = {
    "gene symbol": ["A1CF", "A1BG", "FANCD1", "corrupted"],
    "hgnc id": ["HGNC:24086", "HGNC:5", "HGNC:1101", "corrupted"],
    "ensembl_gene_id": [
        "ENSG00000148584",
        "ENSG00000121410",
        "ENSG00000188389",
        "corrupted",
    ],
}
df_orig = pd.DataFrame(data).set_index("ensembl_gene_id")

df_orig

We require a reference identifier (specified as the reference_id parameter for curate). The list can be looked up using lookup(). Examples are “ontology_id”, which corresponds to the IDs of the ontology terms (e.g. ‘ENSG00000148584’) or “name” which corresponds to the ontology term names (e.g. ‘A1CF’).

To curate the DataFrame into queryable form, we create an index that corresponds to a default identifier. By default we use ensembl_gene_id. The default behavior is to curate the index if a column name is not provided.

First we create a Gene() instance using the default source database and version.

gene_bionty = Gene()

First we can check whether any of our values are mappable against the ontology.

gene_bionty.inspect(df_orig.index, gene_bionty.ensembl_gene_id)

The same procedure is available for gene symbols. First, we inspect which symbols are mappable against the ontology.

gene_bionty.inspect(df_orig["gene symbol"], gene_bionty.symbol)

Apparently 2 of the gene symbols are mappable. Bionty further warns us that some of our symbols can be mapped into standardized symbols.

mapped_symbol_synonyms = gene_bionty.map_synonyms(
    df_orig["gene symbol"], gene_bionty.symbol
)
mapped_symbol_synonyms

We can store them in our DataFrame further use.

df_orig["non-synonymous gene symbol"] = mapped_symbol_synonyms

CellMarker

Depending on how the data was collected and which terminology was used, it is not always possible to curate values. Some values might have used a different standard or be corrupted.

This section will demonstrate how to look up unmatched terms and curate them using CellMarker.

First, we create an examplary DataFrame containing a valid & invalid cell markers (antibody targets) and an additional feature (time) from a flow cytometry dataset.

markers = pd.DataFrame(
    index=[
        "KI67",
        "CCR7x",
        "CD14",
        "CD8",
        "CD45RA",
        "CD4",
        "CD3",
        "CD127",
        "PD1",
        "Invalid-1",
        "Invalid-2",
        "CD66b",
        "Siglec8",
        "Time",
    ]
)

Let’s instantiate the CellMarker ontology with the default database and version.

cell_marker_bionty = CellMarker()

First, we can have a look at the cell marker table that we just loaded.

df = cell_marker_bionty.df()

df.head()

Now let’s check which cell markers from the file can be found in the reference:

cell_marker_bionty.inspect(markers.index, cell_marker_bionty.name)

From the logging, it can be seen that 4 terms were not found in the reference!

Among them Time, Invalid-1 and Invalid-2 are non-marker channels which won’t be curated by cell marker.

Note, certain markers will be converted to synonyms such as PD1 -> PD-1.

We don’t really find CCR7x, let’s check in the lookup with auto-completion:

cell_marker_bionty_lookup = cell_marker_bionty.lookup()

cell_marker_bionty_lookup.CCR7

Indeed we find it should be CCR7, we had a typo there with CCR7x.

Now let’s fix the markers so all of them can be linked:

Tip

Using the .lookup instead of passing a string helps eliminate possible typos!

curated_df = markers.rename(index={"CCR7x": cell_marker_bionty_lookup.CCR7.name})

OK, now we can try to run curate again and all cell markers are linked!

cell_marker_bionty.inspect(curated_df.index, cell_marker_bionty.name)