Skip to content

Quickstart

There are two experiments that you can run immediately. First, make sure you have installed this package.

1. Pick a cohort

manifold-genetics acquire synthetic
manifold-genetics run config.yaml

2,000 samples placed along a branching tree, with 1,000 variants whose allele frequencies drift along it — no single variant carries the tree. Genotypes, labels, colormap and config are written beside each other. Under a minute, nothing downloaded.

It also writes dla_tree_ground_truth.png, the tree the cohort was drawn along. Compare it with outputs/figures/embeddings/project_phate_by_branch.png: the eight branches should be recognisable in both.

manifold-genetics acquire hgdp
manifold-genetics run config.yaml

The public HGDP+1KGP cohort, downloaded already variant-processed. Two plink2 --keep calls select the 4,094 samples that pass QC and the 3,400 of those that are also unrelated. Needs internet and plink2, which is fetched automatically — on a cluster, run acquire on a login node. A few minutes.

Both write into the current directory; --out DIR puts them elsewhere, and neither overwrites an existing config.yaml without --force.

2. Look before you run

manifold-genetics run config.yaml --dry-run

Prints every setting the run will use, marking (default) on anything the package supplied rather than the config file, then exits. This can help you determine what is going on especially during any complex run.

3. Run it

manifold-genetics run config.yaml

If the run is OOM killed, set a memory budget:

manifold-genetics run config.yaml --memory-gb 4

Your own data

acquire custom writes the config and colormap for PLINK files you already have.

One cohort, fitting on a subset of it and embedding that subset:

manifold-genetics acquire custom \
    --fit-plink data/fit_subset \
    --project-plink data/project_subset \
    --labels data/labels.csv \
    --preset subsample

Both sets come from the same cohort: fit_subset is the subset the model is estimated on, project_subset the full cohort the components are computed for.

Two cohorts, projecting yours onto a reference panel — separate label files, because the cohorts need not have the same labels:

manifold-genetics acquire custom \
    --fit-plink data/reference_panel \
    --project-plink data/my_cohort \
    --fit-labels data/reference_labels.csv \
    --project-labels data/my_labels.csv \
    --preset projection

It generates a colour for every label value, and refuses if fewer than half the genotyped samples appear in the label file — a mismatch otherwise shows up as a figure that colours some of its points and looks finished.

See formats if you would rather write both files yourself.

If the PLINK files still need SNP filtering, or you want to project your cohort onto a reference panel it has not been intersected with, see Preprocessing: preprocess filters and intersects, subsample chooses the fit samples, and both write a directory run accepts.

All of Us

For readers working in the All of Us Researcher Workbench with access to the controlled data tier:

manifold-genetics acquire hgdp --archive gs://fc-secure-47ccf5a8-b9ba-460a-aa03-dea8d260953b/Data/1KGPHGDP.tar.gz --out ref/
manifold-genetics acquire aou --out aou/
manifold-genetics preprocess ref/config.yaml aou/config.yaml --preset harmonise --fit-has-chr-prefix --out proj/
manifold-genetics run proj/config.yaml

This is the All of Us experiment from the manuscript: the workbench's HGDP+1KGP panel, the All of Us V8 arrays, harmonised and intersected, then projected. acquire aou has not yet been run inside the workbench (issue

124); the details are in Preprocessing.