Welcome to DIPPER Wiki

Introduction

Overview

DIPPER (DIstance-based Phylogenetic PlacER) is a tool for ultrafast and ultralarge phylogenetic reconstruction on GPUs, designed to maintain high accuracy with a minimal memory footprint. DIPPER introduces several innovations, including a divide-and-conquer strategy, a new placement algorithm, and an on-the-fly distance calculator that dynamically enables selective distance computation. DIPPER consistently outperforms existing distance-based methods in speed, accuracy, and memory efficiency. In addition, DIPPER minimizes branch length underestimation for non-additive distance matrices compared to earlier methods and offers a strict mode that completely eliminates the underestimation.

Figure 1: Overview of DIPPER algorithm

Installation Methods

NOTE: DIPPER is currently supported on systems with NVIDIA GPUs only. Support for additional platforms, including AMD GPUs and CPU-only options for x86-64 and ARM64 architecture, will be added soon. Stay tuned!

1. Using Conda

DIPPER is available on platforms with NVIDIA GPUs via Conda. See DIPPER Bioconda Page for details.

i. Dependencies

1. Conda

ii. Create and activate a Conda environment

conda create -n dipper python=3.11 -y
conda activate dipper
# Set up channels
conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
conda config --set channel_priority strict
# Install DIPPER
conda install bioconda::dipper

iii. Run DIPPER

# Insider conda environment
dipper --help

2. Using Docker Image

To use DIPPER in a docker container, users can create a docker container from a docker image, by following these steps

i. Dependencies

1. Docker

ii. Pull and build the DIPPER docker image from DockerHub

## Note: If the Docker image already exists locally, make sure to pull the latest version using 
## docker pull swalia14/dipper:latest

## If the Docker image does not exist locally, the following command will pull and run the latest version
docker run -it --gpus all swalia14/dipper:latest

iii. Run DIPPER

# Insider docker container (path: /home/Dipper/bin)
./dipper --help

3. Using DockerFile

Docker container with the preinstalled DIPPER program can also be built from a Dockerfile by following these steps.

i. Dependencies

1. Docker
2. Git

ii. Clone the repository and build a docker image

git clone https://github.com/TurakhiaLab/DIPPER.git
cd DIPPER/docker
docker build -t dipper .

iii. Build and run the docker container

docker run -it --gpus all dipper

iv. Run DIPPER

# Insider docker container (path: /home/Dipper/bin)
./dipper --help

4. Using installation script (requires sudo access)

Users without sudo access are advised to install DIPPER via Docker Image or Dockerfile.

Step 1: Clone the repository

git clone https://github.com/TurakhiaLab/DIPPER.git
cd DIPPER

Step 2: Install dependencies (requires sudo access)

DIPPER depends on the following common system libraries, which are typically pre-installed on most development environments:

- wget
- cmake 
- build-essential 
- libboost-all-dev
- libtbb-dev

For Ubuntu users with sudo access, if any of the required libraries are missing, you can install them with:

sudo apt install -y wget cmake build-essential libboost-all-dev  libtbb-dev

Step 3: Build DIPPER

cd install
chmod +x installUbuntu.sh
./installUbuntu.sh
cd ../

Step 4: The DIPPER executable is located in the bin directory and can be run as follows:

cd bin
./dipper --help

Run DIPPER

Functionalities

Table 1: List of functionalities supported by DIPPER

Option	Description
-i, --input-format	Input format (required): d - distance matrix r - raw sequences m - MSA
-o, --output-format	Output format: t - phylogenetic tree in Newick format (default) d - distance matrix in PHYLIP format (coming soon!!)
-I, --input-file	Input file path (required): PHYLIP for distance matrix, FASTA for MSA or raw sequences
-O, --output-file	Output file path (required)
-m, --algorithm	Algorithm selection: 0 - auto (default) 1 - force placement 2 - force NJ 3 - divide-and-conquer
-K, --K-closest	Placement mode: -1 - exact 10 - default
-k, --kmer-size	K-mer size (Valid range: 2–15, default: 15)
-s, --sketch-size	Sketch size (default: 1000)
-d, --distance-type	Distance type: 1 - uncorrected 2 - JC (default) 3 - Tajima-Nei 4 - K2P 5 - Tamura 6 - Jinnei
-a, --add	Add query to a backbone tree using k-closest placement
-t, --input-tree	Input backbone tree in Newick (required with --add option) format
-h, --help	Show help message

Note

All the files in the examples below can be found in the DIPPER/dataset.

Enter into the build directory (assuming $DIPPER_HOME directs to the DIPPER repository directory)

cd $DIPPER_HOME/bin
./dipper -h

De-novo phylogeny construction

DIPPER supports de-novo construction of phylogenies from unaligned/aligned sequences in FASTA format and distance matrix in PHYLIP format.

Default mode

In default mode, DIPPER constructs phylogeny using: 1. Conventional NJ for sequences/tips < 30,000 2. Placement technique for sequences/tips >= 30,000 and < 1,000,000 3. Divide-and-conquer technique for sequences/tips >= 1,000,000

From unaligned sequences

Usage syntax

./dipper -i r -o t -I <path to unaligned sequences FASTA file> -O <path to output file>

Example

./dipper -i r -o t -I ../dataset/t2.unaligned.fa -O tree.nwk

From aligned sequences

Usage syntax (using JC model)

./dipper -i m -o t -d 2 -I <path to aligned sequences FASTA file> -O <path to output file>

Example

./dipper -i m -o t -d 2 -I ../dataset/t1.aligned.fa -O tree.nwk

From distance matrix

Usage syntax

./dipper -i d -o t -I <path to distance matrix PHYLIP file> -O <path to output file>

Example

./dipper -i d -o t -I ../dataset/t2.phy -O tree.nwk

Construct phylogeny using placement technique

DIPPER allows users to construct phylogeny using the forced placement technique by setting the -m option to 1. Below we provide a syntax and an example for input unaligned sequences, but DIPPER also supports aligned sequences and distance matrix as input. Usage syntax

./dipper -i r -o t -m 1 -I <path to unaligned sequences FASTA file> -O <path to output file>

Example

./dipper -i r -o t -m 1 -I ../dataset/t2.unaligned.fa -O tree.nwk

Construct phylogeny using divide-and-conquer technique

DIPPER allows users to construct phylogeny using the forced divide-and-conquer technique by setting the -m option to 3. Below we provide a syntax and an example for input unaligned sequences, but DIPPER also supports aligned sequences and distance matrix as input. Usage syntax

./dipper -i r -o t -m 3 -I <path to unaligned sequences FASTA file> -O <path to output file>

Example

./dipper -i r -o t -m 3 -I ../dataset/t2.unaligned.fa -O tree.nwk

Adding tips (sequences) to a backbone tree

DIPPER allows users to add tips to an existing backbone tree using the placement technique. It requires tip sequences from the backbone tree and input query sequences to be provided in a single file (FASTA format), along with the input tree in Newick format.

Usage syntax

./dipper -i r -o t -m 1 --add -I <path to unaligned/aligned sequences FASTA file (containing backbone tree tip sequences and query sequences)> -O <path to output file> -t <path to input tree>

Example

./dipper -i r -o t -m 1 --add -I ../dataset/t2.unaligned.fa -O tree.nwk -t ../dataset/backbone.nwk

Contributions

We welcome contributions from the community to enhance the capabilities of DIPPER. If you encounter any issues or have suggestions for improvement, please open an issue on DIPPER GitHub page. For general inquiries and support, reach out to our team.

Citing DIPPER

If you use DIPPER in your research or publications, we kindly request that you cite the following paper:
* Sumit Walia, Zexing Chen, Yu-Hsiang Tseng, Yatish Turakhia, "Ultrafast and Ultralarge Distance-Based Phylogenetics Using DIPPER", bioRxiv 2025.08.12.669583; doi: https://doi.org/10.1101/2025.08.12.669583