Diamond-Miner is a Internet topology measurement software that provides statistical guarantees to discover load balanced paths at Internet Scale.

This work has been published to USENIX NSDI 2020.

Link to the paper: https://www.usenix.org/system/files/nsdi20-paper-vermeulen.pdf

These instructions will get you a copy of the project up and running on your local server.

Diamond-Miner has two main components: the core of diamond-miner, containing all of the routines of the algorithm (probing, computation of the next round of probes, database driver) written in C++ and the wrapper which provides a friendly python API to launch D-Miner on multiple vantage points.

You will need to clone both of the projects:

git clone /~https://github.com/dioptra-io/diamond-miner-cpp.git

git clone /~https://github.com/dioptra-io/diamond-miner-wrapper.git

You also need to install several libraries on which diamond-miner relies.

Please install boost program_options libcperm (/~https://github.com/lancealt/libcperm) libtins (http://libtins.github.io)

If you want to be able to run it at > 100K packets per second, you will have to install PF_RING (https://www.ntop.org/guides/pf_ring/)

You finally need to install and start a clickhouse database (https://clickhouse.tech/docs/en/getting_started/)

You first need to configure and compile the core-routines of Diamond-Miner. Diamond-Miner has two main core routines: the prober and the central, where the database computation is made.

cd /where/you/have/cloned/diamond-miner-cpp
mkdir build
cd build
cmake -DPROBER=1 -DCENTRAL=1 ..

If you want to activate PF_RING, you put the PF_RING option in the cmake command:

cmake -DPROBER=1 -DCENTRAL=1 -DPF_RING=1 ..

If you only want the prober component, because you install the database in a single server, remove the CENTRAL option:

cmake -DPROBER=1 -DPF_RING ..

Once it is configured, you can run:

make -j8

No further action are required to install the wrapper.

Once you have installed the core routines and the database, you can run the following command to run Diamond-Miner from the python wrapper directory.

python3 StochasticHeartbeat.py --cpp-heartbeat-dir=/where/core/routines/are/installed --db-host=IP_address_where_the_database_is_installed --table=name_of_the_database_table_to_save_the_replies --remote-probing (if multiple vantage points version) -r 100000 (probing rate) --proto=(udp,tcp)

The different options are described by running:

python3 StochasticHeartbeat.py --help

If you want to run it from multiple vantage points from a centralized server, you just have to modify the file Globals.py and put the vantage points informations.

For example, if you uncomment the line:

("ple1.cesnet.cz", "root", "/root/", "/where/to/store/pcap/")

Diamond-Miner will be run from ple1.cesnet.cz, using the user root, the home directory being /root/ and the directory to store the pcap files is /where/to/store/pcap/

Notice that you can install the core routines on the vantage points of the Globals.py file by running:

python3 Install.py

Depending of the OS of your vantage points, you might want to modify the script install_dependencies.sh, which works for Fedora and CentOS. The install_dependencies_ubuntu.sh works for Ubuntu.

Coming soon.