Skip to content

Latest commit

 

History

History
127 lines (99 loc) · 4.63 KB

CONTRIBUTING.md

File metadata and controls

127 lines (99 loc) · 4.63 KB

Contributing

Set up

Create the shared native tools environment. environment.yml combines the direct dependencies from the CPU and CUDA test environments and adds Charm++ as a local Conda package. It includes GNU Make, GCC, Clang 19, Open MPI, the five external libraries, pkg-config, CMake, Ninja, and CUDA 12.8. Mamba resolves their transitive dependencies. This file also installs the CUDA build components on a CPU-only host.

bash scripts/build_charm_conda.sh
mamba env create --prefix "$PWD/build/_deps/bhlab-depns" -f environment.yml
mamba activate "$PWD/build/_deps/bhlab-depns"
export PKG_CONFIG_PATH="$CONDA_PREFIX/lib/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}"
export LD_LIBRARY_PATH="$CONDA_PREFIX/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

The helper builds Charm++ 8.0.0 into build/conda-channel. Mamba installs and tracks that package with the rest of the environment. Rebuild it on each host. Use a fresh worktree and its own build/_deps/bhlab-depns prefix for each test session. GCC 14 and Clang 19 keep the host compilers within the CUDA 12.8 compatibility range.

Use uv for the Python environment.

uv sync --extra dev
uv run bhlab doctor

Validate

Activate the worktree's bhlab-depns environment and run the Python tests first.

uv run pytest

Build the local native core and run its C++ tests.

uv run bhlab build --platform local-clang
make -f build/Makefile.local-clang check

Run the end-to-end local story.

uv run bhlab demo acceptance

For MPI on one host, install Open MPI and build the mpi-clang profile.

uv run bhlab build --platform mpi-clang
mpirun --oversubscribe -np 2 ./build/bin/bhlab-kmer-mpi \
  --input fixtures/reads/tiny.fq --output /tmp/mpi.tsv \
  --events /tmp/mpi.jsonl --k 15 --layout aos8 --verify-shuffle

For Charm++ on one host, activate the tools environment and build the charm-clang profile. The provided Charm++ package uses charmrun. Use ++local to launch both PEs on the current host.

uv run bhlab build --platform charm-clang --with-charm
charmrun ++local +p2 ./build/bin/bhlab-kmer-charm \
  --input fixtures/reads/tiny.fq --output /tmp/charm.tsv \
  --events /tmp/charm.jsonl --device cpu --k 15 --layout aos8

For CUDA hardware tests, use the designated GPU worktree with a visible GPU, nvcc and nvidia-smi on PATH, and CUDA runtime libraries available to the linker and loader. Install the Python dev dependencies and build build/bin/bhlab-kmer with the gpu-cuda profile first. Select an architecture supported by both the GPU and toolkit. The RTX 5060 Ti uses sm_120 and requires a toolkit that supports it, such as CUDA 12.8.

When Open MPI and CUDA are both detected, build the local target for these tests to avoid the MPI and CUDA link gap.

uv run bhlab build --platform gpu-cuda --emit-only
make -B -f build/Makefile.gpu-cuda local CUDA_ARCH=sm_120
BHLAB_TEST_CUDA=1 uv run pytest tests/test_cuda.py

These tests compare CPU and GPU counts with an independent reference for both layouts and require a nonzero exit when CUDA devices are masked. They create fresh temporary outputs and do not build binaries. Ordinary pytest skips hardware cases without probing the GPU. Once opted in, missing prerequisites or a nonfunctional CUDA binary fail the tests.

Layout

  • src/bhlab/ is the Python CLI, planner, build generator, code generation, and log pipeline.
  • native/ is the C++17 core, device backends, and communication drivers.
  • platforms/ holds build profiles. A profile produces a Makefile, not a second build system.
  • pipelines/ holds the line-based pipeline DSL.
  • rules/ holds anomaly rules.
  • fixtures/ holds deterministic reads and logs.
  • tests/ holds pytest tests.
  • docs/ owns user and design documentation.

Change rules

  • Keep the output checksum identical across layouts, communication backends, and devices for the same input and k. Add a test when that can break.
  • Every run path emits the same LogEvent schema.
  • Auto selection lives only in src/bhlab/planner.py. Do not add backend branching inside commands.
  • Do not add a scheduler, a custom MPI runtime, a web console, or a machine learning model. See the scope document for the current exclusions.
  • Update the canonical document when a command, schema, or workflow changes. The documentation index names the owner of each topic.

Regenerate fixtures

uv run python scripts/make_fixtures.py

Fixtures are deterministic. Regenerate them only when the read or log shape changes, then update the golden expectations.