A headless workflow execution engine for bioinformatics that supports local, cloud, and HPC.
Request Feature
·
Report Bug
·
⭐ Consider starring the repo! ⭐
crankshaft is a headless workflow execution engine written in Rust: it's being
developed in coordination with the sprocket project with the goal of
enabling large-scale bioinformatics analyses. There is no associated
crankshaft command line tool—the end user is really engine developers who
want to include it as a core task execution library in their own command line
tools.
crankshaftaims to be a high-performance workflow execution engine capable of concurrently managing and executing upwards of 20,000 concurrent tasks. The core focus is enabling middle- to large-scale bioinformatics analyses, though it can also be used to design smaller scale execution engines.crankshaftis headless, which means that it doesn't do anything on its own; in fact, it must be driven by some external orchestration code. This allows thecrankshaftlibrary itself to focus on performance improvements that can be enjoyed across the entire community.crankshaftis developed independently of any particular workflow language. Though it's part of the Sprocket project, it's not based on WDL, and, in theory, multiple frontends based on different workflow languages can exist (and we hope this is the case)!
To use crankshaft, you'll need to install Rust.
We recommend using rustup to accomplish this. Once Rust is
installed, you can create a new project and add the latest version of
crankshaft using the following command.
cargo add crankshaftOnce you've added crankshaft to your dependencies, you should head over to the
/examples
to see how you can use the library in your projects.
To bootstrap a development environment, please use the following commands.
# Clone the repository
git clone [email protected]:stjude-rust-labs/crankshaft.git
cd crankshaft
# Build the crate in release mode
cargo build --releaseBefore submitting any pull requests, please make sure the code passes the following checks (from the root directory).
# Run the project's tests.
cargo test --all-features
# Run the tests for the examples.
cargo test --examples --all-features
# Ensure the project doesn't have any linting warnings.
cargo clippy --all-features
# Ensure the project passes `cargo fmt`.
cargo fmt --check
# Ensure the docs build.
cargo docContributions, issues and feature requests are welcome! Feel free to check issues page.
This project is licensed as either Apache 2.0 or MIT at your discretion. Additionally, please see the disclaimer that applies to all crates and command line tools made available by St. Jude Rust Labs.
Copyright © 2024-Present St. Jude Children's Research Hospital.