ZetaSketch is a collection of libraries for single-pass, distributed, approximate aggregation and sketching algorithms.
These algorithms estimate statistics that are often too expensive to compute exactly.
The estimates use far fewer memory resources than exact calculations. For example, the HyperLogLog++ algorithm can estimate daily active users with:
ZetaSketch currently includes libraries to implement the following algorithms:
| Algorithm | Statistics | Libraries |
|---|---|---|
| HyperLogLog++ | Estimates the number of distinct values | Java |
ZetaSketch libraries calculate statistics from sketches. A sketch is a summary of a large data stream. You can extract statistics from a sketch to estimate particular statistics of the original data, or merge sketches to summarize multiple data streams.
After choosing an algorithm, you can use its corresponding libraries to:
- Create sketches
- Add new data to existing sketches
- Merge multiple sketches
- Extract statistics from sketches
The HyperLogLog++ (HLL++) algorithm estimates the number of distinct values in a data stream. HLL++ is based on HyperLogLog; HLL++ more accurately estimates the number of distinct values in very large and small data streams.
// Create a sketch for estimating the number of unique strings in a data stream.
// You can also create sketches for estimating the number of unique byte
// sequences, integers, and longs.
HyperLogLogPlusPlus<String> hll = new HyperLogLogPlusPlus.Builder().buildForStrings();
// You can also set a custom precision. The default normal and sparse precisions
// are 15 and 20, respectively.
HyperLogLogPlusPlus<String> hllCustomPrecision = new HyperLogLogPlusPlus.Builder()
.normalPrecision(13).sparsePrecision(19).buildForStrings();// Add three strings to the `hll` sketch. You must first initialize an empty
// sketch and then add data to it.
hll.add("apple");
hll.add("orange");
hll.add("banana");// Merge `hll2` and `hll3` with `hll`. The sketches must have the same
// original data type and precision.
hll.merge(hll2);
hll.merge(hll3);// Return the estimate of the number of distinct values.
long result = hll.result();Please find the instructions for your build tool on the right side of https://search.maven.org/artifact/com.google.zetasketch/zetasketch
ZetaSketch uses Gradle as its build system. To build the project, simply run:
./gradlew build
We are not currently accepting contributions to this project. Please feel free to file bugs and feature requests using GitHub's issue tracker.
This is not an officially supported Google product.