diff options
| author | Tavian Barnes <tavianator@tavianator.com> | 2023-10-02 10:49:41 -0400 |
|---|---|---|
| committer | Tavian Barnes <tavianator@tavianator.com> | 2023-10-02 10:53:24 -0400 |
| commit | 692098fdb922c464949fad7c5b9e36b531ea6f68 (patch) | |
| tree | ef67e779559c90b317e0a06aaf8301590e8130d9 | |
| parent | fd2a86bb93f42a2c64b202f959dd5c8cbb724c69 (diff) | |
| download | bfs-692098fdb922c464949fad7c5b9e36b531ea6f68.tar.xz | |
bench: Add a README
| -rw-r--r-- | bench/README.md | 51 |
1 files changed, 51 insertions, 0 deletions
diff --git a/bench/README.md b/bench/README.md new file mode 100644 index 0000000..56157a0 --- /dev/null +++ b/bench/README.md @@ -0,0 +1,51 @@ +This directory contains a suite of benchmarks used to evaluate `bfs` and detect performance regressions. +To run them, you'll need the [tailfin] benchmark harness. +You can read the full usage information with + +[tailfin]: https://github.com/tavianator/tailfin + +```console +$ tailfin -n run bench/bench.sh --help +Usage: tailfin run bench/bench.sh [--default] + [--complete] [--early-quit] [--print] [--strategies] + [--build=...] [--bfs] [--find] [--fd] + [--no-clean] [--help] +... +``` + +The benchmarks use various git repositories to have a realistic and reproducible directory structure as a corpus. +Currently, those are the [Linux], [Rust], and [Chromium] repos. +The scripts will automatically clone those repos using [partial clone] filters to avoid downloading the actual file contents, saving bandwidth and space. + +[Linux]: https://github.com/torvalds/linux.git +[Rust]: https://github.com/rust-lang/rust.git +[Chromium]: https://chromium.googlesource.com/chromium/src.git +[partial clone]: https://git-scm.com/docs/partial-clone + +You can try out a quick benchmark by running + +```console +$ tailfin run bench/bench.sh --build=main --complete=linux +``` + +This will build the `main` branch, and measure the complete traversal of the Linux repo. +Results will be both printed to the console and saved in a Markdown file, which you can find by running + +```console +$ tailfin latest +results/2023/09/29/15:32:49 +$ cat results/2023/09/29/15:32:49/runs/1/bench.md +## Complete traversal +... +``` + +To measure performance improvements/regressions of a change, compare the `main` branch to the topic branch on the full benchmark suite: + +```console +$ tailfin run bench/bench.sh --build=main --build=branch --default +``` + +This will take a few minutes. +Results from the full benchmark suite can be seen in performance-related pull requests, for example [#126]. + +[#126]: https://github.com/tavianator/bfs/pull/126 |