diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/BUILDING.md | 201 | ||||
| -rw-r--r-- | docs/CHANGELOG.md | 979 | ||||
| -rw-r--r-- | docs/CONTRIBUTING.md | 61 | ||||
| -rw-r--r-- | docs/RELATED.md | 42 | ||||
| -rw-r--r-- | docs/USAGE.md | 192 | ||||
| -rw-r--r-- | docs/bfs.1 | 934 |
6 files changed, 2409 insertions, 0 deletions
diff --git a/docs/BUILDING.md b/docs/BUILDING.md new file mode 100644 index 0000000..cb33c51 --- /dev/null +++ b/docs/BUILDING.md @@ -0,0 +1,201 @@ +Building `bfs` +============== + +A simple invocation of + + $ ./configure + $ make + +should build `bfs` successfully. + + +Configuration +------------- + +```console +$ ./configure --help +Usage: + + $ ./configure [--enable-*|--disable-*] [CC=...] [CFLAGS=...] [...] + $ make + +... +``` + +### Variables + +Variables set in the environment or on the command line will be picked up: +These variables specify binaries to run during the configuration and build process: + +<pre> +<b>MAKE</b>=<i>make</i> + <a href="https://en.wikipedia.org/wiki/Make_(software)">make</a> implementation +<b>CC</b>=<i>cc</i> + C compiler +<b>INSTALL</b>=<i>install</i> + Copy files during <i>make install</i> +<b>MKDIR</b>="<i>mkdir -p</i>" + Create directories +<b>PKG_CONFIG</b>=<i>pkg-config</i> + Detect external libraries and required build flags +<b>RM</b>="<i>rm -f</i>" + Delete files +</pre> + +These flags will be used by the build process: + +<pre> +<b>CPPFLAGS</b>="<i>-I... -D...</i>" +<b>CFLAGS</b>="<i>-W... -f...</i>" +<b>LDFLAGS</b>="<i>-L... -Wl,...</i>" + Preprocessor/compiler/linker flags + +<b>LDLIBS</b>="<i>-l... -l...</i>" + Dynamic libraries to link + +<b>EXTRA_</b>{<b>CPPFLAGS</b>,<b>CFLAGS</b>,<b>LDFLAGS</b>,<b>LDLIBS</b>}="<i>...</i>" + Adds to the default flags, instead of replacing them +</pre> + +### Build profiles + +The default flags result in a plain debug build. +Other build profiles can be enabled: + +<pre> +--enable-release + Enable optimizations, disable assertions + +--enable-<a href="https://github.com/google/sanitizers/wiki/AddressSanitizer">asan</a> +--enable-<a href="https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#stand-alone-mode">lsan</a> +--enable-<a href="https://github.com/google/sanitizers/wiki/MemorySanitizer">msan</a> +--enable-<a href="https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual">tsan</a> +--enable-<a href="https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html">ubsan</a> + Enable sanitizers + +--enable-<a href="https://gcc.gnu.org/onlinedocs/gcc/gcov/introduction-to-gcov.html">gcov</a> + Enable code coverage instrumentation +</pre> + +You can combine multiple profiles (e.g. `./configure --enable-asan --enable-ubsan`), but not all of them will work together. + +### Dependencies + +`bfs` depends on some system libraries for some of its features. +External dependencies are auto-detected by default, but you can `--enable` or `--disable` them manually: + +<pre> +--enable-<a href="https://savannah.nongnu.org/projects/acl">libacl</a> --disable-libacl +--enable-<a href="https://sites.google.com/site/fullycapable/">libcap</a> --disable-libcap +--enable-<a href="https://github.com/SELinuxProject/selinux">libselinux</a> --disable-libselinux +--enable-<a href="https://github.com/axboe/liburing">liburing</a> --disable-liburing +--enable-<a href="https://github.com/kkos/oniguruma">oniguruma</a> --disable-oniguruma +</pre> + +[`pkg-config`] is used, if available, to detect these libraries and any additional build flags they may require. +If this is undesireable, disable it by setting `PKG_CONFIG` to the empty string (`./configure PKG_CONFIG=""`). + +[`pkg-config`]: https://www.freedesktop.org/wiki/Software/pkg-config/ + +### Out-of-tree builds + +You can set up an out-of-tree build by running the `configure` script from another directory, for example: + + $ mkdir out + $ cd out + $ ../configure + $ make + + +Building +-------- + +### Targets + +The [`Makefile`](/Makefile) supports several different build targets: + +<pre> +make + The default target; builds just the <i>bfs</i> binary +make <b>all</b> + Builds everything, including the tests (but doesn't run them) + +make <b>check</b> + Builds everything, and runs all tests +make <b>unit-tests</b> + Builds and runs the unit tests +make <b>integration-tests</b> + Builds and runs the integration tests +make <b>distcheck</b> + Builds and runs the tests in multiple different configurations + +make <b>install</b> + Installs bfs globally +make <b>uninstall</b> + Uninstalls bfs + +make <b>clean</b> + Deletes all built files +make <b>distclean</b> + Also deletes files generated by ./configure +</pre> + + +Troubleshooting +--------------- + +If the build fails or behaves unexpectedly, start by enabling verbose mode: + + $ ./configure V=1 + $ make V=1 + +This will print the generated configuration and the exact commands that are executed. + +You can also check the file `gen/config.log`, which contains any errors from commands run during the configuration phase. + + +Testing +------- + +`bfs` comes with an extensive test suite which can be run with + + $ make check + +The test harness is implemented in the file [`tests/tests.sh`](/tests/tests.sh). +Individual test cases are found in `tests/*/*.sh`. +Most of them are *snapshot tests* which compare `bfs`'s output to a known-good copy saved under the matching `tests/*/*.out`. + +You can pass the name of a particular test case (or a few) to run just those tests. +For example: + + $ ./tests/tests.sh posix/basic + +If you need to update the reference snapshot, pass `--update`. +It can be handy to generate the snapshot with a different `find` implementation to ensure the output is correct, for example: + + $ ./tests/tests.sh posix/basic --bfs=find --update + +But keep in mind, other `find` implementations may not be correct. +To my knowledge, no other implementation passes even the POSIX-compatible subset of the tests: + + $ ./tests/tests.sh --bfs=find --posix + ... + tests passed: 90 + tests skipped: 3 + tests failed: 6 + +Run + + $ ./tests/tests.sh --help + +for more details. + +### Validation + +A more thorough testsuite is run by the [CI](https://github.com/tavianator/bfs/actions) and to validate releases. +It builds `bfs` in multiple configurations to test for latent bugs, memory leaks, 32-bit compatibility, etc. +You can run it yourself with + + $ make distcheck + +Some of these tests require `sudo`, and will prompt for your password if necessary. diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md new file mode 100644 index 0000000..62b6480 --- /dev/null +++ b/docs/CHANGELOG.md @@ -0,0 +1,979 @@ +3.* +=== + +3.2 +--- + +**May 2, 2024** + +### New features + +- New `-limit N` action that quits immediately after `N` results + +- Implemented `-context` (from GNU find) for matching SELinux contexts ([#27](https://github.com/tavianator/bfs/issues/27)) + +- Implemented `-printf %Z` for printing SELinux contexts + +### Changes + +- The build system has been rewritten, and there is now a configure step: + + $ ./configure + $ make + + See `./configure --help` or [docs/BUILDING.md](/docs/BUILDING.md) for more details. + +- Improved platform support + - Implemented `-acl` on Solaris/Illumos + - Implemented `-xattr` on DragonFly BSD + +### Bug fixes + +- Fixed some rarely-used code paths that clean up after allocation failures + +3.1.3 +----- + +**March 6, 2024** + +### Bug fixes + +- On Linux, the `io_uring` feature probing introduced in `bfs` 3.1.2 only applied to one thread, causing all other threads to avoid using io_uring entirely. + The probe results are now copied to all threads correctly. + ([`f64f76b`](https://github.com/tavianator/bfs/commit/f64f76b55400b71e8576ed7e4a377eb5ef9576aa)) + + +3.1.2 +----- + +**February 29, 2024** + +### Bug fixes + +- On Linux, we now check for supported `io_uring` operations before using them, which should fix `bfs` on 5.X series kernels that support `io_uring` but not all of `openat()`/`close()`/`statx()` ([`8bc72d6`](https://github.com/tavianator/bfs/commit/8bc72d6c20c5e38783c4956c4d9fde9b3ee9140c)) + +- Fixed a test failure triggered by certain filesystem types for `/tmp` ([#131](https://github.com/tavianator/bfs/issues/131)) + +- Fixed parsing and interpretation of timezone offsets for explicit reference times used in `-*since` and `-newerXt` ([`a9f3cde`](https://github.com/tavianator/bfs/commit/a9f3cde30426b546ba6e3172e1a7951213a72049)) + +- Fixed the build on m68k ([`c749c11`](https://github.com/tavianator/bfs/commit/c749c11b04444ca40941dd2ddc5802faed148f6a)) + + +3.1.1 +----- + +**February 16, 2024** + +### Changes + +- Performance and scalability improvements + +- The file count in `bfs -status` now has a thousands separator + + +3.1 +--- + +**February 6, 2024** + +### New features + +- On Linux, `bfs` now uses [io_uring](https://en.wikipedia.org/wiki/Io_uring) for async I/O + +- On all platforms, `bfs` can now perform `stat()` calls in parallel, accelerating queries like `-links`, `-newer`, and `-size`, as well as colorized output + +- On FreeBSD, `-type w` now works to find whiteouts like the system `find` + +### Changes + +- Improved `bfs -j2` performance ([`b2ab7a1`](https://github.com/tavianator/bfs/commit/b2ab7a151fca517f4879e76e626ec85ad3de97c7)) + +- Optimized `-exec` by using `posix_spawn()` when possible, which can avoid the overhead of `fork()` ([`95fbde1`](https://github.com/tavianator/bfs/commit/95fbde17a66377b6fbe7ff1f014301dbbf09270d)) + +- `-execdir` and `-okdir` are now rejected if `$PATH` contains a relative path, matching the behaviour of GNU find ([`163baf1`](https://github.com/tavianator/bfs/commit/163baf1c9af13be0ce705b133e41e0c3d6427398)) + +- Leading whitespace is no longer accepted in integer command line arguments like `-links ' 1'` ([`e0d7dc5`](https://github.com/tavianator/bfs/commit/e0d7dc5dfd7bdaa62b6bc18e9c1cce00bbe08577)) + +### Bug fixes + +- `-quit` and `-exit` could be ignored in the iterative deepening modes (`-S {ids,eds}`). + This is now fixed ([`670ebd9`](https://github.com/tavianator/bfs/commit/670ebd97fb431e830b1500b2e7e8013b121fb2c5)). + The bug was introduced in version 3.0.3 (commit [`5f16169`]). + +- Fixed two possible errors in sort mode (`-s`): + - Too many open files ([`710c083`](https://github.com/tavianator/bfs/commit/710c083ff02eb1cc5b8daa6778784f3d1cd3c08d)) + - Out of memory ([`76ffc8d`](https://github.com/tavianator/bfs/commit/76ffc8d30cb1160d55d855d8ac630a2b9075fbcf)) + +- Fixed handling of FreeBSD union mounts ([`3ac3bee`](https://github.com/tavianator/bfs/commit/3ac3bee7b0d9c9be693415206efa664bf4a7d4a7)) + +- Fixed `NO_COLOR` handling when it's set to the empty string ([`79aee58`](https://github.com/tavianator/bfs/commit/79aee58a4621d01c4b1e98c332775f3b87213ddb)) + +- Fixed some portability issues: + - [OpenBSD](https://github.com/tavianator/bfs/compare/ee200c07643801c8b53e5b80df704ecbf77a884e...79f1521b0e628be72bed3a648f0ae90b62fc69b8) + - [NetBSD](https://github.com/tavianator/bfs/compare/683f2c41c72efcb82ce866e3dcc311ac9bd8b66d...6435684a7d515e18247ae1b3dd9ec8681fee22d0) + - [DragonFly BSD](https://github.com/tavianator/bfs/compare/08867473e75e8e20ca76c7fb181204839e28b271...45fb1d952c3b262278a3b22e9c7d60cca19a5407) + - [Illumos](https://github.com/tavianator/bfs/compare/4010140cb748cc4f7f57b0a3d514485796c665ce...ae94cdc00136685abe61d55e1e357caaa636d785) + + +3.0.4 +----- + +**October 12, 2023** + +### Bug fixes + +- Fixed a segfault when reporting errors under musl ([`d40eb87`]) + +[`d40eb87`]: https://github.com/tavianator/bfs/commit/d40eb87cc00f50a5debb8899eacb7fcf1065badf + + +3.0.3 +----- + +**October 12, 2023** + +### Changes + +- Iterative deepening modes (`-S {ids,eds}`) were optimized by delaying teardown until the very end ([`5f16169`]) + +- Parallel depth-first search (`-S dfs`) was optimized to avoid enqueueing every file separately ([`2572273`]) + +### Bug fixes + +- Iterative deepening modes (`-S {ids,eds}`) were performing iterative *breadth*-first searches since `bfs` 3.0, negating any advantages they may have had over normal breadth-first search. + They now do iterative *depth*-first searches as expected. + ([`a029d95`]) + +- Fixed a linked-list corruption that could lead to an infinite loop on macOS and other non-Linux, non-FreeBSD platforms ([`773f4a4`]) + +[`5f16169`]: https://github.com/tavianator/bfs/commit/5f1616912ba3a7a23ce6bce02df3791b73da38ab +[`2572273`]: https://github.com/tavianator/bfs/commit/257227326fe60fe70e80433fd34d1ebcb2f9f623 +[`a029d95`]: https://github.com/tavianator/bfs/commit/a029d95b5736a74879f32089514a5a6b63d6efbc +[`773f4a4`]: https://github.com/tavianator/bfs/commit/773f4a446f03da62d88e6d17be49fdc0a3e38465 + + +3.0.2 +----- + +**September 6, 2023** + +### Changes + +- `-files0-from` now allows an empty set of paths to be given, matching GNU findutils 4.9.0 + +- Reduced memory consumption in multi-threaded searches + +- Many man page updates + +### Bug fixes + +- Fixed an out-of-bounds memory read that could occur when escaping a string containing an incomplete multi-byte character + + +3.0.1 +----- + +**July 18, 2023** + +### Bug fixes + +- Traversal fixes that mostly affect large directory trees ([#107]) + + - `bfs` could encounter `EMFILE`, close a file, and retry many times, particularly with `-j1` + + - Breadth-first search could become highly unbalanced, negating many of the benefits of `bfs` + + - On non-{Linux,FreeBSD} plaforms, directories could stay open longer than necessary, consuming extra memory + +[#107]: https://github.com/tavianator/bfs/pull/107 + + +3.0 +--- + +**July 13, 2023** + +### New features + +- `bfs` now reads directories asynchronously and in parallel ([#101]). + Performance is significantly improved as a result. + Parallelism is controlled by the new `-j` flag, e.g. `-j1`, `-j2`, etc. + +[#101]: https://github.com/tavianator/bfs/issues/101 + +### Changes + +- `bfs` now uses the [C17] standard version, up from C11 + +- Due to [#101], `bfs` now requires some additional C and POSIX features: + - [Standard C atomics] (`<stdatomic.h>`) + - [POSIX threads] (`<pthread.h>`) + +- `$LS_COLORS` extensions written in different cases (e.g. `*.jpg=35:*.JPG=01;35`) are now matched case-sensitively, to match the new behaviour of GNU ls since coreutils version 9.2 + +- Added a warning/error if `$LS_COLORS` can't be parsed, depending on whether `-color` is requested explicitly + +- Filenames with control characters are now escaped when printing with `-color` + +- Build flags like `WITH_ONIGURUMA` have been renamed to `USE_ONIGURUMA` + +[C17]: https://en.cppreference.com/w/c/17 +[Standard C atomics]: https://en.cppreference.com/w/c/atomic +[POSIX threads]: https://pubs.opengroup.org/onlinepubs/9699919799/idx/threads.html + +### Bug fixes + +- Fixed handling of the "normal text" color (`no` in `$LS_COLORS`) to match GNU ls + + +2.* +=== + +2.6.3 +----- + +**January 31, 2023** + +- Fixed running the tests as root on Linux [`8b24de3`] + +- Fixed some tests on Android [`2724dfb`] [`0a5a80c`] + +- Stopped relying on non-POSIX touch(1) features in the tests. + This should fix the tests on at least OpenBSD. + [`2d5edb3`] + +- User/group caches are now filled lazily instead of eagerly [`b41dca5`] + +- More caches and I/O streams are flushed before -exec/-ok [`f98a1c4`] + +- Fixed various memory safety issues found by fuzzing \ + [`712b137`] [`5ce883d`] [`da02def`] [`c55e855`] + +- Fixed a test failure on certain macOS versions [`8b24de3`] + +- Mitigated a race condition when determining filesystem types ([#97]) + +- Lots of refactoring and optimization + +[`8b24de3`]: https://github.com/tavianator/bfs/commit/8b24de3882ff5a3e33b82ab20bb4eadf134cf559 +[`2724dfb`]: https://github.com/tavianator/bfs/commit/2724dfbd17552f892a0d8b39b96cbe9e49d66fdb +[`0a5a80c`]: https://github.com/tavianator/bfs/commit/0a5a80c98cc7e5d8735b615fa197a6cff2bb08cc +[`2d5edb3`]: https://github.com/tavianator/bfs/commit/2d5edb37b924715b4fbee4d917ac334c773fca61 +[`b41dca5`]: https://github.com/tavianator/bfs/commit/b41dca52762c5188638236ae81b9f4597bb29ac9 +[`f98a1c4`]: https://github.com/tavianator/bfs/commit/f98a1c4a1cf61ff7d6483388ca1fac365fb0b31b +[`712b137`]: https://github.com/tavianator/bfs/commit/712b13756a09014ef730c8f9b96da4dc2f09b762 +[`5ce883d`]: https://github.com/tavianator/bfs/commit/5ce883daaafc69f83b01dac5db0647e9662a6e87 +[`da02def`]: https://github.com/tavianator/bfs/commit/da02defb91c3a1bda0ea7e653d81f997f1c8884a +[`c55e855`]: https://github.com/tavianator/bfs/commit/c55e85580df10c5afdc6fc0710e756a456aa8e93 +[`8b24de3`]: https://github.com/tavianator/bfs/commit/8b24de3882ff5a3e33b82ab20bb4eadf134cf559 +[#97]: https://github.com/tavianator/bfs/issues/97 + + +2.6.2 +----- + +**October 21, 2022** + +- Fixed use of uninitialized memory on parsing errors involving `-fprintf` + +- Fixed Android build issues ([#96]) + +- Refactored the test suite + +[#96]: https://github.com/tavianator/bfs/issues/96 + + +2.6.1 +----- + +**July 7, 2022** + +- Fix `stat()` errors on GNU Hurd systems with glibc older than 2.35 + +- Added fish shell tab completion ([#94]). + Thanks @xfgusta! + +[#94]: https://github.com/tavianator/bfs/pull/94 + + +2.6 +--- + +**May 21, 2022** + +- Fixed deleting large NFS directories on FreeBSD ([#67]). + +- Added support for a `bfs`-specific `BFS_COLORS` environment variable. + +- Refactored the build system, directory structure, and documentation ([#88], [#89], [#91]). + Thanks @ElectronicsArchiver! + +- Added `zsh` completion ([#86]). + Thanks @VorpalBlade! + +- Updated the default color scheme to match GNU coreutils 9.1. + Files with capabilities set are no longer colored differently by default, resulting in a significant performance improvement. + +- Became less aggressive at triggering automounts + +- Added support for out-of-tree builds with `BUILDDIR` + +[#67]: https://github.com/tavianator/bfs/issues/67 +[#86]: https://github.com/tavianator/bfs/issues/86 +[#88]: https://github.com/tavianator/bfs/issues/88 +[#89]: https://github.com/tavianator/bfs/issues/89 +[#91]: https://github.com/tavianator/bfs/issues/91 + + +2.5 +--- + +**March 27, 2022** + +- Added compiler-style context for errors and warnings. + Errors look like this: + + $ bfs -nam needle + bfs: error: bfs -nam needle + bfs: error: ~~~~ + bfs: error: Unknown argument; did you mean -name? + + and warnings look like this: + + $ bfs -print -name 'needle' + bfs: warning: bfs -print -name needle + bfs: warning: ~~~~~~~~~~~~ + bfs: warning: The result of this expression is ignored. + +- Updated from C99 to C11 + +- Fixed the tests when built against musl + +- Fixed a build error reported on Manjaro + + +2.4.1 +----- + +**February 24, 2022** + +- Fixed the build when Oniguruma is not installed in the default search paths ([#82]) + +- Fixed string encoding bugs with Oniguruma enabled + +- Fixed regex error reporting bugs + +[#82]: https://github.com/tavianator/bfs/issues/82 + + +2.4 +--- + +**February 22, 2022** + +- Added the Oniguruma regular expression library as an (optional, but enabled by default) dependency ([#81]). + Oniguruma supports more regular expression syntax types than the POSIX regex API, and often performs better. + To build `bfs` without this new dependency, do `make WITH_ONIGURUMA=` to disable it. + Thanks @data-man! + +- Added support for the `ed`, `emacs`, `grep`, and `sed` regular expression types ([#21]) + +- Before executing a process with `-exec[dir]`/`-ok[dir]`, `bfs` now ensures all output streams are flushed. + Previously, I/O from subprocesses could be interleaved unpredictably with buffered I/O from `bfs` itself. + +[#81]: https://github.com/tavianator/bfs/pull/81 +[#21]: https://github.com/tavianator/bfs/issues/21 + + +2.3.1 +----- + +**January 21, 2022** + +- Fixed the build on Debian kFreeBSD + +- Fixed a crash on GNU Hurd when piping bfs's output + +- Fixed a double-`close()` on non-Linux platforms if `fdopendir()` fails + +- Reduced memory allocations on startup + + +2.3 +--- + +**November 25, 2021** + +- More tweaks to `PAGER` and `LESS` handling for `bfs -help` ([#76]) + +- Use 512-byte blocks for `-ls` when `POSIXLY_CORRECT` is set ([#77]) + +- Implemented `-files0-from FILE` to take a list of `'\0'`-separated starting paths. + GNU find will implement the same feature in an upcoming release. + +- Added colors to `-printf` output ([#62]) + +- Faster recovery from `E2BIG` during `-exec` + +[#76]: https://github.com/tavianator/bfs/issues/76 +[#77]: https://github.com/tavianator/bfs/issues/77 +[#62]: https://github.com/tavianator/bfs/issues/62 + + +2.2.1 +----- + +**June 2, 2021** + +- Fixed some incorrect coloring of broken links when links are being followed (`-L`) + +- Made the tests work when run as root by dropping privileges. + This may be helpful for certain packaging or CI environments, but is not recommended. + +- Treat empty `PAGER` and `LESS` environment variables like they're unset, for `bfs -help` ([#71]). + Thanks @markus-oberhumer! + +- The soft `RLIMIT_NOFILE` is now raised automatically to a fairly large value when possible. + This provides a minor performance benefit for large directory trees. + +- Implemented time units for `-mtime` as found in FreeBSD find ([#75]) + +[#71]: https://github.com/tavianator/bfs/issues/71 +[#75]: https://github.com/tavianator/bfs/issues/75 + + +2.2 +--- + +**March 6, 2021** + +- Fixed `-hidden` on hidden start paths + +- Added a Bash completion script. + Thanks @bmundt6! + +- Fixed rounding in `-used`. + Corresponding fixes were made to GNU find in version 4.8.0. + +- Optimized the open directory representation. + On Linux, much libc overhead is bypassed by issuing syscalls directly. + On all platforms, a few fewer syscalls and open file descriptors will be used. + +- Implemented `-flags` from BSD find + + +2.1 +--- + +**November 11, 2020** + +- Added a new `-status` option that displays the search progress in a bar at the bottom of the terminal + +- Fixed an optimizer bug introduced in version 2.0 that affected some combinations of `-user`/`-group` and `-nouser`/`-nogroup` + + +2.0 +--- + +**October 14, 2020** + +- [#8]: New `-exclude <expression>` syntax to more easily and reliably filter out paths. + For example: + + bfs -name config -exclude -name .git + + will find all files named `config`, without searching any directories (or files) named `.git`. + In this case, the same effect could have been achieved (more awkwardly) with `-prune`: + + bfs ! \( -name .git -prune \) -name config + + But `-exclude` will work in more cases: + + # -exclude works with -depth, while -prune doesn't: + bfs -depth -name config -exclude -name .git + + # -exclude applies even to paths below the minimum depth: + bfs -mindepth 3 -name config -exclude -name .git + +- [#30]: `-nohidden` is now equivalent to `-exclude -hidden`. + This changes the behavior of command lines like + + bfs -type f -nohidden + + to do what was intended. + +- Optimized the iterative deepening (`-S ids`) implementation + +- Added a new search strategy: exponential deepening search (`-S eds`). + This strategy provides many of the benefits of iterative deepening, but much faster due to fewer re-traversals. + +- Fixed an optimizer bug that could skip `-empty`/`-xtype` if they didn't always lead to an action + +- Implemented `-xattrname` to find files with a particular extended attribute (from macOS find) + +- Made `-printf %l` still respect the width specifier (e.g. `%10l`) for non-links, to match GNU find + +- Made `bfs` fail if `-color` is given explicitly and `LS_COLORS` can't be parsed, rather than falling back to non-colored output + +[#8]: https://github.com/tavianator/bfs/issues/8 +[#30]: https://github.com/tavianator/bfs/issues/30 + + +1.* +=== + +1.7 +--- + +**April 22, 2020** + +- Fixed `-ls` printing numeric IDs instead of user/group names in large directory trees +- Cached the user and group tables for a performance boost +- Fixed interpretation of "default" ACLs +- Implemented `-s` flag to sort results + + +1.6 +--- + +**February 25, 2020** + +- Implemented `-newerXt` (explicit reference times), `-since`, `-asince`, etc. +- Fixed `-empty` to skip special files (pipes, devices, sockets, etc.) + + +1.5.2 +----- + +**January 9, 2020** + +- Fixed the build on NetBSD +- Added support for NFSv4 ACLs on FreeBSD +- Added a `+` after the file mode for files with ACLs in `-ls` +- Supported more file types (whiteouts, doors) in symbolic modes for `-ls`/`-printf %M` +- Implemented `-xattr` on FreeBSD + + +1.5.1 +----- + +**September 14, 2019** + +- Added a warning to `-mount`, since it will change behaviour in the next POSIX revision +- Added a workaround for environments that block `statx()` with `seccomp()`, like older Docker +- Fixed coloring of nonexistent leading directories +- Avoided calling `stat()` on all mount points at startup + + +1.5 +--- + +**June 27, 2019** + +- New `-xattr` predicate to find files with extended attributes +- Fixed the `-acl` implementation on macOS +- Implemented depth-first (`-S dfs`) and iterative deepening search (`-S ids`) +- Piped `-help` output into `$PAGER` by default +- Fixed crashes on some invalid `LS_COLORS` values + + +1.4.1 +----- + +**April 5, 2019** + +- Added a nicer error message when the tests are run as root +- Fixed detection of comparison expressions with signs, to match GNU find for things like `-uid ++10` +- Added support for https://no-color.org/ +- Decreased the number of `stat()` calls necessary in some cases + + +1.4 +--- + +**April 15, 2019** + +- New `-unique` option that filters out duplicate files ([#48]) +- Optimized the file coloring implementation +- Fixed the coloring implementation to match GNU ls more closely in many corner cases + - Implemented escape sequence parsing for `LS_COLORS` + - Implemented `ln=target` for coloring links like their targets + - Fixed the order of fallbacks used when some color keys are unset +- Add a workaround for incorrect file types for bind-mounted files on Linux ([#37]) + +[#48]: https://github.com/tavianator/bfs/issues/48 +[#37]: https://github.com/tavianator/bfs/issues/37 + + +1.3.3 +----- + +**February 10, 2019** + +- Fixed unpredictable behaviour for empty responses to `-ok`/`-okdir` caused by an uninitialized string +- Writing to standard output now causes `bfs` to fail if the descriptor was closed +- Fixed incomplete file coloring in error messages +- Added some data flow optimizations +- Fixed `-nogroup`/`-nouser` in big directory trees +- Added `-type w` for whiteouts, as supported by FreeBSD `find` +- Re-wrote the `-help` message and manual page + + +1.3.2 +----- + +**January 11, 2019** + +- Fixed an out-of-bounds read if LS_COLORS doesn't end with a `:` +- Allowed multiple debug flags to be specified like `-D opt,tree` + + +1.3.1 +----- + +**January 3, 2019** + +- Fixed some portability problems affecting FreeBSD + + +1.3 +--- + +**January 2, 2019** + +New features: + +- `-acl` finds files with non-trivial Access Control Lists (from FreeBSD) +- `-capable` finds files with capabilities set +- `-D all` turns on all debugging flags at once + +Fixes: + +- `LS_COLORS` handling has been improved: + - Extension colors are now case-insensitive like GNU `ls` + - `or` (orphan) and `mi` (missing) files are now treated differently + - Default colors can be unset with `di=00` or similar + - Specific colors fall back to more general colors when unspecified in more places + - `LS_COLORS` no longer needs a trailing colon +- `-ls`/`-fls` now prints the major/minor numbers for device nodes +- `-exec ;` is rejected rather than segfaulting +- `bfs` now builds on old Linux versions that require `-lrt` for POSIX timers +- For files whose access/change/modification times can't be read, `bfs` no longer fails unless those times are needed for tests +- The testsuite is now more correct and portable + + +1.2.4 +----- + +**September 24, 2018** + +- GNU find compatibility fixes for `-printf`: + - `%Y` now prints `?` if an error occurs resolving the link + - `%B` is now supported for birth/creation time (as well as `%W`/`%w`) + - All standard `strftime()` formats are supported, not just the ones from the GNU find manual +- Optimizations are now re-run if any expressions are reordered +- `-exec` and friends no longer leave zombie processes around when `exec()` fails + + +1.2.3 +----- + +**July 15, 2018** + +- Fixed `test_depth_error` on filesystems that don't fill in `d_type` +- Fixed the build on Linux architectures that don't have the `statx()` syscall (ia64, sh4) +- Fixed use of AT_EMPTY_PATH for fstatat on systems that don't support it (Hurd) +- Fixed `ARG_MAX` accounting on architectures with large pages (ppc64le) +- Fixed the build against the upcoming glibc 2.28 release that includes its own `statx()` wrapper + + +1.2.2 +----- + +**June 23, 2018** + +- Minor bug fixes: + - Fixed `-exec ... '{}' +` argument size tracking after recovering from `E2BIG` + - Fixed `-fstype` if `/proc` is available but `/etc/mtab` is not + - Fixed an uninitialized variable when given `-perm +rw...` + - Fixed some potential "error: 'path': Success" messages +- Reduced reliance on GNU coreutils in the testsuite +- Refactored and simplified the internals of `bftw()` + + +1.2.1 +----- + +**February 8, 2018** + +- Performance optimizations + + +1.2 +--- + +**January 20, 2018** + +- Added support for the `-perm +7777` syntax deprecated by GNU find (equivalent to `-perm /7777`), for compatibility with BSD finds +- Added support for file birth/creation times on platforms that report it + - `-Bmin`/`-Btime`/`-Bnewer` + - `B` flag for `-newerXY` + - `%w` and `%Wk` directives for `-printf` + - Uses the `statx(2)` system call on new enough Linux kernels +- More robustness to `E2BIG` added to the `-exec` implementation + + +1.1.4 +----- + +**October 27, 2017** + +- Added a man page +- Fixed cases where multiple actions write to the same file +- Report errors that occur when closing files/flushing streams +- Fixed "argument list too long" errors with `-exec ... '{}' +` + + +1.1.3 +----- + +**October 4, 2017** + +- Refactored the optimizer +- Implemented data flow optimizations + + +1.1.2 +----- + +**September 10, 2017** + +- Fixed `-samefile` and similar predicates when passed broken symbolic links +- Implemented `-fstype` on Solaris +- Fixed `-fstype` under musl +- Implemented `-D search` +- Implemented a cost-based optimizer + + +1.1.1 +----- + +**August 10, 2017** + +- Re-licensed under the BSD Zero Clause License +- Fixed some corner cases with `-exec` and `-ok` parsing + + +1.1 +--- + +**July 22, 2017** + +- Implemented some primaries from NetBSD `find`: + - `-exit [STATUS]` (like `-quit`, but with an optional explicit exit status) + - `-printx` (escape special characters for `xargs`) + - `-rm` (alias for `-delete`) +- Warn if `-prune` will have no effect due to `-depth` +- Handle y/n prompts according to the user's locale +- Prompt the user to correct typos without having to re-run `bfs` +- Fixed handling of paths longer than `PATH_MAX` +- Fixed spurious "Inappropriate ioctl for device" errors when redirecting `-exec ... +` output +- Fixed the handling of paths that treat a file as a directory (e.g. `a/b/c` where `a/b` is a regular file) +- Fixed an expression optimizer bug that broke command lines like `bfs -name '*' -o -print` + + +1.0.2 +----- + +**June 15, 2017** + +Bugfix release. + +- Fixed handling of \0 inside -printf format strings +- Fixed `-perm` interpretation of permcopy actions (e.g. `u=rw,g=r`) + + +1.0.1 +----- + +**May 17, 2017** + +Bugfix release. + +- Portability fixes that mostly affect GNU Hurd +- Implemented `-D exec` +- Made `-quit` not disable the implicit `-print` + + +1.0 +--- + +**April 24, 2017** + +This is the first release of bfs with support for all of GNU find's primitives. + +Changes since 0.96: + +- Implemented `-fstype` +- Implemented `-exec/-execdir ... +` +- Implemented BSD's `-X` +- Fixed the tests under Bash 3 (mostly for macOS) +- Some minor optimizations and fixes + + +0.* +=== + + +0.96 +---- + +**March 11, 2017** + +73/76 GNU find features supported. + +- Implemented -nouser and -nogroup +- Implemented -printf and -fprintf +- Implemented -ls and -fls +- Implemented -type with multiple types at once (e.g. -type f,d,l) +- Fixed 32-bit builds +- Fixed -lname on "symlinks" in Linux /proc +- Fixed -quit to take effect as soon as it's reached +- Stopped redirecting standard input from /dev/null for -ok and -okdir, as that violates POSIX +- Many test suite improvements + + +0.88 +---- + +**December 20, 2016** + +67/76 GNU find features supported. + +- Fixed the build on macOS, and some other UNIXes +- Implemented `-regex`, `-iregex`, `-regextype`, and BSD's `-E` +- Implemented `-x` (same as `-mount`/`-xdev`) from BSD +- Implemented `-mnewer` (same as `-newer`) from BSD +- Implemented `-depth N` from BSD +- Implemented `-sparse` from FreeBSD +- Implemented the `T` and `P` suffices for `-size`, for BSD compatibility +- Added support for `-gid NAME` and `-uid NAME` as in BSD + + +0.84.1 +------ + +**November 24, 2016** + +Bugfix release. + +- Fixed [#7] again +- Like GNU find, don't print warnings by default if standard input is not a terminal +- Redirect standard input from /dev/null for -ok and -okdir +- Skip . when -delete'ing +- Fixed -execdir when the root path has no slashes +- Fixed -execdir in / +- Support -perm +MODE for symbolic modes +- Fixed the build on FreeBSD + +[#7]: https://github.com/tavianator/bfs/issues/7 + + +0.84 +---- + +**October 29, 2016** + +64/76 GNU find features supported. + +- Spelling suggestion improvements +- Handle `--` +- (Untested) support for exotic file types like doors, ports, and whiteouts +- Improved robustness in the face of closed std{in,out,err} +- Fixed the build on macOS +- Implement `-ignore_readdir_race`, `-noignore_readdir_race` +- Implement `-perm` + + +0.82 +---- + +**September 4, 2016** + +62/76 GNU find features supported. + +- Rework optimization levels + - `-O1` + - Simple boolean simplification + - `-O2` + - Purity-based optimizations, allowing side-effect-free tests like `-name` or `-type` to be moved or removed + - `-O3` (**default**): + - Re-order tests to reduce the expected cost (TODO) + - `-O4` + - Aggressive optimizations that may have surprising effects on warning/error messages and runtime, but should not otherwise affect the results + - `-Ofast`: + - Always the highest level, currently the same as `-O4` +- Color files with multiple hard links correctly +- Treat `-`, `)`, and `,` as paths when required to by POSIX + - `)` and `,` are only supported before the expression begins +- Implement `-D opt` +- Implement `-D rates` +- Implement `-fprint` +- Implement `-fprint0` +- Implement BSD's `-f` +- Suggest fixes for typo'd arguments + +0.79 +---- + +**May 27, 2016** + +60/76 GNU find features supported. + +- Remove an errant debug `printf()` from `-used` +- Implement the `{} ;` variants of `-exec`, `-execdir`, `-ok`, and `-okdir` + + +0.74 +---- + +**March 12, 2016** + +56/76 GNU find features supported. + +- Color broken symlinks correctly +- Fix [#7] +- Fix `-daystart`'s rounding of midnight +- Implement (most of) `-newerXY` +- Implement `-used` +- Implement `-size` + +[#7]: https://github.com/tavianator/bfs/issues/7 + + +0.70 +---- + +**February 23, 2016** + +53/76 GNU find features supported. + +- New `make install` and `make uninstall` targets +- Squelch non-positional warnings for `-follow` +- Reduce memory footprint by as much as 64% by closing `DIR*`s earlier +- Speed up `bfs` by ~5% by using a better FD cache eviction policy +- Fix infinite recursion when evaluating `! expr` +- Optimize unused pure expressions (e.g. `-empty -a -false`) +- Optimize double-negation (e.g. `! ! -name foo`) +- Implement `-D stat` and `-D tree` +- Implement `-O` + + +0.67 +---- + +**February 14, 2016** + +Initial release. + +51/76 GNU find features supported. diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md new file mode 100644 index 0000000..099157d --- /dev/null +++ b/docs/CONTRIBUTING.md @@ -0,0 +1,61 @@ +Contributing to `bfs` +===================== + +License +------- + +`bfs` is licensed under the [Zero-Clause BSD License](https://opensource.org/licenses/0BSD), a maximally permissive license. +Contributions must use the same license. + +Individual files contain the following tag instead of the full license text: + + SPDX-License-Identifier: 0BSD + +This enables machine processing of license information based on the SPDX License Identifiers that are available here: https://spdx.org/licenses/ + + +Implementation +-------------- + +`bfs` is written in [C](https://en.wikipedia.org/wiki/C_(programming_language)), specifically [C17](https://en.wikipedia.org/wiki/C17_(C_standard_revision)). +You can get a feel for the coding style by skimming the source code. +[`main.c`](/src/main.c) contains an overview of the rest of source files. +A quick summary: + +- Tabs for indentation, spaces for alignment. +- Most types and functions should be namespaced with `bfs_`. + Exceptions are made for things that could be generally useful outside of `bfs`. +- Error handling follows the C standard library conventions: return a nonzero `int` or a `NULL` pointer, with the error code in `errno`. + All failure cases should be handled, including `malloc()` failures. +- `goto` is not considered harmful for cleaning up in error paths. + + +Tests +----- + +`bfs` includes an extensive test suite. +See the [build documentation](BUILDING.md#testing) for details on running the tests. + +Test cases are grouped by the standard or `find` implementation that supports the tested feature(s): + +| Group | Description | +|---------------------------------|---------------------------------------| +| [`tests/posix`](/tests/posix) | POSIX compatibility tests | +| [`tests/bsd`](/tests/bsd) | BSD `find` features | +| [`tests/gnu`](/tests/gnu) | GNU `find` features | +| [`tests/common`](/tests/common) | Features common to BSD and GNU `find` | +| [`tests/bfs`](/tests/bfs) | `bfs`-specific tests | + +Both new features and bug fixes should have associated tests. +To add a test, create a new `*.sh` file in the appropriate group. +Snapshot tests use the `bfs_diff` function to automatically compare the generated and expected outputs. +For example, + +```bash +# posix/something.sh +bfs_diff basic -name something +``` + +`basic` is one of the directory trees generated for test cases; others include `links`, `loops`, `deep`, and `rainbow`. + +Run `./tests/tests.sh posix/something --update` to generate the reference snapshot (and don't forget to `git add` it). diff --git a/docs/RELATED.md b/docs/RELATED.md new file mode 100644 index 0000000..cf52b70 --- /dev/null +++ b/docs/RELATED.md @@ -0,0 +1,42 @@ +# Related utilities + +There are many tools that can be used to find files. +This is a catalogue of some of the most important/interesting ones. + +## `find`-compatible + +### System `find` implementations + +These `find` implementations are commonly installed as the system `find` utility in UNIX-like operating systems: + +- [GNU findutils](https://www.gnu.org/software/findutils/) ([manual](https://www.gnu.org/software/findutils/manual/html_node/find_html/index.html), [source](https://git.savannah.gnu.org/cgit/findutils.git)) +- BSD `find` + - FreeBSD `find` ([manual](https://www.freebsd.org/cgi/man.cgi?find(1)), [source](https://cgit.freebsd.org/src/tree/usr.bin/find)) + - OpenBSD `find` ([manual](https://man.openbsd.org/find.1), [source](https://cvsweb.openbsd.org/src/usr.bin/find/)) + - NetBSD `find` ([manual](https://man.netbsd.org/find.1), [source](http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/find/)) +- macOS `find` ([manual](https://ss64.com/osx/find.html), [source](https://github.com/apple-oss-distributions/shell_cmds/tree/main/find)) +- Solaris `find` + - [Illumos](https://illumos.org/) `find` ([manual](https://illumos.org/man/1/find), [source](https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/find/find.c)) + +### Alternative `find` implementations + +These are not usually installed as the system `find`, but are designed to be `find`-compatible + +- [`bfs`](https://tavianator.com/projects/bfs.html) ([manual](https://man.archlinux.org/man/bfs.1), [source](https://github.com/tavianator/bfs)) +- [schilytools](https://codeberg.org/schilytools/schilytools) `sfind` ([source](https://codeberg.org/schilytools/schilytools/src/branch/master/sfind)) +- [BusyBox](https://busybox.net/) `find` ([manual](https://busybox.net/downloads/BusyBox.html#find), [source](https://git.busybox.net/busybox/tree/findutils/find.c)) +- [ToyBox](http://landley.net/toybox/) `find` ([manual](http://landley.net/toybox/help.html#find), [source](https://github.com/landley/toybox/blob/master/toys/posix/find.c)) +- uutils `find` ([source](https://github.com/uutils/findutils)) + +## `find` alternatives + +These utilities are not `find`-compatible, but serve a similar purpose: + +- [`fd`](https://github.com/sharkdp/fd): A simple, fast and user-friendly alternative to 'find' +- `locate` + - [GNU `locate`](https://www.gnu.org/software/findutils/locate) + - [`mlocate`](https://pagure.io/mlocate) ([manual](), [source](https://pagure.io/mlocate/tree/master)) + - [`plocate`](https://plocate.sesse.net/) ([manual](https://plocate.sesse.net/plocate.1.html), [source](https://git.sesse.net/?p=plocate)) +- [`walk`](https://github.com/google/walk): Plan 9 style utilities to replace find(1) +- [fselect](https://github.com/jhspetersson/fselect): Find files with SQL-like queries +- [rawhide](https://github.com/raforg/rawhide): find files using pretty C expressions diff --git a/docs/USAGE.md b/docs/USAGE.md new file mode 100644 index 0000000..70f8475 --- /dev/null +++ b/docs/USAGE.md @@ -0,0 +1,192 @@ +Using `bfs` +=========== + +`bfs` has the same command line syntax as `find`, and almost any `find` command that works with a major `find` implementation will also work with `bfs`. +When invoked with no arguments, `bfs` will list everything under the current directory recursively, breadth-first: + +```console +$ bfs +. +./LICENSE +./Makefile +./README.md +./completions +./docs +./src +./tests +./completions/bfs.bash +./completions/bfs.zsh +./docs/BUILDING.md +./docs/CHANGELOG.md +./docs/CONTRIBUTING.md +./docs/USAGE.md +./docs/bfs.1 +... +``` + + +Paths +----- + +Arguments that don't begin with `-` are treated as paths to search. +If one or more paths are specified, they are used instead of the current directory: + +```console +$ bfs /usr/bin /usr/lib +/usr/bin +/usr/lib +/usr/bin/bfs +... +/usr/lib/libc.so +... +``` + + +Expressions +----------- + +Arguments that start with `-` form an *expression* which `bfs` evaluates to filter the matched files, and to do things with the files that match. +The most common expression is probably `-name`, which matches filenames against a glob pattern: + +```console +$ bfs -name '*.md' +./README.md +./docs/BUILDING.md +./docs/CHANGELOG.md +./docs/CONTRIBUTING.md +./docs/USAGE.md +``` + +### Operators + +When you put multiple expressions next to each other, both of them must match: + +```console +$ bfs -name '*.md' -name '*ING*' +./docs/BUILDING.md +./docs/CONTRIBUTING.md +``` + +This works because the expressions are implicitly combined with *logical and*. +You could be explicit by writing + +```console +$ bfs -name '*.md' -and -name '*ING'` +``` + +There are other operators like `-or`: + +```console +$ bfs -name '*.md' -or -name 'bfs.*' +./README.md +./completions/bfs.bash +./completions/bfs.fish +./completions/bfs.zsh +./docs/BUILDING.md +./docs/CHANGELOG.md +./docs/CONTRIBUTING.md +./docs/USAGE.md +./docs/bfs.1 +``` + +and `-not`: + +```console +$ bfs -name '*.md' -and -not -name '*ING*' +./README.md +./docs/CHANGELOG.md +./docs/USAGE.md +``` + +### Actions + +Every `bfs` expression returns either `true` or `false`. +For expressions like `-name`, that's all they do. +But some expressions, called *actions*, have other side effects. + +If no actions are included in the expression, `bfs` adds the `-print` action automatically, which is why the above examples actually print any output. +The default `-print` is supressed if any actions are given explicitly. +Available actions include printing with alternate formats (`-ls`, `-printf`, etc.), executing commands (`-exec`, `-execdir`, etc.), deleting files (`-delete`), and stopping the search (`-quit`, `-exit`). + + +Extensions +---------- + +`bfs` implements a few extensions not found in other `find` implementations. + +### `-exclude` + +The `-exclude` operator skips an entire subtree whenever an expression matches. +For example, `-exclude -name .git` will exclude any files or directories named `.git` from the search results. +`-exclude` is easier to use than the standard `-prune` action; compare + + bfs -name config -exclude -name .git + +to the equivalent + + find ! \( -name .git -prune \) -name config + +Unlike `-prune`, `-exclude` even works in combination with `-depth`/`-delete`. + +--- + +### `-limit` + +The `-limit N` action makes `bfs` quit once it gets evaluated `N` times. +Placing it after an action like `-print` limits the number of results that get printed, for example: + +```console +$ bfs -s -type f -name '*.txt' +./1.txt +./2.txt +./3.txt +./4.txt +$ bfs -s -type f -name '*.txt' -print -limit 2 +./1.txt +./2.txt +``` + +This is similar to + +```console +$ bfs -s -type f -name '*.txt' | head -n2 +``` + +but more powerful because you can apply separate limits to different expressions: + +```console +$ bfs \( -name '*.txt' -print -limit 3 -o -name '*.log' -print -limit 4 \) -limit 5 +[At most 3 .txt files, at most 4 .log files, and at most 5 in total] +``` + +and more efficient because it will quit immediately. +When piping to `head`, `bfs` will only quit *after* it tries to output too many results. + +--- + +### `-hidden`/`-nohidden` + +`-hidden` matches "hidden" files (dotfiles). +`bfs -hidden` is effectively shorthand for + + find \( -name '.*' -not -name . -not -name .. \) + +`-nohidden` is equivalent to `-exclude -hidden`. + +--- + +### `-unique` + +This option ensures that `bfs` only visits each file once, even if it's reachable through multiple hard or symbolic links. +It's particularly useful when following symbolic links (`-L`). + +--- + +### `-color`/`-nocolor` + +When printing to a terminal, `bfs` automatically colors paths like GNU `ls`, according to the `LS_COLORS` environment variable. +The `-color` and `-nocolor` options override the automatic behavior, which may be handy when you want to preserve colors through a pipe: + + bfs -color | less -R + +If the [`NO_COLOR`](https://no-color.org/) environment variable is set, colors will be disabled by default. diff --git a/docs/bfs.1 b/docs/bfs.1 new file mode 100644 index 0000000..54166ab --- /dev/null +++ b/docs/bfs.1 @@ -0,0 +1,934 @@ +.TH BFS 1 +.SH NAME +bfs \- breadth-first search for your files +.SH SYNOPSIS +.B bfs +.RB [ flags ...] +.RI [ paths ...] +.RB [ expression ...] +.PP +flags +.RB ( \-H / \-L / \-P +etc.), +.IR paths , +and +.B expressions +may be freely mixed in any order. +.SH DESCRIPTION +.B bfs +is a breadth-first version of the UNIX +.BR find (1) +command. +.PP +.B bfs +supports almost every feature from every major +.BR find (1) +implementation, so your existing command lines should work as-is. +It also adds some features of its own, such as a more forgiving command line parser and some additional options. +.PP +Each +.I path +specified on the command line is treated as a starting path to search through. +If no paths are specified, the current directory +.RI ( . ) +is searched by default. +.PP +Like +.BR find (1), +.B bfs +interprets its arguments as a short-circuiting Boolean expression. +For example, +.PP +.nf +.RS +.B bfs \\\( \-name '*.txt' \-or \-lname '*.txt' \\\\) \-and \-print +.RE +.fi +.PP +will print the all the paths that are either .txt files or symbolic links to .txt files. +.B \-and +is implied between two consecutive expressions, so this is equivalent: +.PP +.nf +.RS +.B bfs \\\( \-name '*.txt' \-or \-lname '*.txt' \\\\) \-print +.RE +.fi +.PP +Finally, +.B \-print +is implied if no actions are specified, so this too is equivalent: +.PP +.nf +.RS +.B bfs \-name '*.txt' \-or \-lname '*.txt' +.RE +.fi +.PP +Most options that take a numeric argument +.I N +will also accept +.I \-N +or +.IR +N . +.IR \-N +means "less than +.IR N ," +and +.I +N +means "greater than +.IR N ." +.SH FLAGS +.TP +.B \-H +Follow symbolic links on the command line, but not while searching. +.TP +.B \-L +Follow all symbolic links. +.TP +.B \-P +Never follow symbolic links (the default). +.TP +.B \-E +Use extended regular expressions (same as \fB\-regextype \fIposix-extended\fR). +.TP +.B \-X +Filter out files with +.RB non- xargs (1)-safe +names. +.TP +.B \-d +Search in post-order (same as +.BR \-depth ). +.TP +.B \-s +Visit directory entries in sorted order. +The sorting takes place within each directory separately, which makes it different from +.B bfs ... | +.BR sort , +but still provides a deterministic ordering. +.TP +.B \-x +Don't descend into other mount points (same as \fB\-xdev\fR). +.TP +\fB\-f \fIPATH\fR +Treat +.I PATH +as a path to search (useful if it begins with a dash). +.PP +.TP +\fB\-D \fIFLAG\fR +Turn on a debugging flag (see +.B \-D +.IR help ). +.PP +\fB\-O\fIN\fR +.RS +Enable optimization level +.I N +(default: +.IR 3 ). +.TP +.BI \-O 0 +Disable all optimizations. +.TP +.BI \-O 1 +Basic logical simplifications. +.TP +.BI \-O 2 +All +.BI \-O 1 +optimizations, plus dead code elimination and data flow analysis. +.TP +.BI \-O 3 +All +.BI \-O 2 +optimizations, plus re-order expressions to reduce expected cost. +.TP +\fB\-O\fI4\fR/\fB\-O\fIfast\fR +All optimizations, including aggressive optimizations that may alter the observed behavior in corner cases. +.RE +.PP +\fB\-S \fIbfs\fR|\fIdfs\fR|\fIids\fR|\fIeds\fR +.RS +Choose the search strategy. +.TP +.I bfs +Breadth-first search (the default). +.TP +.I dfs +Depth-first search. +Uses less memory than breadth-first search, but is typically slower to return relevant results. +.TP +.I ids +Iterative deepening search. +Performs repeated depth-first searches with increasing depth limits. +This gives results in the same order as breadth-first search, but with the reduced memory consumption of depth-first search. +Tends to be very slow in practice, so use it only if you absolutely need breadth-first ordering, but +.B \-S +.I bfs +consumes too much memory. +.TP +.I eds +Exponential deepening search. +A compromise between breadth- and depth-first search, which searches exponentially increasing depth ranges (e.g. 0-1, 1-2, 2-4, 4-8, etc.). +Provides many of the benefits of breadth-first search with depth-first's reduced memory consumption. +Typically far faster than +.B \-S +.IR ids . +.RE +.TP +\fB\-j\fIN\fR +Search with +.I N +threads in parallel (default: number of CPUs, up to +.IR 8 ). +.SH OPERATORS +.TP +\fB( \fIexpression \fB)\fR +Parentheses are used for grouping expressions together. +You'll probably have to write +.B \\\\( +.I expression +.B \\\\) +to avoid the parentheses being interpreted by the shell. +.PP +\fB! \fIexpression\fR +.br +\fB\-not \fIexpression\fR +.RS +The "not" operator: returns the negation of the truth value of the +.IR expression . +You may have to write \fB\\! \fIexpression\fR to avoid \fB!\fR being interpreted by the shell. +.RE +.PP +\fIexpression\fR \fIexpression\fR +.br +\fIexpression \fB\-a \fIexpression\fR +.br +\fIexpression \fB\-and \fIexpression\fR +.RS +Short-circuiting "and" operator: if the left-hand +.I expression +is +.BR true , +returns the right-hand +.IR expression ; +otherwise, returns +.BR false . +.RE +.PP +\fIexpression \fB\-o \fIexpression\fR +.br +\fIexpression \fB\-or \fIexpression\fR +.RS +Short-circuiting "or" operator: if the left-hand +.I expression +is +.BR false , +returns the right-hand +.IR expression ; +otherwise, returns +.BR true . +.RE +.TP +\fIexpression \fB, \fIexpression\fR +The "comma" operator: evaluates the left-hand +.I expression +but discards the result, returning the right-hand +.IR expression . +.SH SPECIAL FORMS +.TP +\fB\-exclude \fIexpression\fR +Exclude all paths matching the +.I expression +from the search. +This is more powerful than +.BR \-prune , +because it applies even when the expression wouldn't otherwise be evaluated, due to +.B \-depth +or +.B \-mindepth +for example. +Exclusions are always applied before other expressions, so it may be least confusing to put them first on the command line. +.PP +.B \-help +.br +.B \-\-help +.RS +Print usage information, and exit immediately (without parsing the rest of the command line or processing any files). +.RE +.PP +.B \-version +.br +.B \-\-version +.RS +Print version information, and exit immediately. +.RE +.SH OPTIONS +.PP +.B \-color +.br +.B \-nocolor +.RS +Turn colors on or off (default: +.B \-color +if outputting to a terminal, +.B \-nocolor +otherwise). +.RE +.TP +.B \-daystart +Measure time relative to the start of today. +.TP +.B \-depth +Search in post-order (descendents first). +.TP +.B \-follow +Follow all symbolic links (same as +.BR \-L ). +.TP +\fB\-files0\-from \fIFILE\fR +Treat the NUL ('\\0')-separated paths in +.I FILE +as starting points for the search. +Pass +.B \-files0\-from +.I \- +to read the paths from standard input. +.PP +\fB\-ignore_readdir_race\fR +.br +\fB\-noignore_readdir_race\fR +.RS +Whether to report an error if +.B bfs +detects that the file tree is modified during the search (default: +.BR \-noignore_readdir_race ). +.RE +.PP +\fB\-maxdepth \fIN\fR +.br +\fB\-mindepth \fIN\fR +.RS +Ignore files deeper/shallower than +.IR N . +.RE +.TP +.B \-mount +Don't descend into other mount points (same as +.B \-xdev +for now, but will skip mount points entirely in the future). +.TP +.B \-nohidden +Exclude hidden files and directories. +.TP +.B \-noleaf +Ignored; for compatibility with GNU find. +.TP +\fB\-regextype \fITYPE\fR +Use +.IR TYPE -flavored +regular expressions. +The possible types are +.RS +.TP +.I posix-basic +POSIX basic regular expressions (the default). +.TP +.I posix-extended +POSIX extended resular expressions. +.TP +.I ed +Like +.BR ed (1) +(same as +.IR posix-basic ). +.TP +.I emacs +Like +.BR emacs (1). +.TP +.I grep +Like +.BR grep (1). +.TP +.I sed +Like +.BR sed (1) +(same as +.IR posix-basic ). +.PP +See +.BR regex (7) +for a description of regular expression syntax. +.RE +.TP +.B \-status +Display a status bar while searching. +.TP +.B \-unique +Skip any files that have already been seen. +Particularly useful along with +.BR \-L . +.PP +.B \-warn +.br +.B \-nowarn +.RS +Turn on or off warnings about the command line. +.RE +.TP +.B \-xdev +Don't descend into other mount points. +.SH TESTS +.TP +.B \-acl +Find files with a non-trivial Access Control List +.RB ( acl (5)). +.PP +\fB\-amin\fR [\fI\-+\fR]\fIN\fR +.br +\fB\-Bmin\fR [\fI\-+\fR]\fIN\fR +.br +\fB\-cmin\fR [\fI\-+\fR]\fIN\fR +.br +\fB\-mmin\fR [\fI\-+\fR]\fIN\fR +.RS +Find files +.BR a ccessed/ B irthed/ c hanged/ m odified +.I N +minutes ago. +.RE +.PP +\fB\-anewer \fIFILE\fR +.br +\fB\-Bnewer \fIFILE\fR +.br +\fB\-cnewer \fIFILE\fR +.br +\fB\-mnewer \fIFILE\fR +.RS +Find files +.BR a ccessed/ B irthed/ c hanged/ m odified +more recently than +.I FILE +was modified. +.RE +.PP +\fB\-asince \fITIME\fR +.br +\fB\-Bsince \fITIME\fR +.br +\fB\-csince \fITIME\fR +.br +\fB\-msince \fITIME\fR +.RS +Find files +.BR a ccessed/ B irthed/ c hanged/ m odified +more recently than the ISO 8601-style timestamp +.IR TIME . +See +.BI \-newer XY +for examples of the timestamp format. +.RE +.PP +\fB\-atime\fR [\fI\-+\fR]\fIN\fR +.br +\fB\-Btime\fR [\fI\-+\fR]\fIN\fR +.br +\fB\-ctime\fR [\fI\-+\fR]\fIN\fR +.br +\fB\-mtime\fR [\fI\-+\fR]\fIN\fR +.RS +Find files +.BR a ccessed/ B irthed/ c hanged/ m odified +.I N +days ago. +.RE +.TP +.B \-capable +Find files with POSIX.1e +.BR capabilities (7) +set. +.TP +\fB\-context \fIGLOB\fR +Find files whose SELinux context matches the +.IR GLOB . +.TP +\fB\-depth\fR [\fI\-+\fR]\fIN\fR +Find files with depth +.IR N . +.TP +.B \-empty +Find empty files/directories. +.PP +.B \-executable +.br +.B \-readable +.br +.B \-writable +.RS +Find files the current user can execute/read/write. +.RE +.PP +.B \-false +.br +.B \-true +.RS +Always false/true. +.RE +.TP +\fB\-fstype \fITYPE\fR +Find files on file systems with the given +.IR TYPE . +.PP +\fB\-gid\fR [\fI\-+\fR]\fIN\fR +.br +\fB\-uid\fR [\fI\-+\fR]\fIN\fR +.RS +Find files owned by group/user ID +.IR N . +.RE +.PP +\fB\-group \fINAME\fR +.br +\fB\-user \fINAME\fR +.RS +Find files owned by the group/user +.IR NAME . +.RE +.TP +.B \-hidden +Find hidden files (those beginning with +.IR . ). +.PP +\fB\-ilname \fIGLOB\fR +.br +\fB\-iname \fIGLOB\fR +.br +\fB\-ipath \fIGLOB\fR +.br +\fB\-iregex \fIREGEX\fR +.br +\fB\-iwholename \fIGLOB\fR +.RS +Case-insensitive versions of +.BR \-lname / \-name / \-path / \-regex / \-wholename . +.RE +.TP +\fB\-inum\fR [\fI\-+\fR]\fIN\fR +Find files with inode number +.IR N . +.TP +\fB\-links\fR [\fI\-+\fR]\fIN\fR +Find files with +.I N +hard links. +.TP +\fB\-lname \fIGLOB\fR +Find symbolic links whose target matches the +.IR GLOB . +.TP +\fB\-name \fIGLOB\fR +Find files whose name matches the +.IR GLOB . +.TP +\fB\-newer \fIFILE\fR +Find files newer than +.IR FILE . +.TP +\fB\-newer\fIXY \fIREFERENCE\fR +Find files whose +.I X +time is newer than the +.I Y +time of +.IR REFERENCE . +.I X +and +.I Y +can be any of +.RI [ aBcm ] +.RI ( a ccess/ B irth/ c hange/ m odification). +.I Y +may also be +.I t +to parse +.I REFERENCE +as an ISO 8601-style timestamp. For example: +.PP +.RS +.nf +\(bu \fI1991-12-14\fR +\(bu \fI1991-12-14T03:00\fR +\(bu \fI1991-12-14T03:00-07:00\fR +\(bu '\fI1991-12-14 10:00Z\fR' +.fi +.RE +.PP +.B \-nogroup +.br +.B \-nouser +.RS +Find files owned by nonexistent groups/users. +.RE +.PP +\fB\-path \fIGLOB\fR +.br +\fB\-wholename \fIGLOB\fR +.RS +Find files whose entire path matches the +.IR GLOB . +.RE +.TP +\fB\-perm\fR [\fI\-\fR]\fIMODE\fR +Find files with a matching mode. +.TP +\fB\-regex \fIREGEX\fR +Find files whose entire path matches the regular expression +.IR REGEX . +.TP +\fB\-samefile \fIFILE\fR +Find hard links to +.IR FILE . +.TP +\fB\-since \fITIME\fR +Find files modified since the ISO 8601-style timestamp +.IR TIME . +See +.BI \-newer XY +for examples of the timestamp format. +.TP +\fB\-size\fR [\fI\-+\fR]\fIN\fR[\fIcwbkMGTP\fR] +Find files with the given size. +The unit can be one of +.PP +.RS +.nf +\(bu \fIc\fRhars (1 byte) +\(bu \fIw\fRords (2 bytes) +\(bu \fIb\fRlocks (512 bytes, the default) +\(bu \fIk\fRiB (1024 bytes) +\(bu \fIM\fRiB (1024 kiB) +\(bu \fIG\fRiB (1024 MiB) +\(bu \fIT\fRiB (1024 GiB) +\(bu \fIP\fRiB (1024 TiB) +.fi +.RE +.TP +.B \-sparse +Find files that occupy fewer disk blocks than expected. +.TP +\fB\-type\fR [\fIbcdlpfswD\fR] +Find files of the given type. +The possible types are +.PP +.RS +\(bu +.IR b lock +device +.br +\(bu +.IR c haracter +device +.br +\(bu +.IR d irectory +.br +\(bu +.IR l ink +(symbolic) +.br +\(bu +.IR p ipe +.br +\(bu +.IR f ile +(regular) +.br +\(bu +.IR s ocket +.br +\(bu +.IR w hiteout +.br +\(bu +.IR D oor +.PP +Multiple types can be given at once, separated by commas. +For example, +.B \-type +.I d,f +matches both directories and regular files. +.RE +.TP +\fB\-used\fR [\fI\-+\fR]\fIN\fR +Find files last accessed +.I N +days after they were changed. +.TP +.B \-xattr +Find files with extended attributes +.RB ( xattr (7)). +.TP +\fB\-xattrname\fR \fINAME\fR +Find files with the extended attribute +.IR NAME . +.TP +\fB\-xtype\fR [\fIbcdlpfswD\fR] +Find files of the given type, following links when +.B \-type +would not, and vice versa. +.SH ACTIONS +.PP +.B \-delete +.br +.B \-rm +.RS +Delete any found files (implies \fB-depth\fR). +.RE +.TP +\fB\-exec \fIcommand ... {} ;\fR +Execute a command. +.TP +\fB\-exec \fIcommand ... {} +\fR +Execute a command with multiple files at once. +.TP +\fB\-ok \fIcommand ... {} ;\fR +Prompt the user whether to execute a command. +.PP +\fB\-execdir \fIcommand ... {} ;\fR +.br +\fB\-execdir \fIcommand ... {} +\fR +.br +\fB\-okdir \fIcommand ... {} ;\fR +.RS +Like +.BR \-exec / \-ok , +but run the command in the same directory as the found file(s). +.RE +.TP +\fB\-exit\fR [\fISTATUS\fR] +Exit immediately with the given status +.RI ( 0 +if unspecified). +.PP +\fB\-fls \fIFILE\fR +.br +\fB\-fprint \fIFILE\fR +.br +\fB\-fprint0 \fIFILE\fR +.br +\fB\-fprintf \fIFILE FORMAT\fR +.RS +Like +.BR \-ls / \-print / \-print0 / \-printf , +but write to +.I FILE +instead of standard output. +.RE +.TP +\fB\-limit \fIN\fR +Quit once this action is evaluated +.I N +times. +.TP +.B \-ls +List files like +.B ls +.IR \-dils . +.TP +.B \-print +Print the path to the found file. +.TP +.B \-print0 +Like +.BR \-print , +but use the null character ('\\0') as a separator rather than newlines. +Useful in conjunction with +.B xargs +.IR \-0 . +.TP +\fB\-printf \fIFORMAT\fR +Print according to a format string (see +.BR find (1)). +These additional format directives are supported: +.RS +.TP +%w +The file's birth time, in the same format as %a/%c/%t. +.TP +.RI %W k +Field +.I k +of the file's birth time, in the same format as +.RI %A k /%C k /%T k . +.RE +.TP +.B \-printx +Like +.BR \-print , +but escape whitespace and quotation characters, to make the output safe for +.BR xargs (1). +Consider using +.B \-print0 +and +.B xargs +.I \-0 +instead. +.TP +.B \-prune +Don't descend into this directory. +This has no effect if +.B \-depth +is enabled (either explicitly, or implicitly by +.BR \-delete ). +Use +.B \-exclude +instead in that case. +.TP +.B \-quit +Quit immediately. +.SH ENVIRONMENT +Certain environment variables affect the behavior of +.BR bfs . +.PP +.B LANG +.br +.B LC_* +.RS +Specifies the +.BR locale (7) +in use for various things. +.B bfs +is not (yet) translated to any languages except English, but the locale will still affect the format of printed values. +Yes/no prompts (e.g. from +.BR \-ok ) +will also be interpreted according to the current locale. +.RE +.PP +.B LS_COLORS +.br +.B BFS_COLORS +.RS +Controls the colors used when displaying file paths if +.B \-color +is enabled. +.B bfs +interprets +.B LS_COLORS +the same way GNU +.BR ls (1) +does (see +.BR dir_colors (5)). +.B BFS_COLORS +can be used to customize +.B bfs +without affecting other commands. +.RE +.TP +.B NO_COLOR +Causes +.B bfs +to default to +.B \-nocolor +if it is set (see https://no-color.org/). +.TP +.B PAGER +Specifies the pager used for +.B \-help +output. +Defaults to +.BR less (1), +if found on the current +.BR PATH , +otherwise +.BR more (1). +.TP +.B PATH +Used to resolve executables for +.BR \-exec [ dir ] +and +.BR \-ok [ dir ]. +.TP +.B POSIXLY_CORRECT +Makes +.B bfs +conform more strictly to the POSIX.1-2017 specification for +.BR find (1). +Currently this has two effects: +.RS +.IP \(bu +Disables warnings by default, because POSIX prohibits writing to standard error (except for the +.B \-ok +prompt), unless the command also fails with a non-zero exit status. +.IP \(bu +Makes +.B \-ls +and +.B \-fls +use 512-byte blocks instead of 1024-byte blocks. +(POSIX does not specify these actions, but BSD +.BR find (1) +implementations use 512-byte blocks, while GNU +.BR find (1) +uses 1024-byte blocks by default.) +.PP +It does not disable +.BR bfs 's +various extensions to the base POSIX functionality. +.B POSIXLY_CORRECT +has the same effects on GNU +.BR find (1). +.RE +.SH EXAMPLES +.TP +.B bfs +With no arguments, +.B bfs +prints all files under the current directory in breadth-first order. +.TP +.B bfs \-name '*.txt' +Prints all the .txt files under the current directory. +.B *.txt +is quoted to ensure the glob is processed by +.B bfs +rather than the shell. +.TP +\fBbfs \-name access_log \-L \fI/var\fR +Finds all files named +.B access_log +under +.IR /var , +following symbolic links. +.B bfs +allows flags and paths to appear anywhere on the command line. +.TP +\fBbfs \fI~ \fB\-not \-user $USER\fR +Prints all files in your home directory not owned by you. +.TP +.B bfs \-xtype l +Finds broken symbolic links. +.TP +.B bfs \-name config \-exclude \-name .git +Finds all files named +.BR config, +skipping every +.B .git +directory. +.TP +.B bfs \-type f \-executable \-exec strip {} + +Runs +.BR strip (1) +on all executable files it finds, passing it multiple files at a time. +.SH BUGS +https://github.com/tavianator/bfs/issues +.SH AUTHOR +Tavian Barnes <tavianator@tavianator.com> +.PP +https://tavianator.com/projects/bfs.html +.SH SEE ALSO +.BR find (1), +.BR locate (1), +.BR xargs (1) |