diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/BUILDING.md | 225 | ||||
| -rw-r--r-- | docs/CHANGELOG.md | 252 | ||||
| -rw-r--r-- | docs/CONTRIBUTING.md (renamed from docs/HACKING.md) | 32 | ||||
| -rw-r--r-- | docs/RELATED.md | 42 | ||||
| -rw-r--r-- | docs/USAGE.md | 56 | ||||
| -rw-r--r-- | docs/bfs.1 | 215 |
6 files changed, 670 insertions, 152 deletions
diff --git a/docs/BUILDING.md b/docs/BUILDING.md index 932845b..cb33c51 100644 --- a/docs/BUILDING.md +++ b/docs/BUILDING.md @@ -1,103 +1,157 @@ Building `bfs` ============== -Compiling ---------- - -`bfs` uses [GNU Make](https://www.gnu.org/software/make/) as its build system. A simple invocation of + $ ./configure $ make -should build `bfs` successfully, with no additional steps necessary. -As usual with `make`, you can run a [parallel build](https://www.gnu.org/software/make/manual/html_node/Parallel.html) with `-j`. -For example, to use all your cores, run `make -j$(nproc)`. +should build `bfs` successfully. -### Targets -| Command | Description | -|------------------|---------------------------------------------------------------| -| `make` | Builds just the `bfs` binary | -| `make all` | Builds everything, including the tests (but doesn't run them) | -| `make check` | Builds everything, and runs the tests | -| `make install` | Installs `bfs` (with man page, shell completions, etc.) | -| `make uninstall` | Uninstalls `bfs` | - -### Flag-like targets - -The build system provides a few shorthand targets for handy configurations: - -| Command | Description | -|----------------|-------------------------------------------------------------| -| `make release` | Build `bfs` with optimizations, LTO, and without assertions | -| `make asan` | Enable [AddressSanitizer] | -| `make lsan` | Enable [LeakSanitizer] | -| `make msan` | Enable [MemorySanitizer] | -| `make tsan` | Enable [ThreadSanitizer] | -| `make ubsan` | Enable [UndefinedBehaviorSanitizer] | -| `make gcov` | Enable [code coverage] | - -[AddressSanitizer]: https://github.com/google/sanitizers/wiki/AddressSanitizer -[LeakSanitizer]: https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#stand-alone-mode -[MemorySanitizer]: https://github.com/google/sanitizers/wiki/MemorySanitizer -[ThreadSanitizer]: https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual -[UndefinedBehaviorSanitizer]: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html -[code coverage]: https://gcc.gnu.org/onlinedocs/gcc/Gcov.html - -You can combine multiple flags and other targets (e.g. `make asan ubsan check`), but not all of them will work together. - -### Flags - -Other flags are controlled with `make` variables and/or environment variables. -Here are some of the common ones; check the [`Makefile`](/Makefile) for more. - -| Flag | Description | -|----------------------------------|---------------------------------------------| -| `CC` | The C compiler to use, e.g. `make CC=clang` | -| `CFLAGS`<br>`EXTRA_CFLAGS` | Override/add to the default compiler flags | -| `LDFLAGS`<br>`EXTRA_LDFLAGS` | Override/add to the linker flags | -| `WITH_ACL`<br>`WITH_ATTR`<br>... | Enable/disable [optional dependencies] | -| `TEST_FLAGS` | `tests.sh` flags for `make check` | -| `BUILDDIR` | The build output directory (default: `.`) | -| `DESTDIR` | The root directory for `make install` | -| `PREFIX` | The installation prefix (default: `/usr`) | -| `MANDIR` | The man page installation directory | - -[optional dependencies]: #dependencies +Configuration +------------- + +```console +$ ./configure --help +Usage: + + $ ./configure [--enable-*|--disable-*] [CC=...] [CFLAGS=...] [...] + $ make + +... +``` + +### Variables + +Variables set in the environment or on the command line will be picked up: +These variables specify binaries to run during the configuration and build process: + +<pre> +<b>MAKE</b>=<i>make</i> + <a href="https://en.wikipedia.org/wiki/Make_(software)">make</a> implementation +<b>CC</b>=<i>cc</i> + C compiler +<b>INSTALL</b>=<i>install</i> + Copy files during <i>make install</i> +<b>MKDIR</b>="<i>mkdir -p</i>" + Create directories +<b>PKG_CONFIG</b>=<i>pkg-config</i> + Detect external libraries and required build flags +<b>RM</b>="<i>rm -f</i>" + Delete files +</pre> + +These flags will be used by the build process: + +<pre> +<b>CPPFLAGS</b>="<i>-I... -D...</i>" +<b>CFLAGS</b>="<i>-W... -f...</i>" +<b>LDFLAGS</b>="<i>-L... -Wl,...</i>" + Preprocessor/compiler/linker flags + +<b>LDLIBS</b>="<i>-l... -l...</i>" + Dynamic libraries to link + +<b>EXTRA_</b>{<b>CPPFLAGS</b>,<b>CFLAGS</b>,<b>LDFLAGS</b>,<b>LDLIBS</b>}="<i>...</i>" + Adds to the default flags, instead of replacing them +</pre> + +### Build profiles + +The default flags result in a plain debug build. +Other build profiles can be enabled: + +<pre> +--enable-release + Enable optimizations, disable assertions + +--enable-<a href="https://github.com/google/sanitizers/wiki/AddressSanitizer">asan</a> +--enable-<a href="https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#stand-alone-mode">lsan</a> +--enable-<a href="https://github.com/google/sanitizers/wiki/MemorySanitizer">msan</a> +--enable-<a href="https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual">tsan</a> +--enable-<a href="https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html">ubsan</a> + Enable sanitizers + +--enable-<a href="https://gcc.gnu.org/onlinedocs/gcc/gcov/introduction-to-gcov.html">gcov</a> + Enable code coverage instrumentation +</pre> + +You can combine multiple profiles (e.g. `./configure --enable-asan --enable-ubsan`), but not all of them will work together. ### Dependencies `bfs` depends on some system libraries for some of its features. -These dependencies are optional, and can be turned off at build time if necessary by setting the appropriate variable to the empty string (e.g. `make WITH_ONIGURUMA=`). +External dependencies are auto-detected by default, but you can `--enable` or `--disable` them manually: -| Dependency | Platforms | `make` flag | -|-------------|------------|------------------| -| [acl] | Linux only | `WITH_ACL` | -| [attr] | Linux only | `WITH_ATTR` | -| [libcap] | Linux only | `WITH_LIBCAP` | -| [Oniguruma] | All | `WITH_ONIGURUMA` | +<pre> +--enable-<a href="https://savannah.nongnu.org/projects/acl">libacl</a> --disable-libacl +--enable-<a href="https://sites.google.com/site/fullycapable/">libcap</a> --disable-libcap +--enable-<a href="https://github.com/SELinuxProject/selinux">libselinux</a> --disable-libselinux +--enable-<a href="https://github.com/axboe/liburing">liburing</a> --disable-liburing +--enable-<a href="https://github.com/kkos/oniguruma">oniguruma</a> --disable-oniguruma +</pre> -[acl]: https://savannah.nongnu.org/projects/acl -[attr]: https://savannah.nongnu.org/projects/attr -[libcap]: https://sites.google.com/site/fullycapable/ -[Oniguruma]: https://github.com/kkos/oniguruma +[`pkg-config`] is used, if available, to detect these libraries and any additional build flags they may require. +If this is undesireable, disable it by setting `PKG_CONFIG` to the empty string (`./configure PKG_CONFIG=""`). -### Dependency tracking +[`pkg-config`]: https://www.freedesktop.org/wiki/Software/pkg-config/ -The build system automatically tracks header dependencies with the `-M` family of compiler options (see `DEPFLAGS` in the [`Makefile`](/Makefile)). -So if you edit a header file, `make` will rebuild the necessary object files ensuring they don't go out of sync. +### Out-of-tree builds -We go one step further than most build systems by tracking the flags that were used for the previous compilation. -That means you can change configurations without having to `make clean`. -For example, +You can set up an out-of-tree build by running the `configure` script from another directory, for example: + $ mkdir out + $ cd out + $ ../configure $ make - $ make release -will build the project in debug mode and then rebuild it in release mode. -A side effect of this may be surprising: `make check` by itself will rebuild the project in the default configuration. -To test a different configuration, you'll have to repeat it (e.g. `make release check`). +Building +-------- + +### Targets + +The [`Makefile`](/Makefile) supports several different build targets: + +<pre> +make + The default target; builds just the <i>bfs</i> binary +make <b>all</b> + Builds everything, including the tests (but doesn't run them) + +make <b>check</b> + Builds everything, and runs all tests +make <b>unit-tests</b> + Builds and runs the unit tests +make <b>integration-tests</b> + Builds and runs the integration tests +make <b>distcheck</b> + Builds and runs the tests in multiple different configurations + +make <b>install</b> + Installs bfs globally +make <b>uninstall</b> + Uninstalls bfs + +make <b>clean</b> + Deletes all built files +make <b>distclean</b> + Also deletes files generated by ./configure +</pre> + + +Troubleshooting +--------------- + +If the build fails or behaves unexpectedly, start by enabling verbose mode: + + $ ./configure V=1 + $ make V=1 + +This will print the generated configuration and the exact commands that are executed. + +You can also check the file `gen/config.log`, which contains any errors from commands run during the configuration phase. Testing @@ -107,27 +161,28 @@ Testing $ make check -Most of the testsuite is implemented in the file [`tests.sh`](/tests.sh). -This script contains hundreds of separate test cases. -Most of them are *snapshot tests* which compare `bfs`'s output to a known-good copy saved under [`tests`](/tests). +The test harness is implemented in the file [`tests/tests.sh`](/tests/tests.sh). +Individual test cases are found in `tests/*/*.sh`. +Most of them are *snapshot tests* which compare `bfs`'s output to a known-good copy saved under the matching `tests/*/*.out`. You can pass the name of a particular test case (or a few) to run just those tests. For example: - $ ./tests/tests.sh test_basic + $ ./tests/tests.sh posix/basic If you need to update the reference snapshot, pass `--update`. It can be handy to generate the snapshot with a different `find` implementation to ensure the output is correct, for example: - $ ./tests/tests.sh test_basic --bfs=find --update + $ ./tests/tests.sh posix/basic --bfs=find --update But keep in mind, other `find` implementations may not be correct. To my knowledge, no other implementation passes even the POSIX-compatible subset of the tests: $ ./tests/tests.sh --bfs=find --posix ... - tests passed: 89 - tests failed: 5 + tests passed: 90 + tests skipped: 3 + tests failed: 6 Run diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 9d02e8e..672c2b4 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -1,6 +1,258 @@ +3.* +=== + +3.1.3 +----- + +**March 6, 2024** + +### Bug fixes + +- On Linux, the `io_uring` feature probing introduced in `bfs` 3.1.2 only applied to one thread, causing all other threads to avoid using io_uring entirely. + The probe results are now copied to all threads correctly. + ([`f64f76b`](https://github.com/tavianator/bfs/commit/f64f76b55400b71e8576ed7e4a377eb5ef9576aa)) + + +3.1.2 +----- + +**February 29, 2024** + +### Bug fixes + +- On Linux, we now check for supported `io_uring` operations before using them, which should fix `bfs` on 5.X series kernels that support `io_uring` but not all of `openat()`/`close()`/`statx()` ([`8bc72d6`](https://github.com/tavianator/bfs/commit/8bc72d6c20c5e38783c4956c4d9fde9b3ee9140c)) + +- Fixed a test failure triggered by certain filesystem types for `/tmp` ([#131](https://github.com/tavianator/bfs/issues/131)) + +- Fixed parsing and interpretation of timezone offsets for explicit reference times used in `-*since` and `-newerXt` ([`a9f3cde`](https://github.com/tavianator/bfs/commit/a9f3cde30426b546ba6e3172e1a7951213a72049)) + +- Fixed the build on m68k ([`c749c11`](https://github.com/tavianator/bfs/commit/c749c11b04444ca40941dd2ddc5802faed148f6a)) + + +3.1.1 +----- + +**February 16, 2024** + +### Changes + +- Performance and scalability improvements + +- The file count in `bfs -status` now has a thousands separator + + +3.1 +--- + +**February 6, 2024** + +### New features + +- On Linux, `bfs` now uses [io_uring](https://en.wikipedia.org/wiki/Io_uring) for async I/O + +- On all platforms, `bfs` can now perform `stat()` calls in parallel, accelerating queries like `-links`, `-newer`, and `-size`, as well as colorized output + +- On FreeBSD, `-type w` now works to find whiteouts like the system `find` + +### Changes + +- Improved `bfs -j2` performance ([`b2ab7a1`](https://github.com/tavianator/bfs/commit/b2ab7a151fca517f4879e76e626ec85ad3de97c7)) + +- Optimized `-exec` by using `posix_spawn()` when possible, which can avoid the overhead of `fork()` ([`95fbde1`](https://github.com/tavianator/bfs/commit/95fbde17a66377b6fbe7ff1f014301dbbf09270d)) + +- `-execdir` and `-okdir` are now rejected if `$PATH` contains a relative path, matching the behaviour of GNU find ([`163baf1`](https://github.com/tavianator/bfs/commit/163baf1c9af13be0ce705b133e41e0c3d6427398)) + +- Leading whitespace is no longer accepted in integer command line arguments like `-links ' 1'` ([`e0d7dc5`](https://github.com/tavianator/bfs/commit/e0d7dc5dfd7bdaa62b6bc18e9c1cce00bbe08577)) + +### Bug fixes + +- `-quit` and `-exit` could be ignored in the iterative deepening modes (`-S {ids,eds}`). + This is now fixed ([`670ebd9`](https://github.com/tavianator/bfs/commit/670ebd97fb431e830b1500b2e7e8013b121fb2c5)). + The bug was introduced in version 3.0.3 (commit [`5f16169`]). + +- Fixed two possible errors in sort mode (`-s`): + - Too many open files ([`710c083`](https://github.com/tavianator/bfs/commit/710c083ff02eb1cc5b8daa6778784f3d1cd3c08d)) + - Out of memory ([`76ffc8d`](https://github.com/tavianator/bfs/commit/76ffc8d30cb1160d55d855d8ac630a2b9075fbcf)) + +- Fixed handling of FreeBSD union mounts ([`3ac3bee`](https://github.com/tavianator/bfs/commit/3ac3bee7b0d9c9be693415206efa664bf4a7d4a7)) + +- Fixed `NO_COLOR` handling when it's set to the empty string ([`79aee58`](https://github.com/tavianator/bfs/commit/79aee58a4621d01c4b1e98c332775f3b87213ddb)) + +- Fixed some portability issues: + - [OpenBSD](https://github.com/tavianator/bfs/compare/ee200c07643801c8b53e5b80df704ecbf77a884e...79f1521b0e628be72bed3a648f0ae90b62fc69b8) + - [NetBSD](https://github.com/tavianator/bfs/compare/683f2c41c72efcb82ce866e3dcc311ac9bd8b66d...6435684a7d515e18247ae1b3dd9ec8681fee22d0) + - [DragonFly BSD](https://github.com/tavianator/bfs/compare/08867473e75e8e20ca76c7fb181204839e28b271...45fb1d952c3b262278a3b22e9c7d60cca19a5407) + - [Illumos](https://github.com/tavianator/bfs/compare/4010140cb748cc4f7f57b0a3d514485796c665ce...ae94cdc00136685abe61d55e1e357caaa636d785) + + +3.0.4 +----- + +**October 12, 2023** + +### Bug fixes + +- Fixed a segfault when reporting errors under musl ([`d40eb87`]) + +[`d40eb87`]: https://github.com/tavianator/bfs/commit/d40eb87cc00f50a5debb8899eacb7fcf1065badf + + +3.0.3 +----- + +**October 12, 2023** + +### Changes + +- Iterative deepening modes (`-S {ids,eds}`) were optimized by delaying teardown until the very end ([`5f16169`]) + +- Parallel depth-first search (`-S dfs`) was optimized to avoid enqueueing every file separately ([`2572273`]) + +### Bug fixes + +- Iterative deepening modes (`-S {ids,eds}`) were performing iterative *breadth*-first searches since `bfs` 3.0, negating any advantages they may have had over normal breadth-first search. + They now do iterative *depth*-first searches as expected. + ([`a029d95`]) + +- Fixed a linked-list corruption that could lead to an infinite loop on macOS and other non-Linux, non-FreeBSD platforms ([`773f4a4`]) + +[`5f16169`]: https://github.com/tavianator/bfs/commit/5f1616912ba3a7a23ce6bce02df3791b73da38ab +[`2572273`]: https://github.com/tavianator/bfs/commit/257227326fe60fe70e80433fd34d1ebcb2f9f623 +[`a029d95`]: https://github.com/tavianator/bfs/commit/a029d95b5736a74879f32089514a5a6b63d6efbc +[`773f4a4`]: https://github.com/tavianator/bfs/commit/773f4a446f03da62d88e6d17be49fdc0a3e38465 + + +3.0.2 +----- + +**September 6, 2023** + +### Changes + +- `-files0-from` now allows an empty set of paths to be given, matching GNU findutils 4.9.0 + +- Reduced memory consumption in multi-threaded searches + +- Many man page updates + +### Bug fixes + +- Fixed an out-of-bounds memory read that could occur when escaping a string containing an incomplete multi-byte character + + +3.0.1 +----- + +**July 18, 2023** + +### Bug fixes + +- Traversal fixes that mostly affect large directory trees ([#107]) + + - `bfs` could encounter `EMFILE`, close a file, and retry many times, particularly with `-j1` + + - Breadth-first search could become highly unbalanced, negating many of the benefits of `bfs` + + - On non-{Linux,FreeBSD} plaforms, directories could stay open longer than necessary, consuming extra memory + +[#107]: https://github.com/tavianator/bfs/pull/107 + + +3.0 +--- + +**July 13, 2023** + +### New features + +- `bfs` now reads directories asynchronously and in parallel ([#101]). + Performance is significantly improved as a result. + Parallelism is controlled by the new `-j` flag, e.g. `-j1`, `-j2`, etc. + +[#101]: https://github.com/tavianator/bfs/issues/101 + +### Changes + +- `bfs` now uses the [C17] standard version, up from C11 + +- Due to [#101], `bfs` now requires some additional C and POSIX features: + - [Standard C atomics] (`<stdatomic.h>`) + - [POSIX threads] (`<pthread.h>`) + +- `$LS_COLORS` extensions written in different cases (e.g. `*.jpg=35:*.JPG=01;35`) are now matched case-sensitively, to match the new behaviour of GNU ls since coreutils version 9.2 + +- Added a warning/error if `$LS_COLORS` can't be parsed, depending on whether `-color` is requested explicitly + +- Filenames with control characters are now escaped when printing with `-color` + +- Build flags like `WITH_ONIGURUMA` have been renamed to `USE_ONIGURUMA` + +[C17]: https://en.cppreference.com/w/c/17 +[Standard C atomics]: https://en.cppreference.com/w/c/atomic +[POSIX threads]: https://pubs.opengroup.org/onlinepubs/9699919799/idx/threads.html + +### Bug fixes + +- Fixed handling of the "normal text" color (`no` in `$LS_COLORS`) to match GNU ls + + 2.* === +2.6.3 +----- + +**January 31, 2023** + +- Fixed running the tests as root on Linux [`8b24de3`] + +- Fixed some tests on Android [`2724dfb`] [`0a5a80c`] + +- Stopped relying on non-POSIX touch(1) features in the tests. + This should fix the tests on at least OpenBSD. + [`2d5edb3`] + +- User/group caches are now filled lazily instead of eagerly [`b41dca5`] + +- More caches and I/O streams are flushed before -exec/-ok [`f98a1c4`] + +- Fixed various memory safety issues found by fuzzing \ + [`712b137`] [`5ce883d`] [`da02def`] [`c55e855`] + +- Fixed a test failure on certain macOS versions [`8b24de3`] + +- Mitigated a race condition when determining filesystem types ([#97]) + +- Lots of refactoring and optimization + +[`8b24de3`]: https://github.com/tavianator/bfs/commit/8b24de3882ff5a3e33b82ab20bb4eadf134cf559 +[`2724dfb`]: https://github.com/tavianator/bfs/commit/2724dfbd17552f892a0d8b39b96cbe9e49d66fdb +[`0a5a80c`]: https://github.com/tavianator/bfs/commit/0a5a80c98cc7e5d8735b615fa197a6cff2bb08cc +[`2d5edb3`]: https://github.com/tavianator/bfs/commit/2d5edb37b924715b4fbee4d917ac334c773fca61 +[`b41dca5`]: https://github.com/tavianator/bfs/commit/b41dca52762c5188638236ae81b9f4597bb29ac9 +[`f98a1c4`]: https://github.com/tavianator/bfs/commit/f98a1c4a1cf61ff7d6483388ca1fac365fb0b31b +[`712b137`]: https://github.com/tavianator/bfs/commit/712b13756a09014ef730c8f9b96da4dc2f09b762 +[`5ce883d`]: https://github.com/tavianator/bfs/commit/5ce883daaafc69f83b01dac5db0647e9662a6e87 +[`da02def`]: https://github.com/tavianator/bfs/commit/da02defb91c3a1bda0ea7e653d81f997f1c8884a +[`c55e855`]: https://github.com/tavianator/bfs/commit/c55e85580df10c5afdc6fc0710e756a456aa8e93 +[`8b24de3`]: https://github.com/tavianator/bfs/commit/8b24de3882ff5a3e33b82ab20bb4eadf134cf559 +[#97]: https://github.com/tavianator/bfs/issues/97 + + +2.6.2 +----- + +**October 21, 2022** + +- Fixed use of uninitialized memory on parsing errors involving `-fprintf` + +- Fixed Android build issues ([#96]) + +- Refactored the test suite + +[#96]: https://github.com/tavianator/bfs/issues/96 + + 2.6.1 ----- diff --git a/docs/HACKING.md b/docs/CONTRIBUTING.md index 08ddac2..099157d 100644 --- a/docs/HACKING.md +++ b/docs/CONTRIBUTING.md @@ -1,5 +1,5 @@ -Hacking on `bfs` -================ +Contributing to `bfs` +===================== License ------- @@ -7,11 +7,17 @@ License `bfs` is licensed under the [Zero-Clause BSD License](https://opensource.org/licenses/0BSD), a maximally permissive license. Contributions must use the same license. +Individual files contain the following tag instead of the full license text: + + SPDX-License-Identifier: 0BSD + +This enables machine processing of license information based on the SPDX License Identifiers that are available here: https://spdx.org/licenses/ + Implementation -------------- -`bfs` is written in [C](https://en.wikipedia.org/wiki/C_(programming_language)), specifically [C11](https://en.wikipedia.org/wiki/C11_(C_standard_revision)). +`bfs` is written in [C](https://en.wikipedia.org/wiki/C_(programming_language)), specifically [C17](https://en.wikipedia.org/wiki/C17_(C_standard_revision)). You can get a feel for the coding style by skimming the source code. [`main.c`](/src/main.c) contains an overview of the rest of source files. A quick summary: @@ -30,18 +36,26 @@ Tests `bfs` includes an extensive test suite. See the [build documentation](BUILDING.md#testing) for details on running the tests. +Test cases are grouped by the standard or `find` implementation that supports the tested feature(s): + +| Group | Description | +|---------------------------------|---------------------------------------| +| [`tests/posix`](/tests/posix) | POSIX compatibility tests | +| [`tests/bsd`](/tests/bsd) | BSD `find` features | +| [`tests/gnu`](/tests/gnu) | GNU `find` features | +| [`tests/common`](/tests/common) | Features common to BSD and GNU `find` | +| [`tests/bfs`](/tests/bfs) | `bfs`-specific tests | + Both new features and bug fixes should have associated tests. -To add a test, create a new function in `tests.sh` called `test_<something>`. +To add a test, create a new `*.sh` file in the appropriate group. Snapshot tests use the `bfs_diff` function to automatically compare the generated and expected outputs. For example, ```bash -function test_something() { - bfs_diff basic -name something -} +# posix/something.sh +bfs_diff basic -name something ``` `basic` is one of the directory trees generated for test cases; others include `links`, `loops`, `deep`, and `rainbow`. -Run `./tests/tests.sh test_something --update` to generate the reference snapshot (and don't forget to `git add` it). -Finally, add the test case to one of the arrays `posix_tests`, `bsd_tests`, `gnu_tests`, or `bfs_tests`, depending on which `find` implementations it should be compatible with. +Run `./tests/tests.sh posix/something --update` to generate the reference snapshot (and don't forget to `git add` it). diff --git a/docs/RELATED.md b/docs/RELATED.md new file mode 100644 index 0000000..cf52b70 --- /dev/null +++ b/docs/RELATED.md @@ -0,0 +1,42 @@ +# Related utilities + +There are many tools that can be used to find files. +This is a catalogue of some of the most important/interesting ones. + +## `find`-compatible + +### System `find` implementations + +These `find` implementations are commonly installed as the system `find` utility in UNIX-like operating systems: + +- [GNU findutils](https://www.gnu.org/software/findutils/) ([manual](https://www.gnu.org/software/findutils/manual/html_node/find_html/index.html), [source](https://git.savannah.gnu.org/cgit/findutils.git)) +- BSD `find` + - FreeBSD `find` ([manual](https://www.freebsd.org/cgi/man.cgi?find(1)), [source](https://cgit.freebsd.org/src/tree/usr.bin/find)) + - OpenBSD `find` ([manual](https://man.openbsd.org/find.1), [source](https://cvsweb.openbsd.org/src/usr.bin/find/)) + - NetBSD `find` ([manual](https://man.netbsd.org/find.1), [source](http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/find/)) +- macOS `find` ([manual](https://ss64.com/osx/find.html), [source](https://github.com/apple-oss-distributions/shell_cmds/tree/main/find)) +- Solaris `find` + - [Illumos](https://illumos.org/) `find` ([manual](https://illumos.org/man/1/find), [source](https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/find/find.c)) + +### Alternative `find` implementations + +These are not usually installed as the system `find`, but are designed to be `find`-compatible + +- [`bfs`](https://tavianator.com/projects/bfs.html) ([manual](https://man.archlinux.org/man/bfs.1), [source](https://github.com/tavianator/bfs)) +- [schilytools](https://codeberg.org/schilytools/schilytools) `sfind` ([source](https://codeberg.org/schilytools/schilytools/src/branch/master/sfind)) +- [BusyBox](https://busybox.net/) `find` ([manual](https://busybox.net/downloads/BusyBox.html#find), [source](https://git.busybox.net/busybox/tree/findutils/find.c)) +- [ToyBox](http://landley.net/toybox/) `find` ([manual](http://landley.net/toybox/help.html#find), [source](https://github.com/landley/toybox/blob/master/toys/posix/find.c)) +- uutils `find` ([source](https://github.com/uutils/findutils)) + +## `find` alternatives + +These utilities are not `find`-compatible, but serve a similar purpose: + +- [`fd`](https://github.com/sharkdp/fd): A simple, fast and user-friendly alternative to 'find' +- `locate` + - [GNU `locate`](https://www.gnu.org/software/findutils/locate) + - [`mlocate`](https://pagure.io/mlocate) ([manual](), [source](https://pagure.io/mlocate/tree/master)) + - [`plocate`](https://plocate.sesse.net/) ([manual](https://plocate.sesse.net/plocate.1.html), [source](https://git.sesse.net/?p=plocate)) +- [`walk`](https://github.com/google/walk): Plan 9 style utilities to replace find(1) +- [fselect](https://github.com/jhspetersson/fselect): Find files with SQL-like queries +- [rawhide](https://github.com/raforg/rawhide): find files using pretty C expressions diff --git a/docs/USAGE.md b/docs/USAGE.md index e2cff44..70f8475 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -18,10 +18,9 @@ $ bfs ./completions/bfs.zsh ./docs/BUILDING.md ./docs/CHANGELOG.md -./docs/HACKING.md +./docs/CONTRIBUTING.md ./docs/USAGE.md ./docs/bfs.1 -./src/bfs.h ... ``` @@ -54,7 +53,7 @@ $ bfs -name '*.md' ./README.md ./docs/BUILDING.md ./docs/CHANGELOG.md -./docs/HACKING.md +./docs/CONTRIBUTING.md ./docs/USAGE.md ``` @@ -65,7 +64,7 @@ When you put multiple expressions next to each other, both of them must match: ```console $ bfs -name '*.md' -name '*ING*' ./docs/BUILDING.md -./docs/HACKING.md +./docs/CONTRIBUTING.md ``` This works because the expressions are implicitly combined with *logical and*. @@ -78,17 +77,16 @@ $ bfs -name '*.md' -and -name '*ING'` There are other operators like `-or`: ```console -$ bfs -name '*.md' -or -name '*.sh' +$ bfs -name '*.md' -or -name 'bfs.*' ./README.md -./tests/find-color.sh -./tests/ls-color.sh -./tests/remove-sibling.sh -./tests/sort-args.sh -./tests/tests.sh -./docs/CHANGELOG.md -./docs/HACKING.md +./completions/bfs.bash +./completions/bfs.fish +./completions/bfs.zsh ./docs/BUILDING.md +./docs/CHANGELOG.md +./docs/CONTRIBUTING.md ./docs/USAGE.md +./docs/bfs.1 ``` and `-not`: @@ -132,6 +130,40 @@ Unlike `-prune`, `-exclude` even works in combination with `-depth`/`-delete`. --- +### `-limit` + +The `-limit N` action makes `bfs` quit once it gets evaluated `N` times. +Placing it after an action like `-print` limits the number of results that get printed, for example: + +```console +$ bfs -s -type f -name '*.txt' +./1.txt +./2.txt +./3.txt +./4.txt +$ bfs -s -type f -name '*.txt' -print -limit 2 +./1.txt +./2.txt +``` + +This is similar to + +```console +$ bfs -s -type f -name '*.txt' | head -n2 +``` + +but more powerful because you can apply separate limits to different expressions: + +```console +$ bfs \( -name '*.txt' -print -limit 3 -o -name '*.log' -print -limit 4 \) -limit 5 +[At most 3 .txt files, at most 4 .log files, and at most 5 in total] +``` + +and more efficient because it will quit immediately. +When piping to `head`, `bfs` will only quit *after* it tries to output too many results. + +--- + ### `-hidden`/`-nohidden` `-hidden` matches "hidden" files (dotfiles). @@ -90,7 +90,7 @@ Follow all symbolic links. Never follow symbolic links (the default). .TP .B \-E -Use extended regular expressions (same as \fB\-regextype posix-extended\fR). +Use extended regular expressions (same as \fB\-regextype \fIposix-extended\fR). .TP .B \-X Filter out files with @@ -171,12 +171,18 @@ consumes too much memory. .TP .I eds Exponential deepening search. -A compromise between breadth- and depth-first search, which searches exponentially increasing depth ranges (e.g 0-1, 1-2, 2-4, 4-8, etc.). +A compromise between breadth- and depth-first search, which searches exponentially increasing depth ranges (e.g. 0-1, 1-2, 2-4, 4-8, etc.). Provides many of the benefits of breadth-first search with depth-first's reduced memory consumption. Typically far faster than .B \-S .IR ids . .RE +.TP +\fB\-j\fIN\fR +Search with +.I N +threads in parallel (default: number of CPUs, up to +.IR 8 ). .SH OPERATORS .TP \fB( \fIexpression \fB)\fR @@ -245,6 +251,20 @@ or .B \-mindepth for example. Exclusions are always applied before other expressions, so it may be least confusing to put them first on the command line. +.PP +.B \-help +.br +.B \-\-help +.RS +Print usage information, and exit immediately (without parsing the rest of the command line or processing any files). +.RE +.PP +.B \-version +.br +.B \-\-version +.RS +Print version information, and exit immediately. +.RE .SH OPTIONS .PP .B \-color @@ -309,11 +329,40 @@ Ignored; for compatibility with GNU find. \fB\-regextype \fITYPE\fR Use .IR TYPE -flavored -regexes (default: -.IR posix-basic ; -see -.B \-regextype -.IR help ). +regular expressions. +The possible types are +.RS +.TP +.I posix-basic +POSIX basic regular expressions (the default). +.TP +.I posix-extended +POSIX extended resular expressions. +.TP +.I ed +Like +.BR ed (1) +(same as +.IR posix-basic ). +.TP +.I emacs +Like +.BR emacs (1). +.TP +.I grep +Like +.BR grep (1). +.TP +.I sed +Like +.BR sed (1) +(same as +.IR posix-basic ). +.PP +See +.BR regex (7) +for a description of regular expression syntax. +.RE .TP .B \-status Display a status bar while searching. @@ -403,6 +452,10 @@ Find files with POSIX.1e .BR capabilities (7) set. .TP +\fB\-context \fIGLOB\fR +Find files whose SELinux context matches the +.IR GLOB . +.TP \fB\-depth\fR [\fI\-+\fR]\fIN\fR Find files with depth .IR N . @@ -426,9 +479,9 @@ Find files the current user can execute/read/write. Always false/true. .RE .TP -.B \-fstype TYPE +\fB\-fstype \fITYPE\fR Find files on file systems with the given -.BR TYPE . +.IR TYPE . .PP \fB\-gid\fR [\fI\-+\fR]\fIN\fR .br @@ -506,13 +559,12 @@ to parse as an ISO 8601-style timestamp. For example: .PP .RS -1991-12-14 -.br -1991-12-14T03:00 -.br -1991-12-14T03:00-07:00 -.br -1991-12-14T10:00Z +.nf +\(bu \fI1991-12-14\fR +\(bu \fI1991-12-14T03:00\fR +\(bu \fI1991-12-14T03:00-07:00\fR +\(bu '\fI1991-12-14 10:00Z\fR' +.fi .RE .PP .B \-nogroup @@ -549,35 +601,67 @@ See for examples of the timestamp format. .TP \fB\-size\fR [\fI\-+\fR]\fIN\fR[\fIcwbkMGTP\fR] -Find files with the given size, in 1-byte -.IR c haracters, -2-byte -.IR w ords, -512-byte -.IR b locks -(default), or -.IR k iB/ M iB/ G iB/ T iB/ P iB. +Find files with the given size. +The unit can be one of +.PP +.RS +.nf +\(bu \fIc\fRhars (1 byte) +\(bu \fIw\fRords (2 bytes) +\(bu \fIb\fRlocks (512 bytes, the default) +\(bu \fIk\fRiB (1024 bytes) +\(bu \fIM\fRiB (1024 kiB) +\(bu \fIG\fRiB (1024 MiB) +\(bu \fIT\fRiB (1024 GiB) +\(bu \fIP\fRiB (1024 TiB) +.fi +.RE .TP .B \-sparse Find files that occupy fewer disk blocks than expected. .TP \fB\-type\fR [\fIbcdlpfswD\fR] Find files of the given type. -Possible types are +The possible types are +.PP +.RS +\(bu .IR b lock -device, +device +.br +\(bu .IR c haracter -device, -.IR d irectory, -symbolic -.IR l ink, -.IR p ipe, -regular -.IR f ile, -.IR s ocket, -.IR w hiteout, -and -.IR D oor. +device +.br +\(bu +.IR d irectory +.br +\(bu +.IR l ink +(symbolic) +.br +\(bu +.IR p ipe +.br +\(bu +.IR f ile +(regular) +.br +\(bu +.IR s ocket +.br +\(bu +.IR w hiteout +.br +\(bu +.IR D oor +.PP +Multiple types can be given at once, separated by commas. +For example, +.B \-type +.I d,f +matches both directories and regular files. +.RE .TP \fB\-used\fR [\fI\-+\fR]\fIN\fR Find files last accessed @@ -626,7 +710,9 @@ but run the command in the same directory as the found file(s). .RE .TP \fB\-exit\fR [\fISTATUS\fR] -Exit immediately with the given status (0 if unspecified). +Exit immediately with the given status +.RI ( 0 +if unspecified). .PP \fB\-fls \fIFILE\fR .br @@ -643,6 +729,11 @@ but write to instead of standard output. .RE .TP +\fB\-limit \fIN\fR +Quit once this action is evaluated +.I N +times. +.TP .B \-ls List files like .B ls @@ -689,15 +780,16 @@ instead. .TP .B \-prune Don't descend into this directory. +This has no effect if +.B \-depth +is enabled (either explicitly, or implicitly by +.BR \-delete ). +Use +.B \-exclude +instead in that case. .TP .B \-quit Quit immediately. -.TP -.B \-version -Print version information. -.TP -.B \-help -Print usage information. .SH ENVIRONMENT Certain environment variables affect the behavior of .BR bfs . @@ -748,17 +840,48 @@ Specifies the pager used for .B \-help output. Defaults to +.BR less (1), +if found on the current +.BR PATH , +otherwise .BR more (1). .TP +.B PATH +Used to resolve executables for +.BR \-exec [ dir ] +and +.BR \-ok [ dir ]. +.TP .B POSIXLY_CORRECT Makes .B bfs conform more strictly to the POSIX.1-2017 specification for .BR find (1). -Currently this just disables warnings by default. +Currently this has two effects: +.RS +.IP \(bu +Disables warnings by default, because POSIX prohibits writing to standard error (except for the +.B \-ok +prompt), unless the command also fails with a non-zero exit status. +.IP \(bu +Makes +.B \-ls +and +.B \-fls +use 512-byte blocks instead of 1024-byte blocks. +(POSIX does not specify these actions, but BSD +.BR find (1) +implementations use 512-byte blocks, while GNU +.BR find (1) +uses 1024-byte blocks by default.) +.PP It does not disable .BR bfs 's various extensions to the base POSIX functionality. +.B POSIXLY_CORRECT +has the same effects on GNU +.BR find (1). +.RE .SH EXAMPLES .TP .B bfs @@ -795,7 +918,7 @@ skipping every .B .git directory. .TP -.B bfs \-type f \-executable \-exec strip '{}' + +.B bfs \-type f \-executable \-exec strip {} + Runs .BR strip (1) on all executable files it finds, passing it multiple files at a time. |