Introduction
What is mdbook-utils? What is it used for?
The mdbook-utils command-line tool manages links, reference definitions, and code blocks in large collections of Markdown files, especially mdbook source directories. It is the companion tool for the Rust How-to book (github).
mdbook-utils is useful for the following:
- centralize all reference definitions in one file to make Markdown files more readable and ease link maintenance,
- replace simple Markdown links by badges,
- identify duplicate or broken links,
- generate a sitemap from published pages listed in SUMMARY.md,
- extract fenced code bocks embedded into the Markdown to separate files for easier formatting, debugging and testing,
- replace code examples by mdbook
#includestatements, - conversely replace includes by the file contents.
mdbook-utils’ underlying library also exposes a public API that may be used from your code.
Key Links
mdbook-utils (GitHub repo) (docs.rs) (crates.io) (user guide - this book)
Installation
Install the command-line tool using cargo:
cargo install mdbook-utils
then invoke mdbook-utils at a shell prompt.
For the bleeding-edge development version, use:
cargo install --git https://github.com/john-cd/mdbook-utils
To uninstall the tool, enter the following in a shell:
cargo uninstall mdbook-utils
Definitions
mdbook is a command-line tool to create books with Markdown. It is commonly used for Rust user guides, such as the Rust book and the Rust How-to book.
Markdown links and reference definitions
Markdown is a lightweight, readable markup language for writing structured documents.
A Markdown link can be an autolink, e.g. <https://example.com>, an inline link like [The user will see this](https://example.com), or a reference-style link: [The user will see this][thisisthelabel].
A reference-style link requires a reference definition with a matching label thisisthelabel: https://example.com/ on a separate line.
Images and badges
Images can be inserted using  or, reference-style, ![Image][1] followed by a reference definition [1]: <http://url/b.jpg>.
More details may be found in the CommonMark documentation.
A status badge is a small image that provides at-a-glance information, for example the build status of a code repository. Badges are commonly displayed on GitHub READMEs and inserted in mdbook documentation as links to a crate’s docs.rs documentation, GitHub repo, or crates.io page. More information about badges may be found in the awesome-badges repo and in the shields.io documentation.
There is no “badge” concept in the Markdown specification. Badges are simply clickable images e.g. [  ](link-to-webpage).
mdbook-utils refdefs generate --badges creates a flat-style GitHub badge by
default. Add --badge-style flat-square, plastic, for-the-badge, or
social to choose another Shields.io style. Repository name
hyphens and underscores are escaped in the badge path so Shields.io renders
them as literal characters. The default flat style omits the style query
parameter, matching Shields.io’s default and preserving existing output for
repository names without separators.
For example, a GitHub repository named my_repo-with-dash uses this badge
image URL with the flat-square style:
https://img.shields.io/badge/my__repo--with--dash-steelblue?logo=github&style=flat-square
Code blocks and includes
Markdown fenced code blocks (we will call them code examples as well) are inserted between two code fences (e.g. sets of triple backticks), with an optional info string (a.k.a. attributes ) after the first backtick group:
```rust
fn main() {}
```
mdbook allows including files into your book via include statements. mdbook interprets included files as Markdown. Since the include syntax is usually used for inserting code snippets and examples, it is often wrapped between two sets of backticks.
```rust
{{# include file.rs}}
```
An include path is relative to the Markdown file containing the directive.
mdBook also supports selecting a line or line range, or the content between
matching ANCHOR and ANCHOR_END markers. A :2:5 selector includes lines
2 through 5, while :setup selects the content inside the setup anchor.
See the mdBook include reference for the complete syntax.
The mdbook-utils markdown extract-code-examples command writes each fenced
Rust block to a deterministic file beneath the requested output directory.
The chapter’s source-relative path is retained in the output path, and blocks
are numbered in source order (block-000.rs, block-001.rs, and so on). This
prevents same-named chapters from overwriting each other’s examples. The
companion replace-code-examples-by-includes command uses the same mapping to
refer back to those files.
Markdown links and reference definitions
Markdown links and reference definitions
Markdown is a lightweight, readable markup language for writing structured documents.
A Markdown link can be an autolink, e.g. <https://example.com>, an inline link like [The user will see this](https://example.com), or a reference-style link: [The user will see this][thisisthelabel].
A reference-style link requires a reference definition with a matching label thisisthelabel: https://example.com/ on a separate line.
Images and badges
Images and badges
Images can be inserted using  or, reference-style, ![Image][1] followed by a reference definition [1]: <http://url/b.jpg>.
More details may be found in the CommonMark documentation.
A status badge is a small image that provides at-a-glance information, for example the build status of a code repository. Badges are commonly displayed on GitHub READMEs and inserted in mdbook documentation as links to a crate’s docs.rs documentation, GitHub repo, or crates.io page. More information about badges may be found in the awesome-badges repo and in the shields.io documentation.
There is no “badge” concept in the Markdown specification. Badges are simply clickable images e.g. [  ](link-to-webpage).
mdbook-utils refdefs generate --badges creates a flat-style GitHub badge by
default. Add --badge-style flat-square, plastic, for-the-badge, or
social to choose another Shields.io style. Repository name
hyphens and underscores are escaped in the badge path so Shields.io renders
them as literal characters. The default flat style omits the style query
parameter, matching Shields.io’s default and preserving existing output for
repository names without separators.
For example, a GitHub repository named my_repo-with-dash uses this badge
image URL with the flat-square style:
https://img.shields.io/badge/my__repo--with--dash-steelblue?logo=github&style=flat-square
Code blocks and includes
Code blocks and includes
Markdown fenced code blocks (we will call them code examples as well) are inserted between two code fences (e.g. sets of triple backticks), with an optional info string (a.k.a. attributes ) after the first backtick group:
```rust
fn main() {}
```
mdbook allows including files into your book via include statements. mdbook interprets included files as Markdown. Since the include syntax is usually used for inserting code snippets and examples, it is often wrapped between two sets of backticks.
```rust
{{# include file.rs}}
```
An include path is relative to the Markdown file containing the directive.
mdBook also supports selecting a line or line range, or the content between
matching ANCHOR and ANCHOR_END markers. A :2:5 selector includes lines
2 through 5, while :setup selects the content inside the setup anchor.
See the mdBook include reference for the complete syntax.
The mdbook-utils markdown extract-code-examples command writes each fenced
Rust block to a deterministic file beneath the requested output directory.
The chapter’s source-relative path is retained in the output path, and blocks
are numbered in source order (block-000.rs, block-001.rs, and so on). This
prevents same-named chapters from overwriting each other’s examples. The
companion replace-code-examples-by-includes command uses the same mapping to
refer back to those files.
Usage
Run the tool without arguments to display the the list of commands:
Tools to manage links, reference definitions, and code examples in Markdown files, especially `mdbook` source directories.
Usage: mdbook-utils [OPTIONS] <COMMAND>
Commands:
refdefs Manage reference definitions
links Manage links
markdown Manage code blocks (embedded examples) and includes
sitemap Generate a sitemap.xml file from published HTML pages linked in SUMMARY.md
debug Parse the entire Markdown code as events and write them to a file
help Print this message or the help of the given subcommand(s)
Options:
-y, --yes Automatically answer `yes` to any user confirmation request
-h, --help Print help
-V, --version Print version
In turn, most commands offer a menu of subcommands.
Reference Definitions
mdbook-utils refdefs offers the following subcommands:
Manage reference definitions
Usage: mdbook-utils refdefs [OPTIONS] <COMMAND>
Commands:
write Write existing reference definitions to a file
generate Generate reference definitions (e.g. badges for Github links or from the dependencies of the code examples)
help Print this message or the help of the given subcommand(s)
Options:
-y, --yes Automatically answer `yes` to any user confirmation request
--log <LOG> Set the logging level (error, warn, info, debug, trace) [env: RUST_LOG=]
-h, --help Print help
Generate GitHub badges with the default flat Shields.io style:
mdbook-utils refdefs generate --badges --markdown-dir ./src --output ./badge_refs.md
Choose another supported style with --badge-style. The style option also
enables badge generation, so --badges may be omitted:
mdbook-utils refdefs generate --badge-style flat-square --markdown-dir ./src --output ./badge_refs.md
Supported values are flat, flat-square, plastic, for-the-badge, and
social. Omitting the option keeps the default flat output and leaves the
style query parameter out of generated URLs.
Links
mdbook-utils links currently offers several subcommands:
Manage links
Usage: mdbook-utils links [OPTIONS] <COMMAND>
Commands:
write-all Write all existing links to a Markdown file
write-inline Write all existing inline / autolinks (i.e., not written as reference-style links) to a Markdown file
duplicate-links Identify links that repeat the same label, URL and title (inline links: the same URL) and write them to a Markdown file
broken-links Identify broken links (i.e. without reference definition) and write to a Markdown file
help Print this message or the help of the given subcommand(s)
Options:
-y, --yes Automatically answer `yes` to any user confirmation request
--log <LOG> Set the logging level (error, warn, info, debug, trace) [env: RUST_LOG=]
-h, --help Print help
links write-inline writes only inline links and autolinks that point to
external http / https sites; relative paths, fragments, other schemes, and
links inside code are excluded. Use --link-format <reference|inline|autolink>
(default reference) to choose the output syntax. The autolink format drops
link text and titles.
Link extraction follows CommonMark link syntax as implemented by
pulldown-cmark, including inline links, reference links, images, and URI
autolinks written in angle brackets. Plain URL text is not treated as an
autolink. Link-like text in fenced or indented code blocks and code spans is
preserved, as are mdBook helpers such as {{# include path/to/file.rs}}.
The parser enables the existing extensions for tables, footnotes, and strikethrough; it does not enable GitHub’s bare-URL autolinks or unrelated Markdown dialects. See the CommonMark specification for destination escaping and link syntax.
Generic website rules classify absolute http and https destinations. They
separate query strings and fragments from the host and final path segment,
preserve Unicode and percent-encoded text for matching, and reject destinations
without a host or with raw whitespace. This is a parsing and classification
boundary: mdBook-utils does not check whether a destination is reachable or
validate every URI component. A destination accepted by CommonMark syntax may
still contain a malformed percent escape and remains its original destination.
Markdown
mdbook-utils markdown deals with fenced code blocks and includes:
Manage code blocks (embedded examples) and includes
Usage: mdbook-utils markdown [OPTIONS] <COMMAND>
Commands:
extract-code-examples Copy Rust code examples from the Markdown into .rs files
replace-code-examples-by-includes Replace Rust code examples from the Markdown by {{#include ...}} statements
replace-includes-by-contents Replace {{#include file.md}} by the file contents
remove-includes Remove {{#include }} statements (and replace them by a hard-coded string)
generate-categories Generate a listing of crates.io dependencies and write to a Markdown file
generate-crates Generate a crate index and write to a Markdown file
identify-files-not-in-summary Identify .md files not in SUMMARY.md
identify-unused-rs-examples Identify .rs examples not used in Markdown files
help Print this message or the help of the given subcommand(s)
Options:
-y, --yes Automatically answer `yes` to any user confirmation request
--log <LOG> Set the logging level (error, warn, info, debug, trace) [env: RUST_LOG=]
-h, --help Print help
To keep Rust examples in separate source files, run extract-code-examples
with the Markdown source directory and an output directory. It writes each
example under a directory derived from the chapter’s path, followed by
block-NNN.rs. For example, chapters/intro.md maps to
code/chapters/intro.md.d/block-000.rs. This keeps chapters with the same
filename separate. Running extraction again is safe when the generated files
still match; the command reports a conflict instead of overwriting an
unrelated file.
After extraction, replace-code-examples-by-includes replaces the fenced Rust
blocks with mdBook include directives that point to those files. Run
replace-includes-by-contents to expand Markdown includes before a tool that
does not process mdBook directives. Include paths resolve from the Markdown
file containing each directive. The expansion command also handles mdBook line
and named-anchor selectors, and leaves *-refs.md includes intact.
identify-files-not-in-summary reads chapter links from SUMMARY.md. Nested
paths are resolved from the Markdown source directory, URL-encoded path
characters are decoded, and query strings or heading fragments do not change
which source file is checked, following mdBook’s SUMMARY.md chapter links.
Links that resolve outside the source directory are ignored. A missing
SUMMARY.md is an error.
identify-unused-rs-examples recognizes the mdBook Rust file helpers
{{# include path/to/example.rs}},
{{# rustdoc_include path/to/example.rs:section}}, and
{{# playground path/to/example.rs editable}}. Their paths are relative to the
Markdown file containing the helper and must resolve inside the requested code
directory. Other .rs text is not treated as a reference, including Rust
string literals and ordinary prose or code that mentions a file path.
Standard-library Rust macros such as include_str! are not interpreted by
this Markdown scan.
These commands print the unmatched Markdown or Rust paths. A chapter omitted
from SUMMARY.md appears in the first command’s output, and a Rust file with
no supported mdBook helper appears in the second command’s output.
{{#include ../refs.md}}
mdbook-utils sitemap and mdbook-utils debug do not have subcommands.
Command-line options
Command-line options vary by subcommand and include -o to set the path of the output file; -m to set the path of the source Markdown directory (./src or ./drafts by default, depending on the subcommand); -c to set the path to the directory containing the Cargo.toml that declares the dependencies (Rust crates) used in your book; and -t to set the path to the destination directory.
-y is a global option that skips confirmation dialogs and is useful when calling mdbook-utils from a script. Commands that modify your Markdown files ask for confirmation first; without a terminal (for example in a script or CI) they fail with an error and change nothing unless -y is given.
--log <LEVEL> is a global option to set the logging level (error, warn, info, debug, trace). It trumps the RUST_LOG environment variable.
Use mdbook-utils <command> <subcommand> --help or help <command> <subcommand> for more details.
The following illustrates options for mdbook-utils sitemap:
Generate a sitemap.xml file from published HTML pages in SUMMARY.md
Usage: mdbook-utils sitemap [OPTIONS]
Options:
-m, --markdown-dir <DIR> Source directory containing the source Markdown files
-b, --base-url <URL> Base URL
-o, --output <FILE> Path of the file to create
--map-index <FROM:TO> Map a filename to another (e.g., 'intro.md' to 'index.md')
-y, --yes Automatically answer `yes` to any user confirmation request
--log <LOG> Set the logging level (error, warn, info, debug, trace) [env: RUST_LOG=]
-h, --help Print help
The sitemap destination must be inside an existing HTML renderer output directory, such as the configured mdBook build directory. A page is listed only when it appears in SUMMARY.md and its rendered HTML file exists in that output directory.
Which tool for what
mdbook-utils and book_tooling overlap. The split is:
| Task | Tool |
|---|---|
Sitemap, book.toml path resolution | mdbook-utils |
Extract, replace, or remove {{#include}} and code examples; unused .rs examples; files not in SUMMARY.md | mdbook-utils |
| Crate and category indexes, dependency reference definitions, GitHub badges | mdbook-utils |
Reports on links (links write-all, duplicate-links, broken-links) | mdbook-utils (read-only reports) |
| Rewriting inline links as reference links in place, reference-definition clean-up, HTTP link checking, search | book_tooling |
mdbook-utils keeps pulldown-cmark for its read-only scans and does not rewrite link syntax in place. Commands that overlap with book_tooling (links write-inline, refdefs write) stay as reports and are not extended further.
Reference Definitions
Reference Definitions
mdbook-utils refdefs offers the following subcommands:
Manage reference definitions
Usage: mdbook-utils refdefs [OPTIONS] <COMMAND>
Commands:
write Write existing reference definitions to a file
generate Generate reference definitions (e.g. badges for Github links or from the dependencies of the code examples)
help Print this message or the help of the given subcommand(s)
Options:
-y, --yes Automatically answer `yes` to any user confirmation request
--log <LOG> Set the logging level (error, warn, info, debug, trace) [env: RUST_LOG=]
-h, --help Print help
Generate GitHub badges with the default flat Shields.io style:
mdbook-utils refdefs generate --badges --markdown-dir ./src --output ./badge_refs.md
Choose another supported style with --badge-style. The style option also
enables badge generation, so --badges may be omitted:
mdbook-utils refdefs generate --badge-style flat-square --markdown-dir ./src --output ./badge_refs.md
Supported values are flat, flat-square, plastic, for-the-badge, and
social. Omitting the option keeps the default flat output and leaves the
style query parameter out of generated URLs.
Links
Links
mdbook-utils links currently offers several subcommands:
Manage links
Usage: mdbook-utils links [OPTIONS] <COMMAND>
Commands:
write-all Write all existing links to a Markdown file
write-inline Write all existing inline / autolinks (i.e., not written as reference-style links) to a Markdown file
duplicate-links Identify links that repeat the same label, URL and title (inline links: the same URL) and write them to a Markdown file
broken-links Identify broken links (i.e. without reference definition) and write to a Markdown file
help Print this message or the help of the given subcommand(s)
Options:
-y, --yes Automatically answer `yes` to any user confirmation request
--log <LOG> Set the logging level (error, warn, info, debug, trace) [env: RUST_LOG=]
-h, --help Print help
links write-inline writes only inline links and autolinks that point to
external http / https sites; relative paths, fragments, other schemes, and
links inside code are excluded. Use --link-format <reference|inline|autolink>
(default reference) to choose the output syntax. The autolink format drops
link text and titles.
Link extraction follows CommonMark link syntax as implemented by
pulldown-cmark, including inline links, reference links, images, and URI
autolinks written in angle brackets. Plain URL text is not treated as an
autolink. Link-like text in fenced or indented code blocks and code spans is
preserved, as are mdBook helpers such as {{# include path/to/file.rs}}.
The parser enables the existing extensions for tables, footnotes, and strikethrough; it does not enable GitHub’s bare-URL autolinks or unrelated Markdown dialects. See the CommonMark specification for destination escaping and link syntax.
Generic website rules classify absolute http and https destinations. They
separate query strings and fragments from the host and final path segment,
preserve Unicode and percent-encoded text for matching, and reject destinations
without a host or with raw whitespace. This is a parsing and classification
boundary: mdBook-utils does not check whether a destination is reachable or
validate every URI component. A destination accepted by CommonMark syntax may
still contain a malformed percent escape and remains its original destination.
Markdown
Markdown
mdbook-utils markdown deals with fenced code blocks and includes:
Manage code blocks (embedded examples) and includes
Usage: mdbook-utils markdown [OPTIONS] <COMMAND>
Commands:
extract-code-examples Copy Rust code examples from the Markdown into .rs files
replace-code-examples-by-includes Replace Rust code examples from the Markdown by {{#include ...}} statements
replace-includes-by-contents Replace {{#include file.md}} by the file contents
remove-includes Remove {{#include }} statements (and replace them by a hard-coded string)
generate-categories Generate a listing of crates.io dependencies and write to a Markdown file
generate-crates Generate a crate index and write to a Markdown file
identify-files-not-in-summary Identify .md files not in SUMMARY.md
identify-unused-rs-examples Identify .rs examples not used in Markdown files
help Print this message or the help of the given subcommand(s)
Options:
-y, --yes Automatically answer `yes` to any user confirmation request
--log <LOG> Set the logging level (error, warn, info, debug, trace) [env: RUST_LOG=]
-h, --help Print help
To keep Rust examples in separate source files, run extract-code-examples
with the Markdown source directory and an output directory. It writes each
example under a directory derived from the chapter’s path, followed by
block-NNN.rs. For example, chapters/intro.md maps to
code/chapters/intro.md.d/block-000.rs. This keeps chapters with the same
filename separate. Running extraction again is safe when the generated files
still match; the command reports a conflict instead of overwriting an
unrelated file.
After extraction, replace-code-examples-by-includes replaces the fenced Rust
blocks with mdBook include directives that point to those files. Run
replace-includes-by-contents to expand Markdown includes before a tool that
does not process mdBook directives. Include paths resolve from the Markdown
file containing each directive. The expansion command also handles mdBook line
and named-anchor selectors, and leaves *-refs.md includes intact.
identify-files-not-in-summary reads chapter links from SUMMARY.md. Nested
paths are resolved from the Markdown source directory, URL-encoded path
characters are decoded, and query strings or heading fragments do not change
which source file is checked, following [mdBook’s SUMMARY.md chapter links][mdbook-summary].
Links that resolve outside the source directory are ignored. A missing
SUMMARY.md is an error.
identify-unused-rs-examples recognizes the mdBook Rust file helpers
{{# include path/to/example.rs}},
{{# rustdoc_include path/to/example.rs:section}}, and
{{# playground path/to/example.rs editable}}. Their paths are relative to the
Markdown file containing the helper and must resolve inside the requested code
directory. Other .rs text is not treated as a reference, including Rust
string literals and ordinary prose or code that mentions a file path.
Standard-library Rust macros such as include_str! are not interpreted by
this Markdown scan.
These commands print the unmatched Markdown or Rust paths. A chapter omitted
from SUMMARY.md appears in the first command’s output, and a Rust file with
no supported mdBook helper appears in the second command’s output.
{{#include ../refs.md}}
Command-line options
Command-line options
Command-line options vary by subcommand and include -o to set the path of the output file; -m to set the path of the source Markdown directory (./src or ./drafts by default, depending on the subcommand); -c to set the path to the directory containing the Cargo.toml that declares the dependencies (Rust crates) used in your book; and -t to set the path to the destination directory.
-y is a global option that skips confirmation dialogs and is useful when calling mdbook-utils from a script. Commands that modify your Markdown files ask for confirmation first; without a terminal (for example in a script or CI) they fail with an error and change nothing unless -y is given.
--log <LEVEL> is a global option to set the logging level (error, warn, info, debug, trace). It trumps the RUST_LOG environment variable.
Use mdbook-utils <command> <subcommand> --help or help <command> <subcommand> for more details.
The following illustrates options for mdbook-utils sitemap:
Generate a sitemap.xml file from published HTML pages in SUMMARY.md
Usage: mdbook-utils sitemap [OPTIONS]
Options:
-m, --markdown-dir <DIR> Source directory containing the source Markdown files
-b, --base-url <URL> Base URL
-o, --output <FILE> Path of the file to create
--map-index <FROM:TO> Map a filename to another (e.g., 'intro.md' to 'index.md')
-y, --yes Automatically answer `yes` to any user confirmation request
--log <LOG> Set the logging level (error, warn, info, debug, trace) [env: RUST_LOG=]
-h, --help Print help
The sitemap destination must be inside an existing HTML renderer output directory, such as the configured mdBook build directory. A page is listed only when it appears in SUMMARY.md and its rendered HTML file exists in that output directory.
Configuration
Path selection follows this order when a command supports the corresponding option: command-line option, matching environment variable, mdBook’s book.toml, then the command’s built-in default. The Markdown source default varies by command. MDBOOK_UTILS_BOOK_ROOT_DIR_PATH selects the directory containing book.toml; it defaults to the current working directory.
mdbook-utils reads the mdBook book.src and build.build-dir settings, and uses the configured [output.*] renderers to resolve renderer output directories. Relative paths in book.toml are relative to the book root. If there are no output tables, mdBook enables HTML by default. One renderer writes into build.build-dir; with multiple renderers, each writes into a subdirectory named after the renderer, such as html or markdown.
The mdBook book.toml is the only TOML configuration source; no separate mdbook-utils TOML file is needed. A missing book.toml is optional. If path resolution reaches an unreadable or malformed book.toml, the command reports the error instead of silently using defaults. When a valid configuration disables the HTML renderer, sitemap requires an explicit destination through --output or MDBOOK_UTILS_BOOK_HTML_BUILD_DIR_PATH.
| Priority | Source | Example |
|---|---|---|
| 1 | Command-line option | --markdown-dir ./src/ |
| 2 | Environment variable (or .env file in the current directory) | MDBOOK_UTILS_MARKDOWN_DIR_PATH |
| 3 | mdBook’s book.toml | book.src, build.build-dir, output.html.site-url |
| 4 | The command’s default | ./src/ |
All environment variables start with MDBOOK_UTILS_. The older unprefixed names (for example BASE_URL or MARKDOWN_DIR_PATH) are still read when the prefixed variable is not set, but print a deprecation warning and will be removed in a later release. Only a .env file in the current directory is loaded; parent directories are not searched.
You may export environment variables manually or store them in a .env file, which will be read automatically:
# Root directory of the book
# `book.toml` is looked up in MDBOOK_UTILS_BOOK_ROOT_DIR_PATH, if set,
# in the current working directory otherwise.
export MDBOOK_UTILS_BOOK_ROOT_DIR_PATH=./test_book/
# Markdown source directory. This overrides `book.src`.
export MDBOOK_UTILS_MARKDOWN_DIR_PATH=./test_book/src/
# Directory where mdBook outputs HTML and JS. This overrides `build.build-dir`
# and renderer-based directory selection.
export MDBOOK_UTILS_BOOK_HTML_BUILD_DIR_PATH=./test_book/book/
# Directory where `mdbook` outputs the book's fully expanded Markdown,
# i.e. with all includes resolved, when `[output.markdown]` is added to `book.toml`.
# It is typically ./book/markdown/.
export MDBOOK_UTILS_BOOK_MARKDOWN_BUILD_DIR_PATH=./test_book/book/markdown/
# Directory where `Cargo.toml` may be found
export MDBOOK_UTILS_CARGO_TOML_DIR_PATH=./test_book/book/code/
# Default destination directory for mdbook-utils outputs.
export MDBOOK_UTILS_DEFAULT_DEST_DIR_PATH=./test_book/temp/
# Base url of the website where the book will be deployed
# (used to build sitemaps). Optional if `book.toml` sets
# `output.html.site-url`; `sitemap` fails when no base URL is configured.
export MDBOOK_UTILS_BASE_URL=http://myexample.com/some_book/
For example, when book.toml contains [output.html] and [output.markdown] and sets build-dir = "dist", mdBook writes HTML to dist/html and expanded Markdown to dist/markdown, relative to the book root. A sole [output.html] writes directly to dist.
You may also set the RUST_LOG environment variable to display the logs.
The CLI options --markdown-dir, --output, --target-dir, --cargo-toml-dir, and --base-url override the matching settings for the current command. See the mdBook general configuration and renderer configuration for the underlying format.
Note: mdbook-utils is not a mdbook mdbook-preprocessor or mdbook-backend at this point.
Public API
The mdbook-utils package provides one library and one command-line binary. The library API is useful when another Rust program needs to run the same processing as a CLI command or compose it with its own workflow.
Add the crate to your Cargo.toml:
cargo add mdbook-utils
The grouped library operations are under mdbook_utils::api. The CLI uses these same functions after its private src/cli layer parses arguments, resolves command options, and reports results. The library owns reusable Markdown processing, link and reference generation, file scans, and sitemap generation.
For example, generate the categories index from a Markdown source directory:
mdbook_utils::api::generate_categories("src", "categories.md")?;
Generate GitHub badges with a selected Shields.io style using
generate_badges_with_style. The existing generate_badges function keeps
the default flat style:
mdbook_utils::api::generate_badges_with_style(
"src",
"badge_refs.md",
mdbook_utils::api::BadgeStyle::FlatSquare,
)?;
Library operations return anyhow::Result, so callers can propagate path, input, parsing, Cargo, and file output errors. Each operation’s rustdoc describes the errors it can return.
The flat mdbook_utils::function_name re-exports remain available for compatibility with existing callers. New code should use the grouped mdbook_utils::api path. The public mdbook_utils::markdown module also exposes the lower-level include and code extraction operations used by the CLI. Internal parsing, filesystem, generation, and link implementation modules remain private. The parser test fixture and hidden development command are not part of the library API.
The current single package already keeps the CLI composition root (src/main.rs) separate from reusable library code (src/lib.rs). A second binary directory or Cargo workspace would add structure without adding a current CLI or library boundary, so the layout stays as it is.
See the crate’s API reference for function signatures and detailed documentation.
Contributing
Pull requests, comments, and issue submissions are actively encouraged!
Repo structure
mdbook-utils is written in Rust and follows the typical cargo package layout.
- The source code is in the
srcfolder. The main executable is inmain.rsand theclimodule. It calls the API inlib.rs. - A simple test
mdbookbook is found intest_book. - The user guide’ sources are in
user_guide. - The Dev Container and Docker (Compose) configuration files are found in
.devcontainer.devcontainer.jsonuses Docker Compose (configured incompose.yamlandcompose.override.yaml), which in turn runs a container fromDockerfile.
.githubcontains the continuous integration (GitHub Actions) workflow.
Development Setup
Using VS Code
Clone the repo and open the folder in VS Code. Edit .devcontainer/.env if needed. VS Code should prompt you to open the code in a Docker container, which installs Rust tooling automatically. Make sure you have previously installed the following:
- Dev Container extension
- Docker Desktop (or at least the Docker engine).
Note that opening the code folder in VS Code using Dev Containers may take a little while the first time around.
Other
If you are not using VS Code, install the Dev Container CLI, use docker compose directly (see below), or simply install the required tools on your local machine.
The following works with Ubuntu (including WSL):
sudo apt-get update
rustup update
rustup component add clippy
rustup toolchain install nightly
rustup component add rustfmt --toolchain nightly
cargo install just
cargo install mdbook
Review .devcontainer/Dockerfile for other optional dependencies.
Build and test the code and user guide
The just command runner is configured to simplify compilation and testing.
Type just at a shell prompt for a list of commands:
just clean # Clean Cargo's `target` and mdbook's `book` folders
just fmt # Format all code
just check # Check whether the code can compile
just build # Build all code and books
just clippy # Scan all code for common mistakes
just test # Test all code and books
just run <command> # Run the tool
just doc # Build and display the `cargo doc` documentation
just serve # Display the user guide
just prep # Run all the steps required before pushing code to GitHub
just update # Update Cargo.lock dependencies
Tests
just test runs all Rust tests (cargo test --all-targets --locked) and the mdBook tests of user_guide/ and test_book/.
- Unit tests are in an inline
#[cfg(test)] mod testsat the bottom of the module they cover, undersrc/. - Integration tests are in
tests/. They run the compiledmdbook-utilsbinary (throughCARGO_BIN_EXE_mdbook-utils) or the public library API against temporary directories, and check exit codes and the files written. Shared sample inputs are intests/fixtures/. - Give each test a descriptive name and a doc comment that states the behavior it verifies. Avoid
unwrap(); returnanyhow::Resultfrom the test or useexpect()with a message.
Run a single integration test file with, for example, cargo test --test cli_links_and_generate.
Docker Compose
Test the Docker Compose setup used during development (which Dev Containers runs) with:
cd ./.devcontainer
docker compose build # uses compose.yaml and compose.override.yaml by default
docker compose up -d
# or simply
docker compose up --build -d
Use the following commands to build and test the code and user guide using the Continuous Integration configuration:
docker compose -f .devcontainer/compose.yaml -f .devcontainer/compose-ci.yaml run \
--build --rm mdbook-utils
Publish to crates.io
- Manual method
- Go to
crates.io, sign in, and create an API token inAccount Settings>API Tokens. - Use
cargo login <token>to save the token in$CARGO_HOME/credentials.toml. just build; just clippy; just run; just doc; cargo package --locked- Review the packaging output in
target/mdbook-utils/packageor usecargo package --list. - When ready,
cargo publish --locked --dry-run; cargo publish --locked
- Docker Compose method
- Pass the
publish.shscript (and required argument-y) as acommandtodocker compose run. - Pass the
CRATES_TOKENenv. variable (which is used bypublish.sh) to Docker Compose using--env.
export CRATES_TOKEN="<token from crates.io>"
docker compose -f .devcontainer/compose.yaml -f .devcontainer/compose-ci.yaml run \
--rm --env CRATES_TOKEN mdbook-utils .devcontainer/publish.sh -y
Release binaries
Pushing a tag such as v0.2.0 runs .github/workflows/release.yml. It builds --release --locked binaries for Linux (x86_64), Windows (x86_64), and macOS (Apple silicon and Intel), packs each with LICENSE and README.md into mdbook-utils-<target>-v<version>.tar.gz (.zip on Windows) with a .sha256 file, runs --help on the unpacked binary, and attaches the archives to a GitHub release. The archive names match [package.metadata.binstall] in Cargo.toml, so users can run cargo binstall mdbook-utils; cargo binstall falls back to building from source when no archive exists.
A manual run of the workflow builds and smoke-tests the archives without creating a release, which is the way to validate packaging without any credentials.
Release checklist and rollback
- Update
RELEASES.mdand the version inCargo.toml, merge todevelop, and wait for CI. - Publish to crates.io with the Build, test, optionally deploy to Pages & publish workflow (or the manual method above).
- Push the tag
vX.Y.Zto create the GitHub release. - Rollback:
cargo yank --version X.Y.Z mdbook-utilshides a bad crates.io release without deleting it; delete the GitHub release and tag if the archives are bad, fix forward, and publish a new patch version.
User guide hosting
The pages.yml workflow builds and tests the guide on every push to develop, writes sitemap.xml into the rendered book with mdbook-utils sitemap --base-url https://john-cd.com/mdbook-utils/, and deploys it to GitHub Pages. The site has no analytics: the guide is documentation for a command-line tool, so no visitor data is collected and no cookie or privacy notice is needed. This can be revisited if usage data becomes useful.
Docker image for CI
CI installs the pinned Rust toolchain and tools directly on the runners (with Swatinem/rust-cache), so there is no published CI image. A prebuilt image would save little time compared to the cache, add a registry to maintain, and need rebuilding for every toolchain update. The .devcontainer image remains for local development and the manual publish method.
Repo structure
Repo structure
mdbook-utils is written in Rust and follows the typical cargo package layout.
- The source code is in the
srcfolder. The main executable is inmain.rsand theclimodule. It calls the API inlib.rs. - A simple test
mdbookbook is found intest_book. - The user guide’ sources are in
user_guide. - The Dev Container and Docker (Compose) configuration files are found in
.devcontainer.devcontainer.jsonuses Docker Compose (configured incompose.yamlandcompose.override.yaml), which in turn runs a container fromDockerfile.
.githubcontains the continuous integration (GitHub Actions) workflow.
Development setup
Development Setup
Using VS Code
Clone the repo and open the folder in VS Code. Edit .devcontainer/.env if needed. VS Code should prompt you to open the code in a Docker container, which installs Rust tooling automatically. Make sure you have previously installed the following:
- Dev Container extension
- Docker Desktop (or at least the Docker engine).
Note that opening the code folder in VS Code using Dev Containers may take a little while the first time around.
Other
If you are not using VS Code, install the Dev Container CLI, use docker compose directly (see below), or simply install the required tools on your local machine.
The following works with Ubuntu (including WSL):
sudo apt-get update
rustup update
rustup component add clippy
rustup toolchain install nightly
rustup component add rustfmt --toolchain nightly
cargo install just
cargo install mdbook
Review .devcontainer/Dockerfile for other optional dependencies.
Build and test the code and user guide
Build and test the code and user guide
The just command runner is configured to simplify compilation and testing.
Type just at a shell prompt for a list of commands:
just clean # Clean Cargo's `target` and mdbook's `book` folders
just fmt # Format all code
just check # Check whether the code can compile
just build # Build all code and books
just clippy # Scan all code for common mistakes
just test # Test all code and books
just run <command> # Run the tool
just doc # Build and display the `cargo doc` documentation
just serve # Display the user guide
just prep # Run all the steps required before pushing code to GitHub
just update # Update Cargo.lock dependencies
Tests
just test runs all Rust tests (cargo test --all-targets --locked) and the mdBook tests of user_guide/ and test_book/.
- Unit tests are in an inline
#[cfg(test)] mod testsat the bottom of the module they cover, undersrc/. - Integration tests are in
tests/. They run the compiledmdbook-utilsbinary (throughCARGO_BIN_EXE_mdbook-utils) or the public library API against temporary directories, and check exit codes and the files written. Shared sample inputs are intests/fixtures/. - Give each test a descriptive name and a doc comment that states the behavior it verifies. Avoid
unwrap(); returnanyhow::Resultfrom the test or useexpect()with a message.
Run a single integration test file with, for example, cargo test --test cli_links_and_generate.
Docker Compose
Test the Docker Compose setup used during development (which Dev Containers runs) with:
cd ./.devcontainer
docker compose build # uses compose.yaml and compose.override.yaml by default
docker compose up -d
# or simply
docker compose up --build -d
Use the following commands to build and test the code and user guide using the Continuous Integration configuration:
docker compose -f .devcontainer/compose.yaml -f .devcontainer/compose-ci.yaml run \
--build --rm mdbook-utils
Publish
Publish to crates.io
- Manual method
- Go to
crates.io, sign in, and create an API token inAccount Settings>API Tokens. - Use
cargo login <token>to save the token in$CARGO_HOME/credentials.toml. just build; just clippy; just run; just doc; cargo package --locked- Review the packaging output in
target/mdbook-utils/packageor usecargo package --list. - When ready,
cargo publish --locked --dry-run; cargo publish --locked
- Docker Compose method
- Pass the
publish.shscript (and required argument-y) as acommandtodocker compose run. - Pass the
CRATES_TOKENenv. variable (which is used bypublish.sh) to Docker Compose using--env.
export CRATES_TOKEN="<token from crates.io>"
docker compose -f .devcontainer/compose.yaml -f .devcontainer/compose-ci.yaml run \
--rm --env CRATES_TOKEN mdbook-utils .devcontainer/publish.sh -y
Release binaries
Pushing a tag such as v0.2.0 runs .github/workflows/release.yml. It builds --release --locked binaries for Linux (x86_64), Windows (x86_64), and macOS (Apple silicon and Intel), packs each with LICENSE and README.md into mdbook-utils-<target>-v<version>.tar.gz (.zip on Windows) with a .sha256 file, runs --help on the unpacked binary, and attaches the archives to a GitHub release. The archive names match [package.metadata.binstall] in Cargo.toml, so users can run cargo binstall mdbook-utils; cargo binstall falls back to building from source when no archive exists.
A manual run of the workflow builds and smoke-tests the archives without creating a release, which is the way to validate packaging without any credentials.
Release checklist and rollback
- Update
RELEASES.mdand the version inCargo.toml, merge todevelop, and wait for CI. - Publish to crates.io with the Build, test, optionally deploy to Pages & publish workflow (or the manual method above).
- Push the tag
vX.Y.Zto create the GitHub release. - Rollback:
cargo yank --version X.Y.Z mdbook-utilshides a bad crates.io release without deleting it; delete the GitHub release and tag if the archives are bad, fix forward, and publish a new patch version.
User guide hosting
The pages.yml workflow builds and tests the guide on every push to develop, writes sitemap.xml into the rendered book with mdbook-utils sitemap --base-url https://john-cd.com/mdbook-utils/, and deploys it to GitHub Pages. The site has no analytics: the guide is documentation for a command-line tool, so no visitor data is collected and no cookie or privacy notice is needed. This can be revisited if usage data becomes useful.
Docker image for CI
CI installs the pinned Rust toolchain and tools directly on the runners (with Swatinem/rust-cache), so there is no published CI image. A prebuilt image would save little time compared to the cache, add a registry to maintain, and need rebuilding for every toolchain update. The .devcontainer image remains for local development and the manual publish method.