Macros

Rust macros enable metaprogramming, allowing you to write code that generates other code. This capability allows for code reuse by reducing boilerplate:

• Macros are commonly used as variadic interfaces: They can handle a variable number of arguments and therefore are more flexible than functions. println!, for example, can take any number of arguments.
• Macros can implement common traits automatically or create domain-specific languages.
• Macros can achieve performance optimizations through compile-time computation and code transformation.

Unlike C or C++ macros, which rely on simple text substitution via the preprocessor and can introduce subtle bugs, Rust macros are part of the language's syntax system and are expanded during compilation, so any errors in the generated code are caught early.

Use Macros

Macros comes in three forms:

• Function-like macros, e.g. println!("Hello, world!");,
• Attribute-like macros e.g. #[foo = "bar"],
• Custom derive macros↗ e.g., #[derive(MyTrait)].

Use Function-like Macros

"Function-like" macros are called most frequently, well, like a function would be, with a list of parameters and within a parent function's body:

fn main() {
    let name = "Alice";
    let age = 30;
    println!("My name is {name} and my age is {}", age);
}

• Note the ! suffix after the macro's name.
• Square or curly brackets ([] or {}) may be used as outer delimiters instead of parentheses e.g., println!{ "Curly" };. This is most often used with vec! to give it an array-like syntax: let _ = vec![1, 2, 3];.
• When a function-like macro is used as an item or a statement (see below) and is not using curly braces, a semicolon is required at the end.

This said, function-like macros are more general and powerful than function calls:

• Macros may accept a variable number of arguments.
• The macro arguments are not restricted to be a comma-separated list of expressions. In fact, it does not need to be valid Rust, which can be exploited to create a domain specific language within Rust code (see section below).
• What a macro generates is not limited to an expression or statement.
• Some macros may be called in places where a function could not, for example outside of a function's body.

The following example demonstrates where function-like macros can be used in Rust. They may appear in expressions or statements; in a pattern, type alias, or declaration; or as an associated item. It is also possible to call macros within a macro definition. Macros, however, may not appear in identifiers, match arms, or struct fields:

use std::cell::RefCell;

// Example uses of function-like macros:
fn examples() {
    // 1. They can be used as expressions.
    // The following defines a vector:
    let _x = vec![1, 2, 3];

    // 2. They can be used as statements.
    // Print a string:
    println!("Hello!");

    // 3. They can be used in a pattern.
    // Let's first write a simple macro-by-example for the next example.
    // It wraps an identifier in `Some()`.
    macro_rules! pat {
        ($i:ident) => {
            Some($i)
        };
    }
    // Destructure an `Option` with the macro above:
    if let pat!(x) = Some(1) {
        assert_eq!(x, 1);
    }
}

// 4. Function-like macros can be used in a type alias.
// Let's first write a macro-by-example that defines a tuple type:
macro_rules! Tuple {
    { $A:ty, $B:ty } => { ($A, $B) };
}

// Define a tuple type alias with the macro above:
type T2 = Tuple!(i32, i32);

/// Print the elements of the tuple type alias:
fn print_tuple(tupl: T2) {
    println!("{} {}", tupl.0, tupl.1);
}

/// 5. Function-like macros can be used in a declaration:
fn use_thread_local() {
    // `thread_local!` declares a new thread-local storage key:
    thread_local!(static FOO: RefCell<u32> = const { RefCell::new(1) });
}

// 6. They can be used as an associated item (here, a `const`).
// This macro creates a constant of a given type and value:
macro_rules! const_maker {
    ($t:ty, $v:tt) => {
        const CONST: $t = $v;
    };
}

#[allow(dead_code)]
trait T {
    const_maker! {i32, 7}
}

// 7. It is possible to call macros within a macro definition.
//
// When used, the outer macro `example` is expanded,
// then the inner macro `println` is expanded.
//
// This macro calls the `println!` macro:
macro_rules! _example {
    () => {
        println!("Macro call in a macro!")
    };
}

fn main() {
    examples();
    print_tuple((1, 3));
    use_thread_local();
}

Apply Attribute-like Macros

Attribute-like macros modify the behavior of items (e.g., functions or modules) they are applied to:

// If `log_calls` is a custom attribute macro, apply it as follows:
#[log_calls]
fn add(a: i32, b: i32) -> i32 {
    a + b
}

Read the procedural macros sections below for more details.

Use Custom Derive Macros

Custom macros automatically generate implementations for structs and enums.

// Automatically implement `MyCustomTrait` for the `User` struct:
#[derive(MyCustomTrait)]
struct User {
    id: u64,
    name: String,
}

Read the procedural macros sections below for more details.

Common Macros from the Standard Library

Print Messages to the Console with println! and friends

println!↗, eprintln!↗, print!↗, and eprint!↗ handle formatted output to standard output (println!, print!) or standard error (eprintln!, eprint!). They work similarly to C's printf and support various formatting specifiers. println! and eprintln! automatically add a newline at the end.

format!↗ is similar to print!, but instead of printing to the console, it constructs and returns a String with the formatted content.

dbg!↗ is a debugging macro. It prints the value of an expression to standard error along with its file, line number, and the expression itself, and then returns the expression's value. This is useful for quickly inspecting values during development.

Concatenate String Literals with concat!

concat!↗ concatenates literals into a static string slice.

See String Concatenation for more details.

Create a Vector with vec!

vec!↗ allows for convenient creation and initialization of vectors (Vec<T>). For example, vec![1, 2, 3] creates a vector with integers 1, 2, and 3.

See Vector for more details.

Immediately Terminate the Current Thread with panic!

panic!↗ immediately terminates the current thread. It is used for unrecoverable errors. You can provide a custom message that will be displayed when the panic occurs.

Assert Conditions in your Tests with assert!, assert_eq!, and assert_ne!

assert!↗, assert_eq!↗, assert_ne!↗ are used for assertions, primarily in testing.

• assert!(condition) panics if the condition is false.
• assert_eq!(left, right) panics if left is not equal to right, providing a helpful message showing both values.
• assert_ne!(left, right) panics if left is equal to right.

There are also debug_assert↗, debug_assert_eq↗, and debug_assert_ne↗ macros that are only enabled in non-optimized builds (i.e. disabled in release builds) by default.

Mark Sections of your Code as Unreachable or Unimplemented

unreachable!↗ is used to mark code paths that should logically never be reached. If the code reaches an unreachable! macro, it will cause a panic, indicating a logical error in the program.

todo!↗ is a placeholder for not-yet-implemented code. If executed, it will cause a panic with a message indicating that the functionality is "not yet implemented." It's useful during development to mark areas that still need attention. You may also use the similar unimplemented!↗.

Conditionally Compile Code with cfg!

cfg!↗ evaluates boolean combinations of configuration flags at compile-time. It is often used with or instead of the #[cfg]↗ attribute. Both use the same syntax↗.

Write Macros

There are two main types of macros in Rust:

• Declarative Macros (a.k.a. "macros-by-example") are the most common. Macros-by-example are defined as a set of pattern-matching rules and are always called similarly to a function (they are "function-like"): a_declarative_macro!(...). Examples in the Standard Library include println!(...), vec![...], and format!(...).

Macros-by-example are expanded at compile-time, meaning they incur no runtime performance cost, and their output is checked syntactically and type-checked.

• Procedural Macros are more advanced and allow you to operate on one or more TokenStream↗ (representing the source code). They are compiled before the main program and can perform more complex code generation. They come in three flavors:
  • Custom derive macros↗ (e.g., #[derive(MyTrait)]) automatically generate trait implementations for structs and enums.
  • Attribute-like macros (outer attribute #[route(GET, "/")], where route is a custom name; or inner attribute #![foo="bar"]) modify the behavior of items (e.g., functions or modules) they are applied to.
  • Function-like procedural macros (my_proc_macro!(...)) have the same call syntax as macros-by-example.

Read the sections below; the macro↗ chapter of the Rust book; and the "Little Book of Rust Macros"↗ for more details.

Write Macros-by-Example

Macros-by-example allow you to define reusable code snippets. They are excellent for simple syntactic transformations and reducing boilerplate based on structural patterns.

Macros-by-example are defined using the macro_rules!↗ syntax and work by pattern matching, where you provide patterns and corresponding code templates. They are expanded into source code that gets compiled with the rest of the program.

The macro_rules! syntax, which resembles a match expression, consists of a name for the macro (e.g., say_hello), followed by a set of rules enclosed in curly braces and separated by semicolons: { <rule1>; <rule2> }. Each rule has a matcher, a pattern within () describing the syntax that it matches, followed by => and a transcriber, code within {} that will be substituted when the macro is invoked: ( matcher ) => { transcriber };.

The transcriber can be an expression, a pattern, a type, zero or more items, or zero or more statements.

For example, the following defines a basic say_hello macro (with one rule without arguments), then calls it:

/// A basic macro-by-example (from the Rust By Example book).
macro_rules! say_hello {
    // The matcher `()` indicates that the macro takes no argument.
    () => {
        // The macro will expand into the contents of this block.
        let w = "World";
        println!("Hello {w}!");
    };
}

fn main() {
    say_hello!();
}

Note: To be more precise, macro_rules! interchangeably accepts {}, () and [] as outer delimiters for rule groups, matchers, and transcribers, but what is described above is conventional.

Capture the Arguments of a Macro-by-Example with Metavariables

You can use metavariables to capture parts of a macro's input. They are prefixed with a dollar sign ($). Inside matchers, $name:fragment-specifier matches a Rust syntax fragment of the kind specified and binds it to the metavariable $name, which can then be used in the transcriber. Valid syntax fragment specifiers↗, also called designators, are:

• block: a block expression e.g. { let x = 1; x + 2 },
• expr: an expression that produces a value, e.g. a + 2 or some_function() or even "a literal",
• ident: a variable or function name, e.g. foo or _identifier or even a raw identifier like r#true; or a keyword.
• item: an item definition, i.e. a function, module, use declaration, type alias, struct, enum, const, trait, impl block...
• lifetime: a lifetime token e.g. 'a,
• literal: a literal e.g. 'c', 42, 1.2, b'bin', "hello", true,
• meta: the contents of an attribute (use the #[$meta:meta] or #![$meta:meta] patterns to match an attribute),
• pat: a pattern (in match, if let, for... expressions) e.g., Message::Move{ x, y: 0 }, Some((a, _)), 0..3 or 0 | 42,
• pat_param: (part of) a pattern without |,
• path: a type path, e.g. std::future::Future or ::std::ops::FnOnce(isize) -> isize,
• stmt: a statement without the trailing semicolon (except for item statements that require semicolons) , e.g. let x = 5;,
• tt: a token tree↗, i.e. a single token (identifier, Rust keyword, operator, or literal) or multiple tokens within matching delimiters (), [], or {},
• ty: a type, e.g. bool, [(i32, i32); 3], impl Trait, dyn Trait + Send + Sync...
• vis: a possibly empty visibility qualifier, e.g. pub, pub(crate)...

In the transcriber (code within {} after =>), metavariables are referred to simply by $name. Metavariables are replaced with the syntax element that matched them.

The special metavariable $crate can be used to refer to the crate defining the macro.

In the following example, $e:expr, $e1:expr, $e2:expr each capture a Rust expression, which must be complete and valid:

macro_rules! print_args {
    // Rule 1: Matches one expression.
    ($e:expr) => {
        println!("Single argument: {:?}", $e);
    };
    // Rule 2: Matches two expressions (separated by a literal comma).
    ($e1:expr, $e2:expr) => {
        println!("Two arguments: {:?} and {:?}", $e1, $e2);
    };
}

fn main() {
    // Note how macros can accept different combinations of
    // arguments by having multiple rules:
    print_args!(10);
    print_args!("hello", true);
}

More complex metavariable expressions↗ support is available in nightly Rust.

Use Repetitions in Macros-by-Example

In a macro matcher, a pattern within () describing the syntax that it matches, repetitions are indicated by placing the tokens to be repeated inside $( ), followed by a repetition operator ( *, +, or ?, similar to regular expressions), optionally with a separator token in-between.

In the example below, $($x:expr),* is read as follows:

• $() groups a pattern.
• $x:expr captures an expression into metavariable $x, as described above.
• , is the literal comma that separates the expressions.
• * is a repetition operator, meaning "zero or more" occurrences of the preceding pattern. Other operators include + (one or more) and ? (zero or one).

Repetition in the transcriber is also possible: the $(...)* syntax repeats the temp += $x; statement for each captured expression $x:

macro_rules! sum {
    // `$x:expr` matches an expression.
    // `$($x:expr),*` matches zero or more occurrences of `$x:expr` separated by commas.
    //
    // To allow for an optional trailing comma, add `$(,)?`. The `?` makes the comma pattern optional.
    ( $( $x:expr ),* ) => {
        {
            let mut temp = 0;
            $(
                temp += $x;
            )*
            temp
        }
    };
}
// Note the double set of { } above.
// One is part of the macro-by-example syntax,
// the other is a block that scopes the `temp` variable
// and avoid name collisions.

fn main() {
    let s = sum!(1, 2, 3, 4);
    println!("{s}");
    assert_eq!(s, 10);
}

Understand the Hygiene of Macros-by-Example

Macros-by-example have "mixed-site hygiene". This means that loop labels, block labels, and local variables are looked up at the macro definition site, while other symbols are looked up at the macro invocation site. Refer to the Rust Reference - Macros↗ for full details.

The following example demonstrates how to use a local variable; and how to define an item in a macro and use it at the invocation site:

macro_rules! foo {
    () => {
        let x = 3; // `x` is local to the macro.
    };
}

// To use a local variable at the macro invocation site, pass it to the macro:
macro_rules! bar {
    ($v:ident) => {
        let $v = 3;
    };
}

// Other symbols are available at the invocation site.
// You may define a function, `impl` block, etc... in a macro.
macro_rules! baz {
    () => {
        fn f() {}
    };
}

fn main() {
    foo!();
    // ERROR println!("{x}");

    bar!(x);
    println!("{x}");

    baz!();
    // The function defined in the macro is available at the invocation site:
    f();
}

Macros-by-Example

Macros operate on syntax, not types. As a result, you can't specify the type of an argument in the matcher (the pattern before =>) of a rule of a macro-by-example.

While there is no type checking during macro expansion, there is after. Your code won't compile if there is a type error:

macro_rules! double {
    ($x:expr) => {
        ($x) * 2
    };
}

fn main() {
    println!("{}", double!(5));
    // ERROR cannot multiply `&str` by `{integer}`
    // println!("{}", double!("hello"));
}

You may therefore use compile-time type checking to ensure that a macro argument is of a specific type:

// 1. If you want to allow expressions but ensure they are numeric, you can
//    enforce type checking:
macro_rules! ensure_numeric_expression {
    ($val:expr) => {{
        let _: f64 = $val; // Forces `$val` to be coercible to `f64`.
        println!("Numeric value: {}", $val);
    }};
}
// You may also use:
// let _ = 0 + $val;

// 2. Accept only literals and rejects expressions or non-numeric values.
macro_rules! ensure_numeric_literal {
    ($val:literal) => {{
        let _num: f64 = $val as f64;
        println!("Numeric value: {}", _num);
    }};
    ($val:expr) => {
        compile_error!("Argument must be a numeric literal.");
    };
}

// 3. Ensure that a literal passed as an argument is a string.
macro_rules! must_be_string {
    ($lit:literal) => {
        // This block acts as a compile-time gate.
        {
            const _: () = {
                // This helper function *only* accepts a string slice.
                const fn must_be_string(_s: &str) {}

                // This line will only compile if `$lit` is a string
                // literal. Any other literal type (numeric, char,
                // bool) will cause a "mismatched types" error.
                must_be_string($lit);
            };
        }
    };
}

// 4. Ensure that only certain types can be processed by a macro.
//    - Define a trait that describes the action we want to perform. This
//      creates a common interface for different types to implement.
trait Processable {
    fn process(&self) {}
}

// B. Create a helper macro to implement our trait for e.g. all integer types at
//    once. This avoids a lot of repetitive boilerplate code.
macro_rules! impl_processable_for_integer {
    // `$t:ty` matches a type. `$(...),*` means "repeat for zero or more, separated by commas".
    ($($t:ty),*) => {
        $(
            // This is the implementation of the trait above.
            // It will be generated for every type passed into the macro.
            impl Processable for $t { }
        )*
    };
}

// C. Run the helper macro for e.g. all primitive integer types.
impl_processable_for_integer!(
    i8, u8, i16, u16, i32, u32, i64, u64, i128, u128, isize, usize
);

// D. Call the trait method.
// This compiles only if there is an `impl` for the type of the expression.
macro_rules! integer_only {
    ( $e:expr ) => {
        $e.process();
    };
}

fn main() {
    ensure_numeric_expression!(2.0 + 2.0);
    // ERROR ensure_numeric_expression!("text");

    ensure_numeric_literal!(42);
    ensure_numeric_literal!(1.234);
    // ERROR ensure_numeric_literal!(2 + 2);
    // ERROR ensure_numeric_literal!("hello");

    must_be_string!("42");
    // ERROR: must_be_string!(42);

    integer_only!(41 + 1);
    // ERROR: integer_only!(4.2_f64);
}

Create a Domain-Specific Language (DSL) with Macros

Macros can be used to create domain-specific languages (DSLs) within Rust code. A DSL is a specialized language tailored to a specific problem domain, making the code easier to write - it may be JSON-like; printf-like; SQL-like; or whatever fits your needs.

Indeed, what is passed to a macro does not need to be valid Rust. Macros-by-example, for example, can accept arbitrary sequences of tokens by matching multiple token trees↗:

macro_rules! accept_single_token_tree {
    ($_:tt) => {};
}

macro_rules! accept_multiple_token_trees {
    ( $( $_:tt )* ) => {};
}

fn main() {
    // A "token tree" can a single lexical token (identifier, Rust keyword,
    // operator, punctuation, or literal):
    accept_single_token_tree!("a");
    accept_single_token_tree!( :: );
    accept_single_token_tree!( , );
    accept_single_token_tree!(for);

    // A token tree can be a list of tokens within balanced delimiters `()`,
    // `[]`, or `{}`:
    accept_single_token_tree!( {"a" ~ async 42} );

    // Use a `$( $_:tt )*` repetition to accept zero or more token trees,
    // here "select", "*", ""from", "table", "where", and "( a > 10 )".
    accept_multiple_token_trees! {
        select * from table where ( a > 10 )
    }
    // Use `$( $name:tt )+` for one or more token trees; `$( $name:tt );*` for
    // semicolon-separated tokens, etc.
}

The following example demonstrates how to create a simple configuration DSL:

//! This macro provides a domain-specific language (DSL) for defining
//! configuration for an application. Instead of using a `HashMap`, or a
//! `struct` directly, we want a more declarative, human-readable way to specify
//! settings.

use std::collections::HashMap;

// Simplified representation of the types of values our configuration can hold:
#[derive(Debug, PartialEq, Clone)]
pub enum ConfigValue {
    String(String),
    Boolean(bool),
    Map(HashMap<String, ConfigValue>),
}

// Our DSL Macro.
//
// The `#[macro_export]` attribute makes the `config!` macro available to other
// crates. See <https://doc.rust-lang.org/reference/macros-by-example.html#path-based-scope>.
#[macro_export]
macro_rules! config {

    // Once the parser begins consuming tokens for a metavariable, it cannot stop or backtrack.
    // You should therefore write macro rules in order from most-specific to least-specific.

    // Base case: no (more) tokens e.g., `config! {}`. Returns an empty `HashMap`.
    // Note that the outer delimiters `()` for the matcher will match any pair of delimiters e.g. `{}`, `[]`.
    () => {
        std::collections::HashMap::new()
    };

    // Rules for a single boolean value (e.g., `config! { key: true }`).
    // - `:` and `true` or `false` are literals in the macro-by-example pattern and transcribed literally.
    // - `stringify!` is used to convert a metavariable or expression into a string slice:
    //   `stringify!(1 + 1) == "1 + 1"`.
    // - The special metavariable `$crate` refers to the crate defining the macro;
    //   It is used to refer to items or macros that are not in scope at the invocation site, here the `enum` defined above.
    //   See <https://doc.rust-lang.org/reference/macros-by-example.html#r-macro.decl.hygiene.crate>.
    ($key:ident : true) => {
        {
            let mut map = std::collections::HashMap::new();
            map.insert(stringify!($key).to_string(), $crate::macro_by_example_dsl::ConfigValue::Boolean(true));
            map
        }
    };
    ($key:ident : false) => {
        {
            let mut map = std::collections::HashMap::new();
            map.insert(stringify!($key).to_string(), $crate::macro_by_example_dsl::ConfigValue::Boolean(false));
            map
        }
    };

    // Rule for a string literal value (e.g., `config!{ key: "value" }`).
    ($key:ident : $value:literal) => {
        {
            let mut map = std::collections::HashMap::new();
            map.insert(stringify!($key).to_string(), $crate::macro_by_example_dsl::ConfigValue::String($value.to_string()));
            map
        }
    };

    // Rule for a boolean value followed by a comma and additional tokens.
    // - Note the use of _recursive_ macro calls on the additional tokens.
    //   Each rule inserts the current key-value pair
    //   in the `HashMap` returned by the recursive call to `config!`.
    //   See <https://lukaswirth.dev/tlborm/decl-macros/patterns/tt-muncher.html>.
    ($key:ident : true, $($rest:tt)*) => {
        {
            let mut map = config!($($rest)*);
            map.insert(stringify!($key).to_string(), $crate::macro_by_example_dsl::ConfigValue::Boolean(true));
            map
        }
    };
    ($key:ident : false, $($rest:tt)*) => {
        {
            let mut map = config!($($rest)*);
            map.insert(stringify!($key).to_string(), $crate::macro_by_example_dsl::ConfigValue::Boolean(false));
            map
        }
    };

    // Rule for a string literal value followed by a comma and additional tokens.
    ($key:ident : $value:literal, $($rest:tt)*) => {
        {
            let mut map = config!($($rest)*);
            map.insert(stringify!($key).to_string(), $crate::macro_by_example_dsl::ConfigValue::String($value.to_string()));
            map
        }
    };

    // Rule for a nested section (identifier followed by a block) without a trailing comma.
    ($key:ident { $($inner_config:tt)* }) => {
        {
            let mut map = std::collections::HashMap::new();
            map.insert(stringify!($key).to_string(), $crate::macro_by_example_dsl::ConfigValue::Map(config!($($inner_config)*)));
            map
        }
    };

    // Rule for a nested section, followed by a comma and additional tokens.
    ($key:ident { $($inner_config:tt)* }, $($rest:tt)*) => {
        {
            let mut map = config!($($rest)*);
            map.insert(stringify!($key).to_string(), $crate::macro_by_example_dsl::ConfigValue::Map(config!($($inner_config)*)));
            map
        }
    };

}

fn main() {
    // Example usage of the `config!` macro to set up application settings in a
    // declarative way.
    //
    // The macro allows for nested structures, making it easy to define complex
    // configurations. The result will be a `HashMap` with keys as strings
    // and values as `ConfigValue` enums, which can represent strings,
    // numbers, booleans, or nested maps.
    let app_config: HashMap<String, ConfigValue> = config! {
        debug_mode: true,
        log_level: "info",
        max_connections: 50,

        database {
            host: "localhost",
            username: "admin",
            password: "secure_password",
        },

        network {
            protocol: "tcp",
            timeout_seconds: 30,
            retries: 3
        }
    };

    println!("{app_config:#?}");

    // You can now access values like this:
    if let Some(ConfigValue::Boolean(debug)) = app_config.get("debug_mode") {
        println!("Debug mode: {debug}");
    }
    if let Some(ConfigValue::Map(db_config)) = app_config.get("database") {
        if let Some(ConfigValue::String(host)) = db_config.get("host") {
            println!("Database host: {host}");
        }
    }

    assert_eq!(
        app_config.get("debug_mode"),
        Some(&ConfigValue::Boolean(true))
    );
    assert_eq!(
        app_config.get("log_level"),
        Some(&ConfigValue::String("info".to_string()))
    );
    assert_eq!(
        app_config.get("max_connections"),
        Some(&ConfigValue::String("50".to_string()))
    );
    assert_eq!(
        app_config.get("network"),
        Some(&ConfigValue::Map(HashMap::from([
            (
                "protocol".to_string(),
                ConfigValue::String("tcp".to_string())
            ),
            (
                "timeout_seconds".to_string(),
                ConfigValue::String("30".into())
            ),
            ("retries".to_string(), ConfigValue::String("3".into())),
        ])))
    );
}

The caveat of DSLs, of course, is that an unfamiliar syntax embedded within Rust code may be confusing to the reader.

To parse Rust-like syntax, refer to the relevant section↗ of the little book of macros or use the syn↗ crate.

Write Procedural Macros

Procedural macros are more flexible than declarative macros (macro_rules!). While macro_rules! macros work by pattern matching and substitution on tokens, procedural macros are essentially Rust functions that run at compile time and receive the Rust code they are applied to as "token streams". They return a new token stream, which the compiler then uses to replace the original code.

Procedural macros are one of the more complex but powerful parts of Rust, allowing for code generation and manipulation of the abstract syntax tree (AST):

• Custom derive attributes (e.g., #[derive(Serialize, Deserialize)]) automatically implement traits for structs and enums.
• Attribute-like macros (e.g., #[route("/users", method = "GET")]) define custom attributes that can be applied to items (functions, structs, modules...) to modify their behavior or generate additional code.
• Function-like macros (e.g., custom!(...)) are similar to macros created with macro_rules!, but with the full power of Rust for parsing and code generation.

Note the following before writing procedural macros:

• Procedural macros must be defined in their own crate. This is because they are compiled for the host (the compiler's environment) rather than the target (where your program will run). The need for a separate crate adds a bit of organizational overhead to projects.

• The proc-macro = true key must be set in Cargo.toml:

  [lib]
  proc-macro = true
  

• Procedural macros must be defined in the root of that crate (in lib.rs).

• You will always use types from the proc_macro↗ crate (provided by the compiler) to interact with token streams (Rust reference)↗.

  • All procedural macros take one or two proc_macro::TokenStream↗ as input and return a proc_macro::TokenStream.
  • A token stream is roughly equivalent to Vec<TokenTree> where a TokenTree↗ can roughly be thought of as a lexical token or a group of tokens within (), [], or {} delimiters. For example foo is an Ident token, . is a Punct token, and 1.2 is a Literal token (Rust reference)↗.

While you could manually parse and generate token streams, it is highly recommended to use helper crates (found on crates.io↗):

• syn↗ is a parser for Rust syntax. It allows you to parse a proc_macro::TokenStream into an Abstract Syntax Tree (AST) that's easy to work with (e.g., syn::ItemStruct↗, syn::FnArg↗).
• quote is a quasi-quoting library that makes it easy to generate Rust code from the parsed AST. It allows you to write Rust code directly and "splice in" variables.

Procedural macros are "unhygienic." They behave as if the output token stream was simply written inline to the code it's next to. This means that they are affected by external items and also affects external imports. Use full absolute paths to items in libraries (for example, ::std::option::Option instead of Option) and make sure that generated functions have names that are unlikely to clash with other functions (like __internal_foo instead of foo) (Rust reference)↗.

Procedural macros can increase compilation times, especially for large projects or complex macros.

Derive Custom Traits with Procedural Macros

Derive macros↗ define new inputs for the derive attribute. These macros can create new items, when applied to a struct, enum, or union.

• Create a separate crate and add the following to its Cargo.toml:

[lib]
proc-macro = true

# Typical dependencies for procedural macros:
[dependencies]
syn = { version = "2.0", features = ["full"] } # The "full" feature is often needed for parsing various Rust constructs.
quote = "1.0"
proc-macro2 = "1.0" # TokenStream manipulation.

• Add the following to lib.rs, the root of the new crate:

// The `proc_macro_derive` attribute marks the public function
// `debug_print_derive` as a custom `derive` macro for the `DebugPrint` trait.
// The input `TokenStream` is the token stream of the struct, enum, or union
// that has the `derive` attribute.
#[proc_macro_derive(DebugPrint)]
pub fn debug_print_derive(
    input: proc_macro::TokenStream,
) -> proc_macro::TokenStream {
    // Parse the input tokens into a syntax tree.
    // `syn`'s `parse_macro_input!` macro converts the input `TokenStream` into
    // a structured `DeriveInput` enum.
    let ast = syn::parse_macro_input!(input as syn::DeriveInput);

    let name = &ast.ident; // The name of the struct/enum.
    let (impl_generics, ty_generics, where_clause) =
        ast.generics.split_for_impl();

    // Generate the implementation of the `DebugPrint` trait.
    // `quote!` allows you to write Rust code directly, and `#ident`,
    // `#ty_generics`, etc. are "splices" that insert the captured syntax
    // elements.
    let expanded = quote::quote! {
        impl #impl_generics DebugPrint for #name #ty_generics #where_clause {
            fn debug_print(&self) {
                // `stringify!(#name)` converts the identifier name into a string literal.
                println!("Debugging {}: {self:?}", stringify!(#name));
            }
        }
    };

    // Return the generated `TokenStream`.
    proc_macro::TokenStream::from(expanded)
}

• To use the derive macro, add it to your main crate's Cargo.toml:

[dependencies]
proc-macros = { path = "../proc_macros" } # Adjust the path as necessary.

• Then, in your main crate, use the macro like this:

use proc_macros::DebugPrint;

// Define the custom trait that the macro will implement.
pub trait DebugPrint {
    fn debug_print(&self);
}

// Apply our custom derive macro.
// This will automatically `impl DebugPrint for` the struct.
// We also derive the standard `Debug` trait.
#[derive(DebugPrint, Debug)]
struct User {
    id: u64,
    name: String,
    active: bool,
}

fn main() {
    let u = User {
        id: 1,
        name: "Alice".to_string(),
        active: true,
    };
    u.debug_print();
}

Note that derive macros can add additional attributes ("derive macro helper attributes") into the scope of the item they are on (Rust reference)↗. This is useful for adding metadata or additional functionality to the item being derived:

[derive(MyCustomDeriveMacro)]
struct MyStruct {
    // `helper` attributes can be added here.
    // They are inert.
    #[helper] field: ()
}

Create Custom Attributes with Procedural Macros

Attribute procedural macros↗ define new outer attributes.

• Create a separate crate and add it to your main crate's Cargo.toml, as above.
• Add the following to lib.rs, the root of the new crate:

use proc_macro::TokenStream;

// Attribute macros are defined by a public function with the
// `proc_macro_attribute` attribute.
#[proc_macro_attribute]
pub fn log_calls(_attr: TokenStream, item: TokenStream) -> TokenStream {
    // When the attribute is written as a bare attribute name, like in this
    // example, the attribute `TokenStream` is empty. If the attribute had
    // arguments, `_attr` would contain anything following the attribute's name,
    // not including the outer delimiters.

    // The second `TokenStream` is the rest of the annotated item, including
    // other attributes on the item. Here, we parse the annotated function:
    let input_fn = syn::parse_macro_input!(item as syn::ItemFn);

    let fn_name = &input_fn.sig.ident; // The function's name.
    let block = &input_fn.block; // The function's body.
    let sig = &input_fn.sig; // The function's signature.
    let vis = &input_fn.vis; // The function's visibility.

    let expanded = quote::quote! {
        vis #sig {
            println!("Calling function `{}`...", stringify!(#fn_name));
            let result = #block; // Execute the original function body.
            println!("Function `{}` finished.", stringify!(#fn_name));
            result // Return the result.
        }
    };

    // The returned `TokenStream` can replace the annotated item with an
    // arbitrary number of items.
    TokenStream::from(expanded)
}

• Then, in your main crate, use the macro like this:

// Import our custom attribute macro.
use proc_macros::log_calls;

// Apply the custom attribute macro.
// This will log the function call.
#[log_calls]
fn add(a: i32, b: i32) -> i32 {
    a + b
}

fn main() {
    let sum = add(5, 7);
    println!("Sum: {sum}");
}

Create Function-like Procedural Macros

Function-like procedural macros↗ are invoked using the macro invocation operator (!).

• Create a separate crate and add it to your main crate's Cargo.toml, as above.
• Add the following to lib.rs, the root of the new crate:

// #[proc_macro] marks `sql` as a function-like procedural macro.
// It must be a public function with signature of `(TokenStream) ->
// TokenStream`. The input `TokenStream` is what is inside the delimiters of the
// macro invocation and the output `TokenStream` replaces the entire macro
// invocation.
#[proc_macro]
pub fn sql(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
    // Here, `input` contains all the tokens passed to the macro call (e.g.,
    // SELECT * FROM users WHERE id = 1).
    // For this simple example, we'll just treat the input as a string literal
    // and wrap it. For robust parsing of complex SQL, you'd use a dedicated
    // parser.
    let sql_query = input.to_string();

    // Generate Rust code that uses the SQL string:
    let expanded = quote::quote! {
        {
            let query = #sql_query;
            println!("Executing SQL query: {}", query);
            // In a real scenario, you'd typically return some type
            // that represents the prepared statement or a query result.
            query
        }
    };

    proc_macro::TokenStream::from(expanded)
}

Then, in your main crate, use the macro like this:

// Import our custom function-like macro.
use proc_macros::sql;

fn main() {
    // Function-like procedural macros are invoked using `!`.
    let user_query = sql!(SELECT * FROM users WHERE id = 1);
    println!("Query result: {user_query}");

    let product_query =
        sql!(INSERT INTO products (name, price) VALUES ("Widget", 9.99));
    println!("Query result: {product_query}");
}

Avoid Common Pitfalls with Macros

Watch out for the following common macro pitfalls:

• Overusing Macros: Macros can obscure logic and make code harder to read or debug, if used unnecessarily.
• Poor Macro Hygiene: If your (procedural) macro introduces variables without care, it can clash with names in the surrounding scope. This can lead to confusing bugs.
• Lack of Testing: If you don't write tests that exercise all of the macro's patterns and edge cases, you might miss subtle bugs.
• Unexpected Expansion Behavior: Macros expand before type checking, so they can produce code that compiles incorrectly or fails in surprising ways.
• Lifetime and Type Issues: Especially in procedural macros, forgetting to handle lifetimes or generic parameters properly can lead to compiler errors that are hard to trace.
• Unclear Error Messages.

Report Errors From and Debug Macros

Macros have two ways of reporting errors. The first is to panic!. The second is to invoke the compile_error! macro.

When something goes wrong inside a macro, the compiler often points to the macro invocation site, not the actual problem. Using the compile_error!↗ macro provides clearer diagnostics:

macro_rules! must_be_an_identifier {
    ($x:ident) => {
        // Your code goes here.
    };
    ($_:tt) => {
        // Causes compilation to fail with the given error message.
        // It is the compiler-level form of `panic!`,
        // but emits an error during compilation rather than at runtime.
        compile_error!("This macro only accepts an identifier.");
    };
}

fn main() {

    must_be_an_identifier!(foo);
    // must_be_an_identifier!(42);
}

Debugging macros can be challenging. To see what a macro expands to and debug it, use the cargo-expand↗ cargo plugin. See also the debugging↗ section of the little book of macros for more suggestions.

References

• Rust Reference - Macros↗.
• Rust by Example - Macros↗.
• The Little Book of Rust Macros↗.
• Macros in Rust: A tutorial with examples↗.

Related Topics

• Development Tools: Procedural Macro Helpers.
  • Compile Macros.
  • Macro Tools.
  • Write Proc Macros.
• Rust Patterns.