Character Sets

Recipe	Crates	Categories
Percent-encode a string
Encode a string as application/x-www-form-urlencoded
Encode and decode hex
Encode and decode base64

Percent-encode a string

Encode an input string with percent_encoding⮳ using the percent_encoding::utf8_percent_encode⮳ function from the percent_encoding crate. Then decode using the percent_encoding::percent_decode⮳ function.

use std::str::Utf8Error;

use percent_encoding::percent_decode;
use percent_encoding::utf8_percent_encode;
use percent_encoding::AsciiSet;
use percent_encoding::CONTROLS;

/// https://url.spec.whatwg.org/#fragment-percent-encode-set
const FRAGMENT: &AsciiSet =
    &CONTROLS.add(b' ').add(b'"').add(b'<').add(b'>').add(b'`');

fn main() -> Result<(), Utf8Error> {
    let input = "confident, productive systems programming";

    let iter = utf8_percent_encode(input, FRAGMENT);
    let encoded: String = iter.collect();
    assert_eq!(encoded, "confident,%20productive%20systems%20programming");

    let iter = percent_decode(encoded.as_bytes());
    let decoded = iter.decode_utf8()?;
    assert_eq!(decoded, "confident, productive systems programming");

    Ok(())
}

The encode set defines which bytes (in addition to non-ASCII and controls) need to be percent-encoded. The choice of this set depends on context. For example, url encodes ? in a URL path but not in a query string.

The return value of encoding is an iterator of &str slices which collect into a std::string::String⮳.

Encode a string as application/x-www-form-urlencoded

Encodes a string into application/x-www-form-urlencoded syntax using the form_urlencoded::byte_serialize⮳ and subsequently decodes it with form_urlencoded::parse⮳. Both functions return iterators that collect into a std::string::String⮳.

use url::form_urlencoded::byte_serialize;
use url::form_urlencoded::parse;

fn main() {
    let urlencoded: String = byte_serialize("What is ❤?".as_bytes()).collect();
    assert_eq!(urlencoded, "What+is+%E2%9D%A4%3F");
    println!("urlencoded:'{}'", urlencoded);

    let decoded: String = parse(urlencoded.as_bytes())
        .map(|(key, val)| [key, val].concat())
        .collect();
    assert_eq!(decoded, "What is ❤?");
    println!("decoded:'{}'", decoded);
}

Encode and decode hex

The data_encoding⮳ crate provides a HEXUPPER::encode method which takes a &[u8] and returns a std::string::String⮳ containing the hexadecimal representation of the data.

Similarly, a HEXUPPER::decode method is provided which takes a &[u8] and returns a Vec<u8> if the input data is successfully decoded.

The example below coverts &[u8] data to hexadecimal equivalent. Compares this value to the expected value.

use data_encoding::DecodeError;
use data_encoding::HEXUPPER;

fn main() -> Result<(), DecodeError> {
    let original = b"The quick brown fox jumps over the lazy dog.";
    let expected = "54686520717569636B2062726F776E20666F78206A756D7073206F76\
        657220746865206C617A7920646F672E";

    let encoded = HEXUPPER.encode(original);
    assert_eq!(encoded, expected);

    let decoded = HEXUPPER.decode(&encoded.into_bytes())?;
    assert_eq!(&decoded[..], &original[..]);

    Ok(())
}

Encode and decode base64

Encodes byte slice into base64 String using base64::encode and decodes it with base64::decode.

use std::str;

use anyhow::Result;
use base64::prelude::*;

fn main() -> Result<()> {
    let hello = b"hello rustaceans";
    let encoded = BASE64_STANDARD.encode(hello);
    let decoded = BASE64_STANDARD.decode(&encoded)?;

    println!("origin: {}", str::from_utf8(hello)?);
    println!("base64 encoded: {}", encoded);
    println!("back to origin: {}", str::from_utf8(&decoded)?);

    Ok(())
}