Parallel Tasks

Recipe	Crates	Categories
Mutate the elements of an array in parallel
Test in parallel if any or all elements of a collection match a given predicate
Search items using given predicate in parallel
Sort a vector in parallel
Map-reduce in parallel
Generate jpg thumbnails in parallel

Mutate the elements of an array in parallel

The example uses the rayon⮳ crate, which is a data parallelism library for Rust. rayon⮳ provides the rayon::iter::IntoParallelRefIterator::par_iter_mut⮳ method for any parallel iterable data type. This is an iterator-like chain that potentially executes in parallel.

use rayon::prelude::*;

fn main() {
    let mut arr = [0, 7, 9, 11];
    arr.par_iter_mut().for_each(|p| *p -= 1);
    println!("{:?}", arr);
}

Test in parallel if any or all elements of a collection match a given predicate

This example demonstrates using the rayon::iter::ParallelIterator::any⮳ and rayon::iter::ParallelIterator::any⮳ methods, which are parallelized counterparts to std::iter::Iterator::any⮳ and std::iter::Iterator::all⮳. rayon::iter::ParallelIterator::any⮳ checks in parallel whether any element of the iterator matches the predicate, and returns as soon as one is found. rayon::iter::ParallelIterator::any⮳ checks in parallel whether all elements of the iterator match the predicate, and returns as soon as a non-matching element is found.

use rayon::prelude::*;

fn main() {
    let mut vec = vec![2, 4, 6, 8];

    assert!(!vec.par_iter().any(|n| (*n % 2) != 0));
    assert!(vec.par_iter().all(|n| (*n % 2) == 0));
    assert!(!vec.par_iter().any(|n| *n > 8));
    assert!(vec.par_iter().all(|n| *n <= 8));

    vec.push(9);

    assert!(vec.par_iter().any(|n| (*n % 2) != 0));
    assert!(!vec.par_iter().all(|n| (*n % 2) == 0));
    assert!(vec.par_iter().any(|n| *n > 8));
    assert!(!vec.par_iter().all(|n| *n <= 8));
}

Search items using given predicate in parallel

This example uses rayon::iter::ParallelIterator::find_any⮳ and rayon::iter::ParallelIterator::find_any⮳ to search a vector in parallel for an element satisfying the predicate in the given closure.

If there are multiple elements satisfying the predicate defined in the closure argument of rayon::iter::ParallelIterator::find_any⮳ rayon⮳ returns the first one found, not necessarily the first one.

Also note that the argument to the closure is a reference to a reference (&&x). See the discussion on std::iter::Iterator::find⮳ for additional details.

use rayon::prelude::*;

fn main() {
    let v = vec![6, 2, 1, 9, 3, 8, 11];

    let f1 = v.par_iter().find_any(|&&x| x == 9);
    let f2 = v.par_iter().find_any(|&&x| x % 2 == 0 && x > 6);
    let f3 = v.par_iter().find_any(|&&x| x > 8);

    assert_eq!(f1, Some(&9));
    assert_eq!(f2, Some(&8));
    assert!(f3 > Some(&8));
}

Sort a vector in parallel

This example will sort in parallel a vector of Strings.

Allocate a vector of empty Strings. par_iter_mut().for_each populates random values in parallel. Although multiple options⮳ exist to sort an enumerable data type, rayon::slice::ParallelSliceMut::par_sort_unstable⮳ is usually faster than stable sort ⮳ algorithms.

use rand::distributions::Alphanumeric;
use rand::thread_rng;
use rand::Rng;
use rayon::prelude::*;

fn main() {
    let mut vec = vec![String::new(); 100];

    vec.par_iter_mut().for_each(|p| {
        let mut rng = thread_rng();
        *p = (0..5).map(|_| rng.sample(Alphanumeric) as char).collect();
    });
    vec.par_sort_unstable();
    println!("{:?}", vec);
}

Map-reduce in parallel

This example uses rayon::iter::ParallelIterator::filter⮳ rayon::iter::ParallelIterator::map⮳ and rayon::iter::ParallelIterator::reduce⮳ to calculate the average age of Person objects whose age is over 30.

rayon::iter::ParallelIterator::filter⮳ returns elements from a collection that satisfy the given predicate. rayon::iter::ParallelIterator::map⮳ performs an operation on every element, creating a new iteration, and rayon::iter::ParallelIterator::reduce⮳ performs an operation given the previous reduction and the current element. Also shows use of rayon::iter::ParallelIterator::sum⮳ which has the same result as the reduce operation in this example.

use rayon::prelude::*;

struct Person {
    age: u32,
}

fn main() {
    let v: Vec<Person> = vec![
        Person { age: 23 },
        Person { age: 19 },
        Person { age: 42 },
        Person { age: 17 },
        Person { age: 17 },
        Person { age: 31 },
        Person { age: 30 },
    ];

    let num_over_30 = v.par_iter().filter(|&x| x.age > 30).count() as f32;
    let sum_over_30 = v
        .par_iter()
        .map(|x| x.age)
        .filter(|&x| x > 30)
        .reduce(|| 0, |x, y| x + y);

    let alt_sum_30: u32 = v.par_iter().map(|x| x.age).filter(|&x| x > 30).sum();

    let avg_over_30 = sum_over_30 as f32 / num_over_30;
    let alt_avg_over_30 = alt_sum_30 as f32 / num_over_30;

    assert!((avg_over_30 - alt_avg_over_30).abs() < f32::EPSILON);
    println!("The average age of people older than 30 is {}", avg_over_30);
}

Generate jpg thumbnails in parallel

This example generates thumbnails for all jpg files in the current directory then saves them in a new folder called thumbnails.

glob::glob_with::glob_with⮳ finds jpeg files in current directory. rayon resizes images in parallel using rayon::iter::IntoParallelRefIterator::par_iter⮳ calling image::DynamicImage::resize⮳

use std::fs::create_dir_all;
use std::path::Path;

use anyhow::Context;
use anyhow::Result;
use glob::glob_with;
use glob::MatchOptions;
use image::imageops::FilterType;
use rayon::prelude::*;

// TODO address the need for test jpg

fn main() -> Result<()> {
    let options: MatchOptions = Default::default();
    let files: Vec<_> = glob_with("*.jpg", options)?
        .filter_map(|x| x.ok())
        .collect();

    if files.is_empty() {
        println!("No .jpg files found in current directory");
        return Ok(()); // or use anyhow::bail!()
    }

    let thumb_dir = "thumbnails";
    create_dir_all(thumb_dir)?;

    println!("Saving {} thumbnails into '{}'...", files.len(), thumb_dir);

    let image_failures: Vec<_> = files
        .par_iter()
        .map(|path| {
            make_thumbnail(path, thumb_dir, 300)
                .with_context(|| path.display().to_string())
        })
        .filter_map(|x| x.err())
        .collect();

    image_failures.iter().for_each(|x| println!("{}", x));

    println!(
        "{} thumbnails saved successfully",
        files.len() - image_failures.len()
    );
    Ok(())
}

fn make_thumbnail<PA, PB>(
    original: PA,
    thumb_dir: PB,
    longest_edge: u32,
) -> Result<()>
where
    PA: AsRef<Path>,
    PB: AsRef<Path>,
{
    let img = image::open(original.as_ref())?;
    let file_path = thumb_dir.as_ref().join(original);

    Ok(img
        .resize(longest_edge, longest_edge, FilterType::Nearest)
        .save(file_path)?)
}