Data Engineering

Develop data analytics applications that process columnar data with Arrow

arrow arrow-crates.io arrow-github arrow-lib.rs

arrow is the official Rust implementation of Apache Arrow

Query in-memory data with datafusion

datafusion datafusion-crates.io datafusion-github datafusion-lib.rs

datafusion is the Apache Arrow DataFusion SQL Query Engine.

Apache DataFusion is an in-memory query engine that uses Apache Arrow as the memory model

DataFusion offers SQL and Dataframe APIs, excellent performance, built-in support for CSV, Parquet, JSON, and Avro, plus extensive customization. DataFusion is great for building projects such as domain specific query engines, new database platforms and data pipelines, query languages and more.

Databend

𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. datafuselabs databend

See also

Rust Data Engineering course By Alfredo Deza et al., O'Reilly