about_BigDatawithRUST
Apache DataFusion
DataFusion is an extensible query engine written in Rust that uses Apache Arrow as its in-memory format.
DataFusion is an extensible query engine written in Rust that uses Apache Arrow as its in-memory format. DataFusion’s target users are developers building fast and feature rich database and analytic systems, customized to particular workloads. See use cases for examples.
“Out of the box,” DataFusion offers SQL and Dataframe APIs, excellent performance, built-in support for CSV, Parquet, JSON, and Avro, extensive customization, and a great community. Python Bindings are also available.
DataFusion features a full query planner, a columnar, streaming, multi-threaded, vectorized execution engine, and partitioned data sources. You can customize DataFusion at almost all points including additional data sources, query languages, functions, custom operators and more. See the Architecture section below for more details.
https://github.com/apache/datafusion
https://docs.rs/datafusion/latest/datafusion/
Composable, Declarative, Stateful Data Streaming System, Fluvio
Fluvio is a lightweight high-performance distributed data streaming system written in Rust and Web Assembly.
Alternative to Kafka + Flink in one product. Powered by Rust & WASM for maximum efficiency, minimum infrastructure overhead, and intuitive developer experience.
https://github.com/infinyon/fluvio?tab=readme-ov-file
https://www.fluvio.io/
https://docs.rs/fluvio/latest/fluvio/
Last updated