Sponsored Content
Skip to main content
Ctrl+K
Apache DataFusion main documentation - Home Apache DataFusion main documentation - Home

ASF Links

  • Apache Software Foundation
  • License
  • Donate
  • Thanks
  • Security

Links

  • GitHub and Issue Tracker
  • crates.io
  • API Docs
  • Blog
  • Code of conduct
  • Download

User Guide

  • Introduction
  • Example Usage
  • Features
  • Concepts, Readings, Events
  • Crate Configuration
  • DataFusion CLI
    • Overview
    • Installation
    • Usage
    • Local Files / Directories
    • CLI Specific Functions
  • DataFrame API
  • Gentle Arrow Introduction
  • Expression API
  • SQL Reference
    • Data Types
    • Struct Type Coercion and Field Mapping
    • SELECT syntax
    • Subqueries
    • DDL
    • DML
    • EXPLAIN
    • Information Schema
    • Operators and Literals
    • Aggregate Functions
    • Window Functions
    • Scalar Functions
    • Special Functions
    • Format Options
    • Prepared Statements
  • Configuration Settings
  • Reading Explain Plans
  • Parquet Content-Defined Chunking
  • Metrics
  • Frequently Asked Questions

Library User Guide

  • Introduction
  • Securing DataFusion
  • Upgrade Guides
    • DataFusion 57.0.0
    • DataFusion 56.0.0
    • DataFusion 55.0.0
    • DataFusion 54.0.0
    • DataFusion 53.0.0
    • DataFusion 52.0.0
    • DataFusion 51.0.0
    • DataFusion 50.0.0
    • DataFusion 49.0.0
    • DataFusion 48.0.1
    • DataFusion 48.0.0
    • DataFusion 47.0.0
    • DataFusion 46.0.0
  • Extensions List
  • Using the SQL API
  • Extending SQL Syntax
  • Working with Exprs
  • Using the DataFrame API
  • Building Logical Plans
  • Catalogs, Schemas, and Tables
  • Functions
    • Adding User Defined Functions: Scalar/Window/Aggregate/Table Functions
    • Spark Compatible Functions
  • Custom Table Provider
  • Table Constraint Enforcement
  • Extending Operators
  • Profiling Cookbook
  • Query Optimizer

Contributor Guide

  • Introduction
  • AI Policy
  • Reviewing Pull Requests
  • Community Communication
  • Development Environment
  • Architecture
  • Workspace Dependency Graph
  • Testing
    • Testing Infrastructure
  • API health policy
  • HOWTOs
  • Release Management
  • Roadmap and Improvement Proposals
  • Governance
  • Inviting New Committers and PMC Members
  • Specifications
    • Invariants
    • Output field name semantics
  • Google Summer of Code (GSOC)
    • GSoC Application Guidelines (2025)
    • GSoC Project Ideas (2025)

DataFusion Subprojects

  • DataFusion Ballista
  • DataFusion Comet
  • DataFusion Java
  • DataFusion Python
  • Functions
  • Spark Compatible Functions

Spark Compatible Functions#

The datafusion-spark crate provides Apache Spark-compatible expressions for use with DataFusion. It is developed in the apache/datafusion-spark repository.

Please see the documentation for the datafusion-spark crate for more details.

previous

Adding User Defined Functions: Scalar/Window/Aggregate/Table Functions

next

Custom Table Provider
Edit on GitHub
Show Source

Apache DataFusion, Apache, the Apache feather logo, and the Apache DataFusion project logo

are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.

Wait! Don't Go Yet 🚀

Get our FREE eBook "10 Programming Tips That Changed Everything" when you subscribe to our newsletter!

No spam. Unsubscribe anytime.