Integrations

This section describes how to enable integrations that connect to other services, such as Apache Airflow, ChatGPT, dbt, Databricks, NVidia DGX, Snowflake, and more.

Additional integrations

A variety of connectors and plugins enable you to integrate Union.ai with additional external services. If you don’t see the integration you need, you can create your own.

AI integrations

• OpenAI connectors Send prompts to ChatGPT and receive responses.

Database integrations

• DuckDB plugin Run analytical queries using DuckDB.
• Google BigQuery connector Query Google BigQuery tables from your workflows.
• SQL plugin Execute SQL queries as tasks.

Data pipeline integrations

• Apache Airflow connector Run Apache Airflow jobs in your workflows.
• Databricks connector Run Databricks jobs in your workflows.
• dbt plugin Run and test your dbt pipelines in Union.ai.
• MemVerge Memory Machine Cloud connector Execute tasks using the MemVerge Memory Machine Cloud connector
• NVIDIA DGX connector Run jobs on the NVIDIA DGX platform.
• Snowflake connector Run Snowflake jobs in your workflow.

Data science tooling integrations

• Modin plugin Scale pandas workflows with modin.
• Papermill plugin Execute Jupyter Notebooks with papermill.

Data validation integrations

• Dolt plugin Version your SQL database with dolt.
• Great Expectations plugin Validate data with great_expectations.
• Pandera plugin Validate pandas dataframes with pandera.
• whylogs plugin The open standard for data logging.

MLOps integrations

• mlflow plugin The open standard for model tracking.
• ONNX plugin Convert ML models to ONNX models seamlessly.

Misc

• File sensor connector Detect files appearing in your local or remote filesystem.