BigQuery connector

Installation

To install the BigQuery connector, run the following command:

This connector is purely a spec. Since SQL is completely portable, there is no need to build a Docker container.

Example usage

For an example query, see BigQuery connector example usage

Local testing

To test the BigQuery connector locally, create a class for the connector task that inherits from AsyncConnectorExecutorMixin. This mixin can handle asynchronous tasks and allows flytekit to mimic FlytePropeller’s behavior in calling the connector.

In some cases, you will need to store credentials in your local environment when testing locally.

Flyte deployment configuration

If you are using a managed deployment of Flyte, you will need to contact your deployment administrator to configure connectors in your deployment.

To enable the BigQuery connector in your Flyte deployment, see the BigQuery connector deployment guide