OpenAI Batch Connector

The Batch API connector allows you to submit requests for asynchronous batch processing on OpenAI. You can provide either a JSONL file or a JSON iterator, and the connector handles the upload to OpenAI, creation of the batch, and downloading of the output and error files.

Installation

To use the OpenAI Batch connector, run the following command:

$ pip install flytekitplugins-openai

Example usage

For a usage example, see OpenAI Batch connector example usage.

Local testing

To test an connector locally, create a class for the connector task that inherits from SyncConnectorExecutorMixin or AsyncConnectorExecutorMixin. These mixins can handle synchronous and synchronous tasks, respectively, and allow flytekit to mimic FlytePropeller’s behavior in calling the connector.

Flyte deployment configuration

If you are using a managed deployment of Flyte, you will need to contact your deployment administrator to configure connectors in your deployment.

To enable the OpenAI Batch connector in your Flyte deployment, refer to the OpenAI Batch connector setup guide