Flyte 2 is available today for local execution - distributed execution coming to open source soon. Preview Flyte 2 for production, hosted on Union.ai

BigQueryConnector

Package: flyteplugins.bigquery

Methods

Method	Description
`create()`	Return a resource meta that can be used to get the status of the task.
`delete()`	Delete the task.
`get()`	Return the status of the task, and return the outputs in some cases.
`get_logs()`	Return the metrics for the task.
`get_metrics()`	Return the metrics for the task.

create()

        
    
def create(
    task_template: flyteidl2.core.tasks_pb2.TaskTemplate,
    inputs: typing.Optional[typing.Dict[str, typing.Any]],
    google_application_credentials: typing.Optional[str],
    kwargs,
) -> flyteplugins.bigquery.connector.BigQueryMetadata

Return a resource meta that can be used to get the status of the task.

Parameter	Type	Description
`task_template`	`flyteidl2.core.tasks_pb2.TaskTemplate`
`inputs`	`typing.Optional[typing.Dict[str, typing.Any]]`
`google_application_credentials`	`typing.Optional[str]`
`kwargs`	`**kwargs`

delete()

        
    
def delete(
    resource_meta: flyteplugins.bigquery.connector.BigQueryMetadata,
    google_application_credentials: typing.Optional[str],
    kwargs,
)

Delete the task. This call should be idempotent. It should raise an error if fails to delete the task.

Parameter	Type	Description
`resource_meta`	`flyteplugins.bigquery.connector.BigQueryMetadata`
`google_application_credentials`	`typing.Optional[str]`
`kwargs`	`**kwargs`

get()

        
    
def get(
    resource_meta: flyteplugins.bigquery.connector.BigQueryMetadata,
    google_application_credentials: typing.Optional[str],
    kwargs,
) -> flyte.connectors._connector.Resource

Return the status of the task, and return the outputs in some cases. For example, bigquery job can’t write the structured dataset to the output location, so it returns the output literals to the propeller, and the propeller will write the structured dataset to the blob store.

Parameter	Type	Description
`resource_meta`	`flyteplugins.bigquery.connector.BigQueryMetadata`
`google_application_credentials`	`typing.Optional[str]`
`kwargs`	`**kwargs`

get_logs()

        
    
def get_logs(
    resource_meta: flyte.connectors._connector.ResourceMeta,
    kwargs,
) -> flyteidl2.connector.connector_pb2.GetTaskLogsResponse

Return the metrics for the task.

Parameter	Type	Description
`resource_meta`	`flyte.connectors._connector.ResourceMeta`
`kwargs`	`**kwargs`

get_metrics()

        
    
def get_metrics(
    resource_meta: flyte.connectors._connector.ResourceMeta,
    kwargs,
) -> flyteidl2.connector.connector_pb2.GetTaskMetricsResponse

Return the metrics for the task.

Parameter	Type	Description
`resource_meta`	`flyte.connectors._connector.ResourceMeta`
`kwargs`	`**kwargs`