1.16.29
Flytekit SDK
These are the Flytekit SDK API docs.
Flytekit is the core Python SDK for the Union and Flyte platforms.
Developing on Flyte
For developing on the Flyte platform you need to add the flytekit package to your project:
$ uv add flytekitThis will install the Flytekit SDK and the pyflyte command-line tool.
When working with the FLytekit SDK you will be using the pyflyte CLI and the Flytekit SDK docs (not the Union SDK docs).
Developing on Union
For developing on the Union platform you need to add the union package to your project:
$ uv add unionThis will install the Union SDK, which is a superset of the Flytekit SDK.
It will also install the union command-line tool.
When working with the Union SDK you will be using the union CLI and both the Flytekit SDK and the Union SDK docs.
Directory
Classes
Protocols
| Protocol | Description |
|---|---|
flytekit.configuration.plugin.FlytekitPluginProtocol |
|
flytekit.core.artifact.ArtifactSerializationHandler |
This protocol defines the interface for serializing artifact-related entities down to Flyte IDL. |
flytekit.core.cache.CachePolicy |
|
flytekit.core.context_manager.SerializableToString |
This protocol is used by the Artifact create_from function. |
flytekit.core.promise.HasFlyteInterface |
|
flytekit.core.promise.LocallyExecutable |
|
flytekit.core.promise.SupportsNodeCreation |
|
flytekit.core.schedule.LaunchPlanTriggerBase |
|
flytekit.deck.renderer.Renderable |
|
flytekit.sensor.base_sensor.SensorConfig |
Functions
| Function | Description |
|---|---|
flytekit.current_context() |
Use this method to get a handle of specific parameters available in a flyte task. |
flytekit.load_implicit_plugins() |
This method allows loading all plugins that have the entrypoint specification. |
flytekit.new_context() |
|
flytekit.bin.entrypoint.get_container_error_timestamp() |
Get timestamp for ContainerError. |
flytekit.bin.entrypoint.get_one_of() |
Helper function to iterate through a series of different environment variables. |
flytekit.bin.entrypoint.get_traceback_str() |
|
flytekit.bin.entrypoint.get_version_message() |
|
flytekit.bin.entrypoint.normalize_inputs() |
|
flytekit.bin.entrypoint.setup_execution() |
|
flytekit.clients.auth.default_html.get_default_success_html() |
|
flytekit.clients.auth.token_client.get_basic_authorization_header() |
This function transforms the client id and the client secret into a header that conforms with http basic auth. |
flytekit.clients.auth.token_client.get_device_code() |
Retrieves the device Authentication code that can be done to authenticate the request using a browser on a separate device. |
flytekit.clients.auth.token_client.get_token() |
retrieved from the IDP, the third is the expiration in seconds. |
flytekit.clients.auth.token_client.poll_token_endpoint() |
|
flytekit.clients.auth_helper.bootstrap_creds_from_server() |
Retrieves the SSL cert from the remote and uses that. should be used only if insecure-skip-verify. |
flytekit.clients.auth_helper.get_authenticated_channel() |
Returns a new channel for the given config that is authenticated. |
flytekit.clients.auth_helper.get_authenticator() |
Returns a new authenticator based on the platform config. |
flytekit.clients.auth_helper.get_channel() |
Creates a new grpc.Channel given a platformConfig. |
flytekit.clients.auth_helper.get_proxy_authenticator() |
|
flytekit.clients.auth_helper.get_session() |
Return a new session for the given platform config. |
flytekit.clients.auth_helper.register_authenticator_plugin() |
Register an authenticator factory by name. |
flytekit.clients.auth_helper.upgrade_channel_to_authenticated() |
Given a grpc.Channel, preferably a secure channel, it returns a composed channel that uses Interceptor to perform an Oauth2.0 Auth flow. |
flytekit.clients.auth_helper.upgrade_channel_to_proxy_authenticated() |
If activated in the platform config, given a grpc.Channel, preferably a secure channel, it returns a composed channel that uses Interceptor to perform authentication with a proxy in front of Flyte. |
flytekit.clients.auth_helper.upgrade_session_to_proxy_authenticated() |
Given a requests.Session, it returns a new session that uses a custom HTTPAdapter to perform authentication with a proxy in front of Flyte. |
flytekit.clients.auth_helper.wrap_exceptions_channel() |
Wraps the input channel with RetryExceptionWrapperInterceptor. |
flytekit.clients.grpc_utils.deadline_interceptor.get_scoped_grpc_deadline() |
|
flytekit.clients.grpc_utils.deadline_interceptor.scoped_grpc_deadline() |
|
flytekit.clients.helpers.iterate_node_executions() |
This returns a generator for node executions. |
flytekit.clients.helpers.iterate_task_executions() |
This returns a generator for task executions, given a node execution identifier. |
flytekit.clis.helpers.display_help_with_error() |
|
flytekit.clis.helpers.hydrate_registration_parameters() |
This is called at registration time to fill out identifier fields (e.g. project, domain, version) that are mutable. |
flytekit.clis.helpers.parse_args_into_dict() |
Takes a tuple like (u’input_b=mystr’, u’input_c=18’) and returns a dictionary of input name to the original string value. |
flytekit.clis.helpers.str2bool() |
bool(‘False’) is True in Python, so we need to do some string parsing. |
flytekit.clis.sdk_in_container.backfill.resolve_backfill_window() |
Resolves the from_date -> to_date. |
flytekit.clis.sdk_in_container.build.build_command() |
Returns a function that is used to implement WorkflowCommand and build an image for flyte workflows. |
flytekit.clis.sdk_in_container.helpers.get_and_save_remote_with_click_context() |
NB: This function will by default mutate the click Context.obj dictionary, adding a remote key with value of the created FlyteRemote object. |
flytekit.clis.sdk_in_container.helpers.parse_copy() |
Helper function to parse cmd line args into enum. |
flytekit.clis.sdk_in_container.helpers.patch_image_config() |
Merge ImageConfig object with images defined in config file. |
flytekit.clis.sdk_in_container.run.dump_flyte_remote_snippet() |
|
flytekit.clis.sdk_in_container.run.get_entities_in_file() |
Returns a list of flyte workflow names and list of Flyte tasks in a file. |
flytekit.clis.sdk_in_container.run.is_optional() |
Checks if the given type is Optional Type. |
flytekit.clis.sdk_in_container.run.load_naive_entity() |
Load the workflow of a script file. |
flytekit.clis.sdk_in_container.run.options_from_run_params() |
|
flytekit.clis.sdk_in_container.run.run_command() |
Returns a function that is used to implement WorkflowCommand and execute a flyte workflow. |
flytekit.clis.sdk_in_container.run.run_remote() |
Helper method that executes the given remote FlyteLaunchplan, FlyteWorkflow or FlyteTask. |
flytekit.clis.sdk_in_container.run.to_click_option() |
This handles converting workflow input types to supported click parameters with callbacks to initialize the input values to their expected types. |
flytekit.clis.sdk_in_container.serialize.serialize_all() |
flyteidl.admin.launch_plan_pb2.LaunchPlan flyteidl.admin.workflow_pb2.WorkflowSpec flyteidl.admin.task_pb2.TaskSpec ```. |
flytekit.clis.sdk_in_container.serve.print_metadata() |
|
flytekit.clis.sdk_in_container.utils.get_option_from_metadata() |
|
flytekit.clis.sdk_in_container.utils.make_click_option_field() |
|
flytekit.clis.sdk_in_container.utils.pretty_print_exception() |
This method will print the exception in a nice way. |
flytekit.clis.sdk_in_container.utils.pretty_print_grpc_error() |
This method will print the grpc error that us more human readable. |
flytekit.clis.sdk_in_container.utils.pretty_print_traceback() |
This method will print the Traceback of an error. |
flytekit.clis.sdk_in_container.utils.remove_unwanted_traceback_frames() |
Custom function to remove certain frames from the traceback. |
flytekit.clis.sdk_in_container.utils.validate_package() |
This method will validate the packages passed in by the user. |
flytekit.configuration.file.bool_transformer() |
|
flytekit.configuration.file.comma_list_transformer() |
|
flytekit.configuration.file.int_transformer() |
|
flytekit.configuration.file.read_file_if_exists() |
Reads the contents of the file if passed a path. |
flytekit.configuration.file.set_if_exists() |
Given a dict d sets the key k with value of config v, if the config value v is set and return the updated dictionary. |
flytekit.configuration.plugin.get_plugin() |
Get current plugin. |
flytekit.core.array_node.array_node() |
ArrayNode implementation that maps over tasks and other Flyte entities. |
flytekit.core.array_node_map_task.array_node_map_task() |
Map task that uses the ArrayNode construct.. |
flytekit.core.array_node_map_task.map_task() |
Wrapper that creates a map task utilizing either the existing ArrayNodeMapTask or the drop in replacement ArrayNode implementation. |
flytekit.core.artifact_utils.idl_partitions_from_dict() |
|
flytekit.core.artifact_utils.idl_time_partition_from_datetime() |
|
flytekit.core.base_task.kwtypes() |
This is a small helper function to convert the keyword arguments to an OrderedDict of types. |
flytekit.core.condition.conditional() |
Use a conditional section to control the flow of a workflow. |
flytekit.core.condition.create_branch_node_promise_var() |
Generates a globally (wf-level) unique id for a variable. |
flytekit.core.condition.merge_promises() |
|
flytekit.core.condition.to_branch_node() |
|
flytekit.core.condition.to_case_block() |
|
flytekit.core.condition.to_ifelse_block() |
|
flytekit.core.condition.transform_to_boolexpr() |
|
flytekit.core.condition.transform_to_comp_expr() |
|
flytekit.core.condition.transform_to_conj_expr() |
|
flytekit.core.condition.transform_to_operand() |
|
flytekit.core.data_persistence.azure_setup_args() |
|
flytekit.core.data_persistence.get_additional_fsspec_call_kwargs() |
These are different from the setup args functions defined above. |
flytekit.core.data_persistence.get_fsspec_storage_options() |
|
flytekit.core.data_persistence.s3_setup_args() |
|
flytekit.core.environment.forge() |
|
flytekit.core.environment.inherit() |
|
flytekit.core.gate.approve() |
Create a Gate object for binary approval. |
flytekit.core.gate.sleep() |
Create a sleep Gate object. |
flytekit.core.gate.wait_for_input() |
Create a Gate object that waits for user input of the specified type. |
flytekit.core.interface.default_output_name() |
|
flytekit.core.interface.detect_artifact() |
If the user wishes to control how Artifacts are created (i.e. naming them, etc.) this is where we pick it up and store it in the interface. |
flytekit.core.interface.extract_return_annotation() |
The purpose of this function is to sort out whether a function is returning one thing, or multiple things, and to name the outputs accordingly, either by using our default name function, or from a typing.NamedTuple. |
flytekit.core.interface.output_name_generator() |
|
flytekit.core.interface.remap_shared_output_descriptions() |
Deals with mixed styles of return value descriptions used in docstrings. |
flytekit.core.interface.repr_kv() |
|
flytekit.core.interface.repr_type_signature() |
Converts an inputs and outputs to a type signature. |
flytekit.core.interface.transform_function_to_interface() |
From the annotations on a task function that the user should have provided, and the output names they want to use for each output parameter, construct the TypedInterface object. |
flytekit.core.interface.transform_inputs_to_parameters() |
Transforms the given interface (with inputs) to a Parameter Map with defaults set. |
flytekit.core.interface.transform_interface_to_list_interface() |
Takes a single task interface and interpolates it to an array interface - to allow performing distributed python map like functions. |
flytekit.core.interface.transform_interface_to_typed_interface() |
Transform the given simple python native interface to FlyteIDL’s interface. |
flytekit.core.interface.transform_type() |
|
flytekit.core.interface.transform_types_to_list_of_type() |
Converts unbound inputs into the equivalent (optional) collections. |
flytekit.core.interface.transform_variable_map() |
Given a map of str (names of inputs for instance) to their Python native types, return a map of the name to a Flyte Variable object with that type. |
flytekit.core.interface.verify_outputs_artifact_bindings() |
|
flytekit.core.launch_plan.reference_launch_plan() |
A reference launch plan is a pointer to a launch plan that already exists on your Flyte installation. |
flytekit.core.legacy_map_task.map_task() |
Use a map task for parallelizable tasks that run across a list of an input type. |
flytekit.core.node.assert_no_promises_in_resources() |
This function will raise an exception if any of the resources have promises in them. |
flytekit.core.node.assert_not_promise() |
This function will raise an exception if the value is a promise. |
flytekit.core.node_creation.create_node() |
This is the function you want to call if you need to specify dependencies between tasks that don’t consume and/or don’t produce outputs. |
flytekit.core.pod_template.serialize_pod_template() |
|
flytekit.core.promise.async_flyte_entity_call_handler() |
This is a limited async version of the main call handler. |
flytekit.core.promise.binding_data_from_python_std() |
|
flytekit.core.promise.binding_from_python_std() |
|
flytekit.core.promise.create_and_link_node() |
|
flytekit.core.promise.create_and_link_node_from_remote() |
This method is used to generate a node with bindings especially when using remote entities, like FlyteWorkflow, FlyteTask and FlyteLaunchplan. |
flytekit.core.promise.create_native_named_tuple() |
Creates and returns a Named tuple with all variables that match the expected named outputs. this makes it possible to run things locally and expect a more native behavior, i.e. address elements of a named tuple by name. |
flytekit.core.promise.create_task_output() |
|
flytekit.core.promise.extract_obj_name() |
Generates a shortened name, without the module information. |
flytekit.core.promise.flyte_entity_call_handler() |
This function is the call handler for tasks, workflows, and launch plans (which redirects to the underlying workflow). |
flytekit.core.promise.get_primitive_val() |
|
flytekit.core.promise.resolve_attr_path_in_dict() |
|
flytekit.core.promise.resolve_attr_path_in_pb_struct() |
Resolves the protobuf struct (e.g. dataclass) with attribute path. |
flytekit.core.promise.resolve_attr_path_in_promise() |
resolve_attr_path_in_promise resolves the attribute path in a promise and returns a new promise with the resolved value This is for local execution only. |
flytekit.core.promise.resolve_attr_path_recursively() |
This function resolves the attribute path in a nested structure recursively. |
flytekit.core.promise.to_binding() |
|
flytekit.core.promise.translate_inputs_to_literals() |
The point of this function is to extract out Literals from a collection of either Python native values (which would be converted into Flyte literals) or Promises (the literals in which would just get extracted). |
flytekit.core.promise.translate_inputs_to_native() |
|
flytekit.core.python_auto_container.get_registerable_container_image() |
Resolve the image to the real image name that should be used for registration. |
flytekit.core.python_auto_container.update_image_spec_copy_handling() |
This helper function is where the relationship between fast register and ImageSpec is codified. |
flytekit.core.reference.get_reference_entity() |
See the documentation for flytekit.reference_task and flytekit.reference_workflow as well. |
flytekit.core.resources.construct_extended_resources() |
Convert public extended resources to idl. |
flytekit.core.resources.convert_resources_to_resource_model() |
Convert flytekit Resources objects to a Resources model. |
flytekit.core.resources.pod_spec_from_resources() |
|
flytekit.core.task.decorate_function() |
Decorates the task with additional functionality if necessary. |
flytekit.core.task.eager() |
Eager workflow decorator. |
flytekit.core.task.reference_task() |
A reference task is a pointer to a task that already exists on your Flyte installation. |
flytekit.core.task.task() |
This is the core decorator to use for any task type in flytekit. |
flytekit.core.testing.patch() |
This is a decorator used for testing. |
flytekit.core.testing.task_mock() |
Use this method to mock a task declaration. |
flytekit.core.tracker.extract_task_module() |
Returns the task-name, absolute module and the string name of the callable. |
flytekit.core.tracker.get_full_module_path() |
|
flytekit.core.tracker.import_module_from_file() |
|
flytekit.core.tracker.is_functools_wrapped_module_level() |
Returns true if the function is a functools.wraps-updated function that is defined in the module-level scope. |
flytekit.core.tracker.is_ipython_or_pickle_exists() |
Returns true if the code is running in an IPython notebook or if a pickle file exists. |
flytekit.core.tracker.isnested() |
Returns true if a function is local to another function and is not accessible through a module. |
flytekit.core.tracker.istestfunction() |
Return true if the function is defined in a test module. |
flytekit.core.type_engine.convert_marshmallow_json_schema_to_python_class() |
Generate a model class based on the provided JSON Schema. |
flytekit.core.type_engine.convert_mashumaro_json_schema_to_python_class() |
Generate a model class based on the provided JSON Schema. |
flytekit.core.type_engine.dataclass_from_dict() |
Utility function to construct a dataclass object from dict. |
flytekit.core.type_engine.generate_attribute_list_from_dataclass_json() |
|
flytekit.core.type_engine.generate_attribute_list_from_dataclass_json_mixin() |
|
flytekit.core.type_engine.get_batch_size() |
|
flytekit.core.type_engine.get_underlying_type() |
Return the underlying type for annotated types or the type itself. |
flytekit.core.type_engine.is_annotated() |
|
flytekit.core.type_engine.modify_literal_uris() |
Modifies the literal object recursively to replace the URIs with the native paths in case they are of type “flyte://”. |
flytekit.core.type_engine.strict_type_hint_matching() |
Try to be smarter about guessing the type of the input (and hence the transformer). |
flytekit.core.type_helpers.load_type_from_tag() |
Loads python type from tag. |
flytekit.core.type_match_checking.literal_types_match() |
Returns if two LiteralTypes are the same. |
flytekit.core.utils.has_return_statement() |
|
flytekit.core.utils.load_proto_from_file() |
|
flytekit.core.utils.str2bool() |
Convert a string to a boolean. |
flytekit.core.utils.write_proto_to_file() |
|
flytekit.core.workflow.construct_input_promises() |
|
flytekit.core.workflow.get_promise() |
This is a helper function that will turn a binding into a Promise object, using a lookup map. |
flytekit.core.workflow.get_promise_map() |
Local execution of imperatively defined workflows is done node by node. |
flytekit.core.workflow.reference_workflow() |
A reference workflow is a pointer to a workflow that already exists on your Flyte installation. |
flytekit.core.workflow.workflow() |
This decorator declares a function to be a Flyte workflow. |
flytekit.deck.deck.generate_time_table() |
|
flytekit.deck.deck.get_deck_template() |
|
flytekit.exceptions.scopes.system_entry_point() |
The reason these two (see the user one below) decorators exist is to categorize non-Flyte exceptions at arbitrary locations. |
flytekit.exceptions.scopes.user_entry_point() |
See the comment for the system_entry_point above as well. |
flytekit.exceptions.utils.annotate_exception_with_code() |
Annotate the exception with the source code, and will be printed in the rich panel. |
flytekit.exceptions.utils.get_source_code_from_fn() |
Get the source code of the function and the column offset of the parameter defined in the input signature. |
flytekit.extend.backend.connector_service.record_connector_metrics() |
|
flytekit.extend.backend.utils.convert_to_flyte_phase() |
Convert the state from the connector to the phase in flyte. |
flytekit.extend.backend.utils.get_agent_secret() |
|
flytekit.extend.backend.utils.get_connector_secret() |
|
flytekit.extend.backend.utils.is_terminal_phase() |
Return true if the phase is terminal. |
flytekit.extend.backend.utils.mirror_async_methods() |
|
flytekit.extend.backend.utils.render_task_template() |
|
flytekit.extras.sqlite3.task.unarchive_file() |
Unarchive given archive and returns the unarchived file name. |
flytekit.extras.tasks.shell.get_raw_shell_task() |
|
flytekit.extras.tasks.shell.subproc_execute() |
Execute a command and capture its stdout and stderr. |
flytekit.image_spec.default_builder.create_docker_context() |
Populate tmp_dir with Dockerfile as specified by the image_spec. |
flytekit.image_spec.default_builder.get_flytekit_for_pypi() |
Get flytekit version on PyPI. |
flytekit.image_spec.default_builder.prepare_poetry_lock_command() |
|
flytekit.image_spec.default_builder.prepare_python_executable() |
|
flytekit.image_spec.default_builder.prepare_python_install() |
|
flytekit.image_spec.default_builder.prepare_uv_lock_command() |
|
flytekit.image_spec.image_spec.validate_container_registry_name() |
Validate Docker container registry name. |
flytekit.interaction.click_types.is_pydantic_basemodel() |
Checks if the python type is a pydantic BaseModel. |
flytekit.interaction.click_types.key_value_callback() |
Callback for click to parse key-value pairs. |
flytekit.interaction.click_types.labels_callback() |
Callback for click to parse labels. |
flytekit.interaction.click_types.literal_type_to_click_type() |
Converts a Flyte LiteralType given a python_type to a click.ParamType. |
flytekit.interaction.click_types.modify_literal_uris() |
Modifies the literal object recursively to replace the URIs with the native paths. |
flytekit.interaction.click_types.resource_callback() |
Click callback to parse resource strings like ‘cpu=1,mem=2Gi’ into a Resources object. |
flytekit.interaction.parse_stdin.parse_stdin_to_literal() |
Parses the user input from stdin and converts it to a literal of the given type. |
flytekit.interaction.string_literals.literal_map_string_repr() |
This method is used to convert a literal map to a string representation. |
flytekit.interaction.string_literals.literal_string_repr() |
This method is used to convert a literal to a string representation. |
flytekit.interaction.string_literals.primitive_to_string() |
This method is used to convert a primitive to a string representation. |
flytekit.interaction.string_literals.scalar_to_string() |
This method is used to convert a scalar to a string representation. |
flytekit.interactive.utils.execute_command() |
Execute a command in the shell. |
flytekit.interactive.utils.get_task_inputs() |
Read task input data from inputs.pb for a specific task function and convert it into Python types and structures. |
flytekit.interactive.utils.load_module_from_path() |
Imports a Python module from a specified file path. |
flytekit.interactive.vscode_lib.decorator.download_file() |
Download a file from a given URL using fsspec. |
flytekit.interactive.vscode_lib.decorator.download_vscode() |
Download vscode server and extension from remote to local and add the directory of binary executable to $PATH. |
flytekit.interactive.vscode_lib.decorator.exit_handler() |
1. |
flytekit.interactive.vscode_lib.decorator.get_code_server_info() |
Returns the code server information based on the system’s architecture. |
flytekit.interactive.vscode_lib.decorator.get_installed_extensions() |
Get the list of installed extensions. |
flytekit.interactive.vscode_lib.decorator.is_extension_installed() |
|
flytekit.interactive.vscode_lib.decorator.prepare_interactive_python() |
1. |
flytekit.interactive.vscode_lib.decorator.prepare_launch_json() |
Generate the launch.json and settings.json for users to easily launch interactive debugging and task resumption. |
flytekit.interactive.vscode_lib.decorator.prepare_resume_task_python() |
Generate a Python script for users to resume the task. |
flytekit.interfaces.random.seed_flyte_random() |
If one wants to influence the pseudo-random behavior of flytekit, this function can be used to seed the flytekit generator. |
flytekit.interfaces.stats.client.get_base_stats() |
|
flytekit.interfaces.stats.client.get_stats() |
|
flytekit.interfaces.stats.taggable.get_stats() |
|
flytekit.lazy_import.lazy_module.is_imported() |
This function is used to check if a module has been imported by the regular import. |
flytekit.lazy_import.lazy_module.lazy_module() |
This function is used to lazily import modules. |
flytekit.loggers.get_level_from_cli_verbosity() |
Converts a verbosity level from the CLI to a logging level. |
flytekit.loggers.initialize_global_loggers() |
Initializes the global loggers to the default configuration. |
flytekit.loggers.is_display_progress_enabled() |
|
flytekit.loggers.is_rich_logging_enabled() |
|
flytekit.loggers.set_developer_properties() |
developer logger is only used for debugging. |
flytekit.loggers.set_flytekit_log_properties() |
flytekit logger, refers to the framework logger. |
flytekit.loggers.set_user_logger_properties() |
user_space logger, refers to the user’s logger. |
flytekit.loggers.upgrade_to_rich_logging() |
|
flytekit.remote.backfill.create_backfill_workflow() |
Generates a new imperative workflow for the launchplan that can be used to backfill the given launchplan. |
flytekit.remote.data.download_literal() |
Download a single literal to a file, if it is a blob or structured dataset. |
flytekit.remote.metrics.aggregate_reference_span() |
|
flytekit.remote.metrics.aggregate_spans() |
|
flytekit.remote.metrics.print_span() |
|
flytekit.remote.remote_fs.get_flyte_fs() |
|
flytekit.tools.fast_registration.compress_tarball() |
Compress code tarball using pigz if available, otherwise gzip. |
flytekit.tools.fast_registration.compute_digest() |
Walks the entirety of the source dir to compute a deterministic md5 hex digest of the dir contents. :return Text. |
flytekit.tools.fast_registration.download_distribution() |
Downloads a remote code distribution and overwrites any local files. |
flytekit.tools.fast_registration.fast_package() |
Takes a source directory and packages everything not covered by common ignores into a tarball named after a hexdigest of the included files. :return os.PathLike. |
flytekit.tools.fast_registration.get_additional_distribution_loc() |
:return Text. |
flytekit.tools.fast_registration.print_ls_tree() |
|
flytekit.tools.interactive.ipython_check() |
Check if interface is launching from iPython (not colab) :return is_ipython (bool): True or False. |
flytekit.tools.module_loader.add_sys_path() |
Temporarily add given path to sys.path. |
flytekit.tools.module_loader.just_load_modules() |
This one differs from the above in that we don’t yield anything, just load all the modules. |
flytekit.tools.module_loader.load_object_from_module() |
TODO: Handle corner cases, like where the first part is [] maybe. |
flytekit.tools.module_loader.module_load_error_handler() |
|
flytekit.tools.repo.find_common_root() |
Given an arbitrary list of folders and files, this function will use the script mode function to walk up the filesystem to find the first folder without an init file. |
flytekit.tools.repo.list_packages_and_modules() |
This is a helper function that returns the input list of python packages/modules as a dot delinated list relative to the given project_root. |
flytekit.tools.repo.package() |
Package the given entities and the source code (if fast is enabled) into a package with the given name in output. |
flytekit.tools.repo.print_registration_status() |
|
flytekit.tools.repo.register() |
Temporarily, for fast register, specify both the fast arg as well as copy_style. fast == True with copy_style == None means use the old fast register tar’ring method. |
flytekit.tools.repo.serialize_and_package() |
Fist serialize and then package all entities Temporarily for fast package, specify both the fast arg as well as copy_style. fast == True with copy_style == None means use the old fast register tar’ring method. |
flytekit.tools.repo.serialize_get_control_plane_entities() |
See flytekit.models.core.identifier.ResourceType to match the trailing index in the file name with the entity type. |
flytekit.tools.repo.serialize_load_only() |
See flytekit.models.core.identifier.ResourceType to match the trailing index in the file name with the entity type. |
flytekit.tools.repo.serialize_to_folder() |
Serialize the given set of python packages to a folder. |
flytekit.tools.script_mode.add_imported_modules_from_source() |
Copies modules into destination that are in modules. |
flytekit.tools.script_mode.compress_scripts() |
Compresses the single script while maintaining the folder structure for that file. |
flytekit.tools.script_mode.get_all_modules() |
Import python file with module_name in source_path and return all modules. |
flytekit.tools.script_mode.list_all_files() |
|
flytekit.tools.script_mode.list_imported_modules_as_files() |
Copies modules into destination that are in modules. |
flytekit.tools.script_mode.ls_files() |
user_modules_and_packages is a list of the Python modules and packages, expressed as absolute paths, that the user has run this pyflyte command with. |
flytekit.tools.script_mode.tar_strip_file_attributes() |
|
flytekit.tools.serialize_helpers.get_registrable_entities() |
Returns all entities that can be serialized and should be sent over to Flyte backend. |
flytekit.tools.serialize_helpers.persist_registrable_entities() |
For protobuf serializable list of entities, writes a file with the name if the entity and enumeration order to the specified folder. |
flytekit.tools.subprocess.check_call() |
|
flytekit.tools.translator.gather_dependent_entities() |
The get_serializable function above takes in an OrderedDict that helps keep track of dependent entities. |
flytekit.tools.translator.get_command_prefix_for_fast_execute() |
|
flytekit.tools.translator.get_reference_spec() |
|
flytekit.tools.translator.get_serializable() |
The flytekit authoring code produces objects representing Flyte entities (tasks, workflows, etc.). |
flytekit.tools.translator.get_serializable_array_node() |
|
flytekit.tools.translator.get_serializable_array_node_map_task() |
|
flytekit.tools.translator.get_serializable_branch_node() |
|
flytekit.tools.translator.get_serializable_flyte_task() |
TODO replace with deep copy. |
flytekit.tools.translator.get_serializable_flyte_workflow() |
TODO replace with deep copy. |
flytekit.tools.translator.get_serializable_launch_plan() |
|
flytekit.tools.translator.get_serializable_node() |
|
flytekit.tools.translator.get_serializable_task() |
|
flytekit.tools.translator.get_serializable_workflow() |
|
flytekit.tools.translator.prefix_with_fast_execute() |
|
flytekit.tools.translator.to_serializable_case() |
|
flytekit.tools.translator.to_serializable_cases() |
|
flytekit.types.directory.types.noop() |
|
flytekit.types.file.file.noop() |
|
flytekit.types.numpy.ndarray.extract_metadata() |
|
flytekit.types.schema.types.generate_ordered_files() |
|
flytekit.types.structured.lazy_import_structured_dataset_handler() |
|
flytekit.types.structured.basic_dfs.get_pandas_storage_options() |
|
flytekit.types.structured.snowflake.get_private_key() |
|
flytekit.types.structured.structured_dataset.convert_schema_type_to_structured_dataset_type() |
|
flytekit.types.structured.structured_dataset.extract_cols_and_format() |
Helper function, just used to iterate through Annotations and extract out the following information. |
flytekit.types.structured.structured_dataset.flatten_dict() |
|
flytekit.types.structured.structured_dataset.get_supported_types() |
|
flytekit.utils.dict_formatter.format_dict() |
Recursively update a dictionary with format strings with values from another dictionary where the keys match the format string. |
flytekit.utils.dict_formatter.get_nested_value() |
Retrieve the nested value from a dictionary based on a list of keys. |
flytekit.utils.dict_formatter.replace_placeholder() |
Replace a placeholder in the original string and handle the specific logic for the sagemaker service and idempotence token. |
flytekit.utils.pbhash.compute_hash() |
Computes a deterministic hash in bytes for the Protobuf object. |
flytekit.utils.pbhash.compute_hash_string() |
Computes a deterministic hash in base64 encoded string for the Protobuf object. |