1.16.14

flytekit.extras.pytorch.checkpoint

Directory

Classes

Class Description
PyTorchCheckpoint This class is helpful to save a checkpoint.
PyTorchCheckpointTransformer TypeTransformer that supports serializing and deserializing checkpoint.

Protocols

Protocol Description
IsDataclass

flytekit.extras.pytorch.checkpoint.IsDataclass 

protocol IsDataclass()

flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint

This class is helpful to save a checkpoint.

class PyTorchCheckpoint(
    module: typing.Optional[torch.nn.modules.module.Module],
    hyperparameters: typing.Union[typing.Dict[str, typing.Any], NamedTuple, flytekit.extras.pytorch.checkpoint.IsDataclass, NoneType],
    optimizer: typing.Optional[torch.optim.optimizer.Optimizer],
)
Parameter Type Description
module typing.Optional[torch.nn.modules.module.Module]
hyperparameters typing.Union[typing.Dict[str, typing.Any], NamedTuple, flytekit.extras.pytorch.checkpoint.IsDataclass, NoneType]
optimizer typing.Optional[torch.optim.optimizer.Optimizer]

Methods

Method Description
from_dict()
from_json()
schema()
to_dict()
to_json()

from_dict() 

def from_dict(
    kvs: typing.Union[dict, list, str, int, float, bool, NoneType],
    infer_missing,
) -> ~A
Parameter Type Description
kvs typing.Union[dict, list, str, int, float, bool, NoneType]
infer_missing

from_json() 

def from_json(
    s: typing.Union[str, bytes, bytearray],
    parse_float,
    parse_int,
    parse_constant,
    infer_missing,
    kw,
) -> ~A
Parameter Type Description
s typing.Union[str, bytes, bytearray]
parse_float
parse_int
parse_constant
infer_missing
kw

schema() 

def schema(
    infer_missing: bool,
    only,
    exclude,
    many: bool,
    context,
    load_only,
    dump_only,
    partial: bool,
    unknown,
) -> SchemaType[A]
Parameter Type Description
infer_missing bool
only
exclude
many bool
context
load_only
dump_only
partial bool
unknown

to_dict() 

def to_dict(
    encode_json,
) -> typing.Dict[str, typing.Union[dict, list, str, int, float, bool, NoneType]]
Parameter Type Description
encode_json

to_json() 

def to_json(
    skipkeys: bool,
    ensure_ascii: bool,
    check_circular: bool,
    allow_nan: bool,
    indent: typing.Union[int, str, NoneType],
    separators: typing.Tuple[str, str],
    default: typing.Callable,
    sort_keys: bool,
    kw,
) -> str
Parameter Type Description
skipkeys bool
ensure_ascii bool
check_circular bool
allow_nan bool
indent typing.Union[int, str, NoneType]
separators typing.Tuple[str, str]
default typing.Callable
sort_keys bool
kw

flytekit.extras.pytorch.checkpoint.PyTorchCheckpointTransformer

TypeTransformer that supports serializing and deserializing checkpoint.

def PyTorchCheckpointTransformer()

Properties

Property Type Description
is_async None
name None
python_type None This returns the python type
type_assertions_enabled None Indicates if the transformer wants type assertions to be enabled at the core type engine layer

Methods

Method Description
assert_type()
from_binary_idl() This function primarily handles deserialization for untyped dicts, dataclasses, Pydantic BaseModels, and attribute access.
from_generic_idl() TODO: Support all Flyte Types.
get_literal_type() Converts the python type to a Flyte LiteralType.
guess_python_type() Converts the Flyte LiteralType to a python object type.
isinstance_generic()
to_html() Converts any python val (dataframe, int, float) to a html string, and it will be wrapped in the HTML div.
to_literal() Converts a given python_val to a Flyte Literal, assuming the given python_val matches the declared python_type.
to_python_value() Converts the given Literal to a Python Type.

assert_type() 

def assert_type(
    t: Type[T],
    v: T,
)
Parameter Type Description
t Type[T]
v T

from_binary_idl() 

def from_binary_idl(
    binary_idl_object: Binary,
    expected_python_type: Type[T],
) -> Optional[T]

This function primarily handles deserialization for untyped dicts, dataclasses, Pydantic BaseModels, and attribute access.｀

For untyped dict, dataclass, and pydantic basemodel: Life Cycle (Untyped Dict as example): python val -> msgpack bytes -> binary literal scalar -> msgpack bytes -> python val (to_literal) (from_binary_idl)

For attribute access: Life Cycle: python val -> msgpack bytes -> binary literal scalar -> resolved golang value -> binary literal scalar -> msgpack bytes -> python val (to_literal) (propeller attribute access) (from_binary_idl)

Parameter Type Description
binary_idl_object Binary
expected_python_type Type[T]

from_generic_idl() 

def from_generic_idl(
    generic: Struct,
    expected_python_type: Type[T],
) -> Optional[T]

TODO: Support all Flyte Types. This is for dataclass attribute access from input created from the Flyte Console.

Note:

  • This can be removed in the future when the Flyte Console support generate Binary IDL Scalar as input.
Parameter Type Description
generic Struct
expected_python_type Type[T]

get_literal_type() 

def get_literal_type(
    t: typing.Type[flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint],
) -> flytekit.models.types.LiteralType

Converts the python type to a Flyte LiteralType

Parameter Type Description
t typing.Type[flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint]

guess_python_type() 

def guess_python_type(
    literal_type: flytekit.models.types.LiteralType,
) -> typing.Type[flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint]

Converts the Flyte LiteralType to a python object type.

Parameter Type Description
literal_type flytekit.models.types.LiteralType

isinstance_generic() 

def isinstance_generic(
    obj,
    generic_alias,
)
Parameter Type Description
obj
generic_alias

to_html() 

def to_html(
    ctx: FlyteContext,
    python_val: T,
    expected_python_type: Type[T],
) -> str

Converts any python val (dataframe, int, float) to a html string, and it will be wrapped in the HTML div

Parameter Type Description
ctx FlyteContext
python_val T
expected_python_type Type[T]

to_literal() 

def to_literal(
    ctx: flytekit.core.context_manager.FlyteContext,
    python_val: flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint,
    python_type: typing.Type[flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint],
    expected: flytekit.models.types.LiteralType,
) -> flytekit.models.literals.Literal

Converts a given python_val to a Flyte Literal, assuming the given python_val matches the declared python_type. Implementers should refrain from using type(python_val) instead rely on the passed in python_type. If these do not match (or are not allowed) the Transformer implementer should raise an AssertionError, clearly stating what was the mismatch

Parameter Type Description
ctx flytekit.core.context_manager.FlyteContext A FlyteContext, useful in accessing the filesystem and other attributes
python_val flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint The actual value to be transformed
python_type typing.Type[flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint] The assumed type of the value (this matches the declared type on the function)
expected flytekit.models.types.LiteralType Expected Literal Type

to_python_value() 

def to_python_value(
    ctx: flytekit.core.context_manager.FlyteContext,
    lv: flytekit.models.literals.Literal,
    expected_python_type: typing.Type[flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint],
) -> flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint

Converts the given Literal to a Python Type. If the conversion cannot be done an AssertionError should be raised

Parameter Type Description
ctx flytekit.core.context_manager.FlyteContext FlyteContext
lv flytekit.models.literals.Literal The received literal Value
expected_python_type typing.Type[flytekit.extras.pytorch.checkpoint.PyTorchCheckpoint] Expected native python type that should be returned