TokenEstimator

Package: flyte.extras

Protocol for records that can estimate their own token count.

Implement this on your record type and the :class:TokenBatcher will call it automatically when no explicit estimated_tokens is passed to :meth:TokenBatcher.submit.

Example::

@dataclass
class Prompt:
    text: str

    def estimate_tokens(self) -> int:
        return len(self.text) // 4 + 1

protocol TokenEstimator()

Methods

estimate_tokens()

def estimate_tokens()