Flyte Datacatalog Configuration
Section: application
grpcPort (int)
On which grpc port to serve Catalog
Default Value:
"8081"grpcServerReflection (bool)
Enable GRPC Server Reflection
Default Value:
"true"grpcMaxRecvMsgSizeMBs (int)
The max receive message size; if unset defaults to gRPC server default value
Default Value:
"0"httpPort (int)
On which http port to serve Catalog
Default Value:
"8080"secure (bool)
Whether to run Catalog in secure mode or not
Default Value:
"false"readHeaderTimeoutSeconds (int)
The amount of time allowed to read request headers.
Default Value:
"32"Section: database
host (string)
Default Value:
""port (int)
Default Value:
"0"dbname (string)
Default Value:
""username (string)
Default Value:
""password (string)
Default Value:
""passwordPath (string)
Default Value:
""options (string)
Default Value:
""debug (bool)
Default Value:
"false"enableForeignKeyConstraintWhenMigrating (bool)
Whether to enable gorm foreign keys when migrating the db
Default Value:
"false"maxIdleConnections (int)
maxIdleConnections sets the maximum number of connections in the idle connection pool.
Default Value:
"10"maxOpenConnections (int)
maxOpenConnections sets the maximum number of open connections to the database.
Default Value:
"100"connMaxLifeTime ( config.Duration)
sets the maximum amount of time a connection may be reused
Default Value:
1h0m0spostgres ( database.PostgresConfig)
Default Value:
dbname: postgres
debug: false
host: localhost
options: sslmode=disable
password: postgres
passwordPath: ""
port: 30001
readReplicaHost: localhost
username: postgressqlite ( database.SQLiteConfig)
Default Value:
file: ""config.Duration
Duration (int64)
Default Value:
1h0m0sdatabase.PostgresConfig
host (string)
The host name of the database server
Default Value:
localhostreadReplicaHost (string)
The host name of the read replica database server
Default Value:
localhostport (int)
The port name of the database server
Default Value:
"30001"dbname (string)
The database name
Default Value:
postgresusername (string)
The database user who is connecting to the server.
Default Value:
postgrespassword (string)
The database password.
Default Value:
postgrespasswordPath (string)
Points to the file containing the database password.
Default Value:
""options (string)
See http://gorm.io/docs/connecting_to_the_database.html for available options passed, in addition to the above.
Default Value:
sslmode=disabledebug (bool)
Whether or not to start the database connection with debug mode enabled.
Default Value:
"false"database.SQLiteConfig
file (string)
The path to the file (existing or new) where the DB should be created / stored. If existing, then this will be reused, else a new will be created
Default Value:
""Section: datacatalog
storage-prefix (string)
StoragePrefix specifies the prefix where DataCatalog stores offloaded ArtifactData in CloudStorage. If not specified, the data will be stored in the base container directly.
Default Value:
metadatametrics-scope (string)
Scope that the metrics will record under.
Default Value:
datacatalogprofiler-port (int)
Port that the profiling service is listening on.
Default Value:
"10254"heartbeat-grace-period-multiplier (int)
Number of heartbeats before a reservation expires without an extension.
Default Value:
"3"max-reservation-heartbeat ( config.Duration)
The maximum available reservation extension heartbeat interval.
Default Value:
10sSection: logger
show-source (bool)
Includes source code location in logs.
Default Value:
"false"mute (bool)
Mutes all logs regardless of severity. Intended for benchmarks/tests only.
Default Value:
"false"level (int)
Sets the minimum logging level.
Default Value:
"3"formatter ( logger.FormatterConfig)
Sets logging format.
Default Value:
type: jsonlogger.FormatterConfig
type (string)
Sets logging format type.
Default Value:
jsonSection: otel
type (string)
Sets the type of exporter to configure [noop/file/jaeger/otlpgrpc/otlphttp].
Default Value:
noopfile ( otelutils.FileConfig)
Configuration for exporting telemetry traces to a file
Default Value:
filename: /tmp/trace.txtjaeger ( otelutils.JaegerConfig)
Configuration for exporting telemetry traces to a jaeger
Default Value:
endpoint: http://localhost:14268/api/tracesotlpgrpc ( otelutils.OtlpGrpcConfig)
Configuration for exporting telemetry traces to an OTLP gRPC collector
Default Value:
endpoint: http://localhost:4317otlphttp ( otelutils.OtlpHttpConfig)
Configuration for exporting telemetry traces to an OTLP HTTP collector
Default Value:
endpoint: http://localhost:4318/v1/tracessampler ( otelutils.SamplerConfig)
Configuration for the sampler to use for the tracer
Default Value:
parentSampler: always
traceIdRatio: 0.01otelutils.FileConfig
filename (string)
Filename to store exported telemetry traces
Default Value:
/tmp/trace.txtotelutils.JaegerConfig
endpoint (string)
Endpoint for the jaeger telemetry trace ingestor
Default Value:
http://localhost:14268/api/tracesotelutils.OtlpGrpcConfig
endpoint (string)
Endpoint for the OTLP telemetry trace collector
Default Value:
http://localhost:4317otelutils.OtlpHttpConfig
endpoint (string)
Endpoint for the OTLP telemetry trace collector
Default Value:
http://localhost:4318/v1/tracesotelutils.SamplerConfig
parentSampler (string)
Sets the parent sampler to use for the tracer
Default Value:
alwaystraceIdRatio (float64)
Default Value:
"0.01"Section: storage
type (string)
Sets the type of storage to configure [s3/minio/local/mem/stow].
Default Value:
s3connection ( storage.ConnectionConfig)
Default Value:
access-key: ""
auth-type: iam
disable-ssl: false
endpoint: ""
region: us-east-1
secret-key: ""stow ( storage.StowConfig)
Storage config for stow backend.
Default Value:
{}container (string)
Initial container (in s3 a bucket) to create -if it doesn't exist-.'
Default Value:
""enable-multicontainer (bool)
If this is true, then the container argument is overlooked and redundant. This config will automatically open new connections to new containers/buckets as they are encountered
Default Value:
"false"cache ( storage.CachingConfig)
Default Value:
max_size_mbs: 0
target_gc_percent: 0limits ( storage.LimitsConfig)
Sets limits for stores.
Default Value:
maxDownloadMBs: 2defaultHttpClient ( storage.HTTPClientConfig)
Sets the default http client config.
Default Value:
headers: null
timeout: 0ssignedUrl ( storage.SignedURLConfig)
Sets config for SignedURL.
Default Value:
{}storage.CachingConfig
max_size_mbs (int)
Maximum size of the cache where the Blob store data is cached in-memory. If not specified or set to 0, cache is not used
Default Value:
"0"target_gc_percent (int)
Sets the garbage collection target percentage.
Default Value:
"0"storage.ConnectionConfig
endpoint ( config.URL)
URL for storage client to connect to.
Default Value:
""auth-type (string)
Auth Type to use [iam,accesskey].
Default Value:
iamaccess-key (string)
Access key to use. Only required when authtype is set to accesskey.
Default Value:
""secret-key (string)
Secret to use when accesskey is set.
Default Value:
""region (string)
Region to connect to.
Default Value:
us-east-1disable-ssl (bool)
Disables SSL connection. Should only be used for development.
Default Value:
"false"config.URL
URL ( url.URL)
Default Value:
ForceQuery: false
Fragment: ""
Host: ""
OmitHost: false
Opaque: ""
Path: ""
RawFragment: ""
RawPath: ""
RawQuery: ""
Scheme: ""
User: nullurl.URL
Scheme (string)
Default Value:
""Opaque (string)
Default Value:
""User (url.Userinfo)
Default Value:
nullHost (string)
Default Value:
""Path (string)
Default Value:
""RawPath (string)
Default Value:
""OmitHost (bool)
Default Value:
"false"ForceQuery (bool)
Default Value:
"false"RawQuery (string)
Default Value:
""Fragment (string)
Default Value:
""RawFragment (string)
Default Value:
""storage.HTTPClientConfig
headers (map[string][]string)
Default Value:
nulltimeout ( config.Duration)
Sets time out on the http client.
Default Value:
0sstorage.LimitsConfig
maxDownloadMBs (int64)
Maximum allowed download size (in MBs) per call.
Default Value:
"2"storage.SignedURLConfig
stowConfigOverride (map[string]string)
Default Value:
nullstorage.StowConfig
kind (string)
Kind of Stow backend to use. Refer to github/flyteorg/stow
Default Value:
""config (map[string]string)
Configuration for stow backend. Refer to github/flyteorg/stow
Default Value:
{}