Run context
Every Flyte run has a run context: a set of invocation-time parameters that control where the run executes, where its outputs are stored, how caching behaves, and more.
There are two sides to run context:
- Write side:
flyte.with_runcontext()sets run parameters before the run starts (programmatic) or via CLI flags. - Read side:
flyte.ctx()accesses run parameters inside a running task.
Configuring a run with flyte.with_runcontext()
flyte.with_runcontext() returns a runner object. Call .run(task, ...) on it to start the run with the specified context:
import flyte
env = flyte.TaskEnvironment("run-context-example")
@env.task
async def process(n: int) -> int:
return n * 2
@env.task
async def root() -> int:
return await process(21)
if __name__ == "__main__":
flyte.init_from_config()
flyte.with_runcontext(
name="my-run",
project="my-project",
domain="development",
).run(root)
All parameters are optional. Unset parameters inherit from the configuration file (config.yaml) or system defaults.
Execution target
| Parameter | Type | Default | Description |
|---|---|---|---|
mode |
"local" | "remote" | "hybrid" |
from config | Where the run executes. "remote" runs on the Flyte backend; "local" runs in-process. |
project |
str |
from config | Project to run in. |
domain |
str |
from config | Domain to run in (e.g. "development", "production"). |
name |
str |
auto-generated | Custom name for the run, visible in the UI. |
version |
str |
from code bundle | Version string for the ephemeral task deployment. |
queue |
str |
from config | Cluster queue to schedule tasks on. |
interruptible |
bool |
per-task setting | Override the interruptible setting for all tasks in the run. True allows spot/preemptible instances; False forces non-interruptible instances. |
with_runcontext() also accepts a debug parameter (bool, default False): launch the run in debug mode, starting a browser-based VS Code session on the task pod so you can step through the task interactively. See
Debug a run.
Storage
| Parameter | Type | Default | Description |
|---|---|---|---|
raw_data_path |
str |
from config | Storage prefix for offloaded data types (
Files,
Dirs,
DataFrames, checkpoints). Accepts s3://, gs://, or local paths. |
run_base_dir |
str |
auto-generated | Base directory for the run’s inputs, outputs, and intermediate per-action artifacts in the data plane object store. Distinct from raw_data_path. |
For the difference between what raw_data_path controls (offloaded values) and what stays at the deployment-configured location (inputs.pb, outputs.pb, Decks) or in the control plane database, see
Where your data lives.
To direct all task outputs to a specific bucket for a run:
if __name__ == "__main__":
flyte.init_from_config()
flyte.with_runcontext(
# Store all task outputs in a dedicated S3 prefix for this run
raw_data_path="s3://my-bucket/runs/experiment-42/",
).run(root)
The equivalent CLI flag is --raw-data-path. See
Run command options for CLI usage.
Caching
| Parameter | Type | Default | Description |
|---|---|---|---|
overwrite_cache |
bool |
False |
Re-execute all tasks even if a cached result exists, and overwrite the cache with new results. |
disable_run_cache |
bool |
False |
Skip cache lookups and writes entirely for this run. |
cache_lookup_scope |
"global" | … |
"global" |
Scope for cache lookups. |
Recovery
| Parameter | Type | Default | Description |
|---|---|---|---|
recover |
bool | str |
False |
Reuse the actions that succeeded in a prior run, re-executing only what failed or changed. A run name recovers from that run and is the only form valid on .run(); True recovers from the run being rerun and is only valid on .rerun(). Remote-only. |
recover_force_rerun_actions |
Sequence[str] |
None |
Names of actions that must execute again even though they succeeded in the source run. Requires recover. |
allow_missing_source_outputs |
bool |
False |
Proceed with a rerun or recovery when the source run’s outputs have been cleaned up from storage, using its inputs URI directly. The new run fails at runtime if the inputs were deleted too. |
Recovery is unaffected by the caching parameters above: it looks up the source run’s results rather than the cache. See Recover a failed run.
Identity and resources
| Parameter | Type | Default | Description |
|---|---|---|---|
service_account |
str |
from config | Kubernetes service account for task pods. |
env_vars |
Dict[str, str] |
None |
Additional environment variables to inject into task containers. |
labels |
Dict[str, str] |
None |
User-defined key=value labels attached to the run — used to filter/organize runs (flyte get run --with-label) and propagated to the task pods as Kubernetes labels. |
annotations |
Dict[str, str] |
None |
User-defined key=value annotations attached to the run and propagated to the task pods as Kubernetes annotations (not filterable; no CLI flag). |
Labels tag a run with arbitrary key=value metadata so you can find and group related runs later, and are also propagated to the run’s task pods as Kubernetes labels (available for cluster-level monitoring, routing, or policies). Set them programmatically with with_runcontext(labels={...}), or from the CLI with the repeatable --label flag:
flyte run --label team=ml --label env=prod my_example.py mainTo list and filter runs by their labels, see Filtering runs by label.
Logging
| Parameter | Type | Default | Description |
|---|---|---|---|
log_level |
int |
from config | Python log level for the framework logger (flyte), e.g. logging.DEBUG. |
user_log_level |
int |
from config | Python log level for the user logger (flyte.user, i.e. flyte.logger). |
log_format |
"console" | "json" |
"console" |
Log output format. |
reset_root_logger |
bool |
False |
If True, clear the root logger’s existing handlers and install Flyte’s own. If False (the default), leave existing root handlers in place and wrap their formatters with the run/action context. |
For setting the logging level (including via environment variables), the framework-vs-user logger split, and capturing third-party library logs as JSON with reset_root_logger, see
Logging.
Code bundling
| Parameter | Type | Default | Description |
|---|---|---|---|
copy_style |
"loaded_modules" | "all" | "none" |
"loaded_modules" |
Code bundling strategy. See Run command options. |
dry_run |
bool |
False |
Build and upload the code bundle without executing the run. |
copy_bundle_to |
Path |
None |
When dry_run=True, copy the bundle to this local path. |
interactive_mode |
bool |
auto-detected | Override interactive mode detection (set automatically for Jupyter notebooks). |
preserve_original_types |
bool |
False |
Keep native DataFrame types (e.g. pd.DataFrame) rather than converting to flyte.io.DataFrame when deserializing outputs. |
Context propagation
| Parameter | Type | Default | Description |
|---|---|---|---|
custom_context |
Dict[str, str] |
None |
Metadata propagated through the entire task hierarchy. Readable inside any task via flyte.ctx().custom_context. See
Custom context. |
Reading context inside a task with flyte.ctx()
Inside a running task, flyte.ctx() returns a TaskContext object with information about the current execution. Outside of a task, it returns None.
@env.task
async def inspect_context() -> str:
ctx = flyte.ctx()
action = ctx.action
return (
f"run={action.run_name}, "
f"action={action.name}, "
f"mode={ctx.mode}, "
f"in_cluster={ctx.is_in_cluster()}"
)
TaskContext fields
| Field | Type | Description |
|---|---|---|
action |
ActionID |
Identity of this specific action (task invocation) within the run. |
mode |
"local" | "remote" | "hybrid" |
Execution mode of the current run. |
version |
str |
Version of the deployed task code bundle. |
raw_data_path |
str |
Storage prefix where offloaded outputs are written. |
run_base_dir |
str |
Base directory for the run’s inputs, outputs, and intermediate per-action artifacts in the data plane object store. |
custom_context |
Dict[str, str] |
Propagated context metadata from with_runcontext(). |
disable_run_cache |
bool |
Whether run caching is disabled for this run. |
is_in_cluster() |
method | Returns True when mode == "remote". Useful for branching local/remote behavior. |
ActionID fields
The ctx.action object identifies this specific task invocation:
| Field | Type | Description |
|---|---|---|
name |
str |
Unique identifier for this action. |
run_name |
str |
Name of the parent run (defaults to name if not set). |
project |
str | None |
Project the action runs in. |
domain |
str | None |
Domain the action runs in. |
org |
str | None |
Organization. |
Naming external resources
ctx.action.run_name is useful for tying external tool runs (experiment trackers, dashboards) to the corresponding Flyte run:
import wandb # type: ignore[import]
@env.task
async def train_model(epochs: int) -> float:
ctx = flyte.ctx()
# Use run_name to tie the W&B run to this Flyte run
run = wandb.init(
project="my-project",
name=ctx.action.run_name,
config={"epochs": epochs},
)
# ... training logic ...
loss = 0.42
run.log({"loss": loss})
run.finish()
return loss
This ensures that when you look up a run in Weights & Biases (or any other tool), its name matches what you see in the Flyte UI.