A3 — Visibility, Search Attributes, and Operating from History (long form of Module 10)

The capstone requirement that gets forgotten until the first incident is "operators need exact execution diagnostics." Event History gives you exact for one run. Visibility gives you find across thousands. This page is about the second, and about which tool answers which question when a run has gone wrong.

What Visibility is

The term Visibility, within the Temporal Platform, refers to the subsystems and APIs that enable an operator to view, filter, and search for Workflow Executions that currently exist within a Temporal Service. Visibility enables the listing, filtering, and sorting of Workflow Executions through a custom SQL-like List Filter, and supports custom Search Attributes for user-defined filtering beyond the default system attributes.

Two properties shape how you use it. First, Visibility is a search index that Temporal updates asynchronously: after a change is recorded it takes some time to propagate, so a List or Count query can briefly return stale results. Treat it as eventually consistent. Second, and following from that, Visibility is built for finding and filtering across many executions, not for reading the current state of one execution. When you need the authoritative, up-to-date state of a specific Workflow Execution, use DescribeWorkflowExecution instead of a Visibility query. Operations that look up a single entity by ID, such as DescribeWorkflowExecution, do not use Visibility.

Upstream is also clear about what not to do with it: to check whether a specific Workflow exists, start it and handle the already-started result, or look it up by ID; to react to a Workflow's progress, follow its Event History or use Child Workflows; to coordinate between Workflows, use Signals, Updates, or Child Workflows rather than reading Visibility from inside Workflow or Activity code. Visibility is for the operator's dashboard, not for AgentRun's control flow.

Search Attributes: default and custom

A Search Attribute is an indexed field used in a List Filter to filter a list of Workflow Executions that have the Search Attribute in their metadata. Each Search Attribute is a key-value pair metadata object included in a Workflow Execution's Visibility information.

Default Search Attributes are created when the initial index is created, are set globally in any Namespace, and are reserved and read-only. The ones you will query for AgentRun (subset; full table upstream, as of 2026-09):

BinaryChecksums and BuildIds are deprecated in favour of the TemporalWorkerDeployment* attributes (removal scheduled for server 1.32–1.34, as of 2026-09; see upstream page).

Custom Search Attributes are keys you create with a name and one of seven types: Bool, Datetime, Double, Int, Keyword, KeywordList, Text. Most custom Search Attributes store structured identifiers, so Keyword is the right default choice: it stores the value as-is and supports =, !=, IN, STARTS_WITH, BETWEEN, and ORDER BY. Use KeywordList for tags, and Text only for prose you want word-level search on — Text tokenises on word boundaries and the = operator does OR matching across tokens, which surprises people who put IDs in it.

Three rules, all upstream:

• Do not put sensitive data or PII in Search Attribute names or values. Values are stored unencrypted in the Visibility store and are not processed by a custom Payload Codec. Owner below is an opaque account ID, never an email.
• They live only as long as the Workflow. Search Attribute values are only available for as long as the Workflow is (retention period). When using Continue-As-New, Search Attribute keys are carried over to the new Workflow Run by default.
• They are not a state store. For business logic that needs a Workflow's state, store it in a local variable and expose it with a Query, or store it externally through Activities. Search Attributes lag; the Query does not.

Limits worth knowing (as of 2026-09; see upstream page): per Namespace on the SQL Visibility stores, 10 Keyword, 3 of each other type, 40 Keyword and 20 of most others on Temporal Cloud; a single value up to 2 KB and 255 characters; 40 KB total per execution. The dev server in the labs uses SQLite, so the SQL limits apply.

How AgentRun exposes AgentStatus, CurrentStep, Owner

Create the keys once per Namespace:

temporal operator search-attribute create --name AgentStatus --type Keyword
temporal operator search-attribute create --name CurrentStep --type Int
temporal operator search-attribute create --name Owner --type Keyword

Declare the same keys in code (names verified against temporalio 1.32.0, as of 2026-09; see upstream page):

# shared.py
from temporalio.common import SearchAttributeKey

AGENT_STATUS = SearchAttributeKey.for_keyword("AgentStatus")
CURRENT_STEP = SearchAttributeKey.for_int("CurrentStep")
OWNER = SearchAttributeKey.for_keyword("Owner")

Set the ones known at start from the Client:

from temporalio.common import SearchAttributePair, TypedSearchAttributes

handle = await client.start_workflow(
    AgentRun.run,
    task,
    id=f"agent-{task.id}",
    task_queue=AGENT_WORKFLOWS,
    search_attributes=TypedSearchAttributes([
        SearchAttributePair(OWNER, task.owner_id),
        SearchAttributePair(AGENT_STATUS, "planning"),
        SearchAttributePair(CURRENT_STEP, 0),
    ]),
)

Upsert the ones that change, from inside the Workflow, at the points where the loop's state changes:

@workflow.defn
class AgentRun:
    def _publish(self, status: str) -> None:
        self.status = status                          # the Query reads this — authoritative
        workflow.upsert_search_attributes([           # the fleet view reads this — eventual
            AGENT_STATUS.value_set(status),
            CURRENT_STEP.value_set(self.step),
        ])

    @workflow.run
    async def run(self, task: Task) -> Result:
        while not self.done:
            self._publish("planning")
            step = self.plan()
            self._publish("calling_llm" if step.kind == "llm" else "calling_tool")
            out = await workflow.execute_activity(...)
            await workflow.wait_condition(lambda: not self.paused)   # Signals pause/resume
            self.step += 1
            if self.step % SNAPSHOT_EVERY == 0:
                workflow.continue_as_new(self.snapshot())            # keys carry over
        self._publish("done")

Set self.paused = True in the pause Signal handler and call self._publish("paused") there too, so a paused run is findable. upsert_search_attributes is a Command like any other: it is recorded in History (UpsertWorkflowSearchAttributes), replays deterministically, and issues no new Command on replay. The Query status remains the strongly consistent answer for one run; the attributes exist so an operator can ask about all runs.

List Filter

The Visibility List API requires you to provide a List Filter as an SQL-like string parameter, made of Search Attribute names, values, and operators. Names are case sensitive, and a single Namespace scopes each List Filter. Supported operators: =, !=, >, >=, <, <=, AND, OR, (), BETWEEN ... AND, IN, STARTS_WITH (Keyword only), IS NULL / IS NOT NULL. ORDER BY is not supported on Temporal Cloud and never on Text attributes (as of 2026-09; see upstream page). The same string works in the Web UI's filter box, temporal workflow list --query, and client.list_workflows().

Queries an operator of the agent runtime actually types:

-- everything paused, across all owners
WorkflowType = 'AgentRun' AND AgentStatus = 'paused'

-- one customer's live runs
Owner = 'acct_7f3a' AND ExecutionStatus = 'Running'

-- runs that have been grinding: past step 500 and still going
WorkflowType = 'AgentRun' AND CurrentStep > 500 AND ExecutionStatus = 'Running'

-- runs that ended badly in the last deploy window
WorkflowType = 'AgentRun' AND ExecutionStatus IN ('Failed', 'TimedOut', 'Terminated')
  AND CloseTime > '2026-09-10T08:00:00Z'

-- which Worker version are the pinned runs on?
WorkflowType = 'AgentRun' AND TemporalWorkflowVersioningBehavior = 'Pinned'
  AND TemporalWorkerDeploymentVersion = 'agent-workflows:v42'

-- the whole family of one job, parent and ResearchAgent children
WorkflowId STARTS_WITH 'agent-task-123'

-- closed runs whose history got long (HistoryLength exists only once closed)
WorkflowType = 'AgentRun' AND ExecutionStatus != 'Running' AND HistoryLength > 1000

temporal workflow list --query "WorkflowType = 'AgentRun' AND AgentStatus = 'paused'"
temporal workflow count --query "WorkflowType = 'AgentRun' GROUP BY ExecutionStatus"

The Count API returns approximate counts, and GROUP BY is supported only there and only on ExecutionStatus (as of 2026-09; see upstream page). From Python, client.list_workflows(query) is an async iterator and client.count_workflows(query) the count. Two idioms to remember: WorkflowId STARTS_WITH 'agent-' for a prefix, and WorkflowId BETWEEN 'agent-' AND 'agent-~' when you need a bounded range. If you ever name a custom attribute the same as a default one's un-prefixed alias (SchedulePaused vs TemporalSchedulePaused), the custom one wins and the Temporal-prefixed name still reaches the default (aliasing needs server 1.30+, as of 2026-09).

The operator's five questions

This section is ours. Module 10's third key example hands you a broken AgentRun history and asks you to diagnose it from the UI alone. The five questions below are the ones to answer, in order, and each has one surface that answers it.

Notice which questions Visibility answers: only the fifth, and only the "how many, which ones" half of it. The first four are answered by History and Describe, per execution, strongly consistent. That split is the boundary the capstone keeps asking you to draw: Temporal's History tells you what execution happened; whether the result was good, and what the LLM provider actually charged, live in systems of record outside it.

Where this is used

Lab 11.2 (labs/11-capstone-durable-agent-runtime/) runs AgentRun for 200 steps with Continue-As-New and has you verify two things with the tools above: that no closed run shows HistoryLength > 700 (temporal workflow list --query "WorkflowType = 'AgentRun' AND ExecutionStatus != 'Running' AND HistoryLength > 700" returns nothing — the lab's measured runs are 516–519 events), and that AgentStatus, CurrentStep, and Owner survive each Continue-As-New and stay queryable while the run is live. Lab 10.2 is the five-question drill on a run you did not start (lab 8.5 is the forensic version, on a broken history with no source); write the five answers in the order of the table, naming the command or UI surface you used for each, and put that page in your Lab 11.4 defense.