State aggregation overview

Track transitions between discrete states with state_agg functions

Since 1.15.0

Track transitions between discrete states for a system or value that switches between them. For example, use state_agg to create a timeline of state transitions, or to calculate the durations of states. state_agg extends the capabilities of compact_state_agg.

state_agg is designed to work with a relatively small number of states. It might not perform well on datasets where states are mostly distinct between rows.

Because state_agg tracks more information, it uses more memory than compact_state_agg. If you want to minimize memory use and don’t need to query the timestamps of state transitions, consider using compact_state_agg instead.

Two-step aggregation

This group of functions uses the two-step aggregation pattern.

Rather than calculating the final result in one step, you first create an intermediate aggregate by using the aggregate function.

Then, use any of the accessors on the intermediate aggregate to calculate a final result. You can also roll up multiple intermediate aggregates with the rollup functions.

The two-step aggregation pattern has several advantages:

1. More efficient because multiple accessors can reuse the same aggregate
2. Easier to reason about performance, because aggregation is separate from final computation
3. Easier to understand when calculations can be rolled up into larger intervals, especially in window functions and continuous aggregates
4. Can perform retrospective analysis even when underlying data is dropped, because the intermediate aggregate stores extra information not available in the final result

To learn more, see the blog post on two-step aggregates.

Functions in this group

Aggregate

• state_agg(): aggregate state data into an intermediate form for further computation

Accessors

• state_at(): get the state at a given time
• duration_in(): get the total duration in the specified states
• interpolated_duration_in(): get the total duration in the specified states, interpolating values at the boundary
• state_periods(): get an array of periods for each state
• state_timeline(): get a timeline of state changes
• interpolated_state_periods(): get an array of periods for each state, interpolating values at the boundary
• interpolated_state_timeline(): get a timeline of state changes, interpolating values at the boundary
• into_values(): return an array of (state, duration) pairs from the aggregate

Rollup

• rollup(): combine multiple intermediate aggregates