Compact state aggregation overview

Track the amount of time spent in each discrete state with compact_state_agg functions

Early access 1.5.0

Track the amount of time a system or value spends in each discrete state. For example, use the compact_state_agg functions to track how much time a system spends in error, running, or starting states.

compact_state_agg is designed to work with a relatively small number of states. It might not perform well on datasets where states are mostly distinct between rows.

If you need to track when each state is entered and exited, use the state_agg functions. If you need to track the liveness of a system based on a heartbeat signal, consider using the heartbeat_agg functions.

Two-step aggregation

This group of functions uses the two-step aggregation pattern.

Rather than calculating the final result in one step, you first create an intermediate aggregate by using the aggregate function.

Then, use any of the accessors on the intermediate aggregate to calculate a final result. You can also roll up multiple intermediate aggregates with the rollup functions.

The two-step aggregation pattern has several advantages:

1. More efficient because multiple accessors can reuse the same aggregate
2. Easier to reason about performance, because aggregation is separate from final computation
3. Easier to understand when calculations can be rolled up into larger intervals, especially in window functions and continuous aggregates
4. Can perform retrospective analysis even when underlying data is dropped, because the intermediate aggregate stores extra information not available in the final result

To learn more, see the blog post on two-step aggregates.

Functions in this group

Aggregate

• compact_state_agg(): aggregate state data into an intermediate form for further computation

Accessors

• duration_in(): get the total duration in the specified states
• interpolated_duration_in(): get the total duration in the specified states, interpolating values at the boundary
• into_values(): return an array of (state, duration) pairs from the aggregate

Rollup

• rollup(): combine multiple intermediate aggregates