Aggregate tables have no retention policy of their own and are never archived to cold storage

Description

Only two tables are ever archived to cold storage: issue_events_issueevent and logs_logevent (spans arrive separately via the promotion path). All six aggregate tables are Postgres-only and are dropped outright by maintain_partitions.

Every raw-data path is tiered. The aggregate layer has no tiers at all.

data	hot (Postgres)	cold (Parquet)	archived?
IssueEvent	30d GLITCHTIP_EVENT_HOT_DAYS	to 90d	yes
LogEvent	7d GLITCHTIP_LOG_HOT_DAYS	to 90d	yes
spans, raw	3d (SpanStaging)	30d GLITCHTIP_SPAN_RAW_RETENTION_DAYS	yes
spans, rollup	—	90d GLITCHTIP_TRANSACTION_RETENTION_DAYS	yes
IssueAggregate	90d	—	no
IssueTag	90d	—	no
IssueEventProjectHourlyStatistic	90d	—	no
TransactionEventProjectHourlyStatistic	90d	—	no
LogProjectHourlyStatistic	90d	—	no
UptimeCheckHourlyStatistic	90d	—	no

The result is inverted with respect to cost. Log events are evicted from Postgres after 7 days because they are expensive to keep. Log statistics — a few hundred bytes per project-hour — sit in Postgres for 90 days and are then deleted permanently. The cheapest, most durable, highest-value-per-byte data in the system has the least considered lifecycle.

To be precise about the current behaviour: aggregates are not expiring earlier than the events they summarise. They use the same *_RETENTION_DAYS value, which is the hot+cold total. The gap is that they cannot expire later, because they have no policy of their own — they inherit the raw data's number by default. That is why "error volume over the last year" is not currently possible: not because it is expensive, but because the two lifetimes were never separated.

The correct principle is already stated in the codebase, once, at apps/performance/maintenance.py:49:

Split retention: raw spans are dropped early (most are never read); the small trend rollups are kept for the long transaction retention.

Spans arrived at this structurally — being built cold-storage-first, the aggregate had to live in Parquet, so "how long do we keep this tier?" was unavoidable. The Postgres-native aggregates never faced the question.

The closed RFC #446 (closed) argued the same premise from the other direction ("users need access to historical statistics (aggregations), they rarely need full fidelity for individual event details"). Cold storage delivered the event half of that; the aggregate half was never built.

Proposed Solution(s)

Give the aggregate layer a lifetime independent of raw retention, defaulting to at least raw retention and permitted to exceed it. Then treat the tables according to what they actually are:

1. Counters — (subject, org, hour) -> count: IssueAggregate, the three *ProjectHourlyStatistic tables, UptimeCheckHourlyStatistic. Small and bounded in cardinality. These can simply stay in Postgres longer; a year of them is not much data. Needs one setting, no new machinery.

2. Group indexes — IssueIndex, TransactionGroup. Current state rather than time series, deleted with their parent object. Not a retention question, listed here only so the distinction is explicit.

3. Dimension breakdowns — IssueTag, LogProjectHourlyStatistic. Cardinality is driven by user-controlled strings, so these are genuinely large and want tiering to Parquet like the span rollups. archive_and_cleanup_partitions is already generic over table name, column types, select SQL and dictionary columns, so most of the machinery exists.

Worth folding in while this area is open: the same counter upsert is currently hand-written five times — update_statistics and update_org_statistics in apps/event_ingest/process_event.py, update_uptime_statistics in apps/uptime/tasks.py, the log statistics writer in apps/logs/process_logs.py, and update_tags. The two in process_event.py are near-identical and differ only in Python input shape and INSERT column order. Each copy independently chose its own deadlock sort key and its own missing-partition behaviour: update_uptime_statistics swallows IntegrityError and logs a warning, while the other four propagate. Consolidating them to one helper would make those two decisions once, deliberately.

This blocks doing the per-environment issue-list fix properly (companion issue), because the durable form of that feature depends on knowing which tier per-environment lifecycle data belongs in.

AI disclosure: investigated and drafted with Claude Code; codebase analysis AI-assisted, reviewed by a maintainer before filing.