Skip to main content

20.9 Production Operations & SRE

Production Datasphere should be operated as a data service with measurable reliability.

SLI examples​

  • pipeline success rate
  • freshness
  • data availability
  • query latency
  • reconciliation pass rate
  • incident recovery time

SLO examples​

Freshness SLO: critical sales data available within agreed window

Pipeline SLO: successful completion above agreed percentage

Incident lifecycle​

Detect -> Triage -> Contain -> Recover -> Validate -> Communicate -> RCA -> Prevent

Runbook structure​

Each critical workload should document:

  1. purpose
  2. owner
  3. dependencies
  4. expected schedule
  5. alerts
  6. common failures
  7. recovery steps
  8. reconciliation query or method
  9. escalation path

MTTD and MTTR​

Track:

  • Mean Time To Detect
  • Mean Time To Recover

A faster pipeline is not automatically better if it is difficult to diagnose or recover.

SRE maturity​

Level 1: reactive support

Level 2: documented operations

Level 3: measurable SLOs

Level 4: automated detection and recovery

Level 5: predictive capacity and systematic reliability engineering