Skip to main content

One post tagged with "High availability"

Keeping monitoring and alerting up when parts of it fail.

View all tags

Prometheus high availability: storage vs alerting

· 7 min read
Eric Qin

Two Prometheus replicas writing to deduplicated storage and sending alerts to a clustered Alertmanager

Ask how to make Prometheus highly available and the usual answer is "run two of them". That is correct, and it is only half the job. Two replicas give you two copies of every alert and two slightly different copies of every time series. Each needs its own fix, and the fixes don't share a configuration.

This post separates the two problems: keeping alerts flowing exactly once, and keeping the data queryable without gaps. It ends with a checklist.