Promscale vs VictoriaMetrics: measuring resource usage in production

Timescale recently published Promscale — an open source long-term remote storage for Prometheus built on top of TimescaleDB. According to the announcement, it should be fast and resource efficient. Let’s compare performance and resource usage on production workload for Promscale and VictoriaMetrics.

Benchmark setup

The following scheme has been constructed for the benchmark:

                                    /--> VictoriaMetrics
2000 x node_exporter <-- vmagent --|
\--> Promscale
global:
scrape_interval: 30s
scrape_configs:
- job_name: node_exporter
static_configs:
{% for n in range(2000) %}
- targets: ['host-node-{{n}}:9100']
labels:
host_number: cfg_{{n}}
role: node-exporter
env: prod
{% endfor %}
synchronous_commit = off
wal_compression = on
docker run --name promscale -d --net=host timescale/promscale:0.1.3 -db-password=secret -log-level=info -db-ssl-mode=disable -async-acks=true
docker run --rm -itd --net=host --name victoria -v `pwd`/data:/data  victoriametrics/victoria-metrics:v1.49.0 -storageDataPath=/data

Benchmark results

Memory usage

Image for post
Image for post
RSS memory usage: VictoriaMetrics vs Promscale
Image for post
Image for post
CPU cores used: VictoriaMetrics vs Promscale
Image for post
Image for post
Disk IO usage: VictoriaMetrics vs Promscale
Image for post
Image for post
Disk IOPS usage: VictoriaMetrics vs Promscale

Disk IO distribution for Promscale between WAL disk and data disk

Image for post
Image for post
Promscale disk IO usage: WAL vs data
Image for post
Image for post
Promscale disk IOPS usage: WAL vs data
Image for post
Image for post
Disk space usage (excluding WAL): VictoriaMetrics vs Promscale
Image for post
Image for post
Promscale WAL size
  • VictoriaMetrics — 0.5 bytes per sample (1.6GB / 3.2 billion samples)

Conclusions

While Promscale and TimescaleDB show good performance on synthetic benchmarks such as TSBS according the these search results, they show not so good performance on production workloads as shown above:

  • Promscale needs 64x more CPU than VictoriaMetrics.
  • Promscale needs 450x more disk IO bandwidth than VictoriaMetrics.
  • Promscale needs a disk with 15000 IOPS, while VictoriaMetrics runs smoothly on a disk with 35 IOPS.
  • Promscale needs up to 92x more disk space than VictoriaMetrics (not counting WAL size).

Founder and core developer at VictoriaMetrics

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store