When size matters — benchmarking VictoriaMetrics vs Timescale and InfluxDB

  • generate the configured number of production-like timeseries;
  • measure insert performance for the generated timeseries;
  • measure select performance for various production-like queries.
  • Timescale
  • InfluxDB
  • MongoDB
  • Cassandra

Adding VictoriaMetrics to TSBS

We liked TSBS, so we quickly hacked support for Prometheus remote write API into TSBS and started using it. Initial results weren’t exciting — VictoriaMetrics was slow on some queries and required a lot of memory during the benchmark execution.

Benchmark preparation

Which competitors to put against VictoriaMetrics?

  • lastpoint - PromQL cannot return last point for each time series;
  • groupby-orderby-limit - PromQL doesn't support order by and limit.

Benchmark results

Insert performance for a billion of datapoints belonging to 40K distinct timeseries:

  • VictoriaMetrics — 1.7M datapoints per second, RAM usage — 0.8GB, data size on HDD — 387MB.
  • InfluxDB — 1.1M datapoints per second, RAM usage — 1.7GB, data size on HDD — 573MB.
  • Timescale — 890K datapoints per second, RAM usage — 0.4GB, data size on HDD — 29GB.
  • VictoriaMetrics wins InfluxDB and Timescale in all the queries by a margin of up to 20x. It especially excels at heavy queries, which scan many millions of datapoints across thousands of distinct timeseries.
  • InfluxDB is on the second place. It wins Timescale on light queries and looses Timescale by up to 3.5x on heavy queries.
  • Timescale is on the third place. Moreover, it was multiple orders of magnitude slower on all the queries when the required data wasn’t in page cache, while VictoriaMetrics and InfluxDB were only marginally slower in these cases.

Analysis

Why Timescale performed so poorly on select queries? The answer is in huge data size (29GB) and on-disk data layout not suited for storage with low iops. Google Cloud HDDs are limited in iops per GB and throughput per GB. 200GB disk is limited by 150 read operations per second and 24MB/s read/write throughput. Simple calculations show that 20 minutes is needed for loading 29GB into page cache at 24MB/s. VictoriaMetrics would load the same amount of data (a billion of datapoints) on the same HDD in 16 seconds.

Conclusion

Sometimes size matters :) It may be more expensive than you expect.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aliaksandr Valialkin

Aliaksandr Valialkin

Founder and core developer at VictoriaMetrics