Evaluating Performance and Correctness: VictoriaMetrics response

Bad compression ratio

sum(vm_data_size_bytes{type=~”storage/.*”}) / sum(vm_rows{type=~”storage/.*”})
  • The time series from the benchmark are far from real world.
  • Real-world measurements usually contain small number of decimal digits on limited range. For instance, the usual temperature range is from -460 F to 1000 F with 0.1 F precision, the usual speed range is from 0 m/s to 10K m/s with 1 m/s precision, the usual qps range is from 0 to 1M with 0.1 qps precision, the usual price range is from $0 to $1M with $0.01 precision.
  • The number of decimal digits becomes even smaller after applying delta-coding, i.e. calculating the difference between adjacent samples. The difference is small for Gauges, since they tend to change slowly. The difference is small for Counters, since their rate is usually limited by relatively small range. The number of decimal digits for Counters can be reduced further by applying double delta-coding to them.

Precision loss

  • The precision loss can occur only on values with more than 12 significant decimal digits. Such values are rare in real world. Even summary counters for nanoseconds shouldn’t lose precision. Of course, if you work in NASA, then you would need up to 15 decimal digits :)
  • Real-world measurements usually contain small number of precise leading decimal digits. The rest of digits are just noise, which has little value because of measurement errors. For instance, the mentioned above metric — go_memstats_gc_cpu_fraction — contains only 4 or 5 precise digits after the point — 0.00308 in the best case — all the other digits are just garbage, which worsens series compression ratio.

Stale timestamps in /api/v1/query results

-search.latencyOffset duration
The time when data points become visible in query results after the collection. Too small value can result in incomplete last points for query results (default 1m0s)

VictoriaMetrics treats vector with a single nameless series as scalar and vice versa

Staleness handling

  • After a special NaN value is found. This value is inserted by Prometheus when the metric disappears from the scrape target or the scrape target is unavailable.
  • After 5 minutes of silence from the previous value.

Slow time series lookups

  • Prometheus:
GOMAXPROCS=1 go test ./tsdb/ -run=111 -bench=BenchmarkHeadPostingForMatchers
  • VictoriaMetrics:
GOMAXPROCS=1 go test ./lib/storage/ -run=111 -bench=BenchmarkHeadPostingForMatchers

Conclusion

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store