Billy: how VictoriaMetrics deals with more than 500 billion rows

What is VictoriaMetrics?

The setup

mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0,discard -O 64bit,huge_file,extent -T huge /dev/nvme0n1mount -o discard,defaults /dev/nvme0n1 /mnt/disks/billy
internal.ip.of.billy-server billy-server
temperature{sensor_id="12345"} 76.23 123456789

Data ingestion

CPU usage during data ingestion
Network usage during data ingestion, bytes/sec
Disk iops during data ingestion
Disk IO usage percent during data ingestion
Disk bandwidth usage during data ingestion
RAM usage during data ingestion
Disk space usage during data ingestion

Data querying: 130 billion of data points

max(max_over_time(temperature[90d]))
Disk space usage during querying of 130 billion of data points
Memory usage during querying of 130 billion of data points
Disk bandwidth usage during querying of 130 billion of data points
Disk iops usage during querying of 130 billion of data points
Disk iops percent usage during querying of 130 billion of data points
CPU usage during querying of 130 billion of data points
System and iowait CPU usage during querying of 130 billion of data points

Data querying: 525.6 billion of data points

Disk space usage during querying of 525.6 billion of data points
Memory usage during querying of 525.6 billion of data points
Disk bandwidth usage during querying of 525.6 billion of data points
Disk bandwidth usage during querying of 525.6 billion of data points (excluding data deletion peaks)
Disk iops when querying 525.6 billion of data points
Disk iops percentage when querying 525.6 billion of data points
CPU usage during querying of 525.6 billion of data points

Query performance analysis

Conclusions

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store