Operations | Monitoring | ITSM | DevOps | Cloud

Time series data is foundational in almost all applications and services. Even if time series isn’t the focus, like in an IoT sensor data centered application, it appears in monitoring data as metrics, logs, and traces. Because of time series data’s unique characteristics, it’s best served in a time series database. InfluxDB is purpose-built to handle the high volume and velocity of time series ingestion, and perform real-time analytics, alerting, and anomaly detection at scale.

This article was originally published on The New Stack and is reposted here with permission. Here is a brief case study that explores the logistics and motivations that would lead a successful company to spend time and resources completely rewriting the core of their flagship product in Rust. Calling a programming language Rust almost seems like a misnomer. Rust is the brittle byproduct of corrosion — not something that would typically inspire confidence.

A few weeks ago, we published some benchmarking that showed performance gains in InfluxDB 3.0 that are orders of magnitude better than previous versions of InfluxDB – and by extension, other databases as well. There are two key factors that influence these gains: 1. Data ingest, and 2. Data compression. This begs the question, just how did we achieve such drastic improvements in our core database? This post sets out to explain how we accomplished these improvements for anyone interested.

InfluxDB and Kafka aren’t competitors – they’re complimentary. Streaming data, and more specifically time series data, travels in high volumes and velocities. Adding InfluxDB to your Kafka cluster provides specialized handling for your time series data. This specialized handling includes real-time queries and analytics, and integration with cutting edge machine learning and artificial intelligence technologies. Companies like as Hulu paired their InfluxDB instances with Kafka.

Any existing InfluxDB user will notice that InfluxDB underwent a transformation with the release of InfluxDB 3.0. InfluxDB v3 provides 45x better write throughput and has 5-25x faster queries compared to previous versions of InfluxDB (see this post for more performance benchmarks). We also deprioritized several features that existed in 2.x to focus on interoperability with existing tools. One of the deprioritized features that existed in InfluxDB v2 is the task engine.

The commercial version of InfluxDB 3.0 is a distributed, scalable time series database built for real-time analytic workloads. It supports infinite cardinality, SQL and InfluxQL as native query languages, and manages data efficiently in object storage as Apache Parquet files. It delivers significant gains in ingest efficiency, scalability, data compression, storage costs, and query performance on higher cardinality data.