MIT Licensed · v0.9.0

Embedded Time-Series
Database for Rust

LSM-based storage with disk-backed roaring postings, optional PromQL, atomic snapshot/restore, and structured observability counters. Embed a full time-series engine directly in your application — no external server required.

6.4M Points/sec insert
0.4B Per data point
64M Points/sec read
40x Compression ratio
main.rs
use std::time::Duration;
use tsink::{DataPoint, Label, Row, StorageBuilder, WalReplayMode, WalSyncMode};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let storage = StorageBuilder::new()
        .with_data_path("./tsink-data")
        .with_wal_sync_mode(WalSyncMode::Periodic(Duration::from_secs(1)))
        .with_wal_replay_mode(WalReplayMode::Salvage)
        .with_memory_limit(512 * 1024 * 1024)
        .build()?;

    storage.insert_rows(&[
        Row::with_labels(
            "cpu_usage",
            vec![Label::new("host", "server-01")],
            DataPoint::new(1_700_000_000, 45.5),
        ),
    ])?;

    let obs = storage.observability_snapshot();
    println!("wal_bytes={}", obs.wal.size_bytes);

    storage.snapshot(std::path::Path::new("./snapshots/tsink-001"))?;

    storage.close()?;
    Ok(())
}
New in 0.9.0

Durability and operability upgrade.

0.9.0 introduces storage format v2, atomic snapshot/restore, deep observability, and tighter safety defaults. Note: persistent data from 0.8.x is not wire-compatible with 0.9.0.

Storage Format v2 + Roaring Index

Persistent series dictionaries moved to series_index.bin with roaring-bitmap postings for faster matcher candidate resolution on reopen and query.

PromQL Compatibility Tightening

Metricless selector semantics, one-to-one vector matching for binary ops, and metric-name handling in vector binary results were aligned in 0.9.0.

Atomic Snapshot/Restore

Create segment-consistent, WAL-aware snapshots from live storage and restore atomically with staged publish semantics.

Observability Snapshot API

observability_snapshot() now exposes structured WAL, flush, compaction, query, and health internals for diagnostics and monitoring.

WAL Replay Policy + Process Lock

Choose Salvage or Strict replay mode for corruption handling, and prevent multi-process write collisions with a per-path .tsink.lock.

Server Hardening + Admin API

tsink-server deepens /metrics and /api/v1/status/tsdb, and adds gated admin snapshot/restore endpoints with optional path-prefix restrictions.

Built for performance.
Designed for simplicity.

Everything you need for embedded time-series workloads, with production-grade durability and introspection.

Gorilla Compression

Adaptive codec selection across Gorilla XOR, delta/bitpack, and RLE; chunk payloads are also zstd-compressed when beneficial. Typical numeric workloads reach ~0.4 bytes per point (~40x).

LSM-Tree Compaction

Tiered L0 → L1 → L2 segment compaction runs in the background. Reduces read amplification while maintaining write throughput with dual numeric/blob lanes.

PromQL Engine

Instant and range queries with 20+ built-in functions, full by/without aggregation support, and 0.9.0 semantic alignment for metricless selectors and vector matching.

Sharded Concurrency

64 internal shards eliminate write contention. Lock-free reads with concurrent writes via configurable worker pools. Full Send + Sync safety.

Typed Values

Six value types: f64, i64, u64, bool, bytes, and string. Per-type codecs, automatic From conversions, and custom bytes aggregation via Codec/Aggregator traits.

Write-Ahead Logging

Segmented WAL with CRC32 checksums and idempotent replay. Choose Periodic or PerAppend sync, and pick replay policy: Salvage (default) or Strict.

Observability Snapshot

Inspect structured WAL, flush, compaction, query, and health internals using observability_snapshot() for dashboards, alerting, and troubleshooting.

Async Storage

Runtime-agnostic AsyncStorage with bounded queues and dedicated workers. Forwards core builder options including WAL replay mode and background fail-fast.

Atomic Snapshot/Restore

Create live, segment-consistent backups with snapshot() and restore atomically with StorageBuilder::restore_from_snapshot.

Server Mode

Prometheus-compatible HTTP server with remote read/write, PromQL API, TLS, bearer auth, admin snapshot/restore endpoints (gated), and deep self-observability.

Series Discovery

Matcher-based series filtering with =, !=, =~, !~. 0.9.0 persists dictionaries and roaring postings to accelerate matcher candidate resolution.

Rich Aggregation

12 built-in aggregations: Sum, Min, Max, Avg, First, Last, Count, Median, Range, Variance, StdDev, and None. Supports downsampling, pagination, and custom bytes aggregation.

Resource Limits

Memory budget with admission backpressure, series cardinality cap, and WAL size limit. 0.9.0 adds incremental shard memory accounting and optional background fail-fast mode.

Container-Aware

Automatic cgroup v1/v2 CPU quota detection sets sensible worker defaults in containers. Override explicitly with TSINK_MAX_CPUS when needed.

Python Bindings

Native Python bindings via UniFFI. Full access to the storage engine from Python with pip install tsink_uniffi — no performance compromise.

Benchmarks that speak
for themselves.

Ballpark figures from cargo bench in quick mode. Run on your hardware for precise numbers.

Single Insert Latency
~1.7µs
~577K points/sec
Batch Insert (1K) Throughput
6.4M pts/s
~155µs per 1,000-point batch
Select 1 Point Query
~114ns
8.8M queries/sec
Select 1K Points Query
~15.4µs
64M points/sec throughput
Select 1M Points Query
~20.9ms
48M points/sec throughput
Compression Ratio Storage
0.4bytes/pt
~40x compression — from 16 bytes to ~0.4 bytes

Get started in minutes.

Add tsink to your Cargo.toml and start storing time-series data.

Add to Cargo.toml
Cargo.toml
# Core library
tsink = "0.9"

# With PromQL support
tsink = { version = "0.9", features = ["promql"] }

# With async storage
tsink = { version = "0.9", features = ["async-storage"] }
basic_usage.rs
use tsink::{DataPoint, Label, Row, StorageBuilder};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let storage = StorageBuilder::new().build()?;

    storage.insert_rows(&[
        Row::new("cpu_usage", DataPoint::new(1_700_000_000, 42.5)),
        Row::new("cpu_usage", DataPoint::new(1_700_000_010, 43.1)),
        Row::with_labels(
            "http_requests",
            vec![Label::new("method", "GET"), Label::new("status", "200")],
            DataPoint::new(1_700_000_000, 120u64),
        ),
    ])?;

    // Time range is [start, end) (end-exclusive)
    let cpu = storage.select("cpu_usage", &[], 1_700_000_000, 1_700_000_100)?;
    assert_eq!(cpu.len(), 2);

    // Label order does not matter for series identity
    let get_200 = storage.select(
        "http_requests",
        &[Label::new("status", "200"), Label::new("method", "GET")],
        1_700_000_000, 1_700_000_100,
    )?;
    assert_eq!(get_200.len(), 1);

    storage.close()?;
    Ok(())
}
persistent_storage.rs
use std::time::Duration;
use tsink::{StorageBuilder, TimestampPrecision, WalReplayMode, WalSyncMode};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let storage = StorageBuilder::new()
        .with_data_path("./tsink-data")
        .with_timestamp_precision(TimestampPrecision::Milliseconds)
        .with_retention(Duration::from_secs(30 * 24 * 3600))          // 30 days
        .with_partition_duration(Duration::from_secs(6 * 3600))     // 6-hour partitions
        .with_chunk_points(4096)
        .with_max_writers(16)
        .with_write_timeout(Duration::from_secs(60))
        .with_memory_limit(1024 * 1024 * 1024)                   // 1 GB
        .with_cardinality_limit(500_000)
        .with_wal_sync_mode(WalSyncMode::Periodic(Duration::from_secs(1)))
        .with_wal_replay_mode(WalReplayMode::Salvage)
        .with_background_fail_fast(true)
        .with_wal_buffer_size(16384)
        .build()?;

    // Data survives restarts. WAL ensures durability on crash.
    // Replay mode controls corruption handling at startup.
    // Background fail-fast can stop new ops after worker failures.

    storage.close()?;
    Ok(())
}
labels.rs
use tsink::{DataPoint, Label, Row, StorageBuilder};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let storage = StorageBuilder::new().build()?;

    // Insert with multi-dimensional labels
    storage.insert_rows(&[
        Row::with_labels(
            "http_requests",
            vec![
                Label::new("method", "GET"),
                Label::new("status", "200"),
                Label::new("endpoint", "/api/users"),
            ],
            DataPoint::new(1_700_000_000, 150.0),
        ),
        Row::with_labels(
            "http_requests",
            vec![
                Label::new("method", "POST"),
                Label::new("status", "201"),
            ],
            DataPoint::new(1_700_000_000, 25.0),
        ),
    ])?;

    // Query all label combinations for a metric
    let all = storage.select_all("http_requests", 1_700_000_000, 1_700_000_100)?;
    for (labels, points) in &all {
        println!("Labels: {:?}, Points: {}", labels, points.len());
    }

    storage.close()?;
    Ok(())
}
aggregation.rs
use tsink::{Aggregation, DataPoint, QueryOptions, Row, StorageBuilder};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let storage = StorageBuilder::new().build()?;

    storage.insert_rows(&[
        Row::new("cpu", DataPoint::new(1_000, 1.0)),
        Row::new("cpu", DataPoint::new(2_000, 2.0)),
        Row::new("cpu", DataPoint::new(3_000, 3.0)),
        Row::new("cpu", DataPoint::new(4_500, 1.5)),
    ])?;

    // Downsample: 2-second buckets with average aggregation
    let opts = QueryOptions::new(1_000, 5_000)
        .with_downsample(2_000, Aggregation::Avg)
        .with_pagination(0, Some(10));

    let buckets = storage.select_with_options("cpu", opts)?;
    assert_eq!(buckets.len(), 2);

    // 12 aggregation functions available:
    // Sum, Min, Max, Avg, First, Last, Count,
    // Median, Range, Variance, StdDev, None

    storage.close()?;
    Ok(())
}
promql_example.rs
// Enable with: tsink = { version = "0.9", features = ["promql"] }
use std::sync::Arc;
use tsink::{StorageBuilder, DataPoint, Row};
use tsink::promql::Engine;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let storage = StorageBuilder::new().build()?;

    storage.insert_rows(&[
        Row::new("http_requests_total", DataPoint::new(1_000, 10.0)),
        Row::new("http_requests_total", DataPoint::new(2_000, 25.0)),
        Row::new("http_requests_total", DataPoint::new(3_000, 50.0)),
    ])?;

    let engine = Engine::new(storage.clone());

    // Instant query — evaluates at a single point in time
    let result = engine.instant_query("http_requests_total", 3_000)?;

    // Range query — evaluates at each step across a time window
    let result = engine.range_query(
        "rate(http_requests_total[1m])",
        1_000, 3_000, 1_000,
    )?;

    // Supports: rate, irate, increase, avg_over_time, sum_over_time,
    // abs, ceil, floor, round, clamp, label_replace, label_join,
    // sort, sort_desc, scalar, vector, time, timestamp, and more

    storage.close()?;
    Ok(())
}
async_usage.rs
// Enable with: tsink = { version = "0.9", features = ["async-storage"] }
use tsink::{AsyncStorageBuilder, DataPoint, Row};

async fn run() -> tsink::Result<()> {
    let storage = AsyncStorageBuilder::new()
        .with_queue_capacity(1024)
        .with_read_workers(4)
        .build()?;

    // Async insert
    storage.insert_rows(vec![
        Row::new("cpu", DataPoint::new(1, 42.0)),
    ]).await?;

    // Async query
    let points = storage.select("cpu", vec![], 0, 10).await?;
    assert_eq!(points.len(), 1);

    // Synchronous introspection available too:
    // storage.memory_used(), storage.memory_budget(),
    // storage.inner(), storage.into_inner()

    storage.close().await?;
    Ok(())
}
series_discovery.rs
use tsink::{SeriesSelection, SeriesMatcher, StorageBuilder};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let storage = StorageBuilder::new().build()?;
    // ... insert data ...

    // Matcher-based series discovery
    let selection = SeriesSelection::new()
        .with_metric("http_requests")
        .with_matcher(SeriesMatcher::equal("method", "GET"))
        .with_matcher(SeriesMatcher::regex_match("status", "2.."))
        .with_time_range(1_700_000_000, 1_700_100_000);

    let series = storage.select_series(&selection)?;

    // Matcher operators: = (equal), != (not_equal),
    //   =~ (regex_match), !~ (regex_no_match)

    // Also: list_metrics() and list_metrics_with_wal()
    let all_series = storage.list_metrics()?;
    for s in &all_series {
        println!("Metric: {}, Labels: {:?}", s.name, s.labels);
    }

    storage.close()?;
    Ok(())
}

Full PromQL engine.
No external dependencies.

Enable with the promql feature flag. Built-in lexer/parser/evaluator with 20+ functions and 0.9.0 semantic alignment updates.

Rate & Counter Functions

rate irate increase

Over-Time Functions

avg_over_time sum_over_time min_over_time max_over_time count_over_time

Math Functions

abs ceil floor round clamp clamp_min clamp_max

Type & Time

scalar vector time timestamp

Sorting & Labels

sort sort_desc label_replace label_join

Aggregation Operators

sum avg min max count topk bottomk

Binary Operators

+ - * / % ^ == != < > <= >= and or unless

With on/ignoring vector matching, by/without grouping, and bool. 0.9.0 enforces stricter one-to-one vector matching semantics.

Prometheus-compatible
server mode.

Run tsink as a standalone HTTP service with remote read/write, PromQL API, TLS, bearer auth, and gated admin snapshot/restore endpoints.

terminal
# Start the server (admin API is opt-in)
$ cargo run -p tsink-server -- server \
    --listen 127.0.0.1:9201 \
    --data-path ./tsink-data \
    --memory-limit 1G \
    --retention 14d \
    --auth-token secret-token \
    --enable-admin-api \
    --admin-path-prefix /srv/tsink-admin

# PromQL instant query
$ curl -H 'Authorization: Bearer secret-token' \
    'http://localhost:9201/api/v1/query?query=up&time=1700000000'

# Create snapshot via admin API
$ curl -X POST http://localhost:9201/api/v1/admin/snapshot \
    -H 'Authorization: Bearer secret-token' \
    -H 'Content-Type: application/json' \
    -d '{"path":"/srv/tsink-admin/snap-001"}'

Endpoints

GET /healthz Health check
GET /ready Readiness probe
GET /metrics Self-monitoring
GET/POST /api/v1/query PromQL instant
GET/POST /api/v1/query_range PromQL range
POST /api/v1/write Remote write
POST /api/v1/read Remote read
GET /api/v1/series Series metadata
GET /api/v1/labels Label names
GET /api/v1/label/<name>/values Values by label
POST /api/v1/import/prometheus Text ingestion
GET /api/v1/status/tsdb TSDB stats
POST /api/v1/admin/snapshot Admin snapshot
POST /api/v1/admin/restore Admin restore
POST /api/v1/admin/delete_series Admin delete (stub 501)
TLS with PEM certificates
Bearer auth (except /healthz, /ready)
Admin API gated by --enable-admin-api + --auth-token
Deep /metrics and /api/v1/status/tsdb internals
Graceful SIGTERM/SIGINT shutdown

Prometheus Integration

prometheus.yml
remote_write:
  - url: http://127.0.0.1:9201/api/v1/write

remote_read:
  - url: http://127.0.0.1:9201/api/v1/read

How it works under the hood.

LSM-style architecture with sharded concurrency, dual encoding lanes, and a disk-backed global series index.

Public API
StorageBuilder
Storage
AsyncStorage
PromQL Engine
Engine (64-shard partitioned)
Writers
N threads, sharded locks
Readers
concurrent, lock-free
Compactor
background merges + recovery markers
Data Pipeline
Active Chunks
in-memory, writable
Immutable Chunks
flushed, read-only
Segments
L0 → L1 → L2 disk
Storage & Durability
WAL
CRC32 + replay policy
Numeric Lane
Gorilla, delta, RLE
Blob Lane
bytes, string + delta block
Series Index v2
persisted roaring postings
Time Partitions Data split by wall-clock intervals (default: 1 hour)
Chunks 2048 points/chunk by default, per-lane encoding, optional zstd payload compression when smaller
Global Series Index Persisted dictionaries and postings in series_index.bin accelerate reopen and matcher-based selection
Background Tasks Flush every 250ms, compact every 5s — with health surfaced via observability snapshot

Adaptive compression codecs.

The encoder picks the most compact lane codec, then optionally wraps chunk payloads with zstd when it reduces size.

Timestamp Codecs

Fixed-step RLE Run-length encoding for fixed-interval timestamps
Delta-of-delta bitpack Primary strategy with bit-packing
Delta varint Varint-encoded deltas for irregular intervals

Value Codecs

Gorilla XOR XOR of IEEE 754 floats (f64)
Zigzag delta bitpack Zigzag + delta + bit-packing (i64)
Delta bitpack Delta encoding + bit-packing (u64)
Constant RLE Run-length for constant values (any numeric)
Bool bitpack 1 bit per value (bool)
Delta block Blob lane compression (bytes, string)
Zstd wrapper (optional) Chunk payload compressed at level 1 when smaller; decoded transparently on reads

Complete API at a glance.

Every type, trait, and method you need to build with tsink.

Storage

trait

The main interface for all time-series operations. Thread-safe (Send + Sync).

insert_rows(&self, rows: &[Row]) -> Result<()> Write one or more data points in a single batch
select(&self, metric, labels, start, end) -> Result<Vec<DataPoint>> Query points by metric name, label filters, and time range
select_into(&self, metric, labels, start, end, buf) -> Result<()> Write into a caller-provided buffer for allocation reuse
select_all(&self, metric, start, end) -> Result<Vec<(Labels, Points)>> Query all label combinations for a metric
select_with_options(&self, metric, opts) -> Result<Vec<DataPoint>> Downsampling, aggregation, custom bytes aggregation, pagination
select_series(&self, selection: &SeriesSelection) -> Result<Vec<MetricSeries>> Matcher-based series discovery (=, !=, =~, !~)
list_metrics(&self) -> Result<Vec<MetricSeries>> Discover all metric series in the database
list_metrics_with_wal(&self) -> Result<Vec<MetricSeries>> Like list_metrics, including WAL-only series
memory_used(&self) / memory_budget(&self) Inspect current/target memory footprint
observability_snapshot(&self) -> StorageObservabilitySnapshot Structured WAL/flush/compaction/query/health internals
snapshot(&self, destination) -> Result<()> Create an atomic on-disk snapshot from a live storage
close(&self) -> Result<()> Gracefully shut down, flushing all buffers and WAL

StorageBuilder

struct

Fluent builder for creating configured Storage instances.

StorageBuilder::new() -> Self Create builder with sensible defaults
.with_data_path(path) -> Self Enable persistent disk storage at the given path
.with_retention(duration) -> Self Set automatic data cleanup policy (default: 14 days)
.with_memory_limit(bytes) -> Self Hard in-memory budget with admission backpressure
.with_cardinality_limit(n) -> Self Cap total unique metric+label series
.with_wal_size_limit(bytes) -> Self Hard cap on total WAL bytes on disk
.with_chunk_points(n) -> Self Target points per chunk before flushing (default: 2048)
.with_wal_replay_mode(mode) -> Self Choose replay behavior on mid-log corruption (Salvage/Strict)
.with_background_fail_fast(bool) -> Self Stop new operations after background worker failures
.build() -> Result<Arc<dyn Storage>> Construct the storage engine
StorageBuilder::restore_from_snapshot(snapshot, data_path) Atomically restore a data directory from snapshot contents

Core Types

structs & enums

The fundamental data structures for time-series storage.

DataPoint { timestamp: i64, value: Value } Timestamp + typed value (f64, i64, u64, bool, bytes, string)
Value::F64 | I64 | U64 | Bool | Bytes | String Six typed variants with auto From conversions
Row::new(metric, data_point) A metric name paired with a single data point
Row::with_labels(metric, labels, data_point) A data point with multi-dimensional label metadata
Label::new(name, value) Key-value pair for dimensional tagging
Codec / Aggregator traits Custom bytes encoding and aggregation logic
StorageObservabilitySnapshot { wal, flush, compaction, query, health } Structured runtime snapshot for diagnostics and monitoring

Querying

structs + enums

Advanced query customization with downsampling, aggregation, and series discovery.

QueryOptions::new(start, end) Create a time-range query
.with_downsample(interval, agg) -> Self Bucket data into time intervals with aggregation
.with_custom_bytes_aggregation(codec, agg) Custom aggregation for non-numeric data
Aggregation::{ None, Sum, Min, Max, Avg, First, Last, Count, Median, Range, Variance, StdDev } 12 built-in aggregation functions + passthrough mode
SeriesSelection / SeriesMatcher Matcher-based series discovery with regex support

Configuration & Errors

enums

Enums for precision, durability, and comprehensive error handling.

TimestampPrecision::{ Nanoseconds, Microseconds, Milliseconds, Seconds } Unit for interpreting timestamp values
WalSyncMode::{ PerAppend, Periodic(Duration) } WAL fsync strategy for durability vs throughput
WalReplayMode::{ Salvage, Strict } Startup replay policy when WAL corruption is encountered
TsinkError::{ InvalidTimeRange, WriteTimeout, MemoryBudgetExceeded, CardinalityLimitExceeded, WalSizeLimitExceeded, ValueTypeMismatch, DataCorruption, ... } Comprehensive error types for all failure modes

PromQL Engine

feature-gated

Full PromQL query engine enabled via the promql feature flag.

promql::Engine::new(storage) -> Engine Create PromQL engine over a storage handle
promql::Engine::with_precision(storage, precision) Construct engine for timestamps not expressed in nanoseconds
engine.instant_query(query, time) -> Result<...> Evaluate PromQL at a single timestamp
engine.range_query(query, start, end, step) -> Result<...> Evaluate PromQL across a time window

Async Storage

feature-gated

Runtime-agnostic async facade enabled via the async-storage feature flag.

AsyncStorageBuilder::new().build() -> Result<AsyncStorage> Runtime-agnostic async facade with bounded queues
.with_wal_replay_mode(mode) / .with_background_fail_fast(bool) Forward durability and fail-fast controls to inner storage builder
async_storage.insert_rows(rows).await Async insert via dedicated worker threads
async_storage.select(metric, labels, start, end).await Async query with configurable read workers
async_storage.snapshot(path).await Create atomic snapshots via async write worker

Fully configurable.

Tune every aspect of the storage engine via the builder API.

Option Description Default
with_data_path(path) Directory for persistent segment files and WAL None (in-memory)
with_chunk_points(n) Target data points per chunk before flushing 2048
with_retention(dur) How long to keep data before auto-cleanup 14 days
with_retention_enforced(bool) Enforce retention window (false keeps data forever) true
with_timestamp_precision(p) Nanoseconds, Microseconds, Milliseconds, or Seconds Nanoseconds
with_max_writers(n) Maximum concurrent write workers (cgroup-aware) CPU count
with_write_timeout(dur) Timeout for write operations 30 seconds
with_partition_duration(dur) Time range covered by each partition 1 hour
with_memory_limit(bytes) Hard in-memory budget with admission backpressure Unlimited
with_cardinality_limit(n) Max unique metric + label-set series Unlimited
with_wal_enabled(bool) Enable write-ahead logging for durability true
with_wal_sync_mode(mode) WAL fsync strategy: Periodic(Duration) or PerAppend Periodic(1s)
with_wal_replay_mode(mode) Replay policy on WAL corruption: Salvage or Strict Salvage
with_wal_buffer_size(n) WAL buffer size in bytes 4096
with_wal_size_limit(bytes) Hard cap on total WAL bytes across all segments Unlimited
with_background_fail_fast(bool) Stop new operations after background flush/compaction worker failures false

Ready to embed a time-series engine?

Add tsink to your Rust project. Store millions of points with adaptive compression, snapshot and restore atomically, inspect runtime internals, and query with PromQL or server mode.