CDviz Collector

CDviz Collector is an open-source event ingestion agent written in Rust. It receives software delivery events from webhooks, Kafka, NATS, SSE, and file sources, normalizes them to the CDEvents standard, and routes them to configured sinks including PostgreSQL, ClickHouse, and HTTP endpoints.

Quick Start

New to CDviz Collector? Get a working setup in 5 minutes:

🚀 Quick Start Guide - Webhook → Transform → Save to files

Learning Paths

📖 Tutorials (Learning-Oriented)

Step-by-step lessons to build understanding:

• Quick Start - 5 min setup with webhook and file output

🔧 How-to Guides (Problem-Oriented)

Practical solutions for specific tasks:

Integrations:

• GitHub - Repository events with signatures
• GitHub Actions - CI/CD workflow events
• GitLab - Repository and pipeline events
• ArgoCD - GitOps deployment events
• Kubernetes - Cluster events via Kubewatch

Common Tasks:

• Troubleshooting - Debug configuration and connectivity issues
• Authentication - Secure outgoing requests
• Validation - Validate incoming webhooks

📚 Reference (Information-Oriented)

Complete technical specifications:

Configuration:

• Configuration Guide - Main config structure and environment variables
• TOML Syntax - Configuration file format help
• CLI Usage - Command-line interface

Commands:

• connect - Launch collector as a server to connect sources to sinks.
• send - Send JSON data directly to a sink for testing and scripting.
• transform - Transform local JSON files using configured transformers or test transformer.

Components:

• Sources - Event collection: Webhook, Files, SSE, Kafka, NATS, Noop
• Parsers - Multi-format input parsing: JSON, XML, TAP, CSV, Text, and more
• Transformers - Event processing with VRL
• Sinks - Event delivery: Database, HTTP, Files, SSE, Kafka, NATS, Debug

🧠 Explanation (Understanding-Oriented)

Concepts and design decisions:

• CDEvents Standard - Why we use CDEvents for standardization

Architecture Concepts:

• Pipeline Flow: External Systems → Sources → Transformers → Sinks → Destinations
• Message Structure: All events have metadata, headers, and body components
• Parallel Processing: Sources, transformers, and sinks run independently
• Deployment Patterns: Single instance (simple) → Multiple instances (scalable) → Hub-and-spoke (enterprise)

Installation

# Quick install
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/cdviz-dev/cdviz-collector/releases/download/v0.6.4/cdviz-collector-installer.sh | sh

# Verify
cdviz-collector --version

Full Installation Guide - Docker, Kubernetes, Homebrew, and more options