Use Cases

Common scenarios and complete configurations for real-world deployments.

About the VRL snippets

The configurations on this page are validated patterns you can adapt. The inline VRL snippets, however, are illustrative excerpts focused on the option being shown. They are not always complete, production-ready transformers. For production use, prefer the maintained transformers from transformers-community (GitHub, ArgoCD, Kubewatch, …) and validate event shapes against the CDEvents specification.

Development & Testing

Local Development

[sources.test_webhook]
enabled = true
[sources.test_webhook.extractor]
type = "webhook"
id = "test"

[sinks.debug]
enabled = true
type = "debug"

[sinks.files]
enabled = true
type = "folder"
kind = "fs"
parameters = { root = "./dev-events" }

Use: Test configurations, debug event flow, local development.

CI/CD Pipeline Testing

[sources.build_events]
enabled = true
transformer_refs = ["to_cdevents"]
[sources.build_events.extractor]
type = "opendal"
kind = "fs"
polling_interval = "10s"
path_patterns = ["build-events/*.json"]
parser = "json"
parameters = { root = "./ci-outputs" }

[transformers.to_cdevents]
type = "vrl"
template_file = "./transforms/ci-events.vrl"

[sinks.validation]
enabled = true
type = "folder"
kind = "fs"
parameters = { root = "./validated-events" }

Use: Validate event transformation in CI pipelines.

Production Deployments

Single Instance (Simple)

One collector handles all event types - good for small to medium scale setups:

GitHub ────┐
           ├─→ CDviz Collector ─→ Database
Jenkins ───┘                      └─→ Archive

Single Collector Configuration

[sources.api_webhooks]
enabled = true
transformer_refs = ["api_to_cdevents"]
[sources.api_webhooks.extractor]
type = "webhook"
id = "api"

[sources.log_files]
enabled = true
transformer_refs = ["logs_to_cdevents"]
[sources.log_files.extractor]
type = "opendal"
kind = "s3"
polling_interval = "5m"
path_patterns = ["logs/**/*.jsonl"]
parser = "jsonl"

[sinks.database]
enabled = true
type = "db"
# URL from environment

[sinks.archive]
enabled = true
type = "folder"
kind = "s3"

Use: Centralized collection for small to medium deployments.

Multiple Instances (Scalable)

Different collectors for different purposes - good for high scale, separation of concerns:

GitHub ─→ Collector A ─┐
                       ├─→ Database
GitLab ─→ Collector B ─┘
                       └─→ Analytics API

Files ──→ Collector C ─→ Archive

Multi-Collector Configuration

# Edge Collector - Regional event collection
[sources.regional_webhooks]
enabled = true
[sources.regional_webhooks.extractor]
type = "webhook"
id = "region-us-east"

[sinks.central_collector]
enabled = true
type = "http"
url = "https://central-collector.internal/webhooks/regional"

# Central Collector - Aggregation and storage
[sources.regional_events]
enabled = true
[sources.regional_events.extractor]
type = "webhook"
id = "regional"

[sinks.master_database]
enabled = true
type = "db"

[sinks.analytics_api]
enabled = true
type = "http"
url = "https://analytics.company.com/events"

Use: Distributed collection with centralized processing.

Specific Integrations

GitHub Repository Events

[sources.github]
enabled = true
transformer_refs = ["github_events"]
[sources.github.extractor]
type = "webhook"
id = "github"
headers_to_keep = ["X-GitHub-Event"]

[sources.github.extractor.headers]
"x-hub-signature-256" = { type = "signature" } # token from environment

[transformers.github_events]
type = "vrl"
template_file = "/etc/cdviz-collector/transformers/github_events.vrl"

Use: Standard GitHub webhook integration. See GitHub Integration.

Kubernetes Cluster Events

[sources.k8s_events]
enabled = true
transformer_refs = ["kubewatch_to_cdevents"]
[sources.k8s_events.extractor]
type = "webhook"
id = "kubewatch"

[transformers.kubewatch_to_cdevents]
type = "vrl"
template = '''
[{
    "body": {
        "context": {
            "type": "dev.cdevents.environment.modified.0.1.1",
            "source": "/k8s/" + (.body.namespace // "default")
        },
        "subject": {
            "id": .body.name,
            "type": .body.kind
        }
    }
}]
'''

Use: Kubernetes cluster monitoring. See Kubernetes Integration.

Build Artifact Processing

[sources.build_artifacts]
enabled = true
transformer_refs = ["artifact_metadata"]
[sources.build_artifacts.extractor]
type = "opendal"
kind = "s3"
polling_interval = "2m"
path_patterns = ["builds/**/*.json", "releases/**/*.json"]
parser = "json"
parameters = { bucket = "ci-artifacts" }

[transformers.artifact_metadata]
type = "vrl"
template = '''
[{
    "body": {
        "context": {
            "type": "dev.cdevents.artifact.published.0.2.0",
            "source": "/ci/artifacts"
        },
        "subject": {
            "id": .body.artifact_id,
            "type": "artifact",
            "content": {
                "url": .body.download_url,
                "digest": { "sha256": .body.sha256 }
            }
        }
    }
}]
'''

[sinks.artifact_db]
enabled = true
type = "db"

[sinks.notifications]
enabled = true
type = "http"
url = "https://notifications.company.com/webhooks/artifacts"

Use: Monitor and track build artifacts across environments.

Advanced Patterns

Multi-Environment with Filtering

# Production events only
[sources.prod_api]
enabled = true
transformer_refs = ["env_filter", "to_cdevents"]
[sources.prod_api.extractor]
type = "webhook"
id = "prod"

[transformers.env_filter]
type = "vrl"
template = '''
if .body.environment != "production" {
    []  # Discard non-production events
} else {
    [.]  # Pass through production events
}
'''

[transformers.to_cdevents]
type = "vrl"
template_file = "./transforms/api-events.vrl"

Use: Environment-specific event processing with filtering.

Event Splitting and Enrichment

[sources.batch_events]
enabled = true
transformer_refs = ["split_batch", "enrich_metadata"]
[sources.batch_events.extractor]
type = "webhook"
id = "batch"

[transformers.split_batch]
type = "vrl"
template = '''
# Split batch into individual events
map_values(.body.events) -> |event| {
    {
        "metadata": .metadata,
        "headers": .headers,
        "body": event
    }
}
'''

[transformers.enrich_metadata]
type = "vrl"
template = '''
[{
    "metadata": merge(.metadata, {
        "region": "${AWS_REGION}",
        "processed_at": now()
    }),
    "headers": .headers,
    "body": .body
}]
'''

Use: Process batch events and add runtime metadata.