Ship logs with Fluent Bit or Vector

Not every app can take an SDK. A vendor binary writes to /var/log, a legacy service only prints to stdout, and a Kubernetes cluster runs a hundred containers you did not write. For those, a log shipper reads the output and forwards it.

For log files and Docker hosts, ZipLogger has no agent of its own. Fluent Bit and Vector both work, because both can post JSON over HTTP and ZipLogger's ingest endpoint accepts it. On Kubernetes you have a second option: the ZipLogger Helm chart ships container logs and also monitors the cluster itself. This post has configs for files, Docker and Kubernetes, and the rules the endpoint applies to what they send.

The endpoint

POST https://app.ziplogger.ai/ingest/v1/logs
X-Api-Key: YOUR_API_KEY

The body can be one JSON object, a JSON array, or NDJSON (one object per line). Both shippers below send NDJSON. The endpoint reads these keys, in any letter case:

Any other top-level key is ignored. That matters for shippers, which add metadata such as host, file or kubernetes at the top level. If you want to keep it, move it under fields.

Content-Encoding: gzip is supported. Keep each request under 1 MB on the wire. A larger body gets a 413 from the proxy in front of the API, with an HTML body rather than JSON. Gzip helps, because the limit counts the compressed bytes. A good answer is 202 with a JSON body like {"accepted": 20, "rejected": 0, ...}.

Five rules for what gets stored

These come from running the endpoint's parsing code against the payloads each shipper produces.

1. A missing message is not an error. The line is accepted and stored with an empty message. Fluent Bit's tail input puts the text in log, so without a rename you get a 202 and a page of blank lines.
2. timestamp must be an ISO 8601 string. A Unix epoch number, or a string like 2026-09-29 10:15:02, fails the whole request with 400 Invalid JSON. Up to nine fractional digits are fine.
3. One bad line fails the batch. If one line of an NDJSON body does not parse, the whole request gets a 400 and nothing in it is stored.
4. Unknown severities become info. ERROR and Warning work, because case is ignored. err, critical and trace are all stored as info. Map them in the shipper.
5. fields must be an object. A string or number under fields fails the request with 400.

Fluent Bit: log files

This is the config from the docs, with a fixed source and gzip added:

[INPUT]
    Name   tail
    Path   /var/log/app/*.log
    Tag    app

[FILTER]
    Name   modify
    Match  app
    Rename log message
    Add    source legacy-app

[OUTPUT]
    Name             http
    Match            *
    Host             app.ziplogger.ai
    Port             443
    TLS              On
    URI              /ingest/v1/logs
    Format           json_lines
    Json_date_key    timestamp
    Json_date_format iso8601
    compress         gzip
    Header           X-Api-Key ${ZIPLOGGER_API_KEY}

The two Json_date_* lines matter. By default, Fluent Bit writes the record time as an epoch number under date. ZipLogger ignores date, so every line would be stamped with its arrival time. Setting only Json_date_key timestamp is worse: the epoch number lands in timestamp and every request is refused (rule 2). Set both.

Every line is stored at info. If the text starts with a level, parse it out with a Fluent Bit parser and rename that key to severity.

Fluent Bit: Kubernetes

If you only need container logs plus cluster health, the ZipLogger Helm chart does both without this config. Use Fluent Bit when you already run it or need its filters.

On Kubernetes, Fluent Bit usually runs as a DaemonSet that tails the container log files on each node. The kubernetes filter adds pod metadata as a nested kubernetes object, which ZipLogger would drop. The nest filter moves it under fields:

[INPUT]
    Name              tail
    Path              /var/log/containers/*.log
    multiline.parser  docker, cri
    Tag               kube.*

[FILTER]
    Name   kubernetes
    Match  kube.*

[FILTER]
    Name   modify
    Match  kube.*
    Rename log message
    Add    source prod-cluster

[FILTER]
    Name       nest
    Match      kube.*
    Operation  nest
    Wildcard   kubernetes
    Wildcard   stream
    Nest_under fields

Use the same [OUTPUT] as above. In ZipLogger the pod name is then fields.kubernetes.pod_name, which you can filter on in the Search page's Fields explorer.

Vector: log files

[sources.app]
type = "file"
include = ["/var/log/app/*.log"]

[transforms.shape]
type = "remap"
inputs = ["app"]
source = '''
.source = "legacy-app"
.severity = "info"
if match(string!(.message), r'\bERROR\b') { .severity = "error" }
if match(string!(.message), r'\bWARN(ING)?\b') { .severity = "warn" }
.fields.file = .file
.fields.host = .host
'''

[sinks.ziplogger]
type = "http"
inputs = ["shape"]
uri = "https://app.ziplogger.ai/ingest/v1/logs"
compression = "gzip"
encoding.codec = "json"
framing.method = "newline_delimited"
request.headers.X-Api-Key = "${ZIPLOGGER_API_KEY}"

Vector's file source already puts the line in .message and the read time in .timestamp, so no rename is needed. The two match lines are a simple way to pull a level out of plain text. Change them to fit your log format. The last two lines keep the file path and host name, which would otherwise be dropped.

Vector: Docker containers

The docker_logs source reads from the Docker socket, so it sees every container's stdout and stderr without touching the containers:

[sources.docker]
type = "docker_logs"

[transforms.shape]
type = "remap"
inputs = ["docker"]
source = '''
.source = .container_name
.fields.stream = .stream
.fields.image = .image
'''

Point the same [sinks.ziplogger] at inputs = ["shape"]. Each container becomes its own source, so the Search page can filter one service at a time. For Kubernetes, Vector's kubernetes_logs source works the same way. Use .source = .kubernetes.container_name and copy the pod fields you care about into .fields.

If nothing shows up

• 401. The header is X-Api-Key, with no Bearer prefix. Create the key under Settings → API keys.
• 400 Invalid JSON. Look at the timestamp, fields and malformed lines (rules 2, 3 and 5). The response has a detail saying which value failed. Retrying the same batch will not help.
• Lines arrive but are blank. The text is under some key other than message (rule 1).
• 429. A plan limit was reached. The response has a Retry-After header. Both shippers retry, which is fine when the whole batch was refused. If only part of a batch was over the limit, the response is still 429, and a retry of the whole batch can store the accepted part twice.
• Do not set a fixed Idempotency-Key header. The endpoint supports one for SDK retries, but it must be unique per batch. A fixed value makes every batch after the first look like a changed resend of the first, and those are refused with 422.

What this does not do

• No managed shipper config. You install and run Fluent Bit or Vector yourself. ZipLogger never sees or manages their config.
• No parsing on the server. Whatever arrives in message is stored as text. Levels, fields and multi-line stack traces have to be parsed in the shipper.
• No other protocols on this endpoint. It does not accept syslog, the Fluentd forward protocol or the Elasticsearch bulk API. If you already run an OpenTelemetry Collector, use its OTLP exporter against /v1/logs instead. See the OpenTelemetry docs.

Ingested bytes count toward your plan's log volume. See pricing. If you can change the app, an SDK gives you structured fields and stack traces without any parsing. The Serilog post and the Python logging post show two of them. Otherwise, create a free workspace, make an API key, and start with the file config above.