Skip to content

out_opentelemetry: add logs_body_key_attributes_exclude option - #12793

Open
cvas22 wants to merge 2 commits into
fluent:masterfrom
cvas22:feat/12735-otel-attributes-exclude
Open

cvas22 wants to merge 2 commits into
fluent:masterfrom
cvas22:feat/12735-otel-attributes-exclude

Conversation

@cvas22

@cvas22 cvas22 commented Nov 15, 2026 •

Copy link
Copy Markdown

With logs_body_key_attributes: on, every field left over after the body becomes an attribute, including the ones already mapped to first-class OTLP fields through logs_trace_id_message_key, logs_severity_text_message_key and so on. So TraceId and SeverityText end up both on the record and in its attributes. Turning the option off isn't a way out, since it also drops the fields people want as attributes.

This adds logs_body_key_attributes_exclude, a comma separated list of record keys that are skipped when the leftover fields are turned into attributes:

outputs:
  - name: opentelemetry
    match: "*"
    logs_body_key: $message
    logs_body_key_attributes: on
    logs_body_key_attributes_exclude: TraceId,SeverityText

Keys are matched by exact name against the top-level record keys. Nothing changes when the option isn't set.

Fixes #12735


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

I'll open the docs PR in fluent-bit-docs.

Backporting

  • Backport to latest stable release.

Notes

I went with the exclude list from the issue rather than having the logs_*_message_key accessors consume their keys, since that would change the output for everyone already using logs_body_key_attributes.

test_out_opentelemetry_logs_body_key_attributes_exclude in the out_opentelemetry integration scenario sends a record with TraceId, SeverityText and thread, and checks the trace ID and severity are set on the record, thread is an attribute, and the two excluded keys aren't. I haven't run it or valgrind locally, I'm relying on CI.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features
    • Added a configuration option to exclude selected record keys from log attributes when body-key attributes are enabled. The log body and other attributes continue to be exported as before.

logs_body_key_attributes adds every record field left over after the body
as an attribute, so fields already mapped to first-class OTLP fields, like
TraceId or SeverityText, are sent twice. Turning the option off drops the
fields that should stay attributes too.

Add logs_body_key_attributes_exclude, a comma separated list of record keys
that are skipped when the leftover fields are turned into attributes.

Signed-off-by: Srinivas Chippagiri <cvas22@gmail.com>
…exclude

Signed-off-by: Srinivas Chippagiri <cvas22@gmail.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T07:39:32.876118Z b9776df PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cvas22

cvas22 commented Nov 15, 2026

Copy link
Copy Markdown
Author

Docs PR: fluent/fluent-bit-docs#2848

@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The OpenTelemetry output plugin adds logs_body_key_attributes_exclude. It skips matching record keys when converting remaining fields into OTLP log attributes. An integration test checks excluded fields and verifies that other attributes remain.

Changes: OpenTelemetry log attribute exclusions

Layer / File(s) Summary
Configure the exclusion list
plugins/out_opentelemetry/opentelemetry.h, plugins/out_opentelemetry/opentelemetry.c
The plugin context stores the exclusion list. The new logs_body_key_attributes_exclude option accepts a comma-separated list of keys.
Skip excluded keys during attribute conversion
plugins/out_opentelemetry/opentelemetry_logs.c, tests/integration/scenarios/out_opentelemetry/config/out_otel_http_logs_attributes_exclude.yaml, tests/integration/scenarios/out_opentelemetry/tests/test_out_opentelemetry_001.py
The plugin skips matching string keys when converting remaining record fields into attributes. The integration test checks that TraceId and SeverityText are excluded, while thread remains an attribute.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature · Severity of issue fixed: Medium

Merge Risk: Minimal · up to b9776

This adds an optional setting to omit selected record keys from OpenTelemetry log attributes. Default behavior is unchanged, and no merge-blocking issues were found.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the change accurately: it adds the logs_body_key_attributes_exclude option to the out_opentelemetry plugin. The raw summary, objectives, and linked issue all describe this exact exclus…
Linked Issues check Passed Issue #12735 asks for a way to keep logs_body_key_attributes enabled while omitting selected record keys from attributes, so keys mapped to first-class OTLP fields are not duplicated. The head adds th…
Out of Scope Changes check Passed All changes from the merge base to head relate to the exclusion option: the config map entry and struct field in opentelemetry.c and opentelemetry.h, the filter helper and include in opentelemetry_log…
Full details: Title check

Explanation

The title names the change accurately: it adds the logs_body_key_attributes_exclude option to the out_opentelemetry plugin. The raw summary, objectives, and linked issue all describe this exact exclusion option for keys used with logs_body_key_attributes, so the title matches the main change.

Full details: Linked Issues check

Explanation

Issue #12735 asks for a way to keep logs_body_key_attributes enabled while omitting selected record keys from attributes, so keys mapped to first-class OTLP fields are not duplicated. The head adds the logs_body_key_attributes_exclude CLIST option in opentelemetry.c (lines near 1797). In opentelemetry_logs.c, log_record_set_attributes() calls the new log_record_attribute_is_excluded() before msgpack_kv_to_otlp_any_value() for each leftover map entry, so excluded keys are skipped. The check matches exact key length and content and returns false when the list is NULL, so behavior is unchanged when the option is unset. The proposal in the issue explicitly allows this exclusion-list approach. The integration test scenario is described in the PR, but it was not run; this does not affect the code-level compliance verdict. Documentation is out of this check's coding scope.

Full details: Out of Scope Changes check

Explanation

All changes from the merge base to head relate to the exclusion option: the config map entry and struct field in opentelemetry.c and opentelemetry.h, the filter helper and include in opentelemetry_logs.c, and the integration config and test. The flb_slist.h include supports the list traversal in the helper. No unrelated behavior change was found in the inspected diffs of opentelemetry.c and opentelemetry_logs.c.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7e615fda-aff2-41db-967b-607ce64afdca
Commits

Reviewing files that changed from the base of the PR and between 27d5624 and b9776df.

Files selected for processing: 5

  • plugins/out_opentelemetry/opentelemetry.c
  • plugins/out_opentelemetry/opentelemetry.h
  • plugins/out_opentelemetry/opentelemetry_logs.c
  • tests/integration/scenarios/out_opentelemetry/config/out_otel_http_logs_attributes_exclude.yaml
  • tests/integration/scenarios/out_opentelemetry/tests/test_out_opentelemetry_001.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

out_opentelemetry: option to exclude specific body keys from log attributes (avoid duplication with logs_*_message_key)

1 participant