Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
33c6dd4
fix: fixed size exemplar reservoir
proost Aug 14, 2026
deb01d6
doc: update changelog
proost Aug 14, 2026
fd8e429
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
proost Aug 14, 2026
a978029
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
proost Aug 16, 2026
c7864c3
fix: uniform distributed random
proost Aug 20, 2026
6c4e5f6
test: benchmark for lock contention
proost Aug 21, 2026
5e65718
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
proost Aug 21, 2026
ee3c93e
test: add preview benchmark
proost Aug 21, 2026
980ef1e
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
proost Aug 21, 2026
64c406f
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
proost Aug 24, 2026
dcff7a2
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
proost Aug 29, 2026
84e4fef
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
proost Sep 9, 2026
916b364
Merge branch 'main' into fix-fixed-size-exemplar-reservoir
proost Sep 17, 2026
71ac538
doc: update changelog
proost Sep 17, 2026
dffd08c
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
proost Oct 12, 2026
4eed868
Merge branch 'fix-fixed-size-exemplar-reservoir' of github.com:proost…
proost Oct 16, 2026
7880394
Merge branch 'main' into fix-fixed-size-exemplar-reservoir
marcalff Oct 16, 2026
8b7facc
Merge branch 'main' of github.com:open-telemetry/opentelemetry-cpp in…
proost Oct 19, 2026
470b90c
Merge branch 'fix-fixed-size-exemplar-reservoir' of github.com:proost…
proost Oct 19, 2026
8c80ba1
Merge branch 'main' of github.com:open-telemetry/opentelemetry-cpp in…
proost Oct 24, 2026
1697700
doc: add section
proost Oct 24, 2026
a936f6b
doc: wrong section
proost Oct 24, 2026
ba93b11
doc: wrong location
proost Oct 24, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,15 @@ Increment the:

## [Unreleased]

Changes:

* [METRICS SDK] Fix preview exemplar reservoirs to serialize concurrent
measurement offers and collection, reset stored cells and sampling state
between collection intervals, remain usable after collection, and omit empty
cells from collected results. Correct the simple fixed-size reservoir's
sampling bounds so the current measurement can be discarded.
[#4429](https://github.com/open-telemetry/opentelemetry-cpp/pull/4429)

## [1.30.0] 2026-10-22

This release contains:
Expand Down
6 changes: 3 additions & 3 deletions bazel/otel_cc_benchmark.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
load("@rules_cc//cc:cc_binary.bzl", "cc_binary")
load("@rules_cc//cc:cc_test.bzl", "cc_test")

def otel_cc_benchmark(name, srcs, deps, tags = [""]):
def otel_cc_benchmark(name, srcs, deps, tags = [""], defines = []):
"""
Creates targets for the benchmark and related targets.

Expand All @@ -30,7 +30,7 @@ def otel_cc_benchmark(name, srcs, deps, tags = [""]):
srcs = srcs,
deps = deps + ["@com_github_google_benchmark//:benchmark"],
tags = tags + ["manual"],
defines = ["BAZEL_BUILD"],
defines = ["BAZEL_BUILD"] + defines,
)

# The result of running the benchmark, captured into a text file.
Expand All @@ -51,5 +51,5 @@ def otel_cc_benchmark(name, srcs, deps, tags = [""]):
deps = deps + ["@com_github_google_benchmark//:benchmark"],
args = ["--benchmark_min_time=1x"],
tags = tags + ["benchmark"],
defines = ["BAZEL_BUILD"],
defines = ["BAZEL_BUILD"] + defines,
)
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
#ifdef ENABLE_METRICS_EXEMPLAR_PREVIEW

# include <memory>
# include <mutex>
# include <utility>
# include <vector>

# include "opentelemetry/context/context.h"
Expand Down Expand Up @@ -44,6 +46,9 @@ class FixedSizeExemplarReservoir : public ExemplarReservoir
{
return;
}

std::lock_guard<std::mutex> lock{mutex_};

@lalitb lalitb Aug 19, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This serializes offers and collection for each reservoir. That is the right correctness fix, and the feature is still ENABLE_METRICS_EXEMPLAR_PREVIEW-gated. It does add contention to the exemplar-enabled record path, so please mention that tradeoff in the PR description and open a follow-up to benchmark or explore a less contended design before exemplars become stable.

@proost proost Aug 19, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because resetting "reservoir_cell_selector_" is called in "CollectAndReset". "CollectAndReset" is never called in the this repo, But i can't find what is contract about thread-safe.

It does add contention to the exemplar-enabled record path

Absolutely right. I bit more digging out atomic operation way.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm considering using atomic variable, But it is not easy to change it and it needs broader change of ReservoirCell and ReservoirCellSelector.

So i added benchmark to the description.

6c4e5f6


auto idx =
reservoir_cell_selector_->ReservoirCellIndexFor(storage_, value, attributes, context);
if (idx != -1)
Expand All @@ -60,6 +65,9 @@ class FixedSizeExemplarReservoir : public ExemplarReservoir
{
return;
}

std::lock_guard<std::mutex> lock{mutex_};

auto idx =
reservoir_cell_selector_->ReservoirCellIndexFor(storage_, value, attributes, context);
if (idx != -1)
Expand All @@ -76,17 +84,26 @@ class FixedSizeExemplarReservoir : public ExemplarReservoir
{
return results;
}

std::lock_guard<std::mutex> lock{mutex_};

if (!map_and_reset_cell_)
{
reservoir_cell_selector_.reset();
reservoir_cell_selector_->reset();
return results;
}
for (auto reservoirCell : storage_)

results.reserve(storage_.size());
for (auto &reservoirCell : storage_)
{
auto result = (reservoirCell.*(map_and_reset_cell_))(pointAttributes);
results.push_back(result);
if (result)
{
results.emplace_back(std::move(result));
}
}
reservoir_cell_selector_.reset();

reservoir_cell_selector_->reset();
return results;
}

Expand All @@ -95,6 +112,7 @@ class FixedSizeExemplarReservoir : public ExemplarReservoir
std::vector<ReservoirCell> storage_;
std::shared_ptr<ReservoirCellSelector> reservoir_cell_selector_;
MapAndResetCellType map_and_reset_cell_{nullptr};
std::mutex mutex_;
};

} // namespace metrics
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

#ifdef ENABLE_METRICS_EXEMPLAR_PREVIEW

# include <cstdint>
# include <memory>
# include <vector>

Expand Down Expand Up @@ -68,6 +69,11 @@ class SimpleFixedSizeExemplarReservoir : public FixedSizeExemplarReservoir
// https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#simplefixedsizeexemplarreservoir
//

if (size_ == 0)
{
return -1;
}

size_t measurement_num = measurements_seen_++;
size_t index = static_cast<size_t>(-1);

Expand All @@ -77,22 +83,26 @@ class SimpleFixedSizeExemplarReservoir : public FixedSizeExemplarReservoir
}
else
{
size_t random_index = sdk::common::Random::GenerateRandom64() % measurement_num;

if (random_index < size_)
{
index = random_index;
}
return GetRandomCellIndex(size_, measurement_num, sdk::common::Random::GenerateRandom64());
}

return static_cast<int>(index);
}

void reset() override {}
void reset() override { measurements_seen_ = 0; }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reset is correct, but the sampling calculation just above is still off by one. measurement_num is zero-based, so the random choice must cover [0, measurement_num]. With a size-one reservoir, % measurement_num makes the second measurement replace the first every time instead of with 50% probability; size 0 also reaches modulo zero. Could we fix that here and add a deterministic test for the selection bounds?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

I updated: c7864c3


private:
static int GetRandomCellIndex(size_t size,
size_t measurement_num,
uint64_t random_value) noexcept
{
size_t random_index = random_value % (measurement_num + 1);
return random_index < size ? static_cast<int>(random_index) : -1;
}

size_t measurements_seen_ = 0;
size_t size_;
friend class SimpleFixedSizeCellSelectorTestPeer;
}; // class SimpleFixedSizeCellSelector

}; // class SimpleFixedSizeExemplarReservoir
Expand Down
36 changes: 36 additions & 0 deletions sdk/test/metrics/exemplar/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,25 @@
# SPDX-License-Identifier: Apache-2.0

load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//bazel:otel_cc_benchmark.bzl", "otel_cc_benchmark")

otel_cc_benchmark(
name = "fixed_size_exemplar_reservoir_benchmark",
srcs = [
"fixed_size_exemplar_reservoir_benchmark.cc",
],
defines = ["ENABLE_METRICS_EXEMPLAR_PREVIEW"],
tags = [
"benchmark",
"metrics",
"test",
],
deps = [
"//api",
"//sdk:headers",
"//sdk/src/common:random",
],
)

cc_test(
name = "no_exemplar_reservoir_test",
Expand Down Expand Up @@ -70,3 +89,20 @@ cc_test(
"@com_google_googletest//:gtest_main",
],
)

cc_test(
name = "fixed_size_exemplar_reservoir_test",
srcs = [
"fixed_size_exemplar_reservoir_test.cc",
],
tags = [
"metrics",
"test",
],
deps = [
"//api",
"//sdk:headers",
"//sdk/src/metrics",
"@com_google_googletest//:gtest_main",
],
)
10 changes: 9 additions & 1 deletion sdk/test/metrics/exemplar/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,18 @@
# Copyright The OpenTelemetry Authors
# SPDX-License-Identifier: Apache-2.0

if(WITH_BENCHMARK)
add_executable(fixed_size_exemplar_reservoir_benchmark
fixed_size_exemplar_reservoir_benchmark.cc)
target_link_libraries(
fixed_size_exemplar_reservoir_benchmark benchmark::benchmark
${CMAKE_THREAD_LIBS_INIT} opentelemetry_common)
endif()

foreach(
testname
no_exemplar_reservoir_test aligned_histogram_bucket_exemplar_reservoir_test
reservoir_cell_test filter_predicate_test)
reservoir_cell_test filter_predicate_test fixed_size_exemplar_reservoir_test)
add_executable(${testname} "${testname}.cc")
target_link_libraries(
${testname} ${GTEST_BOTH_LIBRARIES} ${CMAKE_THREAD_LIBS_INIT}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

#ifdef ENABLE_METRICS_EXEMPLAR_PREVIEW

# include <benchmark/benchmark.h>
# include <cstdint>
# include <string>
# include <utility>

# include "opentelemetry/context/context.h"
# include "opentelemetry/nostd/shared_ptr.h"
# include "opentelemetry/sdk/metrics/data/exemplar_data.h"
# include "opentelemetry/sdk/metrics/exemplar/reservoir.h"
# include "opentelemetry/sdk/metrics/exemplar/reservoir_cell.h"
# include "opentelemetry/sdk/metrics/exemplar/simple_fixed_size_exemplar_reservoir.h"

namespace
{

using opentelemetry::context::Context;
using opentelemetry::nostd::shared_ptr;
using opentelemetry::sdk::metrics::ExemplarReservoir;
using opentelemetry::sdk::metrics::MetricAttributes;
using opentelemetry::sdk::metrics::ReservoirCell;
using opentelemetry::sdk::metrics::SimpleFixedSizeExemplarReservoir;

class SharedSimpleFixedSizeExemplarReservoirFixture : public benchmark::Fixture
{
public:
using benchmark::Fixture::SetUp;
using benchmark::Fixture::TearDown;

void SetUp(benchmark::State &state) override
{
if (state.thread_index() != 0)
{
return;
}

auto selector = SimpleFixedSizeExemplarReservoir::GetSimpleFixedSizeCellSelector(1);
reservoir_ = shared_ptr<ExemplarReservoir>{
new SimpleFixedSizeExemplarReservoir{1, selector, &ReservoirCell::GetAndResetDouble}};

// Model a reservoir partway through a normal collection interval. This
// keeps the benchmark focused on the steady-state offer path.
for (int64_t i = 0; i < 1024; ++i)
{
reservoir_->OfferMeasurement(static_cast<double>(i), attributes_, context_);
}
}

void TearDown(benchmark::State &state) override
{
if (state.thread_index() == 0)
{
reservoir_ = nullptr;
}
}

protected:
shared_ptr<ExemplarReservoir> reservoir_;
MetricAttributes attributes_{{"http.request.method", "GET"},
{"http.route", "/checkout"},
{"http.response.status_code", int64_t{200}},
{"service.version", "1.42.0"}};
Context context_;
};

BENCHMARK_DEFINE_F(SharedSimpleFixedSizeExemplarReservoirFixture, OfferMeasurement)
(benchmark::State &state)
{
double value = 10.0 + static_cast<double>(state.thread_index());
for (auto _ : state)
{
reservoir_->OfferMeasurement(value, attributes_, context_);
value += 0.001;
}

benchmark::DoNotOptimize(value);
state.SetItemsProcessed(state.iterations());
}

BENCHMARK_REGISTER_F(SharedSimpleFixedSizeExemplarReservoirFixture, OfferMeasurement)
->ThreadRange(1, 8)
->UseRealTime()
->Unit(benchmark::kNanosecond);

} // namespace

BENCHMARK_MAIN();

#endif // ENABLE_METRICS_EXEMPLAR_PREVIEW
Loading
Loading