What happens
A KPI totals one value per source, so when two data sources report the same event, a sum KPI
adds both of them up. The KPI then shows a number that no single source ever reported, and that no
point on the chart shows either.
Example
One sensor, one event (a single day), reported by two sources — say a forecaster's value that was
later corrected by an upload, or two forecasters with different configurations:
| event_start |
source |
value |
| 2030-03-15 |
source A |
100.0 |
| 2030-03-15 |
source B |
80.0 |
with
asset.sensors_to_show_as_kpis = [
{"title": "Daily costs", "sensor": sensor.id, "function": "sum"}
]
[GET] /assets/(id)/kpis over that single day returns 180.0. Neither source said 180: one said
100 and the other said 80. Verified against a development database on main.
The same total is what the chart shows the sum of, because it draws one series per source — so the
KPI is consistent with the chart, but the number is not a quantity anyone asked for. It is a total
across sources, where each source is a separate claim about the same event, not a separate
contribution to it.
Why it happens
get_kpis (flexmeasures/api/v3_0/assets.py) reads:
beliefs = sensor.search_beliefs(
event_starts_after=start,
event_ends_before=end,
most_recent_beliefs_only=True,
)
most_recent_beliefs_only is per source, so this returns one row per event per source. The
default use_latest_version_per_event=True does collapse sources sharing a (name, type, model),
but only within the same belief time, unless one_deterministic_belief_per_event is set — which the
KPI query does not set. Two sources believing the same event at different moments therefore both
survive into values, and sum adds them.
Is it intended?
Partly, and that is what makes it worth a decision rather than a quick patch. The code comment above
that call says the opposite of what the call does:
# The beliefs the chart draws: one value per event, the most recent one.
# Aggregating belief rows instead would count a revision on top of what it revised,
# and would count each source separately when several report the same sensor.
The second line describes exactly what still happens across sources. Meanwhile
test_kpi_reports_what_the_chart_draws deliberately asserts a total across two sources — but there
each source reports a different event, so the sum is meaningful. The case above, two sources on the
same event, is not covered by any test.
So the principle "a KPI must describe the same beliefs as the chart" is right; what is missing is
what a KPI should do when the chart draws two points for one event.
Possible directions
- Pick one source per event. Pass
one_deterministic_belief_per_event=True in the KPI query,
which keeps the highest-priority source per event (by version, then id). One line, and it matches
the comment's intent. It changes what a KPI reports for sensors that legitimately carry several
sources today, and it makes the KPI stop agreeing with a chart that still draws both.
- Let the KPI say which source it means, via a
source key in the sensors_to_show_as_kpis
entry, defaulting to today's behaviour. Explicit, but it asks the user to know about sources.
- Leave the total as is and make the chart and the KPI agree visibly — for instance by having a
multi-source KPI say so, rather than presenting one number as if it were unambiguous.
I would lean to 1 for sum in particular, since summing two claims about one event has no reading
under which it is the right answer, whereas mean, min and max at least degrade gracefully.
How to reproduce
Add to flexmeasures/api/v3_0/tests/test_assets_api.py, beside test_kpi_reports_what_the_chart_draws:
@pytest.mark.parametrize("requesting_user", ["test_admin_user@seita.nl"], indirect=True)
def test_kpi_two_sources_same_event(
db, client, setup_api_test_data, setup_sources, requesting_user
):
asset_type = db.session.query(GenericAssetType).filter_by(name="battery").one_or_none()
asset = GenericAsset(
name="kpi two sources same event",
generic_asset_type=asset_type,
account_id=requesting_user.account_id,
)
db.session.add(asset)
db.session.flush()
sensor = Sensor(
name="kpi two sources sensor",
generic_asset=asset,
event_resolution=timedelta(days=1),
unit="EUR",
)
db.session.add(sensor)
db.session.flush()
sources = list(setup_sources.values())
a, b = sources[0], sources[-1]
window_start = datetime(2030, 3, 15, tzinfo=utc)
db.session.bulk_insert_mappings(
TimedBelief,
[
dict(event_start=window_start, belief_horizon=timedelta(days=2),
event_value=100.0, sensor_id=sensor.id, source_id=a.id,
cumulative_probability=0.5),
dict(event_start=window_start, belief_horizon=timedelta(days=1),
event_value=80.0, sensor_id=sensor.id, source_id=b.id,
cumulative_probability=0.5),
],
)
asset.sensors_to_show_as_kpis = [
{"title": "Daily costs", "sensor": sensor.id, "function": "sum"}
]
db.session.flush()
total = _kpi_total(
client, asset, window_start.isoformat(),
(window_start + timedelta(days=1)).isoformat(),
)
assert total == pytest.approx(180.0) # what happens today
Why now
This is pre-existing behaviour, and reproduces on main independently of any open branch. It comes up
now because PR #2464 makes a scheduled sensor able to carry schedules from several sources: after a
change to an asset's flex config, the sensor holds the schedule computed under each configuration.
Summing two schedules for the same period is meaningless in exactly the way described above, so the
window in which this bites gets wider.
Raised out of the discussion on #2464.
What happens
A KPI totals one value per source, so when two data sources report the same event, a
sumKPIadds both of them up. The KPI then shows a number that no single source ever reported, and that no
point on the chart shows either.
Example
One sensor, one event (a single day), reported by two sources — say a forecaster's value that was
later corrected by an upload, or two forecasters with different configurations:
with
[GET] /assets/(id)/kpisover that single day returns 180.0. Neither source said 180: one said100 and the other said 80. Verified against a development database on
main.The same total is what the chart shows the sum of, because it draws one series per source — so the
KPI is consistent with the chart, but the number is not a quantity anyone asked for. It is a total
across sources, where each source is a separate claim about the same event, not a separate
contribution to it.
Why it happens
get_kpis(flexmeasures/api/v3_0/assets.py) reads:most_recent_beliefs_onlyis per source, so this returns one row per event per source. Thedefault
use_latest_version_per_event=Truedoes collapse sources sharing a(name, type, model),but only within the same belief time, unless
one_deterministic_belief_per_eventis set — which theKPI query does not set. Two sources believing the same event at different moments therefore both
survive into
values, andsumadds them.Is it intended?
Partly, and that is what makes it worth a decision rather than a quick patch. The code comment above
that call says the opposite of what the call does:
The second line describes exactly what still happens across sources. Meanwhile
test_kpi_reports_what_the_chart_drawsdeliberately asserts a total across two sources — but thereeach source reports a different event, so the sum is meaningful. The case above, two sources on the
same event, is not covered by any test.
So the principle "a KPI must describe the same beliefs as the chart" is right; what is missing is
what a KPI should do when the chart draws two points for one event.
Possible directions
one_deterministic_belief_per_event=Truein the KPI query,which keeps the highest-priority source per event (by version, then id). One line, and it matches
the comment's intent. It changes what a KPI reports for sensors that legitimately carry several
sources today, and it makes the KPI stop agreeing with a chart that still draws both.
sourcekey in thesensors_to_show_as_kpisentry, defaulting to today's behaviour. Explicit, but it asks the user to know about sources.
multi-source KPI say so, rather than presenting one number as if it were unambiguous.
I would lean to 1 for
sumin particular, since summing two claims about one event has no readingunder which it is the right answer, whereas
mean,minandmaxat least degrade gracefully.How to reproduce
Add to
flexmeasures/api/v3_0/tests/test_assets_api.py, besidetest_kpi_reports_what_the_chart_draws:Why now
This is pre-existing behaviour, and reproduces on
mainindependently of any open branch. It comes upnow because PR #2464 makes a scheduled sensor able to carry schedules from several sources: after a
change to an asset's flex config, the sensor holds the schedule computed under each configuration.
Summing two schedules for the same period is meaningless in exactly the way described above, so the
window in which this bites gets wider.
Raised out of the discussion on #2464.