From 26fbf72895dd4ad3371a6b22ce75f5b8672b8e16 Mon Sep 17 00:00:00 2001 From: "F.N. Claessen" Date: Fri, 31 Jul 2026 17:53:42 +0200 Subject: [PATCH 1/8] feat: model inflexible devices as assets; drop unreleased flex-context group field An inflexible (unschedulable) device can now be declared in a flex-model entry via a single inflexible-consumption/inflexible-production sensor reference (its own asset). Such a device joins a group through the ordinary flex-model 'group' field, exactly like a flexible member, so its fixed load counts towards the group's intermediate power constraint -- and it is gathered upward with the rest of the flex-model/asset tree, so no downward flex-context traversal is needed. This replaces the unreleased approach from #2358, which let a flex-context inflexible entry carry a 'group' field (removed here, together with the SensorReference.group carrier and the inventory plumbing behind it). The flat flex-context inflexible-consumption/production lists remain for site base load. InflexibleDeviceSchema moves to schemas/sensors.py so both the flex-context and the flex-model can use it without an import cycle. group_to_devices already folds inflexible members in, and FlexDevice.group_key reverts to reading the flex-model 'group' -- so a flex-model inflexible device's membership uses the normal path. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TaMepbxRYzJxQKFtu6Hq4p Signed-off-by: F.N. Claessen --- documentation/changelog.rst | 3 +- documentation/features/scheduling.rst | 8 +- flexmeasures/data/models/planning/devices.py | 178 +++++++++++------- .../planning/tests/test_device_inventory.py | 132 +++++++------ .../planning/tests/test_group_constraints.py | 67 +++---- .../data/schemas/scheduling/__init__.py | 72 ++----- .../data/schemas/scheduling/metadata.py | 4 +- .../data/schemas/scheduling/storage.py | 113 +++++++++++ flexmeasures/data/schemas/sensors.py | 46 ++++- .../data/schemas/tests/test_scheduling.py | 99 +++++----- flexmeasures/ui/static/openapi-specs.json | 59 +++--- 11 files changed, 478 insertions(+), 303 deletions(-) diff --git a/documentation/changelog.rst b/documentation/changelog.rst index f2ce661c91..073cbb06eb 100644 --- a/documentation/changelog.rst +++ b/documentation/changelog.rst @@ -15,7 +15,8 @@ v1.0.0 | July XX, 2026 New features ------------- -* New ``inflexible-consumption`` and ``inflexible-production`` flex-context fields make explicit how the sign of each inflexible device's power data should be read (positive values denote consumption resp. production), accepting sensor references with optional source filters; they replace the now-deprecated ``inflexible-device-sensors`` field (bare sensor IDs, sign read from each sensor's ``consumption_is_positive`` attribute), which remains supported. Each inflexible device may also carry a ``group`` field, so that inflexible (measured) load counts towards the intermediate power constraint of the group it belongs to [see `PR #2358 `_] +* New ``inflexible-consumption`` and ``inflexible-production`` flex-context fields make explicit how the sign of each inflexible device's power data should be read (positive values denote consumption resp. production), accepting sensor references with optional source filters; they replace the now-deprecated ``inflexible-device-sensors`` field (bare sensor IDs, sign read from each sensor's ``consumption_is_positive`` attribute), which remains supported [see `PR #2358 `_] +* An inflexible (unschedulable) device can be modelled as its own asset by giving its flex-model entry a single ``inflexible-consumption`` or ``inflexible-production`` sensor reference; such a device joins a ``group`` like any other member, so its fixed (measured) load counts towards the group's intermediate power constraint [see `PR #2371 `_] * Forecasting regressors can filter their input beliefs by data source, source type, excluded source type, or source organisation [see `PR #2347 `_] * When multiple selected sources record a belief about the same event at the same belief time, forecasting pipelines now resolve the collision deterministically: the order of an explicit ``sources`` list decides precedence (first listed wins), and otherwise the highest source ID wins after selecting the latest version within each source family [see `PR #2347 `_] * Filter organisations by account role in the Accounts API and organisation list UI [see `PR #2353 `_] diff --git a/documentation/features/scheduling.rst b/documentation/features/scheduling.rst index cdbc5577cd..5b889d8627 100644 --- a/documentation/features/scheduling.rst +++ b/documentation/features/scheduling.rst @@ -298,6 +298,12 @@ For more details on the possible formats for field values, see :ref:`variable_qu * - ``group`` - |GROUP.example| - .. include:: ../_autodoc/GROUP.rst + * - ``inflexible-consumption`` + - ``{"sensor": 3}`` + - .. include:: ../_autodoc/INFLEXIBLE_CONSUMPTION.rst + * - ``inflexible-production`` + - ``{"sensor": 3}`` + - .. include:: ../_autodoc/INFLEXIBLE_PRODUCTION.rst .. [#quantity_field] Can only be set as a fixed quantity. @@ -349,7 +355,7 @@ The sensor-referenced form is convenient when you pass the whole flex-model in o Here, the battery and PV installation may each individually schedule up to 2 kW, but their combined power flowing through the shared inverter is hard-limited to 2.5 kW. -Inflexible (measured) devices can be group members too. An ``inflexible-consumption`` or ``inflexible-production`` entry in the flex-context may carry the same ``group`` field, so that its fixed load or supply counts towards the group's intermediate power constraint — for example, an unschedulable base load sitting behind the same inverter or feeder as a battery. As with flexible members, the recommended form is an ``{"asset": }`` reference to the equipment node the device sits behind (a ``{"sensor": }`` reference is also accepted), and the group's own flex-model entry (defining its capacities) must still be present. +Inflexible (measured) devices can be group members too — for example, an unschedulable base load sitting behind the same inverter or feeder as a battery. To do so, model the inflexible device as its own asset and give its flex-model entry a single ``inflexible-consumption`` or ``inflexible-production`` reference to the sensor recording its power (the field name sets the sign convention, and source filters may be added). Such an entry carries no schedulable-device fields; it simply declares a fixed device whose power is accounted for. It then joins a group through the ordinary ``group`` field, exactly like a flexible member (with the group's own flex-model entry, defining its capacities, still present), so its fixed load or supply counts towards the group's intermediate power constraint. Site-wide inflexible load that does not belong to any group is instead listed in the flex-context's ``inflexible-consumption``/``inflexible-production`` fields (as a list). Usually, not the whole flexibility model is needed. diff --git a/flexmeasures/data/models/planning/devices.py b/flexmeasures/data/models/planning/devices.py index 3faec52786..527d402746 100644 --- a/flexmeasures/data/models/planning/devices.py +++ b/flexmeasures/data/models/planning/devices.py @@ -84,7 +84,9 @@ class FlexDevice: role: DeviceRole #: Canonical solver device index; None for stock-only entries. index: int | None - #: The deserialized flex-model entry (with underscore keys); None for inflexible devices. + #: The deserialized flex-model entry (with underscore keys). Set for flexible, + #: stock-only, group and flex-model-sourced inflexible entries; None only for the + #: flat-list inflexible devices that come from the flex-context. flex_model: dict | None #: The device's power sensor, resolved from the entry's top-level "sensor" key, else from a nested consumption/production output reference. #: None for entries that reference no power sensor at all (e.g. asset-only entries). @@ -104,9 +106,6 @@ class FlexDevice: #: For inflexible devices given as a sensor reference with source filters: #: the SensorReference (a schema-layer object, hence untyped here); None otherwise. sensor_reference: Any | None = None - #: For inflexible devices assigned to a group: the raw ``{"sensor": ...}``/``{"asset": ...}`` - #: group reference (flexible devices carry theirs inside ``flex_model["group"]`` instead). - group_reference: Any | None = None @property def sensor_id(self) -> int | None: @@ -133,13 +132,12 @@ def production_sensor(self) -> Sensor | None: def group_key(self) -> tuple[str, int] | None: """The key of the group this entry belongs to (via its "group" field), if any. - Flexible/stock/group entries carry their group inside ``flex_model["group"]``; - inflexible devices (which have no flex-model entry) carry it on - ``group_reference`` instead. + Every kind of entry that can join a group carries its group inside + ``flex_model["group"]`` (including flex-model-sourced inflexible devices). + Flat-list inflexible devices from the flex-context have no flex-model entry, so + they never belong to a group. """ - if self.flex_model is not None: - return resolve_group_key(self.flex_model) - return resolve_group_reference(self.group_reference) + return resolve_group_key(self.flex_model) def _resolve_output_sensor(flex_model: dict | None, output_field: str) -> Sensor | None: @@ -208,9 +206,9 @@ def resolve_group_reference(group: Any | None) -> tuple[str, int] | None: """Return a normalized ("sensor", id) or ("asset", id) key for a "group" reference, or None. The reference may be a ``{"sensor": ...}``/``{"asset": ...}`` dict (as produced by - :class:`~flexmeasures.data.schemas.scheduling.groups.GroupReferenceSchema`, used - both by flex-model entries' ``group`` field and by inflexible-device flex-context - entries), or -- for backwards compatibility -- a raw sensor id/object. + :class:`~flexmeasures.data.schemas.scheduling.groups.GroupReferenceSchema`, used by + flex-model entries' ``group`` field), or -- for backwards compatibility -- a raw + sensor id/object. """ if not group: return None @@ -289,36 +287,6 @@ def _collect_referenced_group_keys( return referenced_group_keys -def _collect_inflexible_referenced_group_keys( - flex_context: dict, is_single_sensor_mode: bool -) -> set[tuple[str, int]]: - """Collect the group keys referenced by inflexible devices' "group" fields. - - Scans the top-level (electricity) inflexible devices and those of each commodity - context. A group referenced only by inflexible devices still needs its flex-model - entry classified as a group entry, so these keys are unioned into the inventory's - ``referenced_group_keys`` before the flex-model entries are classified. - - :raises ValueError: When an inflexible device carries a "group" field in - single-sensor mode (groups need a multi-device flex-model to - hold the group entry that defines the group's capacity). - """ - referenced_group_keys: set[tuple[str, int]] = set() - contexts = [flex_context, *flex_context.get("commodity_contexts", [])] - for context in contexts: - for field_name, _ in INFLEXIBLE_DEVICE_FIELDS: - for entry in context.get(field_name, []): - group_key = resolve_group_reference(getattr(entry, "group", None)) - if group_key is None: - continue - if is_single_sensor_mode: - raise ValueError( - "The 'group' field is only supported in multi-device flex-models." - ) - referenced_group_keys.add(group_key) - return referenced_group_keys - - def _classify_group_entry(inventory: DeviceInventory, fm: dict) -> bool: """Classify a flex-model entry as a group entry, if its own sensor/asset is referenced as a group. @@ -346,6 +314,70 @@ def _classify_group_entry(inventory: DeviceInventory, fm: dict) -> bool: return True +def _classify_flex_model_inflexible_entry(fm: dict) -> "FlexDevice | None": + """Classify a flex-model entry that declares an inflexible device, or return None. + + An inflexible device is declared by a single ``inflexible-consumption`` or + ``inflexible-production`` sensor reference on its own flex-model entry (typically + the flex-model of the inflexible device's asset). Its fixed power is accounted for + but not scheduled; its group membership (if any) is read from its own ``group`` + field, exactly like a flexible member. Schema validation guarantees at most one of + the two fields is present and that the entry carries no schedulable-device fields. + + The returned device's ``index`` is None; it is assigned into the inflexible tail + (after the flexible devices) once their count is known. + """ + for field_name, consumption_is_positive in ( + ("inflexible_consumption", True), + ("inflexible_production", False), + ): + entry_ref = fm.get(field_name) + if entry_ref is None: + continue + # Tolerate SensorReference-like objects (with a `.sensor`) and plain Sensors. + reference_sensor = getattr(entry_ref, "sensor", None) + if reference_sensor is not None: + sensor, sensor_reference = reference_sensor, entry_ref + else: + sensor, sensor_reference = entry_ref, None + return FlexDevice( + role=DeviceRole.INFLEXIBLE, + index=None, + flex_model=fm, + power_sensor=sensor, + asset=(getattr(sensor, "asset", None) or fm.get("asset")), + commodity=fm.get("commodity", "electricity"), + consumption_is_positive=consumption_is_positive, + sensor_reference=sensor_reference, + ) + return None + + +def _register_flex_model_inflexible( + inventory: DeviceInventory, + pending: list, + fm: dict, + is_single_sensor_mode: bool, +) -> bool: + """Classify and register a flex-model inflexible-device entry, if this is one. + + Groups (and the inflexible-device entries that join them) need a multi-device + flex-model, so this is a no-op in single-sensor mode. The registered device is + appended to ``inventory.entries`` now, and to ``pending`` for index assignment into + the inflexible tail once the flexible-device count is known. + + :returns: True if the entry was an inflexible device (and got registered). + """ + if is_single_sensor_mode: + return False + device = _classify_flex_model_inflexible_entry(fm) + if device is None: + return False + inventory.entries.append(device) + pending.append(device) + return True + + #: Flex-model fields that make a device entry (with a state-of-charge sensor) #: also carry the SoC parameters of its stock. SOC_PARAM_FIELDS = ("soc_at_start", "soc_min", "soc_max", "soc_targets") @@ -358,8 +390,10 @@ class DeviceInventory: The canonical device enumeration is: 1. flexible devices (flex-model entries with role DEVICE), in flex-model order, - 2. top-level (electricity) inflexible devices from the flex-context, in order, - 3. each commodity context's own inflexible devices, in the order the + 2. inflexible devices declared in the flex-model (as their own asset), in + flex-model order, + 3. top-level (electricity) inflexible devices from the flex-context, in order, + 4. each commodity context's own inflexible devices, in the order the commodity contexts are given. Within each context, inflexible devices are enumerated per field in @@ -375,7 +409,7 @@ class DeviceInventory: entries: list[FlexDevice] = field(default_factory=list) #: The schedulable devices; ``devices[d].index == d``. devices: list[FlexDevice] = field(default_factory=list) - #: The inflexible devices from the flex-context, with indices following the devices. + #: The inflexible devices (flex-model-declared and flex-context), with indices following the devices. inflexible_devices: list[FlexDevice] = field(default_factory=list) #: SoC parameters per stock key. Keys are shared with :attr:`stock_groups`. stock_entries: dict[int, dict] = field(default_factory=dict) @@ -407,15 +441,12 @@ def from_flex_config( inventory = cls(is_single_sensor_mode=is_single_sensor_mode) - # Collect the group keys referenced by entries' "group" fields (both flex-model - # entries and inflexible devices from the flex-context); the entries whose own - # sensor/asset matches a referenced key are classified as group entries below. - # Inflexible devices are folded in first, so a group referenced only by an - # inflexible device still gets its flex-model entry classified as a group entry. + # Collect the group keys referenced by flex-model entries' "group" fields; the + # entries whose own sensor/asset matches a referenced key are classified as group + # entries below. Flex-model-sourced inflexible devices carry their "group" the + # same way, so this one scan covers them too. inventory.referenced_group_keys = _collect_referenced_group_keys( flex_model_list, is_single_sensor_mode - ) | _collect_inflexible_referenced_group_keys( - flex_context, is_single_sensor_mode ) # One counter yields the synthetic stock keys for devices without a @@ -432,12 +463,26 @@ def register_stock_params(stock_key: int, fm: dict) -> None: ) inventory.stock_entries[stock_key] = fm + # Inflexible devices declared in the flex-model (as their own asset), collected + # here and assigned indices in the inflexible tail once the flexible-device count + # is known (see below). + pending_inflexible: list[FlexDevice] = [] + for fm in flex_model_list: # Group entry (multi-device mode only): this entry's own sensor/asset is # the aggregate sensor/asset referenced by another entry's "group" field. if _classify_group_entry(inventory, fm): continue + # Inflexible-device entry (multi-device mode only): declares a single + # inflexible-consumption/production reference, so it is an inflexible device + # (fixed power), not a schedulable one. Classify before the device logic so + # its power sensor doesn't make it pass for a schedulable device. + if _register_flex_model_inflexible( + inventory, pending_inflexible, fm, is_single_sensor_mode + ): + continue + if is_single_sensor_mode: power_sensor = sensor else: @@ -495,10 +540,15 @@ def register_stock_params(stock_key: int, fm: dict) -> None: inventory.entries.append(device) inventory.devices.append(device) - # Inflexible devices from the flex-context: top-level (electricity) sensors - # first, then each commodity context's own sensors, in context order. - # Within each context, fields are read in INFLEXIBLE_DEVICE_FIELDS order. + # Inflexible devices, in canonical order: those declared in the flex-model (in + # flex-model order) first, then the flat flex-context lists -- top-level + # (electricity) sensors, then each commodity context's own sensors, in context + # order (per-field in INFLEXIBLE_DEVICE_FIELDS order within each). index = len(inventory.devices) + for device in pending_inflexible: + device.index = index + inventory.inflexible_devices.append(device) + index += 1 index = inventory._register_inflexible_devices( flex_context, "electricity", index ) @@ -510,6 +560,10 @@ def register_stock_params(stock_key: int, fm: dict) -> None: assert all( device.index == d for d, device in enumerate(inventory.devices) ), "Device indices must match their position among the schedulable devices." + assert all( + device.index == len(inventory.devices) + i + for i, device in enumerate(inventory.inflexible_devices) + ), "Inflexible device indices must follow the flexible devices contiguously." return inventory def _register_inflexible_devices( @@ -528,10 +582,8 @@ def _register_inflexible_devices( reference_sensor = getattr(entry, "sensor", None) if reference_sensor is not None: sensor, sensor_reference = reference_sensor, entry - group_reference = getattr(entry, "group", None) else: sensor, sensor_reference = entry, None - group_reference = None self.inflexible_devices.append( FlexDevice( role=DeviceRole.INFLEXIBLE, @@ -542,7 +594,6 @@ def _register_inflexible_devices( commodity=commodity, consumption_is_positive=consumption_is_positive, sensor_reference=sensor_reference, - group_reference=group_reference, ) ) index += 1 @@ -598,10 +649,11 @@ def stock_constraint_device(self, stock_key: int) -> int | None: def group_to_devices(self) -> dict[tuple[str, int], list[int]]: """Map each group key to the indices of the (leaf) member devices of that group. - Members are the flexible devices *and* the inflexible devices (from the - flex-context) that reference the group. Including the latter is what lets a - group's intermediate power constraint also account for inflexible (measured) - load sitting behind the same node. + Members are the flexible devices *and* the flex-model-declared inflexible + devices that reference the group. Including the latter is what lets a group's + intermediate power constraint also account for inflexible (measured) load + sitting behind the same node. (Flat-list inflexible devices from the + flex-context have no flex-model entry, so they never belong to a group.) Membership is resolved transitively: a group entry may itself belong to another group (via its own "group" field), diff --git a/flexmeasures/data/models/planning/tests/test_device_inventory.py b/flexmeasures/data/models/planning/tests/test_device_inventory.py index 88930de5b9..1a67e54cb5 100644 --- a/flexmeasures/data/models/planning/tests/test_device_inventory.py +++ b/flexmeasures/data/models/planning/tests/test_device_inventory.py @@ -391,70 +391,69 @@ def test_inflexible_field_enumeration_and_signs(): assert inventory.inflexible_sensors == [legacy, load, pv] -def test_inflexible_device_group_membership(): - """An inflexible device carrying a ``group`` field becomes a member of that group, - so its (fixed) load counts towards the group's intermediate power constraint.""" - from flexmeasures.data.schemas.sensors import SensorReference - +def test_flex_model_inflexible_device_classified(): + """A flex-model entry declaring a single inflexible-consumption/production reference + is classified as an inflexible device (not a schedulable one), in the inflexible + tail after the flexible devices.""" battery = make_sensor(1) load = make_sensor(12) - group_sensor = make_sensor(10) - group_entry = {"sensor": group_sensor, "power_capacity_in_mw": 0.001} + solar = make_sensor(13) inventory = DeviceInventory.from_flex_config( [ - {"sensor": battery, "group": {"sensor": group_sensor}}, - group_entry, - ], - { - "inflexible_consumption": [ - SensorReference(sensor=load, group={"sensor": group_sensor}) - ] - }, + {"sensor": battery}, + {"asset": object(), "inflexible_consumption": load}, + {"asset": object(), "inflexible_production": solar}, + ] ) + assert [entry.role for entry in inventory.entries] == [ + DeviceRole.DEVICE, + DeviceRole.INFLEXIBLE, + DeviceRole.INFLEXIBLE, + ] assert inventory.num_flexible == 1 - # The inflexible load follows the flexible battery (index 0), at index 1. - assert [device.index for device in inventory.inflexible_devices] == [1] - assert inventory.group_entries == {("sensor", group_sensor.id): group_entry} - # The group contains both the flexible battery (0) and the inflexible load (1). - assert inventory.group_to_devices == {("sensor", group_sensor.id): [0, 1]} - assert inventory.by_index(1).commodity == "electricity" - + assert inventory.num_scheduled == 3 + # Inflexible devices follow the flexible battery (index 0), at indices 1, 2. + assert [d.index for d in inventory.inflexible_devices] == [1, 2] + assert [d.sensor_id for d in inventory.inflexible_devices] == [12, 13] + # The sign convention comes from the field the device was declared under. + assert [d.consumption_is_positive for d in inventory.inflexible_devices] == [ + True, + False, + ] -def test_group_referenced_only_by_inflexible_device(): - """A group referenced solely by an inflexible device still gets its flex-model - entry classified as a group entry, with the inflexible device as its member.""" - from flexmeasures.data.schemas.sensors import SensorReference - battery = make_sensor(1) # not in the group +def test_flex_model_inflexible_device_group_membership(): + """A flex-model inflexible entry joins a group through the normal ``group`` field, + exactly like a flexible member, so its fixed load counts towards the group.""" + battery = make_sensor(1) load = make_sensor(12) group_sensor = make_sensor(10) group_entry = {"sensor": group_sensor, "power_capacity_in_mw": 0.001} inventory = DeviceInventory.from_flex_config( [ - {"sensor": battery}, + {"sensor": battery, "group": {"sensor": group_sensor}}, + { + "asset": object(), + "inflexible_consumption": load, + "group": {"sensor": group_sensor}, + }, group_entry, - ], - { - "inflexible_consumption": [ - SensorReference(sensor=load, group={"sensor": group_sensor}) - ] - }, + ] ) - assert [entry.role for entry in inventory.entries] == [ - DeviceRole.DEVICE, - DeviceRole.GROUP, - ] - assert inventory.group_to_devices == {("sensor", group_sensor.id): [1]} + assert inventory.num_flexible == 1 + assert inventory.group_entries == {("sensor", group_sensor.id): group_entry} + # The group contains both the flexible battery (0) and the inflexible load (1). + assert inventory.group_to_devices == {("sensor", group_sensor.id): [0, 1]} + assert inventory.by_index(1).group_key == ("sensor", group_sensor.id) -def test_inflexible_device_asset_group_membership(): - """An inflexible device may reference its group by asset, mirroring the asset-tree - way of defining groups (see PR #2277).""" - from flexmeasures.data.schemas.sensors import SensorReference +def test_flex_model_inflexible_device_asset_group(): + """The group may be referenced by asset, mirroring the asset-tree way of defining + groups (PR #2277).""" class _FakeAsset: def __init__(self, asset_id: int): @@ -468,39 +467,36 @@ def __init__(self, asset_id: int): inventory = DeviceInventory.from_flex_config( [ {"sensor": battery, "group": {"asset": inverter}}, + { + "asset": object(), + "inflexible_production": load, + "group": {"asset": inverter}, + }, group_entry, - ], - { - "inflexible_production": [ - SensorReference(sensor=load, group={"asset": inverter}) - ] - }, + ] ) assert inventory.group_to_devices == {("asset", inverter.id): [0, 1]} -def test_inflexible_group_rejected_in_single_sensor_mode(): - """Groups need a multi-device flex-model to hold the group entry, so an inflexible - device with a ``group`` field in single-sensor mode is rejected.""" - from flexmeasures.data.schemas.sensors import SensorReference - - with pytest.raises(ValueError, match="multi-device"): - DeviceInventory.from_flex_config( - {"soc_at_start": 0.0}, - { - "inflexible_consumption": [ - SensorReference( - sensor=make_sensor(12), group={"sensor": make_sensor(10)} - ) - ] - }, - sensor=make_sensor(1), - ) +def test_flex_model_inflexible_precedes_flat_context_inflexible(): + """Canonical order: flex-model inflexible devices come before the flat-list + inflexible devices from the flex-context.""" + inventory = DeviceInventory.from_flex_config( + [ + {"sensor": make_sensor(1)}, + {"asset": object(), "inflexible_consumption": make_sensor(12)}, + ], + flex_context={"inflexible_consumption": [make_sensor(20)]}, + ) + # index 0 = flexible; 1 = flex-model inflexible; 2 = flat-context inflexible. + assert [d.sensor_id for d in inventory.inflexible_devices] == [12, 20] + assert [d.index for d in inventory.inflexible_devices] == [1, 2] -def test_inflexible_device_without_group_has_no_membership(): - """A plain-sensor inflexible device (no group) forms no group membership.""" +def test_flat_context_inflexible_device_has_no_group(): + """A flat-list inflexible device from the flex-context has no flex-model entry, so + it never belongs to a group.""" inventory = DeviceInventory.from_flex_config( [{"sensor": make_sensor(1)}], {"inflexible_consumption": [make_sensor(12)]}, diff --git a/flexmeasures/data/models/planning/tests/test_group_constraints.py b/flexmeasures/data/models/planning/tests/test_group_constraints.py index 95957f0246..88f8e119c7 100644 --- a/flexmeasures/data/models/planning/tests/test_group_constraints.py +++ b/flexmeasures/data/models/planning/tests/test_group_constraints.py @@ -8,7 +8,6 @@ from flexmeasures.data.models.generic_assets import GenericAsset, GenericAssetType from flexmeasures.data.models.time_series import Sensor, TimedBelief from flexmeasures.data.models.planning.storage import StorageScheduler -from flexmeasures.data.schemas.sensors import SensorReference from flexmeasures.utils.unit_utils import ur @@ -197,9 +196,10 @@ def _write_constant_forecast(db, sensor, value_mw: float, hours: int = 4): def test_inflexible_device_counts_towards_group(db, building): - """Point 5 of the #2276 discussion: an inflexible (measured) load assigned to a - group counts towards that group's intermediate power constraint, so the flexible - members get less headroom than the group cap alone would give them. + """Point 5 of the #2276 discussion: an inflexible (measured) load modelled as its + own asset (a flex-model entry with a single ``inflexible-consumption`` reference) + and assigned to a group counts towards that group's intermediate power constraint, + so the flexible members get less headroom than the group cap alone would give them. A battery that is paid to consume would use its full 2 MW capacity, but a 2 MW group cap shared with a fixed 1 MW inflexible load leaves it only 1 MW. @@ -209,39 +209,40 @@ def test_inflexible_device_counts_towards_group(db, building): load = make_group_sensor(db, building) # a 1 MW inflexible consumer _write_constant_forecast(db, load, value_mw=1.0) - flex_model = [ - { - "sensor": battery, - "soc_at_start": 0.0, - "soc_min": 0.0, - "soc_max": 100.0, - "power_capacity_in_mw": ur.Quantity("2 MW"), - "consumption_capacity": ur.Quantity("2 MW"), - "production_capacity": ur.Quantity("2 MW"), - "group": {"sensor": group_sensor}, - }, - {"sensor": group_sensor, "power_capacity_in_mw": ur.Quantity("2 MW")}, - ] + flex_context = { + "consumption_price": ur.Quantity("-100 EUR/MWh"), # paid to consume + "production_price": ur.Quantity("100 EUR/MWh"), + "shared_currency_unit": "EUR", + } - def flex_context_with(load_in_group: bool): - return { - "consumption_price": ur.Quantity("-100 EUR/MWh"), # paid to consume - "production_price": ur.Quantity("100 EUR/MWh"), - "shared_currency_unit": "EUR", - "inflexible_consumption": [ - SensorReference( - sensor=load, - group={"sensor": group_sensor} if load_in_group else None, - ) - ], - } + def flex_model(load_in_group: bool): + # The inflexible load is a flex-model entry declaring a single + # inflexible-consumption reference; it joins the group through the normal + # `group` field, exactly like the flexible battery. + load_entry = {"inflexible_consumption": load} + if load_in_group: + load_entry["group"] = {"sensor": group_sensor} + return [ + { + "sensor": battery, + "soc_at_start": 0.0, + "soc_min": 0.0, + "soc_max": 100.0, + "power_capacity_in_mw": ur.Quantity("2 MW"), + "consumption_capacity": ur.Quantity("2 MW"), + "production_capacity": ur.Quantity("2 MW"), + "group": {"sensor": group_sensor}, + }, + load_entry, + {"sensor": group_sensor, "power_capacity_in_mw": ur.Quantity("2 MW")}, + ] # With the load in the group, battery consumption is capped at 1 MW (2 MW group # cap minus the fixed 1 MW load), and the saved aggregate includes the load. results = run_scheduler( building, - flex_model, - flex_context_with(True), + flex_model(load_in_group=True), + flex_context, return_multiple=True, ).compute() schedules = { @@ -258,8 +259,8 @@ def flex_context_with(load_in_group: bool): # Control: with the same load NOT in the group, the battery uses the full 2 MW. results_control = run_scheduler( building, - flex_model, - flex_context_with(False), + flex_model(load_in_group=False), + flex_context, return_multiple=True, ).compute() schedules_control = { diff --git a/flexmeasures/data/schemas/scheduling/__init__.py b/flexmeasures/data/schemas/scheduling/__init__.py index dfa2c43775..980878efe0 100644 --- a/flexmeasures/data/schemas/scheduling/__init__.py +++ b/flexmeasures/data/schemas/scheduling/__init__.py @@ -25,15 +25,11 @@ VariableQuantityField, SensorIdField, SensorReference, - SensorReferenceSchema, + InflexibleDeviceSchema, OutputSensorReferenceSchema, PriceField, ) from flexmeasures.data.schemas.scheduling import metadata -from flexmeasures.data.schemas.scheduling.groups import ( - GroupReferenceSchema, - validate_group_sensor_is_power_sensor, -) from flexmeasures.data.schemas.units import UnitField from flexmeasures.utils.doc_utils import rst_to_openapi from flexmeasures.data.schemas.times import ( @@ -184,54 +180,6 @@ class DBCommitmentSchema(CommitmentSchema, NoTimeSeriesSpecs): pass -class InflexibleDeviceSchema(SensorReferenceSchema): - """One inflexible device: a sensor reference with optional source filters, - optionally assigned to a group (so its measured load counts towards that group's - intermediate power constraint).""" - - class Meta: - description = "Sensor reference from which to look up an inflexible device's power (or energy) data." - - group = fields.Nested( - GroupReferenceSchema, - data_key="group", - required=False, - metadata=metadata.GROUP.to_dict(), - ) - - @validates("group") - def validate_group(self, group: dict, **kwargs): - validate_group_sensor_is_power_sensor(group) - - @post_load - def to_sensor_or_reference(self, data: dict, **kwargs) -> Sensor | SensorReference: - """Return a plain Sensor when neither source filters nor a group are given - (backward-compatible shape downstream), and a SensorReference otherwise (see - VariableQuantityField._deserialize_dict). A group is carried on the - SensorReference (belief queries ignore it) so the device inventory can resolve - the device's group membership. - """ - group = data.get("group") - if group is None and not any( - data.get(key) - for key in ( - "source_types", - "exclude_source_types", - "sources", - "source_account", - ) - ): - return data["sensor"] - return SensorReference( - sensor=data["sensor"], - source_types=data.get("source_types"), - exclude_source_types=data.get("exclude_source_types"), - sources=data.get("sources"), - source_account=data.get("source_account"), - group=group, - ) - - class SharedSchema(Schema): """Shared schema for fields common across commodities in flex-context and commodity-context.""" @@ -1428,6 +1376,24 @@ def check_prices(self, data: dict, original_data: dict, **kwargs): }, "example-units": EXAMPLE_UNIT_TYPES["commodity"], }, + "inflexible-consumption": { + "default": None, + "description": rst_to_openapi(metadata.INFLEXIBLE_CONSUMPTION.description), + "types": { + "backend": "typeTwo", + "ui": "A sensor recording this inflexible device's power (positive is consumption).", + }, + "example-units": EXAMPLE_UNIT_TYPES["power"], + }, + "inflexible-production": { + "default": None, + "description": rst_to_openapi(metadata.INFLEXIBLE_PRODUCTION.description), + "types": { + "backend": "typeTwo", + "ui": "A sensor recording this inflexible device's power (positive is production).", + }, + "example-units": EXAMPLE_UNIT_TYPES["power"], + }, } diff --git a/flexmeasures/data/schemas/scheduling/metadata.py b/flexmeasures/data/schemas/scheduling/metadata.py index 8959e18149..fdd9a8a1d4 100644 --- a/flexmeasures/data/schemas/scheduling/metadata.py +++ b/flexmeasures/data/schemas/scheduling/metadata.py @@ -49,7 +49,7 @@ def to_dict(self): The sign convention is determined by the key name: positive values denote consumption. Sensors that explicitly record consumption as negative values (``consumption_is_positive`` attribute set to false) are rejected here; list them under ``inflexible-production`` instead. -Must be a list of sensor references, optionally with source filters, and optionally each assigned to a ``group`` (so the device's measured load counts towards that group's intermediate power constraint). +Each entry is a sensor reference, optionally with source filters. In the flex-context this is a list of such references (site-level base load); in a flex-model entry it is a single reference, so that an inflexible device modelled as its own asset can join a ``group`` like any other member. """, example=[{"sensor": 3}, {"sensor": 4}], ) @@ -61,7 +61,7 @@ def to_dict(self): The sign convention is determined by the key name: positive values denote production (the FlexMeasures default). Sensors that explicitly record production as negative values (``consumption_is_positive`` attribute set to true) are rejected here; list them under ``inflexible-consumption`` instead. -Must be a list of sensor references, optionally with source filters, and optionally each assigned to a ``group`` (so the device's measured supply counts towards that group's intermediate power constraint). +Each entry is a sensor reference, optionally with source filters. In the flex-context this is a list of such references (site-level base generation); in a flex-model entry it is a single reference, so that an inflexible device modelled as its own asset can join a ``group`` like any other member. """, example=[{"sensor": 3}, {"sensor": 4}], ) diff --git a/flexmeasures/data/schemas/scheduling/storage.py b/flexmeasures/data/schemas/scheduling/storage.py index 590645a6ce..216ad4953d 100644 --- a/flexmeasures/data/schemas/scheduling/storage.py +++ b/flexmeasures/data/schemas/scheduling/storage.py @@ -24,6 +24,7 @@ ) from flexmeasures.data.schemas.sensors import ( SensorReference, + InflexibleDeviceSchema, OutputSensorReferenceSchema, VariableQuantityField, ) @@ -36,6 +37,81 @@ ALLOWED_COMMODITIES = {"electricity", "gas"} +#: User-facing (hyphenated) flex-model keys that make an entry a schedulable device; +#: an entry that declares an inflexible device must carry none of them, so +#: device-inventory classification stays unambiguous. Keyed on data-keys (consistent +#: across StorageFlexModelSchema and DBStorageFlexModelSchema, whose attribute names +#: differ). ``group``, ``commodity``, ``asset``, ``sensor`` and the activation +#: preferences may co-exist with an inflexible declaration. +_SCHEDULABLE_DEVICE_DATA_KEYS = ( + "consumption", + "production", + "state-of-charge", + "soc-at-start", + "soc-min", + "soc-max", + "soc-minima", + "soc-maxima", + "soc-targets", + "soc-gain", + "soc-usage", + "power-capacity", + "consumption-capacity", + "production-capacity", + "roundtrip-efficiency", + "charging-efficiency", + "discharging-efficiency", + "storage-efficiency", + "operation-modes", +) + + +def validate_inflexible_flex_model_entry(data: dict, original_data: dict): + """Validate a flex-model entry that declares an inflexible device. + + An inflexible device is declared by a single ``inflexible-consumption`` or + ``inflexible-production`` sensor reference. Such an entry must not declare both + signs, must not use a sensor whose explicit ``consumption_is_positive`` attribute + contradicts the field's sign convention, and must not also carry schedulable-device + fields (so it is unambiguously classified as an inflexible device). The last check + inspects the original (hyphenated) input keys, so it works for both flex-model + schemas and ignores load-default fills. + """ + has_consumption = "inflexible_consumption" in data + has_production = "inflexible_production" in data + if not has_consumption and not has_production: + return + if has_consumption and has_production: + raise ValidationError( + "An inflexible device entry must declare either inflexible-consumption or" + " inflexible-production, not both.", + field_name="inflexible-consumption", + ) + field, data_key, consumption_is_positive = ( + ("inflexible_consumption", "inflexible-consumption", True) + if has_consumption + else ("inflexible_production", "inflexible-production", False) + ) + entry = data[field] + sensor = entry.sensor if isinstance(entry, SensorReference) else entry + explicit_attribute = (getattr(sensor, "attributes", None) or {}).get( + "consumption_is_positive" + ) + if explicit_attribute is not None and explicit_attribute != consumption_is_positive: + raise ValidationError( + f"Sensor {sensor.id} has `consumption_is_positive={explicit_attribute}`," + f" which conflicts with the sign convention of the `{data_key}` field.", + field_name=data_key, + ) + offending = [k for k in _SCHEDULABLE_DEVICE_DATA_KEYS if k in original_data] + if offending: + raise ValidationError( + f"An inflexible device entry (`{data_key}`) must not also carry" + f" schedulable-device field(s) {offending}.", + field_name=data_key, + ) + + # Telling type hints what to expect after schema parsing SoCTarget = TypedDict( "SoCTarget", @@ -289,6 +365,22 @@ class StorageFlexModelSchema(Schema): metadata=metadata.GROUP.to_dict(), ) + # An entry may instead describe an inflexible (unschedulable) device: a single + # sensor reference whose fixed power is accounted for (and, via `group`, counted + # towards a group's intermediate power constraint), rather than being scheduled. + inflexible_consumption = fields.Nested( + InflexibleDeviceSchema, + data_key="inflexible-consumption", + required=False, + metadata=metadata.INFLEXIBLE_CONSUMPTION.to_dict(), + ) + inflexible_production = fields.Nested( + InflexibleDeviceSchema, + data_key="inflexible-production", + required=False, + metadata=metadata.INFLEXIBLE_PRODUCTION.to_dict(), + ) + # Activation prices prefer_curtailing_later = fields.Bool( data_key="prefer-curtailing-later", @@ -461,6 +553,10 @@ def validate_state_of_charge( def validate_group(self, group: dict, **kwargs): validate_group_sensor_is_power_sensor(group) + @validates_schema(pass_original=True) + def validate_inflexible_device(self, data: dict, original_data: dict, **kwargs): + validate_inflexible_flex_model_entry(data, original_data) + @validates("asset") def validate_asset(self, asset: Asset, **kwargs): if self.sensor is not None and self.sensor.asset != asset: @@ -555,6 +651,19 @@ class DBStorageFlexModelSchema(Schema): metadata=metadata.GROUP.to_dict(), ) + inflexible_consumption = fields.Nested( + InflexibleDeviceSchema, + data_key="inflexible-consumption", + required=False, + metadata=metadata.INFLEXIBLE_CONSUMPTION.to_dict(), + ) + inflexible_production = fields.Nested( + InflexibleDeviceSchema, + data_key="inflexible-production", + required=False, + metadata=metadata.INFLEXIBLE_PRODUCTION.to_dict(), + ) + soc_min = VariableQuantityField( to_unit="MWh", data_key="soc-min", @@ -699,6 +808,10 @@ def __init__(self, *args, **kwargs): def validate_group(self, group: dict, **kwargs): validate_group_sensor_is_power_sensor(group) + @validates_schema(pass_original=True) + def validate_inflexible_device(self, data: dict, original_data: dict, **kwargs): + validate_inflexible_flex_model_entry(data, original_data) + @validates_schema def forbid_time_series_specs(self, data: dict, **kwargs): """Do not allow time series specs for the flex-model fields saved in the db.""" diff --git a/flexmeasures/data/schemas/sensors.py b/flexmeasures/data/schemas/sensors.py index cac24cff6d..875844cb58 100644 --- a/flexmeasures/data/schemas/sensors.py +++ b/flexmeasures/data/schemas/sensors.py @@ -970,12 +970,6 @@ class SensorReference: properties works without modification. The source filters are passed through to :meth:`TimedBelief.search ` in :func:`~flexmeasures.data.models.planning.utils.get_series_from_quantity_or_sensor`. - - The optional ``group`` carries an inflexible device's group membership (a - ``{"sensor": ...}`` / ``{"asset": ...}`` reference, see - :class:`~flexmeasures.data.schemas.scheduling.groups.GroupReferenceSchema`); it is - only meaningful for inflexible-device flex-context entries and is ignored by belief - queries. """ sensor: Sensor @@ -983,10 +977,6 @@ class SensorReference: exclude_source_types: list[str] | None = field(default=None) sources: list[DataSource] | None = field(default=None) source_account: list[Account] | None = field(default=None) - #: Inflexible-device group membership (a ``{"sensor": ...}``/``{"asset": ...}`` - #: reference); ignored by belief queries. None when the reference is not an - #: inflexible device or does not belong to a group. - group: dict | None = field(default=None) @property def unit(self) -> str: @@ -1059,6 +1049,42 @@ class Meta: ) +class InflexibleDeviceSchema(SensorReferenceSchema): + """One inflexible device: a sensor reference with optional source filters. + + Used both in the flex-context (as a list, for site-level inflexible load) and in a + flex-model entry (as a single reference, when an inflexible device is modelled as + its own asset). Deserializes to a plain :class:`Sensor` when no source filters are + given (a backward-compatible shape downstream) and to a :class:`SensorReference` + otherwise. + """ + + class Meta: + description = "Sensor reference from which to look up an inflexible device's power (or energy) data." + + @post_load + def to_sensor_or_reference( + self, data: dict, **kwargs + ) -> "Sensor | SensorReference": + if not any( + data.get(key) + for key in ( + "source_types", + "exclude_source_types", + "sources", + "source_account", + ) + ): + return data["sensor"] + return SensorReference( + sensor=data["sensor"], + source_types=data.get("source_types"), + exclude_source_types=data.get("exclude_source_types"), + sources=data.get("sources"), + source_account=data.get("source_account"), + ) + + class SensorIdOrReferenceField(fields.Raw): """Field accepting either a sensor ID or a source-filtered sensor reference.""" diff --git a/flexmeasures/data/schemas/tests/test_scheduling.py b/flexmeasures/data/schemas/tests/test_scheduling.py index 21eff55735..59851ff992 100644 --- a/flexmeasures/data/schemas/tests/test_scheduling.py +++ b/flexmeasures/data/schemas/tests/test_scheduling.py @@ -1707,64 +1707,61 @@ def test_flex_context_schema_inflexible_devices_deserialization( assert reference.source_types == ["forecaster"] -def test_flex_context_schema_inflexible_device_group( - db, app, setup_inflexible_sensors, setup_price_sensors, dummy_asset +def test_storage_flex_model_inflexible_device_field( + db, app, setup_dummy_sensors, setup_inflexible_sensors ): - """An inflexible device may carry a `group` reference; it deserializes to a - SensorReference (even without source filters) carrying that group, and the group - must reference a power sensor (or an asset).""" - load = setup_inflexible_sensors["attributeless power"] - group_sensor = setup_inflexible_sensors["consumption-positive power"] + """A flex-model entry can declare an inflexible device via a single + inflexible-consumption/production reference. Both signs on one entry, a sensor + whose explicit sign contradicts the field, and a co-existing schedulable field are + all rejected.""" + consumption_positive = setup_inflexible_sensors["consumption-positive power"] + production_positive = setup_inflexible_sensors["production-positive power"] + attributeless = setup_inflexible_sensors["attributeless power"] - data = FlexContextSchema().load( - { - "inflexible-consumption": [ - {"sensor": load.id, "group": {"sensor": group_sensor.id}} - ] - } - ) - entry = data["inflexible_consumption"][0] - assert isinstance(entry, SensorReference) - assert entry.sensor == load - assert entry.group == {"sensor": group_sensor} - assert entry.source_types is None # no source filters given - - # An asset-referenced group is also accepted. - data_asset = FlexContextSchema().load( - { - "inflexible-production": [ - {"sensor": load.id, "group": {"asset": dummy_asset.id}} - ] - } - ) - assert data_asset["inflexible_production"][0].group == {"asset": dummy_asset} + for schema in ( + StorageFlexModelSchema( + start=datetime(2026, 6, 1, tzinfo=pytz.UTC), sensor=None + ), + DBStorageFlexModelSchema(), + ): + # A plain reference deserializes to a Sensor. + loaded = schema.load({"inflexible-consumption": {"sensor": attributeless.id}}) + assert loaded["inflexible_consumption"] == attributeless - # The group must reference a power sensor. - price_sensor = setup_price_sensors["consumption-price in SEK/kWh"] - with pytest.raises(ValidationError, match="power unit"): - FlexContextSchema().load( + # A source-filtered reference deserializes to a SensorReference. + loaded = schema.load( { - "inflexible-consumption": [ - {"sensor": load.id, "group": {"sensor": price_sensor.id}} - ] + "inflexible-production": { + "sensor": attributeless.id, + "source-types": ["forecaster"], + } } ) + assert isinstance(loaded["inflexible_production"], SensorReference) - # Exactly one of 'sensor'/'asset' must be given. - with pytest.raises(ValidationError, match="exactly one"): - FlexContextSchema().load( - { - "inflexible-consumption": [ - { - "sensor": load.id, - "group": { - "sensor": group_sensor.id, - "asset": dummy_asset.id, - }, - } - ] - } - ) + # Declaring both signs on one entry is rejected. + with pytest.raises(ValidationError, match="not both"): + schema.load( + { + "inflexible-consumption": {"sensor": attributeless.id}, + "inflexible-production": {"sensor": attributeless.id}, + } + ) + + # A sensor whose explicit consumption_is_positive contradicts the field. + with pytest.raises(ValidationError, match="conflicts with the sign convention"): + schema.load({"inflexible-consumption": {"sensor": production_positive.id}}) + with pytest.raises(ValidationError, match="conflicts with the sign convention"): + schema.load({"inflexible-production": {"sensor": consumption_positive.id}}) + + # An inflexible entry must not also carry a schedulable-device field. + with pytest.raises(ValidationError, match="schedulable-device field"): + schema.load( + { + "inflexible-consumption": {"sensor": attributeless.id}, + "power-capacity": "1 MW", + } + ) def test_db_flex_context_schema_inflexible_devices( diff --git a/flexmeasures/ui/static/openapi-specs.json b/flexmeasures/ui/static/openapi-specs.json index bd61b2fa80..3e09b0cbc6 100644 --- a/flexmeasures/ui/static/openapi-specs.json +++ b/flexmeasures/ui/static/openapi-specs.json @@ -4941,18 +4941,6 @@ ], "additionalProperties": false }, - "GroupReference": { - "type": "object", - "properties": { - "sensor": { - "type": "integer" - }, - "asset": { - "type": "integer" - } - }, - "description": "Reference to a group of devices whose aggregate power is constrained." - }, "InflexibleDevice": { "type": "object", "properties": { @@ -5003,13 +4991,6 @@ "items": { "type": "integer" } - }, - "group": { - "description": "Reference to a group of devices whose aggregate power is constrained. The recommended form identifies the group by the ``asset`` that represents the shared equipment (``{\"asset\": }``, e.g. a sub-EMS asset in the tree); a power ``sensor`` (``{\"sensor\": }``) is also accepted - give exactly one of the two.\nThe referenced asset or sensor should itself get its own flex-model entry defining the group's ``power-capacity`` (hard constraint) and/or ``consumption-capacity``/``production-capacity`` (soft constraints with default breach prices).\nWhen the group is referenced by ``asset``, the group entry defines no power sensor of its own; the group's aggregate power is saved via that entry's own ``consumption`` and/or ``production`` output sensors, following the usual output-sensor conventions.\nWhen the group is referenced by ``sensor``, the group's scheduled aggregate power is saved directly to that group sensor.\n", - "example": { - "asset": 7 - }, - "$ref": "#/components/schemas/GroupReference" } }, "required": [ @@ -5179,7 +5160,7 @@ }, "inflexible-consumption": { "type": "array", - "description": "Power (or energy) sensors representing loads that are relevant, but not flexible in the timing of their demand.\nFor example, a sensor recording the power of a base load that is connected behind the main meter, and whose consumption falls under the same contract as the flexible device(s) being scheduled.\nTheir power demand cannot be adjusted but still matters for finding the best schedule for other devices.\n\nThe sign convention is determined by the key name: positive values denote consumption.\nSensors that explicitly record consumption as negative values (consumption_is_positive attribute set to false) are rejected here; list them under inflexible-production instead.\n\nMust be a list of sensor references, optionally with source filters, and optionally each assigned to a group (so the device's measured load counts towards that group's intermediate power constraint).\n", + "description": "Power (or energy) sensors representing loads that are relevant, but not flexible in the timing of their demand.\nFor example, a sensor recording the power of a base load that is connected behind the main meter, and whose consumption falls under the same contract as the flexible device(s) being scheduled.\nTheir power demand cannot be adjusted but still matters for finding the best schedule for other devices.\n\nThe sign convention is determined by the key name: positive values denote consumption.\nSensors that explicitly record consumption as negative values (consumption_is_positive attribute set to false) are rejected here; list them under inflexible-production instead.\n\nEach entry is a sensor reference, optionally with source filters. In the flex-context this is a list of such references (site-level base load); in a flex-model entry it is a single reference, so that an inflexible device modelled as its own asset can join a group like any other member.\n", "example": [ { "sensor": 3 @@ -5194,7 +5175,7 @@ }, "inflexible-production": { "type": "array", - "description": "Power (or energy) sensors representing generators that are relevant, but not flexible in the timing of their supply.\nFor example, a sensor recording rooftop solar power that is connected behind the main meter, and whose production falls under the same contract as the flexible device(s) being scheduled.\nTheir power supply cannot be adjusted but still matters for finding the best schedule for other devices.\n\nThe sign convention is determined by the key name: positive values denote production (the FlexMeasures default).\nSensors that explicitly record production as negative values (consumption_is_positive attribute set to true) are rejected here; list them under inflexible-consumption instead.\n\nMust be a list of sensor references, optionally with source filters, and optionally each assigned to a group (so the device's measured supply counts towards that group's intermediate power constraint).\n", + "description": "Power (or energy) sensors representing generators that are relevant, but not flexible in the timing of their supply.\nFor example, a sensor recording rooftop solar power that is connected behind the main meter, and whose production falls under the same contract as the flexible device(s) being scheduled.\nTheir power supply cannot be adjusted but still matters for finding the best schedule for other devices.\n\nThe sign convention is determined by the key name: positive values denote production (the FlexMeasures default).\nSensors that explicitly record production as negative values (consumption_is_positive attribute set to true) are rejected here; list them under inflexible-consumption instead.\n\nEach entry is a sensor reference, optionally with source filters. In the flex-context this is a list of such references (site-level base generation); in a flex-model entry it is a single reference, so that an inflexible device modelled as its own asset can join a group like any other member.\n", "example": [ { "sensor": 3 @@ -6768,6 +6749,18 @@ } ] }, + "GroupReference": { + "type": "object", + "properties": { + "sensor": { + "type": "integer" + }, + "asset": { + "type": "integer" + } + }, + "description": "Reference to a group of devices whose aggregate power is constrained." + }, "StorageFlexModelSchemaOpenAPI": { "type": "object", "properties": { @@ -6856,6 +6849,30 @@ }, "$ref": "#/components/schemas/GroupReference" }, + "inflexible-consumption": { + "description": "Power (or energy) sensors representing loads that are relevant, but not flexible in the timing of their demand.\nFor example, a sensor recording the power of a base load that is connected behind the main meter, and whose consumption falls under the same contract as the flexible device(s) being scheduled.\nTheir power demand cannot be adjusted but still matters for finding the best schedule for other devices.\n\nThe sign convention is determined by the key name: positive values denote consumption.\nSensors that explicitly record consumption as negative values (consumption_is_positive attribute set to false) are rejected here; list them under inflexible-production instead.\n\nEach entry is a sensor reference, optionally with source filters. In the flex-context this is a list of such references (site-level base load); in a flex-model entry it is a single reference, so that an inflexible device modelled as its own asset can join a group like any other member.\n", + "example": [ + { + "sensor": 3 + }, + { + "sensor": 4 + } + ], + "$ref": "#/components/schemas/InflexibleDevice" + }, + "inflexible-production": { + "description": "Power (or energy) sensors representing generators that are relevant, but not flexible in the timing of their supply.\nFor example, a sensor recording rooftop solar power that is connected behind the main meter, and whose production falls under the same contract as the flexible device(s) being scheduled.\nTheir power supply cannot be adjusted but still matters for finding the best schedule for other devices.\n\nThe sign convention is determined by the key name: positive values denote production (the FlexMeasures default).\nSensors that explicitly record production as negative values (consumption_is_positive attribute set to true) are rejected here; list them under inflexible-consumption instead.\n\nEach entry is a sensor reference, optionally with source filters. In the flex-context this is a list of such references (site-level base generation); in a flex-model entry it is a single reference, so that an inflexible device modelled as its own asset can join a group like any other member.\n", + "example": [ + { + "sensor": 3 + }, + { + "sensor": 4 + } + ], + "$ref": "#/components/schemas/InflexibleDevice" + }, "prefer-curtailing-later": { "type": "boolean", "default": true, From 114ff8518715a0da22dd58f50547411b229d555b Mon Sep 17 00:00:00 2001 From: "F.N. Claessen" Date: Fri, 31 Jul 2026 17:55:10 +0200 Subject: [PATCH 2/8] docs: fill in PR number in changelog Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TaMepbxRYzJxQKFtu6Hq4p Signed-off-by: F.N. Claessen --- documentation/changelog.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/changelog.rst b/documentation/changelog.rst index 073cbb06eb..27498b70f4 100644 --- a/documentation/changelog.rst +++ b/documentation/changelog.rst @@ -16,7 +16,7 @@ New features ------------- * New ``inflexible-consumption`` and ``inflexible-production`` flex-context fields make explicit how the sign of each inflexible device's power data should be read (positive values denote consumption resp. production), accepting sensor references with optional source filters; they replace the now-deprecated ``inflexible-device-sensors`` field (bare sensor IDs, sign read from each sensor's ``consumption_is_positive`` attribute), which remains supported [see `PR #2358 `_] -* An inflexible (unschedulable) device can be modelled as its own asset by giving its flex-model entry a single ``inflexible-consumption`` or ``inflexible-production`` sensor reference; such a device joins a ``group`` like any other member, so its fixed (measured) load counts towards the group's intermediate power constraint [see `PR #2371 `_] +* An inflexible (unschedulable) device can be modelled as its own asset by giving its flex-model entry a single ``inflexible-consumption`` or ``inflexible-production`` sensor reference; such a device joins a ``group`` like any other member, so its fixed (measured) load counts towards the group's intermediate power constraint [see `PR #2374 `_] * Forecasting regressors can filter their input beliefs by data source, source type, excluded source type, or source organisation [see `PR #2347 `_] * When multiple selected sources record a belief about the same event at the same belief time, forecasting pipelines now resolve the collision deterministically: the order of an explicit ``sources`` list decides precedence (first listed wins), and otherwise the highest source ID wins after selecting the latest version within each source family [see `PR #2347 `_] * Filter organisations by account role in the Accounts API and organisation list UI [see `PR #2353 `_] From c4de6ddb57bac1127ff8d21b33cd022045d730bc Mon Sep 17 00:00:00 2001 From: "F.N. Claessen" Date: Fri, 31 Jul 2026 18:52:16 +0200 Subject: [PATCH 3/8] fix: single-object example for the flex-model inflexible fields The flex-model inflexible-consumption/inflexible-production fields are a single sensor reference, but reused the shared metadata whose example is a list (that form fits the flex-context list fields). Override the example to a single object on the flex-model declarations so the generated OpenAPI no longer shows an array for an object-valued field. Addresses Copilot's review on #2374. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TaMepbxRYzJxQKFtu6Hq4p Signed-off-by: F.N. Claessen --- .../data/schemas/scheduling/storage.py | 22 +++++++++++++++---- flexmeasures/ui/static/openapi-specs.json | 22 +++++-------------- 2 files changed, 24 insertions(+), 20 deletions(-) diff --git a/flexmeasures/data/schemas/scheduling/storage.py b/flexmeasures/data/schemas/scheduling/storage.py index 216ad4953d..86a7462f23 100644 --- a/flexmeasures/data/schemas/scheduling/storage.py +++ b/flexmeasures/data/schemas/scheduling/storage.py @@ -112,6 +112,20 @@ def validate_inflexible_flex_model_entry(data: dict, original_data: dict): ) +# In a flex-model entry the inflexible fields are a *single* sensor reference (unlike +# the flex-context, where they are lists), so override the shared metadata's list +# example with a single object -- otherwise the generated OpenAPI/docs would show an +# array for an object-valued field. +_INFLEXIBLE_CONSUMPTION_FLEX_MODEL_META = { + **metadata.INFLEXIBLE_CONSUMPTION.to_dict(), + "example": {"sensor": 3}, +} +_INFLEXIBLE_PRODUCTION_FLEX_MODEL_META = { + **metadata.INFLEXIBLE_PRODUCTION.to_dict(), + "example": {"sensor": 3}, +} + + # Telling type hints what to expect after schema parsing SoCTarget = TypedDict( "SoCTarget", @@ -372,13 +386,13 @@ class StorageFlexModelSchema(Schema): InflexibleDeviceSchema, data_key="inflexible-consumption", required=False, - metadata=metadata.INFLEXIBLE_CONSUMPTION.to_dict(), + metadata=_INFLEXIBLE_CONSUMPTION_FLEX_MODEL_META, ) inflexible_production = fields.Nested( InflexibleDeviceSchema, data_key="inflexible-production", required=False, - metadata=metadata.INFLEXIBLE_PRODUCTION.to_dict(), + metadata=_INFLEXIBLE_PRODUCTION_FLEX_MODEL_META, ) # Activation prices @@ -655,13 +669,13 @@ class DBStorageFlexModelSchema(Schema): InflexibleDeviceSchema, data_key="inflexible-consumption", required=False, - metadata=metadata.INFLEXIBLE_CONSUMPTION.to_dict(), + metadata=_INFLEXIBLE_CONSUMPTION_FLEX_MODEL_META, ) inflexible_production = fields.Nested( InflexibleDeviceSchema, data_key="inflexible-production", required=False, - metadata=metadata.INFLEXIBLE_PRODUCTION.to_dict(), + metadata=_INFLEXIBLE_PRODUCTION_FLEX_MODEL_META, ) soc_min = VariableQuantityField( diff --git a/flexmeasures/ui/static/openapi-specs.json b/flexmeasures/ui/static/openapi-specs.json index 3e09b0cbc6..f61a713a6d 100644 --- a/flexmeasures/ui/static/openapi-specs.json +++ b/flexmeasures/ui/static/openapi-specs.json @@ -6851,26 +6851,16 @@ }, "inflexible-consumption": { "description": "Power (or energy) sensors representing loads that are relevant, but not flexible in the timing of their demand.\nFor example, a sensor recording the power of a base load that is connected behind the main meter, and whose consumption falls under the same contract as the flexible device(s) being scheduled.\nTheir power demand cannot be adjusted but still matters for finding the best schedule for other devices.\n\nThe sign convention is determined by the key name: positive values denote consumption.\nSensors that explicitly record consumption as negative values (consumption_is_positive attribute set to false) are rejected here; list them under inflexible-production instead.\n\nEach entry is a sensor reference, optionally with source filters. In the flex-context this is a list of such references (site-level base load); in a flex-model entry it is a single reference, so that an inflexible device modelled as its own asset can join a group like any other member.\n", - "example": [ - { - "sensor": 3 - }, - { - "sensor": 4 - } - ], + "example": { + "sensor": 3 + }, "$ref": "#/components/schemas/InflexibleDevice" }, "inflexible-production": { "description": "Power (or energy) sensors representing generators that are relevant, but not flexible in the timing of their supply.\nFor example, a sensor recording rooftop solar power that is connected behind the main meter, and whose production falls under the same contract as the flexible device(s) being scheduled.\nTheir power supply cannot be adjusted but still matters for finding the best schedule for other devices.\n\nThe sign convention is determined by the key name: positive values denote production (the FlexMeasures default).\nSensors that explicitly record production as negative values (consumption_is_positive attribute set to true) are rejected here; list them under inflexible-consumption instead.\n\nEach entry is a sensor reference, optionally with source filters. In the flex-context this is a list of such references (site-level base generation); in a flex-model entry it is a single reference, so that an inflexible device modelled as its own asset can join a group like any other member.\n", - "example": [ - { - "sensor": 3 - }, - { - "sensor": 4 - } - ], + "example": { + "sensor": 3 + }, "$ref": "#/components/schemas/InflexibleDevice" }, "prefer-curtailing-later": { From fd78a771ad271323c32ef756e5f1ac1e85cc6cb3 Mon Sep 17 00:00:00 2001 From: "F.N. Claessen" Date: Fri, 31 Jul 2026 19:01:24 +0200 Subject: [PATCH 4/8] fix: whitelist keys allowed alongside an inflexible flex-model declaration The schedulable-field check used a hand-maintained blacklist that missed at least soc-unit (Copilot review on #2374), so a contradictory entry like {inflexible-consumption, soc-unit} slipped through. Replace it with a whitelist of the keys allowed alongside an inflexible declaration (identity/group/commodity and the defaulted activation preferences), rejecting every other declared field. This stays complete as new device fields are added. Adds a soc-unit test guard. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TaMepbxRYzJxQKFtu6Hq4p Signed-off-by: F.N. Claessen --- .../data/schemas/scheduling/storage.py | 52 ++++++++----------- .../data/schemas/tests/test_scheduling.py | 15 +++++- 2 files changed, 37 insertions(+), 30 deletions(-) diff --git a/flexmeasures/data/schemas/scheduling/storage.py b/flexmeasures/data/schemas/scheduling/storage.py index 86a7462f23..e2f051af2e 100644 --- a/flexmeasures/data/schemas/scheduling/storage.py +++ b/flexmeasures/data/schemas/scheduling/storage.py @@ -37,32 +37,25 @@ ALLOWED_COMMODITIES = {"electricity", "gas"} -#: User-facing (hyphenated) flex-model keys that make an entry a schedulable device; -#: an entry that declares an inflexible device must carry none of them, so -#: device-inventory classification stays unambiguous. Keyed on data-keys (consistent -#: across StorageFlexModelSchema and DBStorageFlexModelSchema, whose attribute names -#: differ). ``group``, ``commodity``, ``asset``, ``sensor`` and the activation -#: preferences may co-exist with an inflexible declaration. -_SCHEDULABLE_DEVICE_DATA_KEYS = ( - "consumption", - "production", - "state-of-charge", - "soc-at-start", - "soc-min", - "soc-max", - "soc-minima", - "soc-maxima", - "soc-targets", - "soc-gain", - "soc-usage", - "power-capacity", - "consumption-capacity", - "production-capacity", - "roundtrip-efficiency", - "charging-efficiency", - "discharging-efficiency", - "storage-efficiency", - "operation-modes", +#: User-facing (hyphenated) flex-model keys that may co-exist with an inflexible-device +#: declaration: its own identity/grouping/commodity and the always-defaulted activation +#: preferences. Any other declared field is a schedulable-device field, so an entry +#: carrying one alongside ``inflexible-consumption``/``inflexible-production`` is +#: rejected -- this is a whitelist (rather than a blacklist of schedulable keys) so it +#: stays complete as new device fields are added. Keyed on data-keys, which are the +#: same across StorageFlexModelSchema and DBStorageFlexModelSchema (their *attribute* +#: names differ). +_INFLEXIBLE_ALLOWED_DATA_KEYS = frozenset( + { + "inflexible-consumption", + "inflexible-production", + "group", + "commodity", + "asset", + "sensor", + "prefer-charging-sooner", + "prefer-curtailing-later", + } ) @@ -74,8 +67,9 @@ def validate_inflexible_flex_model_entry(data: dict, original_data: dict): signs, must not use a sensor whose explicit ``consumption_is_positive`` attribute contradicts the field's sign convention, and must not also carry schedulable-device fields (so it is unambiguously classified as an inflexible device). The last check - inspects the original (hyphenated) input keys, so it works for both flex-model - schemas and ignores load-default fills. + inspects the original (hyphenated) input keys against a whitelist of keys allowed + alongside an inflexible declaration, so it works for both flex-model schemas, stays + complete as device fields are added, and ignores load-default fills. """ has_consumption = "inflexible_consumption" in data has_production = "inflexible_production" in data @@ -103,7 +97,7 @@ def validate_inflexible_flex_model_entry(data: dict, original_data: dict): f" which conflicts with the sign convention of the `{data_key}` field.", field_name=data_key, ) - offending = [k for k in _SCHEDULABLE_DEVICE_DATA_KEYS if k in original_data] + offending = sorted(set(original_data) - _INFLEXIBLE_ALLOWED_DATA_KEYS) if offending: raise ValidationError( f"An inflexible device entry (`{data_key}`) must not also carry" diff --git a/flexmeasures/data/schemas/tests/test_scheduling.py b/flexmeasures/data/schemas/tests/test_scheduling.py index 59851ff992..af4bdd8313 100644 --- a/flexmeasures/data/schemas/tests/test_scheduling.py +++ b/flexmeasures/data/schemas/tests/test_scheduling.py @@ -1754,7 +1754,9 @@ def test_storage_flex_model_inflexible_device_field( with pytest.raises(ValidationError, match="conflicts with the sign convention"): schema.load({"inflexible-production": {"sensor": consumption_positive.id}}) - # An inflexible entry must not also carry a schedulable-device field. + # An inflexible entry must not also carry a schedulable-device field. The check + # is a whitelist, so it also catches less-obvious device fields (e.g. soc-unit), + # not just an enumerated blacklist. with pytest.raises(ValidationError, match="schedulable-device field"): schema.load( { @@ -1763,6 +1765,17 @@ def test_storage_flex_model_inflexible_device_field( } ) + # soc-unit exists only on StorageFlexModelSchema, and is also rejected. + with pytest.raises(ValidationError, match="schedulable-device field"): + StorageFlexModelSchema( + start=datetime(2026, 6, 1, tzinfo=pytz.UTC), sensor=None + ).load( + { + "inflexible-consumption": {"sensor": attributeless.id}, + "soc-unit": "kWh", + } + ) + def test_db_flex_context_schema_inflexible_devices( db, app, setup_inflexible_sensors, setup_price_sensors From dafcee3dda479207506a3b3e8f4e0842d8dd7da5 Mon Sep 17 00:00:00 2001 From: "F.N. Claessen" Date: Fri, 31 Jul 2026 19:12:24 +0200 Subject: [PATCH 5/8] docs+test: punctuation line breaks in new docstrings; cover source-filtered flex-model inflexible Addresses Copilot's third-round review on #2374: - Reflow the new docstrings (InflexibleDeviceSchema and the inflexible-device helpers/validator) to break only after punctuation, matching the repo docstring convention and the surrounding code. - Add a device-inventory regression test that a flex-model inflexible entry given as a source-filtered SensorReference keeps the reference on FlexDevice.sensor_reference (so source filters reach the solver's power lookup). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TaMepbxRYzJxQKFtu6Hq4p Signed-off-by: F.N. Claessen --- flexmeasures/data/models/planning/devices.py | 26 +++++++++---------- .../planning/tests/test_device_inventory.py | 23 ++++++++++++++++ .../data/schemas/scheduling/storage.py | 14 +++++----- flexmeasures/data/schemas/sensors.py | 9 +++---- 4 files changed, 46 insertions(+), 26 deletions(-) diff --git a/flexmeasures/data/models/planning/devices.py b/flexmeasures/data/models/planning/devices.py index 527d402746..f15268a14b 100644 --- a/flexmeasures/data/models/planning/devices.py +++ b/flexmeasures/data/models/planning/devices.py @@ -317,15 +317,15 @@ def _classify_group_entry(inventory: DeviceInventory, fm: dict) -> bool: def _classify_flex_model_inflexible_entry(fm: dict) -> "FlexDevice | None": """Classify a flex-model entry that declares an inflexible device, or return None. - An inflexible device is declared by a single ``inflexible-consumption`` or - ``inflexible-production`` sensor reference on its own flex-model entry (typically - the flex-model of the inflexible device's asset). Its fixed power is accounted for - but not scheduled; its group membership (if any) is read from its own ``group`` - field, exactly like a flexible member. Schema validation guarantees at most one of - the two fields is present and that the entry carries no schedulable-device fields. - - The returned device's ``index`` is None; it is assigned into the inflexible tail - (after the flexible devices) once their count is known. + An inflexible device is declared by a single ``inflexible-consumption`` or ``inflexible-production`` sensor reference on its own flex-model entry, + typically the flex-model of the inflexible device's asset. + Its fixed power is accounted for but not scheduled; + its group membership (if any) is read from its own ``group`` field, exactly like a flexible member. + Schema validation guarantees at most one of the two fields is present, + and that the entry carries no schedulable-device fields. + + The returned device's ``index`` is None; + it is assigned into the inflexible tail (after the flexible devices) once their count is known. """ for field_name, consumption_is_positive in ( ("inflexible_consumption", True), @@ -361,10 +361,10 @@ def _register_flex_model_inflexible( ) -> bool: """Classify and register a flex-model inflexible-device entry, if this is one. - Groups (and the inflexible-device entries that join them) need a multi-device - flex-model, so this is a no-op in single-sensor mode. The registered device is - appended to ``inventory.entries`` now, and to ``pending`` for index assignment into - the inflexible tail once the flexible-device count is known. + Groups (and the inflexible-device entries that join them) need a multi-device flex-model, + so this is a no-op in single-sensor mode. + The registered device is appended to ``inventory.entries`` now, + and to ``pending`` for index assignment into the inflexible tail once the flexible-device count is known. :returns: True if the entry was an inflexible device (and got registered). """ diff --git a/flexmeasures/data/models/planning/tests/test_device_inventory.py b/flexmeasures/data/models/planning/tests/test_device_inventory.py index 1a67e54cb5..a462aaa5e8 100644 --- a/flexmeasures/data/models/planning/tests/test_device_inventory.py +++ b/flexmeasures/data/models/planning/tests/test_device_inventory.py @@ -494,6 +494,29 @@ def test_flex_model_inflexible_precedes_flat_context_inflexible(): assert [d.index for d in inventory.inflexible_devices] == [1, 2] +def test_flex_model_inflexible_source_filtered_reference(): + """A flex-model inflexible entry given as a source-filtered SensorReference keeps the + reference on FlexDevice.sensor_reference, so its source filters reach the solver's + power lookup (not just the flat flex-context list).""" + from flexmeasures.data.schemas.sensors import SensorReference + + battery = make_sensor(1) + load = make_sensor(12) + reference = SensorReference(sensor=load, source_types=["forecaster"]) + + inventory = DeviceInventory.from_flex_config( + [ + {"sensor": battery}, + {"asset": object(), "inflexible_consumption": reference}, + ] + ) + + device = inventory.inflexible_devices[0] + assert device.power_sensor is load + assert device.sensor_reference is reference + assert device.consumption_is_positive is True + + def test_flat_context_inflexible_device_has_no_group(): """A flat-list inflexible device from the flex-context has no flex-model entry, so it never belongs to a group.""" diff --git a/flexmeasures/data/schemas/scheduling/storage.py b/flexmeasures/data/schemas/scheduling/storage.py index e2f051af2e..601d90dfe0 100644 --- a/flexmeasures/data/schemas/scheduling/storage.py +++ b/flexmeasures/data/schemas/scheduling/storage.py @@ -62,14 +62,12 @@ def validate_inflexible_flex_model_entry(data: dict, original_data: dict): """Validate a flex-model entry that declares an inflexible device. - An inflexible device is declared by a single ``inflexible-consumption`` or - ``inflexible-production`` sensor reference. Such an entry must not declare both - signs, must not use a sensor whose explicit ``consumption_is_positive`` attribute - contradicts the field's sign convention, and must not also carry schedulable-device - fields (so it is unambiguously classified as an inflexible device). The last check - inspects the original (hyphenated) input keys against a whitelist of keys allowed - alongside an inflexible declaration, so it works for both flex-model schemas, stays - complete as device fields are added, and ignores load-default fills. + An inflexible device is declared by a single ``inflexible-consumption`` or ``inflexible-production`` sensor reference. + Such an entry must not declare both signs, + must not use a sensor whose explicit ``consumption_is_positive`` attribute contradicts the field's sign convention, + and must not also carry schedulable-device fields (so it is unambiguously classified as an inflexible device). + The last check inspects the original (hyphenated) input keys against a whitelist of keys allowed alongside an inflexible declaration, + so it works for both flex-model schemas, stays complete as device fields are added, and ignores load-default fills. """ has_consumption = "inflexible_consumption" in data has_production = "inflexible_production" in data diff --git a/flexmeasures/data/schemas/sensors.py b/flexmeasures/data/schemas/sensors.py index 875844cb58..ecfde3e581 100644 --- a/flexmeasures/data/schemas/sensors.py +++ b/flexmeasures/data/schemas/sensors.py @@ -1052,11 +1052,10 @@ class Meta: class InflexibleDeviceSchema(SensorReferenceSchema): """One inflexible device: a sensor reference with optional source filters. - Used both in the flex-context (as a list, for site-level inflexible load) and in a - flex-model entry (as a single reference, when an inflexible device is modelled as - its own asset). Deserializes to a plain :class:`Sensor` when no source filters are - given (a backward-compatible shape downstream) and to a :class:`SensorReference` - otherwise. + Used both in the flex-context (as a list, for site-level inflexible load), + and in a flex-model entry (as a single reference, when an inflexible device is modelled as its own asset). + Deserializes to a plain :class:`Sensor` when no source filters are given (a backward-compatible shape downstream), + and to a :class:`SensorReference` otherwise. """ class Meta: From c0c7c685a88ebfb4bcbea4f2d1db18cc3d9da391 Mon Sep 17 00:00:00 2001 From: "F.N. Claessen" Date: Fri, 31 Jul 2026 19:23:07 +0200 Subject: [PATCH 6/8] fix: reject inflexible fields in a single-sensor flex-model Addresses Copilot's fourth-round review on #2374: a single-sensor flex-model dict containing inflexible-consumption/inflexible-production was a no-op in classification, so it would be silently scheduled as a normal device. Now raise 'only supported in multi-device flex-models' (mirroring the group field), and add a regression test. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TaMepbxRYzJxQKFtu6Hq4p Signed-off-by: F.N. Claessen --- flexmeasures/data/models/planning/devices.py | 13 +++++++++---- .../models/planning/tests/test_device_inventory.py | 10 ++++++++++ 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/flexmeasures/data/models/planning/devices.py b/flexmeasures/data/models/planning/devices.py index f15268a14b..e0eb1852bd 100644 --- a/flexmeasures/data/models/planning/devices.py +++ b/flexmeasures/data/models/planning/devices.py @@ -361,18 +361,23 @@ def _register_flex_model_inflexible( ) -> bool: """Classify and register a flex-model inflexible-device entry, if this is one. - Groups (and the inflexible-device entries that join them) need a multi-device flex-model, - so this is a no-op in single-sensor mode. + Inflexible-device fields need a multi-device flex-model, so a single-sensor flex-model + that declares one is rejected (rather than being silently scheduled as a normal device), + mirroring how the ``group`` field is handled. The registered device is appended to ``inventory.entries`` now, and to ``pending`` for index assignment into the inflexible tail once the flexible-device count is known. :returns: True if the entry was an inflexible device (and got registered). + :raises ValueError: When a single-sensor flex-model declares an inflexible device. """ - if is_single_sensor_mode: - return False device = _classify_flex_model_inflexible_entry(fm) if device is None: return False + if is_single_sensor_mode: + raise ValueError( + "The 'inflexible-consumption'/'inflexible-production' fields are only" + " supported in multi-device flex-models." + ) inventory.entries.append(device) pending.append(device) return True diff --git a/flexmeasures/data/models/planning/tests/test_device_inventory.py b/flexmeasures/data/models/planning/tests/test_device_inventory.py index a462aaa5e8..4be8d9aaf3 100644 --- a/flexmeasures/data/models/planning/tests/test_device_inventory.py +++ b/flexmeasures/data/models/planning/tests/test_device_inventory.py @@ -517,6 +517,16 @@ def test_flex_model_inflexible_source_filtered_reference(): assert device.consumption_is_positive is True +def test_flex_model_inflexible_rejected_in_single_sensor_mode(): + """Inflexible-device fields need a multi-device flex-model; a single-sensor dict that + declares one is rejected, rather than silently scheduled as a normal device.""" + with pytest.raises(ValueError, match="multi-device"): + DeviceInventory.from_flex_config( + {"inflexible_consumption": make_sensor(12)}, + sensor=make_sensor(1), + ) + + def test_flat_context_inflexible_device_has_no_group(): """A flat-list inflexible device from the flex-context has no flex-model entry, so it never belongs to a group.""" From 24663b9cc7c29f987453ad680204c06aaa018bf5 Mon Sep 17 00:00:00 2001 From: "F.N. Claessen" Date: Fri, 31 Jul 2026 20:06:21 +0200 Subject: [PATCH 7/8] docs: clarify that group is optional on a flex-model inflexible device, and it may set a commodity Addresses review questions on #2374: without a group, a flex-model inflexible device is simply accounted for under the grid connection (like the flex-context list, only declared on the asset); with a group it additionally counts towards that group's constraint. It may also set a commodity like any device entry. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TaMepbxRYzJxQKFtu6Hq4p Signed-off-by: F.N. Claessen --- documentation/features/scheduling.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/documentation/features/scheduling.rst b/documentation/features/scheduling.rst index 5b889d8627..bfbec970d3 100644 --- a/documentation/features/scheduling.rst +++ b/documentation/features/scheduling.rst @@ -355,7 +355,9 @@ The sensor-referenced form is convenient when you pass the whole flex-model in o Here, the battery and PV installation may each individually schedule up to 2 kW, but their combined power flowing through the shared inverter is hard-limited to 2.5 kW. -Inflexible (measured) devices can be group members too — for example, an unschedulable base load sitting behind the same inverter or feeder as a battery. To do so, model the inflexible device as its own asset and give its flex-model entry a single ``inflexible-consumption`` or ``inflexible-production`` reference to the sensor recording its power (the field name sets the sign convention, and source filters may be added). Such an entry carries no schedulable-device fields; it simply declares a fixed device whose power is accounted for. It then joins a group through the ordinary ``group`` field, exactly like a flexible member (with the group's own flex-model entry, defining its capacities, still present), so its fixed load or supply counts towards the group's intermediate power constraint. Site-wide inflexible load that does not belong to any group is instead listed in the flex-context's ``inflexible-consumption``/``inflexible-production`` fields (as a list). +Inflexible (measured) devices can be modelled in the flex-model too — for example, an unschedulable base load. To do so, model the inflexible device as its own asset and give its flex-model entry a single ``inflexible-consumption`` or ``inflexible-production`` reference to the sensor recording its power (the field name sets the sign convention, and source filters may be added). Such an entry carries no schedulable-device fields; it simply declares a fixed device whose power is accounted for. Like any device entry, it may set a ``commodity`` (defaulting to electricity), and its fixed power is then netted into that commodity's grid connection. + +The ``group`` field is optional on such an entry. Without it, the device is simply accounted for under the grid connection (just like listing its sensor in the flex-context's ``inflexible-consumption``/``inflexible-production`` fields, only declared on the asset instead). With it, the device *also* joins that group through the ordinary ``group`` field, exactly like a flexible member (the group's own flex-model entry, defining its capacities, must still be present), so that its fixed load or supply additionally counts towards the group's intermediate power constraint — for example, a base load sitting behind the same inverter or feeder as a battery. Usually, not the whole flexibility model is needed. From a95773b974405ae8c205917e6e12a974bdbd6f16 Mon Sep 17 00:00:00 2001 From: "F.N. Claessen" Date: Fri, 31 Jul 2026 20:47:44 +0200 Subject: [PATCH 8/8] test: cover a non-electricity commodity on a flex-model inflexible device The flex-model inflexible tests all used the default (electricity) commodity; only the flat flex-context lists had per-commodity coverage. Add a test that a flex-model inflexible entry with commodity=gas is classified with that commodity and joins the gas device group (not electricity). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01TaMepbxRYzJxQKFtu6Hq4p Signed-off-by: F.N. Claessen --- .../planning/tests/test_device_inventory.py | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/flexmeasures/data/models/planning/tests/test_device_inventory.py b/flexmeasures/data/models/planning/tests/test_device_inventory.py index 4be8d9aaf3..6705e1070b 100644 --- a/flexmeasures/data/models/planning/tests/test_device_inventory.py +++ b/flexmeasures/data/models/planning/tests/test_device_inventory.py @@ -494,6 +494,28 @@ def test_flex_model_inflexible_precedes_flat_context_inflexible(): assert [d.index for d in inventory.inflexible_devices] == [1, 2] +def test_flex_model_inflexible_device_commodity(): + """A flex-model inflexible device may set a commodity (defaulting to electricity); + it then joins that commodity's device group, so its fixed power nets under that + commodity's grid connection rather than the electricity one.""" + battery = make_sensor(1) # electricity (device 0) + gas_load = make_sensor(12) + + inventory = DeviceInventory.from_flex_config( + [ + {"sensor": battery}, + {"asset": object(), "inflexible_production": gas_load, "commodity": "gas"}, + ] + ) + + device = inventory.inflexible_devices[0] + assert device.commodity == "gas" + assert device.consumption_is_positive is False + # The gas inflexible device joins the gas group, not the electricity one. + assert inventory.commodity_to_devices["gas"] == [device.index] + assert device.index not in inventory.commodity_to_devices["electricity"] + + def test_flex_model_inflexible_source_filtered_reference(): """A flex-model inflexible entry given as a source-filtered SensorReference keeps the reference on FlexDevice.sensor_reference, so its source filters reach the solver's