Automations UI polish: edit action and batch job stats

documentation/api/change_log.rst

@@ -9,14 +9,15 @@ v3.0-33 | September 1, 2026
 """""""""""""""""""""""""""
 - Added ``GET /api/v3_0/assets/<id>/automations`` and ``GET /api/v3_0/assets/<id>/automations/<automation_id>`` for listing and inspecting forecast automations, including the sensors an automation reads from and writes to. Each automation shows the IANA ``timezone`` in which its cron expression is interpreted, and a ``cursor``: the offset-aware UTC time of the most recent run it committed to. The cursor advances just before queueing, so it does not indicate that queueing or the forecast itself succeeded. Asset job entries now include ``created_via`` provenance; automation identity is included only when the caller may read that automation.
 - Added ``GET /api/v3_0/sources/<id>`` to show the full record of one data source, including the attributes in which data generators store their configuration.
+- Automation list entries now include recent ``job_stats`` counts, collected in one batched cache pass. If Redis is unavailable, the list remains available with empty counts and a ``redis_connection_err`` message.
+- Added ``POST /api/v3_0/assets/<id>/automations``, ``PATCH /api/v3_0/assets/<id>/automations/<automation_id>`` and ``DELETE /api/v3_0/assets/<id>/automations/<automation_id>`` for managing an asset's automations. They require account admin or consultant rights, and an automation may only involve sensors that its creator can access: read access to the sensors it reads data from, and permission to record data on the sensors it writes to (a ``403`` otherwise). Both the creation and the update accept a ``timezone``, in which the automation's cron expression is interpreted; it defaults to the server's ``FLEXMEASURES_TIMEZONE``.
 
 v3.0-32 | August 11, 2026
 """""""""""""""""""""""""
 - API endpoints are now rate-limited. A request which exceeds a limit is answered with a ``429 (Too Many Requests)`` status code and a ``Retry-After`` header stating how many seconds to wait. Responses also carry ``X-RateLimit-*`` headers, describing the limit that applied, how much of it is left, and when it resets. A stricter limit applies to ``POST /assets/<id>/schedules/trigger``, ``POST /sensors/<id>/schedules/trigger`` and ``POST /sensors/<id>/forecasts/trigger`` than to other endpoints; the health endpoints are exempt. Per-account overrides are set by assigning the account a plan (a ``Plan`` database row), rather than through an account attribute.
 - Introduced the ``inflexible-consumption`` and ``inflexible-production`` flex-context fields, which make explicit how the sign of each inflexible device's power data should be read: positive values denote consumption resp. production. Each entry is a sensor reference (``{"sensor": <id>}``), optionally with source filters (``source-types``, ``exclude-source-types``, ``sources``, ``source-account``). Deprecated the ``inflexible-device-sensors`` field (a list of bare sensor IDs, whose sign convention is read from each sensor's ``consumption_is_positive`` attribute); it remains supported, but cannot be combined with the new fields in one flex-context.
 - Added a ``role`` query parameter to ``GET /api/v3_0/accounts`` for filtering accessible organisations by account role.
 - Sensor references on variable-quantity flex-model and flex-context fields (such as ``soc-minima``, ``soc-maxima``, the capacity fields and the price fields) may now include a ``default`` fallback quantity, e.g. ``{"sensor": 50, "default": "0 kWh"}``. It fills the time slots for which the referenced sensor holds no value. Note that this fills *every* such slot, so a sensor recording only occasional setpoints becomes densely constrained. The field is not (yet) applied to sensor references on ``inflexible-consumption``/``inflexible-production`` or to forecaster regressors. Take particular care with a fallback of ``0`` on a ``consumption-capacity`` or ``production-capacity``: if the sensor holds no value for the whole scheduling window, the resulting all-zero capacity is read as a physical statement about the device and enforced strictly (see :ref:`the flex-model capacity fields <flex_models_and_schedulers>`), rather than as a limit that may be breached at a price.
-- Added ``POST /api/v3_0/assets/<id>/automations``, ``PATCH /api/v3_0/assets/<id>/automations/<automation_id>`` and ``DELETE /api/v3_0/assets/<id>/automations/<automation_id>`` for managing an asset's automations. They require account admin or consultant rights, and an automation may only involve sensors that its creator can access: read access to the sensors it reads data from, and permission to record data on the sensors it writes to (a ``403`` otherwise). Both the creation and the update accept a ``timezone``, in which the automation's cron expression is interpreted; it defaults to the server's ``FLEXMEASURES_TIMEZONE``.
 - Extended ``GET /api/v3_0/jobs/<uuid>`` with a ``result`` field containing ``unresolved`` and ``resolved`` arrays, each keyed by asset ID. For scheduling jobs, this surfaces soft state-of-charge constraint analysis: ``soc-minima`` and ``soc-maxima`` violations (with a ``violation`` magnitude) or satisfied constraints (with a ``margin`` headroom). Both arrays are empty when no SoC constraints were defined.
 - **Field canonicalization** for background job tracking:
   * The ``job`` field is now the canonical way to identify background jobs returned by `/sensors/<id>/schedules/trigger`, `/assets/<id>/schedules/trigger`, and `/sensors/<id>/forecasts/trigger` endpoints. If applicable, the triggered response now also returns a ``results-url`` pointing to the sensor-specific results endpoint, alongside the generic ``job-url``.

documentation/changelog.rst

@@ -58,6 +58,7 @@ New features
 * Automations can also compute schedules on a recurring basis (``flexmeasures add automation --type schedules``), with the schedule start defaulting to each run's time [see `PR #2293 <https://www.github.com/FlexMeasures/flexmeasures/pull/2293>`_]
 * Automations can be created, edited and deleted in the UI and through new API endpoints (``[POST|PATCH|DELETE] /assets/(id)/automations``), by organisation admins and consultants, with their recurrence expressed in a selectable IANA timezone, and only involving sensors they can access themselves (read access to the sensors an automation reads, and permission to record data on the sensors it writes to) [see `PR #2294 <https://www.github.com/FlexMeasures/flexmeasures/pull/2294>`_]
 * Reports can run as background jobs (``flexmeasures add report --as-job``, processed by workers of the new ``reporting`` queue) and be computed on a recurring basis by automations, with a rolling report window expressed as Pandas offsets or defaulting to the last cron period [see `PR #2297 <https://www.github.com/FlexMeasures/flexmeasures/pull/2297>`_]
+* Show automation job counts in one batched asset request, load full automation details on demand, and support editing an automation's name, recurrence, timezone and activation state in the UI [see `PR #2299 <https://www.github.com/FlexMeasures/flexmeasures/pull/2299>`_]
 * ``flexmeasures show data-sources`` now shows which organisation a data source belongs to, and can list the sensors holding data recorded by a given source [see `PR #2401 <https://www.github.com/FlexMeasures/flexmeasures/pull/2401>`_]
 * The flex-context can now define multiple commodities, each specifying their own prices and grid capacities [see `PR #1946 <https://www.github.com/FlexMeasures/flexmeasures/pull/1946>`_, `PR #2172 <https://www.github.com/FlexMeasures/flexmeasures/pull/2172>`_, `PR #2235 <https://www.github.com/FlexMeasures/flexmeasures/pull/2235>`_, `PR #2271 <https://www.github.com/FlexMeasures/flexmeasures/pull/2271>`_, `PR #2355 <https://www.github.com/FlexMeasures/flexmeasures/pull/2355>`_ and `PR #2380 <https://www.github.com/FlexMeasures/flexmeasures/pull/2380>`_]
 * Support multiple feeders to a shared storage [see `PR #2001 <https://www.github.com/FlexMeasures/flexmeasures/pull/2001>`_, `PR #2321 <https://www.github.com/FlexMeasures/flexmeasures/pull/2321>`_, `PR #2322 <https://www.github.com/FlexMeasures/flexmeasures/pull/2322>`_, `PR #2325 <https://www.github.com/FlexMeasures/flexmeasures/pull/2325>`_ and `PR #2431 <https://www.github.com/FlexMeasures/flexmeasures/pull/2431>`_]

flexmeasures/api/v3_0/assets.py

@@ -13,6 +13,7 @@
 from flask_sqlalchemy.pagination import SelectPagination
 
 from marshmallow import fields, post_load, ValidationError, Schema, validate
+from redis.exceptions import RedisError
 
 from webargs.flaskparser import use_kwargs, use_args
 from sqlalchemy import select, func, or_
@@ -59,6 +60,7 @@
     create_automation,
     delete_automation as remove_automation,
     describe_cronstr,
+    get_asset_automations_job_stats,
     get_automation_job_stats,
     resolve_automation_sensors,
     update_automation,
@@ -1396,10 +1398,11 @@ def get_automations(self, id: int, asset: GenericAsset):
         get:
           summary: Get all automations defined on an asset.
           description: |
-            The response will be a list of automations: recurring forecasting or scheduling tasks
+            The response will be a list of automations: recurring forecasting, scheduling or reporting tasks
             defined on the asset. Each entry shows the automation's ID, when it was created,
-            its type, name, activation status, and its recurrence, both as a cron string
-            and described in natural language. Each entry also shows the IANA timezone in which its cron expression is interpreted, and its cursor.
+            its type, name, activation status, recurrence, IANA timezone, cursor,
+            and counts of recently created jobs per job status. Jobs in Redis have a limited TTL,
+            so not all past jobs are counted.
           security:
             - ApiKeyAuth: []
           parameters:
@@ -1429,6 +1432,9 @@ def get_automations(self, id: int, asset: GenericAsset):
                             cursor: "2026-07-11T04:00:00+00:00"
                             recurrence_description: "At 06:00"
                             active: true
+                            job_stats:
+                              finished: 3
+                        redis_connection_err: null
             400:
               description: INVALID_REQUEST, REQUIRED_INFO_MISSING, UNEXPECTED_PARAMS
             401:
@@ -1440,14 +1446,33 @@ def get_automations(self, id: int, asset: GenericAsset):
           tags:
             - Assets
         """
+        redis_connection_err = None
+        try:
+            job_stats = get_asset_automations_job_stats(asset)
+        except NoRedisConfigured as e:
+            job_stats = {}
+            redis_connection_err = e.args[0]
+        except RedisError:
+            current_app.logger.warning(
+                "Could not load automation job statistics because Redis is unavailable.",
+                exc_info=True,
+            )
+            job_stats = {}
+            redis_connection_err = (
+                "Redis is unavailable; job statistics could not be loaded."
+            )
         automations_data = []
         for automation in asset.automations:
             automation_data = automation_schema.dump(automation)
             automation_data["recurrence_description"] = describe_cronstr(
                 automation.cronstr
             )
+            automation_data["job_stats"] = job_stats.get(automation.id, {})
             automations_data.append(automation_data)
-        return {"automations": automations_data}, 200
+        return {
+            "automations": automations_data,
+            "redis_connection_err": redis_connection_err,
+        }, 200
 
     @route("/<id>/automations/<int:automation_id>", methods=["GET"])
     @use_kwargs(
@@ -1580,6 +1605,15 @@ def get_automation(self, id: int, automation_id: int, asset: GenericAsset):
         except NoRedisConfigured as e:
             automation_data["job_stats"] = {}
             redis_connection_err = e.args[0]
+        except RedisError:
+            current_app.logger.warning(
+                "Could not load automation job statistics because Redis is unavailable.",
+                exc_info=True,
+            )
+            automation_data["job_stats"] = {}
+            redis_connection_err = (
+                "Redis is unavailable; job statistics could not be loaded."
+            )
         automation_data["redis_connection_err"] = redis_connection_err
         return automation_data, 200
 

flexmeasures/api/v3_0/tests/test_automations_api.py

@@ -6,6 +6,7 @@
 
 import pytest
 from flask import url_for
+from redis.exceptions import TimeoutError as RedisTimeoutError
 from sqlalchemy import select
 
 from flexmeasures.data.models.automations import Automation
@@ -99,12 +100,48 @@ def test_get_automations(
     assert day_ahead["recurrence_description"] == "At 06:00"
     assert day_ahead["active"] is True
     assert day_ahead["created_at"] is not None
+    assert day_ahead["job_stats"] == {}  # this automation has not queued any jobs
     # generator and parameters are not listed
     assert "generator_id" not in day_ahead
     assert "generator" not in day_ahead
     assert "parameters" not in day_ahead
 
 
+@pytest.mark.parametrize(
+    "requesting_user", ["test_prosumer_user@seita.nl"], indirect=True
+)
+def test_get_automations_when_redis_times_out(
+    app,
+    add_battery_assets_fresh_db,
+    add_automations,
+    requesting_user,
+    monkeypatch,
+):
+    """The automation list remains available when Redis times out."""
+    battery = add_battery_assets_fresh_db["Test battery"]
+
+    def raise_redis_timeout(asset):
+        raise RedisTimeoutError("Redis timed out at private-host.example")
+
+    monkeypatch.setattr(
+        "flexmeasures.api.v3_0.assets.get_asset_automations_job_stats",
+        raise_redis_timeout,
+    )
+
+    with app.test_client() as client:
+        response = client.get(url_for("AssetAPI:get_automations", id=battery.id))
+
+    assert response.status_code == 200
+    assert len(response.json["automations"]) == 2
+    assert all(
+        automation["job_stats"] == {} for automation in response.json["automations"]
+    )
+    assert response.json["redis_connection_err"] == (
+        "Redis is unavailable; job statistics could not be loaded."
+    )
+    assert "private-host.example" not in response.text
+
+
 @pytest.mark.parametrize(
     "requesting_user", ["test_prosumer_user@seita.nl"], indirect=True
 )

flexmeasures/data/services/automations.py

@@ -34,6 +34,7 @@
 from flexmeasures.data.queries.generic_assets import (
     asset_and_ancestor_ids,
     asset_is_in_subtree,
+    descendants_cte,
 )
 from flexmeasures.utils.time_utils import apply_offset_chain, get_timezone, server_now
 
@@ -765,30 +766,30 @@ def get_automations_involving_sensor(sensor: Sensor) -> list[Automation]:
     return involved
 
 
-def get_automation_job_stats(automation: Automation) -> dict[str, int]:
-    """Count the jobs created by this automation, per job status.
+def _asset_subtree_sensor_ids(asset_id: int) -> set[int]:
+    """Return all sensor IDs on an asset and its descendants."""
+    tree = descendants_cte(root_asset_id=asset_id, max_depth=None)
+    return set(
+        db.session.scalars(
+            select(Sensor.id).where(Sensor.generic_asset_id.in_(select(tree.c.id)))
+        ).all()
+    )
 
-    Note that jobs in Redis have a limited TTL, so this only counts fairly recent jobs.
-    """
-    # Determine the job cache entries to scan. Forecasting and reporting jobs
-    # are cached under their target/output sensor(s), which may belong to a
-    # different asset than the automation's own asset.
+
+def _job_cache_refs(
+    automation: Automation, schedule_sensor_ids: set[int] | None = None
+) -> set[tuple[int, str, str]]:
+    """Return the job-cache entries in which an automation's jobs may live."""
     parameters = automation.parameters or {}
     if automation.type == "schedules":
         # Scheduling jobs are cached under the asset (multi-device wrap-up jobs)
         # and under individual device sensors (per-device jobs), which may belong
         # to child assets rather than the automation's own (site) asset.
-        sensor_ids = _relevant_sensor_ids(
-            automation,
-            [
-                entry.get("sensor")
-                for entry in parameters.get("flex-model", []) or []
-                if isinstance(entry, dict)
-            ],
-        )
-        cache_refs = [(automation.asset_id, "scheduling", "asset")] + [
-            (sensor_id, "scheduling", "sensor") for sensor_id in sensor_ids
-        ]
+        if schedule_sensor_ids is None:
+            schedule_sensor_ids = _asset_subtree_sensor_ids(automation.asset_id)
+        return {(automation.asset_id, "scheduling", "asset")} | {
+            (sensor_id, "scheduling", "sensor") for sensor_id in schedule_sensor_ids
+        }
     elif automation.type == "reports":
         sensor_ids = _relevant_sensor_ids(
             automation,
@@ -798,27 +799,60 @@ def get_automation_job_stats(automation: Automation) -> dict[str, int]:
                 if isinstance(output, dict)
             ],
         )
-        cache_refs = [(sensor_id, "reporting", "sensor") for sensor_id in sensor_ids]
+        return {(sensor_id, "reporting", "sensor") for sensor_id in sensor_ids}
     else:
         sensor_ids = _relevant_sensor_ids(
             automation,
             [parameters.get("sensor"), parameters.get("sensor-to-save")],
         )
-        cache_refs = [(sensor_id, "forecasting", "sensor") for sensor_id in sensor_ids]
+        return {(sensor_id, "forecasting", "sensor") for sensor_id in sensor_ids}
 
-    counts: dict[str, int] = {}
+
+def _count_automation_jobs(
+    cache_refs: set[tuple[int, str, str]], automation_ids: set[int]
+) -> dict[int, dict[str, int]]:
+    """Count jobs per automation and status in one pass over the cache entries."""
+    counts: dict[int, dict[str, int]] = {
+        automation_id: {} for automation_id in automation_ids
+    }
     seen_job_ids: set[str] = set()
     for entity_id, queue, asset_or_sensor_type in cache_refs:
         for job in current_app.job_cache.get(entity_id, queue, asset_or_sensor_type):
             if job.id in seen_job_ids:
                 continue
             seen_job_ids.add(job.id)
-            if job.meta.get("trigger", {}).get("automation_id") == automation.id:
+            automation_id = job.meta.get("trigger", {}).get("automation_id")
+            if automation_id in counts:
                 status = str(job.get_status().value)
-                counts[status] = counts.get(status, 0) + 1
+                counts[automation_id][status] = counts[automation_id].get(status, 0) + 1
     return counts
 
 
+def get_automation_job_stats(automation: Automation) -> dict[str, int]:
+    """Count the recent jobs created by this automation, per job status."""
+    return _count_automation_jobs(_job_cache_refs(automation), {automation.id})[
+        automation.id
+    ]
+
+
+def get_asset_automations_job_stats(asset) -> dict[int, dict[str, int]]:
+    """Count recent jobs for all of an asset's automations in one cache pass."""
+    automations = asset.automations
+    if not automations:
+        return {}
+    schedule_sensor_ids = (
+        _asset_subtree_sensor_ids(asset.id)
+        if any(automation.type == "schedules" for automation in automations)
+        else None
+    )
+    cache_refs: set[tuple[int, str, str]] = set()
+    for automation in automations:
+        cache_refs |= _job_cache_refs(automation, schedule_sensor_ids)
+    return _count_automation_jobs(
+        cache_refs, {automation.id for automation in automations}
+    )
+
+
 def _prepare_forecast_automation(
     asset, parameters: dict, generator_class: str | None, config: dict | None, source
 ) -> tuple[int, list[str]]: