Fixes #39648 - Improve live job output updates

[jakduch]

The output for an expanded host is currently refreshed every five seconds by fetching the complete template invocation details. This repeatedly transfers the full accumulated output together with template, task, proxy, and permission data, then parses and renders the entire console again.

Use the existing per-host output API with the since parameter to request only chunks newer than the last displayed timestamp. Poll once per second while the output is open, append new chunks while preserving split lines, pause requests in hidden tabs, and refresh the complete invocation details once after the host finishes.

Keep existing output line sets stable and memoize their parsed ANSI output so long-running jobs only process newly appended content.

Tests:

• incremental requests use the latest displayed timestamp
• new output is appended and split lines are preserved
• polling stops after completion, collapse, or unmount
• hidden tabs pause output requests
• existing output rendering and ANSI colors remain unchanged

This pull request was assisted by Codex 5.6 Sol High.

[kmalyjur]

Thank you, that's a good idea and it works well. I have some comments.

Right now it polls every one sec, but there could be some kind of "adaptive polling" to reduce the load when the job is running but not producing output.

Simplified "solution" to that:

// In handleSuccess:
  const nextInterval = newChunks.length > 0
    ? OUTPUT_REFRESH_INTERVAL_MS      // Got output, stay fast (1s)
    : OUTPUT_REFRESH_INTERVAL_MS * 3; // Idle, slow down (3s)

  if (data?.refresh) {
    timeoutId = setTimeout(poll, nextInterval);
  }

[kmalyjur]

This hook has a lot of state management: 5 refs/state vars, useEffect just to sync props - local state, two similar fetch functions (scheduleOutputPoll, fetchDetails) etc. It could be simplified. What do you think?

The suggestion:

export const useTemplateInvocationOutputPolling = ({
  hostID,
  jobID,
  isExpanded,
  initialOutput = [],
}) => {
  const dispatch = useDispatch();
  const [output, setOutput] = useState(initialOutput);
  const lastTimestampRef = useRef(getLastOutputTimestamp(initialOutput));

  useEffect(() => {
    if (!isExpanded) return;

    let cancelled = false;
    let timeoutId = null;
    const outputURL = templateInvocationOutputUrl(hostID, jobID);

    const poll = () => {
      if (cancelled) return;

      // Pause when tab hidden
      if (document.visibilityState === 'hidden') {
        timeoutId = setTimeout(poll, OUTPUT_REFRESH_INTERVAL_MS);
        return;
      }

      dispatch(
        APIActions.get({
          url: outputURL,
          key: `${GET_TEMPLATE_INVOCATION_OUTPUT}_${hostID}`,
          params: lastTimestampRef.current ? { since: lastTimestampRef.current } : {},
          handleSuccess: ({ data }) => {
            if (cancelled) return;

            const newChunks = data?.output || [];
            if (newChunks.length > 0) {
              setOutput(prev => {
                const merged = mergeOutput(prev, newChunks);
                lastTimestampRef.current = getLastOutputTimestamp(merged);
                return merged;
              });
            }

            // Continue polling or fetch final details
            if (data?.refresh) {
              timeoutId = setTimeout(poll, OUTPUT_REFRESH_INTERVAL_MS);
            } else {
              // Host finished - fetch complete details once
              dispatch(
                APIActions.get({
                  url: showTemplateInvocationUrl(hostID, jobID),
                  key: `${GET_TEMPLATE_INVOCATION}_${hostID}`,
                })
              );
            }
          },
          handleError: () => {
            if (cancelled) return;
          },
        })
      );
    };

    poll();

    return () => {
      cancelled = true;
      clearTimeout(timeoutId);
    };
  }, [isExpanded, hostID, jobID, dispatch]);

  return output;
};

[jakduch]

I simplified the hook while keeping the initial/final details refresh and terminal-state handling. Polling now gradually backs off from 1s to 3s while idle and returns to 1s as soon as new output arrives. I added coverage for both cases in 21a15f9.