Fix: Handle empty retained partitions

[ngyngcphu]

Issue

When reading topic messages, Console can calculate an invalid start offset if retention has already deleted all records from a partition. One example is a topic with:

Size:               0 B
Estimated messages: 0
Cleanup Policy:     Delete
Retention Time:     ~3 days
Retention Size:     Infinite

After the retention window passes, Kafka can delete all messages from the topic. At that point the partition has no readable records.

Error from backend:

{"time":"2026-08-03T15:21:21.059645+07:00","level":"INFO","msg":"assigning partitions","logger":"kafka_client","why":"new assignments from direct consumer","how":0,"input":{"code.review.jobs":{"0":{"At":149,"Epoch":-1,"CurrentEpoch":0},"1":{"At":91,"Epoch":-1,"CurrentEpoch":0}}}}
{"time":"2026-08-03T15:21:21.094099+07:00","level":"INFO","msg":"received OFFSET_OUT_OF_RANGE on the first fetch, resetting to the configured ConsumeResetOffset","logger":"kafka_client","broker":"1","topic":"code.review.jobs","partition":1,"prior_offset":91}
{"time":"2026-08-03T15:21:21.094099+07:00","level":"INFO","msg":"received OFFSET_OUT_OF_RANGE on the first fetch, resetting to the configured ConsumeResetOffset","logger":"kafka_client","broker":"2","topic":"code.review.jobs","partition":0,"prior_offset":149}
{"time":"2026-08-03T15:21:21.094161+07:00","level":"INFO","msg":"immediate metadata update triggered","logger":"kafka_client","why":"fetch had inner topic errors from broker 1: OFFSET_OUT_OF_RANGE{code.review.jobs[1]}"}
{"time":"2026-08-03T15:21:55.956222+07:00","level":"ERROR","msg":"errors while fetching records","topic_name":"","partition":-1,"error":"context deadline exceeded"}

In that state Kafka reports the same earliest and latest offsets, for example:

earliest offset = 150
latest offset   = 150

This PR treats partitions with latest <= earliest as empty before choosing a read offset, so Console does not request an offset outside the retained range.
Before fix, run test TestCalculateConsumeRequests_TimestampEmptyRetainedPartition, got error:

go test -count=1 ./pkg/console -run 'TestCalculateConsumeRequests_TimestampEmptyRetainedPartition'
--- FAIL: TestCalculateConsumeRequests_TimestampEmptyRetainedPartition (0.00s)
    list_messages_test.go:114:
                Error Trace:    /Users/lap60627/Documents/PTO/console/backend/pkg/console/list_messages_test.go:114
                Error:          Should be empty, but was map[0:0x182bbc2def90]
                Test:           TestCalculateConsumeRequests_TimestampEmptyRetainedPartition
FAIL
FAIL    github.com/redpanda-data/console/backend/pkg/console    1.737s
FAIL

After fix, run test TestCalculateConsumeRequests_TimestampEmptyRetainedPartition successfully:

go test -count=1 ./pkg/console -run 'TestCalculateConsumeRequests_TimestampEmptyRetainedPartition'
ok      github.com/redpanda-data/console/backend/pkg/console    1.079s

[Copilot]

Pull request overview

This PR addresses a Kafka edge case where a partition can become “logically empty” after retention deletes all records (earliest == latest), which previously could lead Console to compute an invalid start offset and trigger offset-out-of-range / stalled consumption behavior.

Changes:

• Treat partitions with latest <= earliest as empty when calculating timestamp-based start offsets.
• Clamp timestamp-resolved offsets to the retained range.
• Add a regression test for timestamp lookup returning -1 with equal low/high watermarks.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
backend/pkg/console/list_messages.go	Adjusts consume-request start offset selection for timestamp mode when watermarks indicate an empty retained partition.
backend/pkg/console/list_messages_test.go	Adds a regression test using a fake cluster to reproduce timestamp lookup returning -1 on an empty retained partition.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.