Skip to content

Add API endpoint and UI button to compact repo - #468

Open
Forceu wants to merge 3 commits into
Ravinou:feat/compactfrom
Forceu:featureCompact
Open

Add API endpoint and UI button to compact repo#468
Forceu wants to merge 3 commits into
Ravinou:feat/compactfrom
Forceu:featureCompact

Conversation

@Forceu

@Forceu Forceu commented May 14, 2025

Copy link
Copy Markdown
Contributor

This PR adds a button and an API endpoint to compact a repo and will close #222

The new api endpoint is POST /api/v1/repositories/[id]/compact. By default the call only initiates compacting and returns 200, if the parameter await is set to true, it will only return after compacting and an additional storage calculation is done.

Compacting can be disabled on append-only repos with the env DISABLE_COMPACT_APPEND_ONLY.

One drawback with compacting is however, that after the process is finished, it registers as a change. This means, a notification might not be send, if a future backup is not made and the notification period was exceeded. I think it is the same with borgbase and there is not really a way around it (other than saving the last date before compacting).

No tests or documentation were created / altered in this PR.

@Forceu
Forceu marked this pull request as draft May 14, 2025 12:38
@Forceu
Forceu marked this pull request as ready for review May 15, 2025 13:16
@Forceu Forceu changed the title WIP: Add button to compact repo from the UI Add API endpoint and UI button to compact repo May 15, 2025
@Forceu

Forceu commented May 15, 2025

Copy link
Copy Markdown
Contributor Author

@Ravinou The PR is ready for review now. Please note that I changed the gear icon to a pencil in the repo overview, as I think with two buttons it looks better. In case you prefer the old gear icon, let me know and I will revert it back.

…, added DISABLE_COMPACT_APPEND_ONLY env, replaced gear with pencil icon
@Ravinou
Ravinou changed the base branch from main to feat/compact May 18, 2025 09:59
@Ravinou
Ravinou changed the base branch from feat/compact to main May 18, 2025 10:00
@Ravinou

Ravinou commented May 18, 2025

Copy link
Copy Markdown
Owner

Thanks, I will study this later (no more time today :D ).

Note that :

  • shell must be tested with bats, like all other shells
  • API must be testes with vitest, like all other api

We have to manage the competing compacts well.

As for the modification date, this will modify the last backup date. After that, it's indicated everywhere that it's a date of last “modification” in borgwarehouse, and compacting is still a manual action. So I don't see any problem with that, I'll add it to the documentation.

Finally, I'll have to do some tests with the borgbackup CLI and read their documentation to be sure of the different types of return from this command.

It's a big feature. I've created the feat/compact branch.

@Ravinou
Ravinou changed the base branch from main to feat/compact May 18, 2025 10:00
@Ravinou Ravinou self-assigned this May 18, 2025
@Ravinou
Ravinou self-requested a review May 18, 2025 10:02
Comment thread helpers/shells/compactRepo.sh
Comment thread pages/api/v1/repositories/[slug]/compact/index.ts
@Forceu

Forceu commented May 18, 2025

Copy link
Copy Markdown
Contributor Author
  • shell must be tested with bats, like all other shells
  • API must be testes with vitest, like all other api

I can give that a try. I have never used bats or vitest (and don't have a lot of experience with TS either), but I guess I can look through the existing scripts

We have to manage the competing compacts well.

I assume during a compacting action, the repository is locked and no other action can be performed. I will test that later on. However it would be nice to have an indication once compaction is done, if an async call was made.

As for the modification date, this will modify the last backup date. After that, it's indicated everywhere that it's a date of last “modification” in borgwarehouse, and compacting is still a manual action. So I don't see any problem with that, I'll add it to the documentation.

The only problem would be if someone was running a cronjob fairly regularly to compact the repos with the API call. But a note in the documentation should be sufficient I assume.

@Forceu
Forceu requested a review from Ravinou May 19, 2025 13:41
@Forceu

Forceu commented Jun 10, 2025

Copy link
Copy Markdown
Contributor Author

@Ravinou Are the tests sufficient or shall I change anything? :)

@Ravinou

Ravinou commented Jul 5, 2025

Copy link
Copy Markdown
Owner

I haven't forgotten you. This summer's vacation, coming up in a few weeks, should help me find some time to move forward. Thank you for your patience 🙏

@hozza

hozza commented Aug 8, 2025

Copy link
Copy Markdown

Hope you had a good holiday! Is there anything I can do to help with this PR?

I'm excited to see it in action and reclaim some much-needed storage space.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regularely disable append-only mode automatically (optionally)

3 participants