Skip to content

fix(multi.kagane): work around Cloudflare challenge on all API routes - #679

Open
CtrlAltElite97 wants to merge 10 commits into
Aidoku-Community:mainfrom
CtrlAltElite97:main
Open

fix(multi.kagane): work around Cloudflare challenge on all API routes#679
CtrlAltElite97 wants to merge 10 commits into
Aidoku-Community:mainfrom
CtrlAltElite97:main

Conversation

@CtrlAltElite97

Copy link
Copy Markdown
Contributor

Problem

  • kagane.to now returns a Cloudflare managed challenge (403 + "Just a moment…" HTML) on every API route — search, series, and integrity.
  • The source got HTML where it expected JSON → JsonParseError on every call → "No Result" everywhere.

Why the simple fix doesn't work

  • Solving the challenge in a WebView and retrying with a normal Request fails.
  • The cf_clearance cookie stays in the web view's own cookie store; the app's HTTP client never sees it.
  • Verified on device: after a successful solve, the retry still returned 403 cf-mitigated: challenge.

Changes

  • New ApiClient (helpers.rs) — holds a WebView in a RefCell, solves the challenge once per session, and runs every API call as a fetch() inside that web view so requests carry its cookies.
  • Polling instead of eval_async — eval_async returned JsError(MissingResult) on device. Uses the same pattern as en.comix: kick the promise off with a synchronous eval that stashes its result on window, then poll. 30s abort guard included.
  • lib.rs — search, details, listings, home, and the integrity/page-manifest exchange all route through ApiClient.
  • Covers as data: URIs — covers are behind the same challenge but loaded by the app's image pipeline, which never has clearance. Now fetched through the web view and inlined; all covers per page in one round trip via Promise.all; individual failures keep their original URL instead of failing the listing.
  • Cover URL fixed → /api/v2/image/{id}/compressed (the path the site actually uses; 60KB vs 382KB).
  • source.json — minAppVersion → 0.8.4 (matches en.comix, the other WebView source), version → 3.

Testing

  • Tested on device via aidoku serve: browse, search, listings, details (status, rating, authors/artists, tags, chapters), and covers all load.
  • cargo fmt --check, cargo clippy, cargo build --target wasm32-unknown-unknown, and aidoku verify all clean.

Known tradeoff

  • Data-URI covers add ~80KB per cover to the manga DB (~3MB per 35-item page) and slow list loads.
  • Root cause is app-side: CloudflareHandler.shouldHandle() misses managed challenges because it scrapes markup instead of checking the cf-mitigated: challenge header. If that's fixed upstream, this workaround can revert to plain URLs.

@kkantan kkantan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have seen with other sources doing this that this is not a good solution. the while loop eval to wait for an async result leads to deadlocks in the app. at the very least, we would want to use the async eval api (though it appears to be broken current, so we would need to wait until it is fixed). however, having to spawn a web view and run all this js for requests is not a good solution regardless, since I believe things like rakuyomi don't support it.

the cf_clearance cookie not being given to the source after passing cloudflare with the app sounds like an app issue to me. is kagane handling cloudflare in a special way, or does this need to be reported to aidoku?

@CtrlAltElite97

Copy link
Copy Markdown
Contributor Author

I think we have seen with other sources doing this that this is not a good solution. the while loop eval to wait for an async result leads to deadlocks in the app. at the very least, we would want to use the async eval api (though it appears to be broken current, so we would need to wait until it is fixed). however, having to spawn a web view and run all this js for requests is not a good solution regardless, since I believe things like rakuyomi don't support it.

the cf_clearance cookie not being given to the source after passing cloudflare with the app sounds like an app issue to me. is kagane handling cloudflare in a special way, or does this need to be reported to aidoku?

Thanks for the review.

The Cloudflare challenge is site-wide rather than page-specific. If I open kagane.to in a fresh browser session, it clears itself after a few seconds without any interaction. Once that's done, everything the source uses, including search, series pages, covers, and the chapter API, works fine for the rest of that session. So the idea of solving it once and then reusing that session is valid.

I also looked into whether we could solve the challenge once and then make normal requests using the clearance cookie instead of routing everything through the web view. Unfortunately, that doesn't seem possible. The cf_clearance cookie is intentionally inaccessible to JavaScript, and as far as I can tell there's no way to retrieve it through Aidoku's current source API either. Given those limitations, keeping requests inside the web view isn't really a design choice. It's the only approach that's currently possible.

As for the busy-wait loop, I agree that's not something we should merge as-is. I've been experimenting with replacing it using the web view's async APIs instead of polling in a loop, but I haven't verified that implementation yet. I didn't want to push something untested, so I'll update the PR once I have a solution I'm confident in.

@CtrlAltElite97
CtrlAltElite97 requested a review from kkantan August 6, 2026 21:32
@jianmingyong

Copy link
Copy Markdown
Contributor
IMG_2840

It works for me just fine without this PR… I guess you are also affected by the new cloudflare bug. The webview is really just a workaround that I used long ago too but it does cause some issues.

0.8.4 update seems to have worked for some people as that would get the correct cf cookie automatically but it might still reject some devices if they choose a stronger challenge that requires a webview to resolve.

(One way is also to use the fake login button to intercept the cf_cleareance cookie but that’s really ugly as well)

@CtrlAltElite97

Copy link
Copy Markdown
Contributor Author
IMG_2840

It works for me just fine without this PR… I guess you are also affected by the new cloudflare bug. The webview is really just a workaround that I used long ago too but it does cause some issues.

0.8.4 update seems to have worked for some people as that would get the correct cf cookie automatically but it might still reject some devices if they choose a stronger challenge that requires a webview to resolve.

(One way is also to use the fake login button to intercept the cf_cleareance cookie but that’s really ugly as well)

image

It doesn't work for me. 🫠

@jianmingyong

Copy link
Copy Markdown
Contributor

It is funny because the internal webview and the safari doesn’t work for me. But the aidoku app works.

@jianmingyong

Copy link
Copy Markdown
Contributor

If you want to do this, make an option for opt-in because the invisible webview cannot solve captcha. That will prevent some people from using it. Of course cf challenges changes for everyone so some can, some cannot. Best to be optional so everyone can use it.

@jianmingyong

Copy link
Copy Markdown
Contributor

@CtrlAltElite97 Have you tried clearing source cache? I did play around a bit switching between my mobile carrier and wifi. The error would continue to persist until I clear the cache to reset all cookies. Of course to trigger the webview, you had to click retry once.

@CtrlAltElite97

Copy link
Copy Markdown
Contributor Author

@CtrlAltElite97 Have you tried clearing source cache? I did play around a bit switching between my mobile carrier and wifi. The error would continue to persist until I clear the cache to reset all cookies. Of course to trigger the webview, you had to click retry once.

Did everything you mentioned resulting in no success which let me to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants