Support negative indices with --range and --chapter-range (#5402) - #7605
Open
thatDudo wants to merge 6636 commits into
Open
Support negative indices with --range and --chapter-range (#5402)#7605thatDudo wants to merge 6636 commits into
thatDudo wants to merge 6636 commits into
Conversation
rename 'tag_names' to 'tags'
allow legacy domains by default
* [arca.live] Add extractor skeleton * [arcalive] update names and formatting * [arcalive] implement initial file extraction code * [arcalive] improve '_extract_media()' performance compile and cache regex on demand * [arcalive] improve image extraction - extract 'data-originalurl' URLs if available - replace URL query strings with 'type=orig' - ignore emoticons by default * [arcalive] update defaults - include 'title' in filenames - use 0.5-1.5s delay between requests * [arcalive] use ext from 'data-orig' if available * [arcalive] update docs/supportedsites * [arcalive] add tests * [arcalive] update 'board' extractor pattern so it doesn't also match 'post' URLs --------- Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
follow redirect instead of rewriting them to deviantart.com/stash/…
…f links [tiktok] Move avatar download to the user extractor, which results in more accurate metadata output (it would previously write the metadata of the video which the avatar was scraped from) [tiktok] Fix tests and remove redundant user profile test
[DeviantArt] Adding support for sub-folders
add 'archives' option for additional data
to prevent "403 Forbidden" errors when downloading
don't use a hardcoded '.part' filename extension fixes regression introduced in 29d315e
Make Extractor.request(…, fatal=False) actually non-fatal by returning an empty response instead of raising an exception when a request fails due to connection issues.
category changes: - kemonoparty -> kemono - coomerparty -> coomer - koharu -> schalenetwork also wanted to rename '2chan' -> 'sturdychan', but the site's main page is still titled '2chen'
primarily to achieve some form of backwards compatibility for 922c296
Contributor
Author
|
I suppose it might have been possible to do this with filter predicates by using page and count. |
Owner
|
Another problem is that It also doesn't work for We should introduce a new field that either holds the total number of files / items ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
17 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It seems it wasnt possible to download just the last pages of each chapter.
I have looked through the code and the main difficulty is that _kwdict["count"] is not always defined if the chapter extractor is returning an iterator rather than a list.
If that is the case, it will treat -n:-m as (0,sys.maxsize).
If you want I can cache some of the results rather than recalculating them on each call.
Fixes #5402