| Commit message (Collapse) | Author | Lines |
|
|
|
It might be that we get a valid (maybe empty) response from the API, in
which case we do not want to simply crash because we expect the
'gmetadata' field in the response. Instead, throw a proper ScrapeError
for it.
|
|
|
|
Non-H usually has nothing to censor, so this should be a safe default.
We have not come across anything where this would have been a false
positive.
|
|
Now that we have this in our utility suite, we can make use of it in the
built-in scraper plugins. This increases coverage and removes a lot of
duplicate code.
|
|
If a parser function returned None we yield it regardless, even though
it won't have any impact further down the line. Instead clean up the
collect() stream as early as possible.
|
|
|
|
|
|
|
|
We can expect a number of scraper sources to either give languages as
ISO 639-3 or as their English name, so it makes sense to implement a
simple parser method on our side.
|
|
Even though our base class here is abstract, this method is not, so we
can ignore B027 [1].
[1] https://docs.astral.sh/ruff/rules/empty-method-without-abstract-decorator/
|
|
This fixes flake8-bugbear's B904 [1].
[1] https://docs.astral.sh/ruff/rules/raise-without-from-inside-except/
|
|
This cuts down on code duplication and also fixes B023 [1].
[1] https://docs.astral.sh/ruff/rules/function-uses-loop-variable/#function-uses-loop-variable-b023
|
|
See https://docs.astral.sh/ruff/rules/mutable-argument-default/
|
|
|
|
|
|
This was uncovered by bugbear, but did not seem to have tripped our
test. Fix it anyway.
|
|
|
|
GraphQL integers are 32-bit as per spec [1] [2]. Implementations may
therefore error on large numbers. Since an archive's size can reasonably
exceed this value, make sure to report it as a float instead.
[1] https://graphql.org/learn/schema/
[2] https://github.com/graphql/graphql-js/issues/292#issuecomment-186702763
|