| Commit message (Collapse) | Author | Lines |
|---|
|
This commit enables ruff's flake8-builtin linter that emits warnings
when builtin functions are shadowed. This is useful for builtins like
"dict", "list", or "str" which we use often.
Given the nature of this program we historically rely a lot on the usage
of "id", "hash", and "filter" as variable names which also shadow Python
builtins. For now let's ignore those, we have not used any of them in
our code and the impact to the codebase would be considerable. This
might be revisited in the future.
|
|
Sadly we couldn't find out what this was meant to do, but we believe it
was added in error. All tests pass with this removed, so we can drop it.
|
|
This makes the code clearer and saves a whole indentation level.
|
|
|
|
It might be that we get a valid (maybe empty) response from the API, in
which case we do not want to simply crash because we expect the
'gmetadata' field in the response. Instead, throw a proper ScrapeError
for it.
|
|
|
|
Non-H usually has nothing to censor, so this should be a safe default.
We have not come across anything where this would have been a false
positive.
|
|
Now that we have this in our utility suite, we can make use of it in the
built-in scraper plugins. This increases coverage and removes a lot of
duplicate code.
|
|
If a parser function returned None we yield it regardless, even though
it won't have any impact further down the line. Instead clean up the
collect() stream as early as possible.
|
|
|
|
|
|
|
|
We can expect a number of scraper sources to either give languages as
ISO 639-3 or as their English name, so it makes sense to implement a
simple parser method on our side.
|
|
Even though our base class here is abstract, this method is not, so we
can ignore B027 [1].
[1] https://docs.astral.sh/ruff/rules/empty-method-without-abstract-decorator/
|
|
This fixes flake8-bugbear's B904 [1].
[1] https://docs.astral.sh/ruff/rules/raise-without-from-inside-except/
|
|
This cuts down on code duplication and also fixes B023 [1].
[1] https://docs.astral.sh/ruff/rules/function-uses-loop-variable/#function-uses-loop-variable-b023
|
|
See https://docs.astral.sh/ruff/rules/mutable-argument-default/
|
|
|
|
|
|
This was uncovered by bugbear, but did not seem to have tripped our
test. Fix it anyway.
|
|
|
|
GraphQL integers are 32-bit as per spec [1] [2]. Implementations may
therefore error on large numbers. Since an archive's size can reasonably
exceed this value, make sure to report it as a float instead.
[1] https://graphql.org/learn/schema/
[2] https://github.com/graphql/graphql-js/issues/292#issuecomment-186702763
|
