specifying a version breaks file generation with `-mod=vendor`, which is
used by tooling like nixpkgs.
this commit replaces the `go:generate` statement with ones found in
other files (which builds successfully) for consistency.
Signed-off-by: Cassie Cheung <me@soopy.moe>
* fix: Correctly format listener address (https://github.com/TecharoHQ/anubis/issues/93)
Handle addresses that include a hostname, not just ports. If
the address starts with a colon, assume it's just a port and
prefix it with "http://localhost". Otherwise, prefix the
entire address with "http://". This ensures that the listener
URL is correctly formatted regardless of whether it includes
a hostname or just a port.
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* chore(docs): add changelog entry
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
---------
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* web/js: add project license in the JavaScript used by Anubis
This will allow LibreJS users to pass the captcha without problems
without having to whitelist anubis manually.
* Update docs/docs/CHANGELOG.md
Co-authored-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Fijxu <fijxu@nadeko.net>
---------
Signed-off-by: Fijxu <fijxu@nadeko.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* cmd/anubis: add a debug option for benchmarking hashrate
Having the ability to benchmark different proof-of-work implementations
is useful for extending Anubis. This adds a flag `--debug-benchmark-js`
(and its associated environment variable `DEBUG_BENCHMARK_JS`) for
serving a tool to do so.
Internally, a there is a new policy action, "DEBUG_BENCHMARK", which
serves the benchmarking tool instead of a challenge. The flag then
replaces all bot rules with a special rule matching every request
to that action. The benchmark page makes heavy use of inline styles,
because currently all global styles are shared across all pages. This
could be fixed, but I wanted to avoid major changes to the templates.
* web/js: add signal for aborting an active proof-of-work algorithm
Both proof-of-work algorithms now take an optional `AbortSignal`, which
immediately terminates all workers and returns `false` if aborted before
the challenge is complete.
* web/js: add algorithm comparison to the benchmark page
"Compare:" is added to the benchmark page for testing the relative
performance between two algorithms. Since benchmark runs generally have
high variance, it may take a while for the averages to converge on a
stable difference.
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* Add periodic cleanup job for DecayMap
see https://github.com/TecharoHQ/anubis/issues/8
* Refactor: Improve DecayMap cleanup tests and add Len method
- Refactored DecayMap cleanup tests to use the new Len method
for more precise assertions.
- Added a Len method to DecayMap to retrieve the number of
entries.
- Simplified conditional checks in Get method.
* chore(changelog): add entry
* fix(tests): Use Impl.expire for decaymap cleanup
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
---------
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
Using TrimRight will remove all characters from `*dockerRepo` from right
to left that match a character contained on `"/"+filepath.Base(*dockerRepo)`
(the cutset) until it doesn't matches anymore.
So for example, if `dockerRepo` is `example.com/fijxu/anubis`, and
`"/"+filepath.Base(*dockerRepo)` is `/anubis`, it will remove
`u/anubis` and not just `/anubis` from `dockerRepo` because `u` is a character inside the
cutoff.
Since the challenge is done off of the main thread, there is no simple
way to report the progress done towards completing it. This change
adds a callback parameter, `progressCallback`, which is called with
the most recently attempted nonce every ~1024 iterations (should this
be configurable?). For the single-threaded "slow" algorithm, this is
exactly every 1024 iterations. For the multi-threaded "fast" algorithm,
threads take turns reporting in a round-robin as then notice they
have passed a multiple of 1024. This complexity is to avoid individual
threads falling behind their siblings due to the overhead of messaging
the main thread. To minimize this overhead as much as possible, a
regular number is sent instead of an object.
With the new information provided by the callback, a hash rate display
is added to the challenge page. This display is updated at most once
per second and set with tabular numbers to avoid the constantly changing
value being too visually distracting.
* web: show a progress bar based on completion probability
To provide more feedback to the user, the spinner is replaced with a
progress bar of the probability the challenge is complete. Since it
looks a little weird that a progress bar would fill up a quarter of the
way and then jump to the end (even though the probability would make
that happen 1 in 4 times), the bar is mapped with a quadratic easing
function to move faster at the beginning and then slow down as the
probability of redirection increases. If the probability exceeds 90%,
a message appears letting the user know things are taking longer than
expected and to continue being patient.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* Add middleware to set Cache-Control header for challenge HTML
* Add `NoStoreCache` middleware function in `internal/headers.go` to set Cache-Control: no-store header
* Apply `NoStoreCache` middleware in `cmd/anubis/main.go` to set Cache-Control header for challenge HTML
* docs: Add no-cache header information for challenge page
* docs: Update changelog to reflect no-store Cache-Control header addition for challenge page
* refactor: rename variable for clarity and update caching middleware in RenderIndex
* chore: move changes to the unreleased section
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
---------
Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com>
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
Closes#125Closes#40
Among other things, this moves all of the asset generation to run within
the context of an npm script. Developer documentation stubs have been
added so that people can get started more easily.
The top-level Dockerfile (which is no longer used in production) has
been removed as its presence has been causing confusion. This changeset
will break it anyways.
These changes will make for less "repo churn" as the static assets are
built and rebuilt, at the cost of making the build step more complicated
for downstream packagers. If this becomes a burden, we can explore
making a "release tarball" that contains pre-massaged outputs.
* Change how to make Anubis work without a reverse proxy
* Apply suggestions from code review
Co-authored-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Henri Vasserman <henv@hot.ee>
* add support for unix sockets.
* add env var docs
* lib: fix tests
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Henri Vasserman <henv@hot.ee>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
Closes#109
This was a hack I did on stream. I thought this would have a positive
effect, but a combination of real-world testing from people using Anubis
in prod and gray-hat testing has proven this is an unfeature and is
probably causing more harm than good at this stage.
In the future I'll probably make the `dnsbl` block more flexible so that
you can specify your own lists and rules around them.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(docs): Make dark mode diff lines readable
If using dark mode, these lines are not legible at all. I separated the colors into variables and added
more contrasting colors for the dark mode.
* chore: add to changelog
* cmd/anubis: allow setting key bytes in flag/envvar
Docs are updated to generate a random key on load and when people press
the recycle button.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* review feedback fixups
Signed-off-by: Xe Iaso <me@xeiaso.net>
* Update cmd/anubis/main.go
Signed-off-by: Xe Iaso <me@xeiaso.net>
* Apply suggestions from code review
Co-authored-by: Ryan Cao <70191398+ryanccn@users.noreply.github.com>
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Ryan Cao <70191398+ryanccn@users.noreply.github.com>
Removes `main` from the repo and updates `.gitignore` to help prevent
accidents like this again. `.test` comes from go test binaries, which I
feel is also worth excluding
Signed-off-by: Jared Allard <jaredallard@users.noreply.github.com>
* Refactor anubis to split business logic into a lib, and cmd to just be direct usage.
* Post-rebase fixes.
* Update changelog, remove unnecessary one.
* lib: refactor this
This is mostly based on my personal preferences for how Go code should
be laid out. I'm not sold on the package name "lib" (I'd call it anubis
but that would stutter), but people are probably gonna import it as
libanubis so it's likely fine.
Packages have been "flattened" to centralize implementation with area of
concern. This goes against the Java-esque style that many people like,
but I think this helps make things simple.
Most notably: the dnsbl client (which is a hack) is an internal package
until it's made more generic. Then it can be made external.
I also fixed the logic such that `go generate` works and rebased on
main.
* internal/test: run tests iff npx exists and DONT_USE_NETWORK is not set
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: install deps
Signed-off-by: Xe Iaso <me@xeiaso.net>
* .github/workflows: verbose go tests?
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: sleep 2
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: nix this test so CI works
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: warmup per browser?
Signed-off-by: Xe Iaso <me@xeiaso.net>
* internal/test: disable for now :(
Signed-off-by: Xe Iaso <me@xeiaso.net>
* lib/anubis: do not apply bot rules if address check fails
Closes#83
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* Cleanup regex
Were were going overkill on the escape characters
* Update docs/docs/CHANGELOG.md
Co-authored-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Dennis ten Hoove <36002865+dennis1248@users.noreply.github.com>
---------
Signed-off-by: Dennis ten Hoove <36002865+dennis1248@users.noreply.github.com>
Co-authored-by: Xe Iaso <me@xeiaso.net>
The example/default bot policy document had a rule to allow RSS readers
through based on paths that end with ".rss", ".xml", ".atom", or
".json". Frameworks like Rails will treat these specially, meaning that
going to /things/12345-whateverhaha.json could bypass Anubis.
I checked the history of this rule and it was present in the original
example policy file in Xe/x. This rule is likely a mistake and it has
been removed. I think it was for making my blog still work with RSS
readers.
Thanks to Graham Sutherland for reporting this over email.
Signed-off-by: Xe Iaso <me@xeiaso.net>
hash.Write never returns error so removing it from
the results simplifies usage and eliminates dead error handling.
Signed-off-by: Alexander Yastrebov <yastrebov.alex@gmail.com>
* Added the possibility to define rules for remote addresses
* Added change in changelog
* Added check for X-Real-Ip and X-Forwarded-For when checking for remote address filtering
* cmd/anubis: refine IP filtering logic
* Optimize the configuration so that the IP trie is created once at
application start instead of dynamically being created every request.
* Document the changes in the changelog and docs site.
* Allow pure IP range filtering.
* Allow user agent based IP range filtering.
* Allow path based IP range filtering.
* Create --debug-x-real-ip-default flag for testing Anubis locally
without a HTTP load balancer.
---------
Co-authored-by: Xe Iaso <me@xeiaso.net>
Closes#30
Introduces the "challenge" field in bot rule definitions:
```json
{
"name": "generic-bot-catchall",
"user_agent_regex": "(?i:bot|crawler)",
"action": "CHALLENGE",
"challenge": {
"difficulty": 16,
"report_as": 4,
"algorithm": "slow"
}
}
```
This makes Anubis return a challenge page for every user agent with
"bot" or "crawler" in it (case-insensitively) with difficulty 16 using
the old "slow" algorithm but reporting in the client as difficulty 4.
This is useful when you want to make certain clients in particular
suffer.
Additional validation and testing logic has been added to make sure
that users do not define "impossible" challenge settings.
If no algorithm is specified, Anubis defaults to the "fast" algorithm.
Signed-off-by: Xe Iaso <me@xeiaso.net>
