mirror of
https://github.com/TecharoHQ/anubis.git
synced 2025-08-03 01:38:14 -04:00

* feat: Add Open Graph tag support (og-tags) Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Fix: Prevent nil pointer dereference in test (og-tags) Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat!: Implement Open Graph tag caching and passthrough functionality (WIP) I'm going to sleep. currently tags are passed to renderIndex. see https://github.com/TecharoHQ/anubis/issues/131 Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat: Add configuration for air tool with build and logger settings Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat: Move OG tags to base template (og-tags) Moves the Open Graph (OG) tags from the index template to the base template. This allows OG tags to be set on any page, not just the index. Also adds a BaseWithOGTags function to the web package to allow passing OG tags to the base template. Removes the ogTags parameter from the Index function and template. Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Delete CHANGELOG.md Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat: Add language attribute to HTML tag in template Signed-off-by: Jason Cameron <git@jasoncameron.dev> * fix(tests): Fix nil pointer ref Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat(og-tags): Add timeout to http client (og-tags) Signed-off-by: Jason Cameron <git@jasoncameron.dev> * style: fix line endings & indentation Signed-off-by: Jason Cameron <git@jasoncameron.dev> * style: add inspection comment for GoBoolExpressions in UnchangingCache Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat(og-tags): Implement Open Graph tag fetching and caching Signed-off-by: Jason Cameron <git@jasoncameron.dev> * fix(og-tags): Simplify Open Graph tag extraction logic Signed-off-by: Jason Cameron <git@jasoncameron.dev> * fix(og-tags): Add nil check in isOGMetaTag and enhance test cases Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat(og-tags): Add approved tags and prefixes for Open Graph extraction Signed-off-by: Jason Cameron <git@jasoncameron.dev> * test(og-tags): Update tests with approved tags and improve clarity Signed-off-by: Jason Cameron <git@jasoncameron.dev> * chore: Add changelog notes Signed-off-by: Jason Cameron <git@jasoncameron.dev> * fix: Improve stability of the target fetcher? Signed-off-by: Jason Cameron <git@jasoncameron.dev> * fix: Update template error handling and improve Open Graph tag integration Signed-off-by: Jason Cameron <git@jasoncameron.dev> * style: format files and remove deubg logs Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat: Credit CELPHASE for mascot design (og-tags) Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat: Credit CELPHASE for mascot design (og-tags) Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat: Allow twitter prefixed OG tags by default Signed-off-by: Jason Cameron <git@jasoncameron.dev> * chore: replace /tmp with /var Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Update docs/docs/CHANGELOG.md Co-authored-by: Xe Iaso <me@xeiaso.net> Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com> * Update docs/docs/admin/configuration/open-graph.mdx Co-authored-by: Xe Iaso <me@xeiaso.net> Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com> * chore: add fediverse to default prefixes (#og-tags) Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat(og-tags): Remove og-query-distinct flag This commit removes the `og-query-distinct` flag and associated logic. URLs with different query parameters will now always be treated as the same cache key for Open Graph tags. This simplifies the caching logic and improves performance. Additionally, the http client used for fetching OG tags is now a member of the OGTagCache struct, rather than a global variable. This improves testability and allows for more flexible configuration in the future. Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Update docs/docs/admin/configuration/open-graph.mdx Co-authored-by: Xe Iaso <me@xeiaso.net> Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com> * docs: remove og tags references Signed-off-by: Jason Cameron <git@jasoncameron.dev> * refactor: rename url > u to not overlap package name Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Update internal/ogtags/cache.go Co-authored-by: Xe Iaso <me@xeiaso.net> Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com> * Update internal/ogtags/cache.go Co-authored-by: Xe Iaso <me@xeiaso.net> Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com> * fix(tests): Don't use network when network access is disabled Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Fix: Handle nil URL in GetOGTags (og-tags) Signed-off-by: Jason Cameron <git@jasoncameron.dev> * chore: sort installation docs alphabetically Signed-off-by: Jason Cameron <git@jasoncameron.dev> * fix(tests): validate that no duplicate requests are made Signed-off-by: Jason Cameron <git@jasoncameron.dev> * style(tests): remove unused ok var Signed-off-by: Jason Cameron <git@jasoncameron.dev> * docs: convert to table fmt Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat(og-tags): Enhance OG tag fetching and caching Adds additional approved OG tags (`keywords`, `author`), improves Signed-off-by: Jason Cameron <git@jasoncameron.dev> * chore: update generated templ's after format Signed-off-by: Jason Cameron <git@jasoncameron.dev> * fix(tests): update integration_test.go to reflect the new behavior of fetchHTMLDocument Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Revert "data/botPolicies: allow iMessage scraper by default (#178)" This reverts commit 21a9d777 Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Fix: Simplify ogTags access in cache test. Didn't know this was possible! wow! Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Fix: Handle request timeouts when fetching OG tags (#og-tags) Cache a nil result for half the TTL to avoid repeatedly requesting a timed-out URL. Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Fix: make OG tags passthrough option function. Signed-off-by: Jason Cameron <git@jasoncameron.dev> * Fix: Handle timeouts and non-200 responses when fetching OG tags (og-tags) - Cache empty results for timeouts and non-200 status codes to avoid spamming the server. - Use a non-nil empty map to represent empty results in the cache, as nil would be a cache miss. Signed-off-by: Jason Cameron <git@jasoncameron.dev> * feat(og-tags): switch to http.MaxBytesReader Signed-off-by: Jason Cameron <git@jasoncameron.dev> * chore(og-tags): add noindex, nofollow meta tag and update error line numbers Signed-off-by: Jason Cameron <git@jasoncameron.dev> --------- Signed-off-by: Jason Cameron <git@jasoncameron.dev> Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com> Co-authored-by: Xe Iaso <me@xeiaso.net>
86 lines
2.4 KiB
Go
86 lines
2.4 KiB
Go
package internal
|
|
|
|
import (
|
|
"log/slog"
|
|
"net"
|
|
"net/http"
|
|
"strings"
|
|
|
|
"github.com/TecharoHQ/anubis"
|
|
"github.com/sebest/xff"
|
|
)
|
|
|
|
// UnchangingCache sets the Cache-Control header to cache a response for 1 year if
|
|
// and only if the application is compiled in "release" mode by Docker.
|
|
func UnchangingCache(next http.Handler) http.Handler {
|
|
//goland:noinspection GoBoolExpressions
|
|
if anubis.Version == "devel" {
|
|
return next
|
|
}
|
|
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.Header().Set("Cache-Control", "public, max-age=31536000")
|
|
next.ServeHTTP(w, r)
|
|
})
|
|
}
|
|
|
|
// RemoteXRealIP sets the X-Real-Ip header to the request's real IP if
|
|
// the setting is enabled by the user.
|
|
func RemoteXRealIP(useRemoteAddress bool, bindNetwork string, next http.Handler) http.Handler {
|
|
if !useRemoteAddress {
|
|
slog.Debug("skipping middleware, useRemoteAddress is empty")
|
|
return next
|
|
}
|
|
|
|
if bindNetwork == "unix" {
|
|
// For local sockets there is no real remote address but the localhost
|
|
// address should be sensible.
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
r.Header.Set("X-Real-Ip", "127.0.0.1")
|
|
next.ServeHTTP(w, r)
|
|
})
|
|
}
|
|
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
host, _, err := net.SplitHostPort(r.RemoteAddr)
|
|
if err != nil {
|
|
panic(err) // this should never happen
|
|
}
|
|
r.Header.Set("X-Real-Ip", host)
|
|
next.ServeHTTP(w, r)
|
|
})
|
|
}
|
|
|
|
// XForwardedForToXRealIP sets the X-Real-Ip header based on the contents
|
|
// of the X-Forwarded-For header.
|
|
func XForwardedForToXRealIP(next http.Handler) http.Handler {
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
if xffHeader := r.Header.Get("X-Forwarded-For"); r.Header.Get("X-Real-Ip") == "" && xffHeader != "" {
|
|
ip := xff.Parse(xffHeader)
|
|
slog.Debug("setting x-real-ip", "val", ip)
|
|
r.Header.Set("X-Real-Ip", ip)
|
|
}
|
|
|
|
next.ServeHTTP(w, r)
|
|
})
|
|
}
|
|
|
|
// NoStoreCache sets the Cache-Control header to no-store for the response.
|
|
func NoStoreCache(next http.Handler) http.Handler {
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.Header().Set("Cache-Control", "no-store")
|
|
next.ServeHTTP(w, r)
|
|
})
|
|
}
|
|
|
|
// Do not allow browsing directory listings in paths that end with /
|
|
func NoBrowsing(next http.Handler) http.Handler {
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
if strings.HasSuffix(r.URL.Path, "/") {
|
|
http.NotFound(w, r)
|
|
return
|
|
}
|
|
next.ServeHTTP(w, r)
|
|
})
|
|
}
|