Commit graph

1760 commits

Author SHA1 Message Date
kim 2300d5e73b
[bugfix] function queue memory pools limitlessly grow (#2882)
* updates the simple queue memory pool to actually self-clean + limit growth

* update memory pool cleaning frequency
2024-05-01 13:30:43 +02:00
kim eb61c783ed
[bugfix] flaky paging test (#2888) 2024-05-01 13:29:42 +02:00
kim a8254a40e7
[bugfix] further paging mishaps (#2884)
* FURTHER paging shenanigans 🥲

* remove cursor logic from ToLinkURL()

* fix up paging tests

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-04-30 16:22:23 +02:00
kim ec7c983e46
[bugfix] retry on http 500 errors *inclusive* (#2886) 2024-04-30 16:18:32 +02:00
kim ec334ece20
[chore] include attemptno in httpclient logs (#2887)
* include request attempt number in httpclient logs

* slightly nicer attempt number formatting
2024-04-30 16:15:50 +02:00
Daenney 39b3a27c82
[docs] Remove last references to RPi (#2885)
This updates the documentation to remove the last stray references to
the copaganda Pi. It now uses the the term single-board computer. GtS
can run fine on all kinds of SBCs and isn't limited to that one
particular fruit version.
2024-04-30 14:27:44 +01:00
kim 4f87ef246c
[bugfix] paging rel links (#2883)
* fix paging so it uses correct cursor query parameter name

* improved code comment

* whoops, flip the cursoring 🤦

* fix the broken test
2024-04-30 12:19:33 +02:00
dependabot[bot] bfc21e4850
[chore]: Bump github.com/tdewolff/minify/v2 from 2.20.19 to 2.20.20 (#2875) 2024-04-29 10:45:17 +00:00
dependabot[bot] d3bac8bbec
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.69 to 7.0.70 (#2877) 2024-04-29 10:44:11 +00:00
tobi 40ece19055
[chore] Fix conflict in workers tests (#2880)
* [chore] Fix conflict in workers tests

* commenty-wenty
2024-04-29 11:43:18 +02:00
dependabot[bot] 1375a86919
[chore]: Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc (#2878) 2024-04-29 09:18:16 +00:00
dependabot[bot] b2eecd9dc7
[chore]: Bump go.opentelemetry.io/otel/sdk from 1.25.0 to 1.26.0 (#2879) 2024-04-29 09:02:12 +00:00
kim 48b91ca239
[bugfix] Fix error string typo (#2873) 2024-04-26 15:20:56 +02:00
kim c9c0773f2c
[performance] update remaining worker pools to use queues (#2865)
* start replacing client + federator + media workers with new worker + queue types

* refactor federatingDB.Delete(), drop queued messages when deleting account / status

* move all queue purging to the processor workers

* undo toolchain updates

* code comments, ensure dereferencer worker pool gets started

* update gruf libraries in readme

* start the job scheduler separately to the worker pools

* reshuffle ordering or server.go + remove duplicate worker start / stop

* update go-list version

* fix vendoring

* move queue invalidation to before wipeing / deletion, to ensure queued work not dropped

* add logging to worker processing functions in testrig, don't start workers in unexpected places

* update go-structr to add (+then rely on) QueueCtx{} type

* ensure more worker pools get started properly in tests

* fix remaining broken tests relying on worker queue logic

* fix account test suite queue popping logic, ensure noop workers do not pull from queue

* move back accidentally shuffled account deletion order

* ensure error (non nil!!) gets passed in refactored federatingDB{}.Delete()

* silently drop deletes from accounts not permitted to

* don't warn log on forwarded deletes

* make if else clauses easier to parse

* use getFederatorMsg()

* improved code comment

* improved code comment re: requesting account delete checks

* remove boolean result from worker start / stop since false = already running or already stopped

* remove optional passed-in http.client

* remove worker starting from the admin CLI commands (we don't need to handle side-effects)

* update prune cli to start scheduler but not all of the workers

* fix rebase issues

* remove redundant return statements

* i'm sorry sir linter
2024-04-26 13:50:46 +01:00
tobi ba4f51ce2f
[chore] update Docker container to use new go swagger hash (#2872) 2024-04-26 12:12:24 +02:00
Daenney 83d24658a8
[chore] Update the flags passed to goreleaser (#2869)
* [chore] Update the flags passed to goreleaser

--rm-dist has been deprecated, the new flag is called --clean.

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-04-26 11:42:33 +02:00
Daenney 6c9bc26a6d
[chore] Update setting testrig loglevel (#2870)
cmp.Or was introduced in Go 1.22 and picks the first value that's not
the zero value for the type. For a string, the zero value is the empty
string, which is what os.Getenv will return if the environment variable
is not set. That then results in "error" being returned instead.

This allows loading an environment variable with a default without
having to do the check and write out the conditional.
2024-04-26 11:31:26 +02:00
tobi fd8a724e77
[chore] Bump go swagger (#2871)
* bump go swagger version

* bump swagger version
2024-04-26 11:31:10 +02:00
Daenney 3a369d834a
[chore] Upgrade our Go version to 1.22 (#2862)
* [chore] Upgrade our Go version to 1.22

With Go 1.22 having been released at the start of February, it's now
been a few months. No major issues have shown up, and the two point
release since then have primarily been security fixes plus some general
bug fixing.

This sets the required Go version to 1.22, as there's nothing in 1.22.1
or 1.22.2 that we would explicitly require. It sets the toolchain to the
latest point release, to ensure we pick up any fixes from there when
building releases etc.

* [chore] Update CI to Go 1.22

* [chore] Update golangci-lint to 1.25.7

Newer version should know about Go 1.22 and run fine.

* [chore] Update Docker container to Go 1.22

* [chore] Update Dockerfile to newer Alpine version

* sign drone.yml

* add missing license header

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-04-26 10:40:29 +02:00
tobi aecf74951c
[chore] Settings refactor 2: the re-refactoring-ing (#2866)
* [chore] Bit more refactoring of settings panel

* fix up some remaining things

* groovy baby yeah!

* remove unused Suspense
2024-04-25 18:24:24 +02:00
tobi 7a1e639483
[chore] Refactor settings panel routing (and other fixes) (#2864) 2024-04-24 11:12:47 +01:00
dependabot[bot] 62788aa116
[chore]: Bump codeberg.org/gruf/go-mutexes from 1.4.0 to 1.4.1 (#2860)
Bumps codeberg.org/gruf/go-mutexes from 1.4.0 to 1.4.1.

---
updated-dependencies:
- dependency-name: codeberg.org/gruf/go-mutexes
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-22 13:43:45 +01:00
dependabot[bot] a57dd15a8e
[chore]: Bump github.com/miekg/dns from 1.1.58 to 1.1.59 (#2861) 2024-04-22 09:11:53 +00:00
kim 12a7eba01f
[chore] bump modernc.org/sqlite to v1.29.8 (with our concurrency workaround) (#2855) 2024-04-22 11:02:11 +02:00
Daenney dcab555a6b
[chore] Update robots.txt (#2856)
This updates the robots.txt based on the list of the ai.robots.txt
repository. We can look at automating that at some point.

It's worth pointing out that some robots, namely the ones by Bytedance,
are known to ignore robots.txt entirely.
2024-04-22 11:01:37 +02:00
dependabot[bot] 0db9e34b69
[chore]: Bump github.com/KimMachineGun/automemlimit from 0.5.0 to 0.6.0 (#2859) 2024-04-22 08:59:30 +00:00
tobi b7c629a18a
[bugfix] Fix incorrect field name for status source, add helpful message (#2854)
* [bugfix] Fix incorrect field name for status source, add helpful message

* swagger

* yyammm
2024-04-18 13:22:55 +02:00
tobi 431505b3e4
[feature] Stub conversations endpoint (#2853) 2024-04-18 12:36:02 +02:00
tobi 34d0159ad5
[feature] Stub account mutes endpoint (#2852)
* [feature] Stub account mutes endpoint

* swagger? i barely know 'er!
2024-04-18 09:59:47 +01:00
kim b3f2d44143
bump to modernc.org/sqlite v1.29.7 (#2850) 2024-04-17 17:10:51 +01:00
kim c67bbe5ba0
update to set requesting account when deleting status (#2849) 2024-04-17 16:54:40 +01:00
tobi cef9924d9a
[feature] Status source endpoint (#2848)
* [feature] statusSource endpoint

* finish up
2024-04-17 13:49:20 +01:00
tobi ef16919d4a
[feature] Stub status history endpoint (#2847) 2024-04-17 13:06:49 +01:00
tobi 8cf685fbe9
[bugfix] Fix minor API issue w/ boosted statuses (#2846) 2024-04-17 12:41:40 +02:00
tobi 6de5717d7f
[bugfix] fix get all tokens (#2841) 2024-04-16 13:25:47 +02:00
tobi 3cceed11b2
[feature/performance] Store account stats in separate table (#2831)
* [feature/performance] Store account stats in separate table, get stats from remote

* test account stats

* add some missing increment / decrement calls

* change stats function signatures

* rejig logging a bit

* use lock when updating stats
2024-04-16 13:10:13 +02:00
kim f79d50b9b2
[performance] cached oauth database types (#2838)
* update token + client code to use struct caches

* add code comments

* slight tweak to default mem ratios

* fix envparsing

* add appropriate invalidate hooks

* update the tokenstore sweeping function to rely on caches

* update to use PutClient()

* add ClientID to list of token struct indices
2024-04-15 14:22:21 +01:00
tobi 8b30709791
[chore] Turn accounts-registration-open false by default (#2839) 2024-04-15 14:41:15 +02:00
kim 1018cde107
[chore] bump bun library versions (#2837) 2024-04-15 12:01:20 +02:00
dependabot[bot] 6bb43f3f9b
[chore]: Bump golang.org/x/net from 0.23.0 to 0.24.0 (#2834) 2024-04-15 08:44:29 +00:00
dependabot[bot] 66e4510bf1
[chore]: Bump golang.org/x/crypto from 0.21.0 to 0.22.0 (#2835) 2024-04-15 08:34:49 +00:00
Bishochiparaa fdd23cb9ed
[chore] Delete the unnecessary #, because this # lead to the wrong URL (#2830) 2024-04-14 09:31:45 +02:00
tobi 89e0cfd874
[feature] Admin accounts endpoints; approve/reject sign-ups (#2826)
* update settings panels, add pending overview + approve/deny functions

* add admin accounts get, approve, reject

* send approved/rejected emails

* use signup URL

* docs!

* email

* swagger

* web linting

* fix email tests

* wee lil fixerinos

* use new paging logic for GetAccounts() series of admin endpoints, small changes to query building

* shuffle useAccountIDIn check *before* adding to query

* fix parse from toot react error

* use `netip.Addr`

* put valid slices in globals

* optimistic updates for account state

---------

Co-authored-by: kim <grufwub@gmail.com>
2024-04-13 13:25:10 +02:00
kim 1439042104
[performance] update GetAccountsByIDs() to use the new multi cache loader endpoint (#2828)
* update GetAccountsByIDs() to use the new multi cache loader endpoint

* fix broken import
2024-04-13 11:36:10 +02:00
tobi a5c6dc9716
[bugfix] Include MIME email headers to avoid mangling non-ascii text (#2827) 2024-04-12 15:18:14 +01:00
dependabot[bot] c097745c38
[chore]: Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc (#2818)
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc](https://github.com/open-telemetry/opentelemetry-go) from 1.24.0 to 1.25.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.24.0...v1.25.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-11 11:46:18 +02:00
kim db2dcc3455
[chore] update go-structr => v0.6.2 (fixes nested field ptr following) (#2822)
* update go-structr => v0.6.1 (fixes nested field ptr following)

* bump to v0.6.2
2024-04-11 11:46:08 +02:00
tobi 9fb8a78f91
[feature] New user sign-up via web page (#2796)
* [feature] User sign-up form and admin notifs

* add chosen + filtered languages to migration

* remove stray comment

* chosen languages schmosen schmanguages

* proper error on local account missing
2024-04-11 11:45:53 +02:00
kim a483bd9e38
[performance] massively improved ActivityPub delivery worker efficiency (#2812)
* add delivery worker type that pulls from queue to httpclient package

* finish up some code commenting, bodge a vendored activity library change, integrate the deliverypool changes into transportcontroller

* hook up queue deletion logic

* support deleting queued http requests by target ID

* don't index APRequest by hostname in the queue

* use gorun

* use the original context's values when wrapping msg type as delivery{}

* actually log in the AP delivery worker ...

* add uncommitted changes

* use errors.AsV2()

* use errorsv2.AsV2()

* finish adding some code comments, add bad host handling to delivery workers

* slightly tweak deliveryworkerpool API, use advanced sender multiplier

* remove PopCtx() method, let others instead rely on Wait()

* shuffle things around to move delivery stuff into transport/ subpkg

* remove dead code

* formatting

* validate request before queueing for delivery

* finish adding code comments, fix up backoff code

* finish adding more code comments

* clamp minimum no. senders to 1

* add start/stop logging to delivery worker, some slight changes

* remove double logging

* use worker ptrs

* expose the embedded log fields in httpclient.Request{}

* ensure request context values are preserved when updating ctx

* add delivery worker tests

* fix linter issues

* ensure delivery worker gets inited in testrig

* fix tests to delivering messages to check worker delivery queue

* update error type to use ptr instead of value receiver

* fix test calling Workers{}.Start() instead of testrig.StartWorkers()

* update docs for advanced-sender-multiplier

* update to the latest activity library version

* add comment about not using httptest.Server{}
2024-04-11 11:45:35 +02:00
dependabot[bot] 15733cddb2
[chore]: Bump github.com/yuin/goldmark from 1.7.0 to 1.7.1 (#2819)
Bumps [github.com/yuin/goldmark](https://github.com/yuin/goldmark) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/yuin/goldmark/releases)
- [Commits](https://github.com/yuin/goldmark/compare/v1.7.0...v1.7.1)

---
updated-dependencies:
- dependency-name: github.com/yuin/goldmark
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-10 12:04:27 +01:00