dullbananas
4ba6221e04
Move SQL triggers from migrations into reusable sql file ( #4333 )
...
* stuff
* stuff including batch_upsert function
* stuff
* do things
* stuff
* different timestamps
* stuff
* Revert changes to comment.rs
* Update comment.rs
* Update comment.rs
* Update post_view.rs
* Update utils.rs
* Update up.sql
* Update up.sql
* Update down.sql
* Update up.sql
* Update main.rs
* use anyhow macro
* Create down.sql
* Create up.sql
* Create replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update utils.rs
* Update .woodpecker.yml
* Update sql_format_check.sh
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Create dump_schema.sh
* Update start_dev_db.sh
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* stuff
* Update replaceable_schema.sql
* Update .pg_format
* fmt
* stuff
* stuff (#21 )
* Update replaceable_schema.sql
* Update up.sql
* Update replaceable_schema.sql
* fmt
* update cargo.lock
* stuff
* Update replaceable_schema.sql
* Remove truncate trigger because truncate is already restricted by foreign keys
* Update replaceable_schema.sql
* fix some things
* Update replaceable_schema.sql
* Update replaceable_schema.sql
* Update .woodpecker.yml
* stuff
* fix TG_OP
* Psql env vars
* try to fix combine_transition_tables parse error
* Revert "try to fix combine_transition_tables parse error"
This reverts commit 75d00a4626a5bcd0834b0066a4b42b
2024-04-17 20:58:44 -04:00
TechVest
b0370ae2fd
chore: fix some comments ( #4637 )
...
Signed-off-by: TechVest <techdashen@qq.com>
2024-04-17 14:35:54 +02:00
Dessalines
5dea21d531
Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613 ( #4614 )
...
* Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613
* Fixing clippy.
2024-04-10 10:14:11 -04:00
Dessalines
705e86eb4c
Fixing docker release ( #4592 )
...
* Re-add notif on tag failure.
* Upping rust version.
* Version 0.19.4-beta.1a
* Try again.
* Version 0.19.4-beta.1b
* Removing unstable inspect.
* Version 0.19.4-beta.1c
* Remove use release cache.
* Trying to fix cargo publish 1.
* Version 0.19.4-beta.1d
* Re-adding publish release
* Version 0.19.4-beta.2
* Fixing workspace for lemmy_federate
2024-04-04 16:14:59 +02:00
Dessalines
95069d7648
Fixing some clippy and woodpecker lints. ( #4565 )
...
* Fixing some clippy and woodpecker lints.
* Try fixing woodpecker 1.
* Revert "Try fixing woodpecker 1."
This reverts commit 7c2020a08d
2024-03-26 10:17:42 +01:00
Dessalines
890565ca14
Deleting denied local_users older than a week. Fixes #4434 ( #4448 )
...
* Deleting denied local_users older than a week. Fixes #4434
* Addressing PR comments.
* Upping rust to 1.76
* Delete the person rows also.
2024-02-15 07:50:53 -05:00
Nutomic
b58da11fb7
Mark instance as alive after successful activity send ( fixes #4039 ) ( #4377 )
...
* Mark instance as alive after successful activity send (fixes #4039 )
* clippy
* Instance::update
* domain
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-19 09:40:12 -05:00
ALEX11BR
a5289dd4cf
Use pretty_assertions for assertions ( #4347 )
...
* Use `pretty_assertions` for assertions
* fixed ordering of `use`s
* ci
---------
Co-authored-by: Felix Ableitner <me@nutomic.com>
2024-01-04 04:47:18 -05:00
dullbananas
8e2cbc9a0f
Remove id column and use different primary key on some tables ( #4093 )
...
* post_saved
* fmt
* remove unique and not null
* put person_id first in primary key and remove index
* use post_saved.find
* change captcha_answer
* remove removal of not null
* comment_aggregates
* comment_like
* comment_saved
* aggregates
* remove "\"
* deduplicate site_aggregates
* person_post_aggregates
* community_moderator
* community_block
* community_person_ban
* custom_emoji_keyword
* federation allow/block list
* federation_queue_state
* instance_block
* local_site_rate_limit, local_user_language, login_token
* person_ban, person_block, person_follower, post_like, post_read, received_activity
* community_follower, community_language, site_language
* fmt
* image_upload
* remove unused newtypes
* remove more indexes
* use .find
* merge
* fix site_aggregates_site function
* fmt
* Primary keys dess (#17 )
* Also order reports by oldest first (ref #4123 ) (#4129 )
* Support signed fetch for federation (fixes #868 ) (#4125 )
* Support signed fetch for federation (fixes #868 )
* taplo
* add federation queue state to get_federated_instances api (#4104 )
* add federation queue state to get_federated_instances api
* feature gate
* move retry sleep function
* move stuff around
* Add UI setting for collapsing bot comments. Fixes #3838 (#4098 )
* Add UI setting for collapsing bot comments. Fixes #3838
* Fixing clippy check.
* Only keep sent and received activities for 7 days (fixes #4113 , fixes #4110 ) (#4131 )
* Only check auth secure on release mode. (#4127 )
* Only check auth secure on release mode.
* Fixing wrong js-client.
* Adding is_debug_mode var.
* Fixing the desktop image on the README. (#4135 )
* Delete dupes and add possibly missing unique constraint on person_aggregates.
* Fixing clippy lints.
---------
Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: phiresky <phireskyde+git@gmail.com>
* fmt
* Update community_block.rs
* Update instance_block.rs
* Update person_block.rs
* Update person_block.rs
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: phiresky <phireskyde+git@gmail.com>
2023-11-13 08:14:07 -05:00
Nutomic
cb01427dcf
Only keep sent and received activities for 7 days ( fixes #4113 , fixes #4110 ) ( #4131 )
2023-11-06 16:17:53 -05:00
dullbananas
a14657d124
Refactor rate limiter and improve rate limit bucket cleanup ( #3937 )
...
* Update rate_limiter.rs
* Update mod.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update mod.rs
* Update scheduled_tasks.rs
* Shrink `RateLimitBucket`
* Update rate_limiter.rs
* Update mod.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update mod.rs
* Update rate_limiter.rs
* fmt
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* Update rate_limiter.rs
* rerun ci
* Update rate_limiter.rs
* Undo changes to fields
* Manually undo changes to RateLimitBucket fields
* fmt
* Bucket cleanup loop in rate_limit/mod.rs
* Remove rate limit bucket cleanup from scheduled_tasks.rs
* Remove ;
* Remove UNINITIALIZED_TOKEN_AMOUNT
* Update rate_limiter.rs
* fmt
* Update rate_limiter.rs
* fmt
* Update rate_limiter.rs
* fmt
* Update rate_limiter.rs
* stuff
* MapLevel trait
* fix merge
* Prevent negative numbers in buckets
* Clean up MapLevel::check
* MapLevel::remove_full_buckets
* stuff
* Use remove_full_buckets to avoid allocations
* stuff
* remove tx
* Remove RateLimitConfig
* Rename settings_updated_channel to rate_limit_cell
* Remove global rate limit cell
* impl Default for RateLimitCell
* bucket_configs doc comment to explain EnumMap
* improve test_rate_limiter
* rename default to with_test_config
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2023-10-19 15:31:51 +02:00
Alex Roman
6735a98d35
Async scheduler ( #3949 )
...
* fix: switch to async scheduler
* fix: pass context to scheduled tasks
* Merge remote-tracking branch 'upstream/main' into async-scheduler
* retrigger ci
* retrigger ci
2023-09-11 11:12:16 +02:00
Dessalines
9785b20843
Adding a scaled sort, to boost smaller communities. ( #3907 )
...
* Adding a scaled sort, to boost smaller communities.
- Previously referred to as *best* .
- Fixes #3622
* Fixing scheduled task update.
* Converting hot_rank integers to floats.
* Altering hot_rank psql function to default to zero after a week.
* Setting scaled_rank to zero, where hot_rank is zero.
* Adding image_upload table.
2023-09-06 13:43:27 -04:00
phiresky
514f2222e0
Fix time zone handling ( #3496 )
...
* convert naive time to utc time
* compounding fixes
* cargo fmt
* fix the rest
* fix down migration
* fix migrations
* fix after merge
* clippy fix
* ap-fed 0.5.0
---------
Co-authored-by: Nutomic <me@nutomic.com>
2023-08-24 11:27:00 -04:00
Alex Roman
ab828b81e4
fix: add client connect timeout to scheduled task ( #3883 )
2023-08-21 10:53:35 +02:00
Denis Dzyubenko
55e383ae38
Fix fetch instance software version from nodeinfo ( #3772 )
...
Fixes #3771
2023-08-01 10:55:28 +02:00
Sander Saarend
77a8e3b897
Don't panic when scheduled tasks can't connect to database ( #3634 )
...
* Don't panic when scheduled tasks can't connect to database
* Use map & map_err instead of match
* Empty commit for Woodpecker re-run
2023-07-19 15:00:44 +02:00
cetra3
9256895635
Cache & Optimize Woodpecker CI ( #3450 )
...
* Try using drone cache plugin
* Try another path
* Include volume
* Fix formatting
* Include fmt
* Exclude cargo dir from prettier
* Don't override cargo
* Just do check
* Add cache key
* Use different cache plugin
* Add clippy
* Try minio
* Add quotes
* Try adding secrets
* Try again
* Again
* Use correct secret formation
* Add back clippy
* Use secret for the root bucket name
* Try drone cache instead
* Add region
* Add path-style option
* Include cargo clippy
* Include everything again
* Fix formatting
* Don't run clippy twice
* Add `allow` statements for tests to pass
* Adjust endpoint to be a secret
* Fix prettier
* Merge & fix tests
* Try to restart the woodpecker test
* Change the ENV var name
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-17 11:04:14 -04:00
Sander Saarend
6688a8a5d4
Optimize hot rank updates ( #3617 )
2023-07-17 11:05:55 +02:00
Nutomic
e9e76549a8
Split activity table into sent and received parts ( fixes #3103 ) ( #3583 )
...
* Split activity table into sent and received parts (fixes #3103 )
The received activities are only stored in order to avoid processing
the same incoming activity multiple times. For this purpose it is
completely unnecessary to store the data. So we can split the
table into sent_activity and received_activity parts, where
only sent_activity table needs to store activity data. This should
reduce storage use significantly.
Also reduces activity storage duration to three months, we can reduce
this further if necessary.
Additionally the id columns of activity tables are removed because
they are completely unused and risk overflowing (fixes #3560 ).
* address review
* move insert_received_activity() methods to verify handlers
* remove unnecessary conflict line
* clippy
* use on conflict, add tests
2023-07-14 11:17:06 -04:00
Nutomic
7d8cb93b53
Check for dead federated instances ( fixes #2221 ) ( #3427 )
...
* Check for dead federated instances (fixes #2221 )
* move to apub crate, use timestamp
* make it compile
* clippy
* use moka to cache blocklists, dead instances, restore orig scheduled tasks
* remove leftover last_alive var
* error handling
* wip
* fix alive check for instances without nodeinfo, add coalesce
* clippy
* move federation blocklist cache to #3486
* unused deps
2023-07-13 16:12:01 +02:00
Sander Saarend
651f2747ee
Only update site_aggregates for local site ( #3516 )
2023-07-10 11:20:39 -04:00
TKilFree
2aef6a5a33
feat: re-added captcha checks ( #3289 )
2023-06-27 06:38:53 -04:00
Sander Saarend
211e76dc27
Batch hot rank updates ( #3175 )
2023-06-27 10:13:51 +02:00
Vijay Ramesh
4db65c191c
after 30 days post deletion, replace comment.content and post.body with 'Permanently Deleted'
2023-06-21 12:46:56 -07:00
dullbananas
45818fb4c5
Reduce memory usage of rate limiting ( #3111 )
...
* Reduce Vec allocations
* Optimize stuff
* Move embedded migrations to separate crate
* Revert "Move embedded migrations to separate crate"
This reverts commit 44b1049970b4528e5b8508e835d487#1 )
* Update rate_limiter.rs
* Update mod.rs
* Update scheduled_tasks.rs
* Fix rate_limiter.rs
* Dullbananas patch 1 (#2 )
* Update rate_limiter.rs
* Update mod.rs
* Update scheduled_tasks.rs
* Fix rate_limiter.rs
* Rate limit IPv6 addresses in groups
* Fmt lib.rs
* woodpicker trigger
* Refactor and comment `check_rate_limit_full`
* Add `test_split_ipv6`
* Replace -2.0 with UNINITIALIZED_TOKEN_AMOUNT
* Add `test_rate_limiter`
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-06-21 10:28:20 +02:00
Dessalines
981c111567
Move connection creation into scheduler. ( #3120 )
...
* Move connection creation into scheduler.
- #3076
* Fix clippy.
2023-06-20 11:33:03 +02:00
Sander Saarend
68d814b9b1
Handle scheduled task errors ( #3090 )
...
* Add --disable-scheduled-tasks CLI flag
* Add error handling for scheduled tasks
2023-06-15 11:29:12 +02:00
Dessalines
8cb5939f50
Adding hot_rank columns in place of function sorting. ( #2952 )
...
* Adding hot_rank columns in place of function sorting.
- Creates hot_rank columns for post, comment, and community.
- Fixes #2932
* Updating all hot ranks on startup.
* Fixing post.url migration.
* Removing update_instance_software from startup.
* Adding post_rank query
2023-06-08 16:15:15 -04:00
Dessalines
1b5437cbe3
Addressing slow profile queries. #2777 ( #2830 )
...
* Addressing slow profile queries. #2777
* Addressing PR comments.
2023-04-25 19:28:06 -04:00
Dessalines
209c8a9185
Switch from bb8 to deadpool. Fixes #2765 ( #2768 )
...
* Switch from bb8 to deadpool. Fixes #2765
* Remove unecessary deadpool dependency.
* Ignoring nodeinfo test.
2023-02-28 22:45:37 +01:00
Dessalines
3735c6fabf
Adding instance software and version. Fixes #2222 ( #2733 )
...
* Adding instance software and version. Fixes #2222
* Fix clippy.
* Fix clippy 2
* Fixing some more issues.
2023-02-18 09:36:12 -05:00
Dessalines
a610211557
Fixing .drone.yml ( #2677 )
...
* Try to fix docker/drone plugin
* Trying to use one rust image.
* Fixing drone 1.
* Fixing drone 2.
* Add drone notify.
* Fixing drone 3.
* Fixing drone 4.
* Fix clippy.
* Remove uninlined format lint.
* Combine all cargo tasks
* Fixing drone 5.
* Fixing drone 6.
* Fixing drone 7.
* Fixing drone 8.
* Fixing drone 9.
* Fixing drone 10.
* Fixing drone 12.
* Fixing drone 13.
* Fixing drone 14.
* Fixing drone 15.
* Fixing drone 16.
* Fixing drone 17.
* Fixing drone 18.
* Fixing drone 19.
* Fixing drone 20.
* Fixing drone 21.
* Fixing drone 22.
* Fixing drone 23.
* Fixing drone 24.
* Fixing drone 25.
* Fixing drone 26.
* Fixing drone 27.
* Fixing drone 28.
* Fixing drone 29.
* Fixing drone 30.
* Fixing drone 31.
* Fixing drone 32.
* Fixing drone 33.
* Fixing drone 34.
* Fixing drone 35.
* Fixing drone 36.
* Fixing drone 37.
* Fixing drone 38.
* Fixing drone 39.
* Fixing drone 40.
* Fixing drone 41.
* Fixing drone 43.
* Fixing drone 44.
* Fixing drone 45.
* Last cleanup.
* Fixing drone 46.
* Separate ci steps (#2679 )
* separate ci steps
* fix 1
* add comments
* dont add rustfmt explicitly
* Revert "dont add rustfmt explicitly"
This reverts commit 358ce3302a134b7ac88d90a854079356995e9725.
* dont use all features for tests
---------
Co-authored-by: Nutomic <me@nutomic.com>
2023-01-30 14:17:24 -05:00
Nutomic
6f3bf4634b
Various pedantic clippy fixes ( #2568 )
...
* Various pedantic clippy fixes
* more clippy pedantic fixes
* try to fix ci
* add fix clippy script, use rust 1.65
* fix clippy
2022-11-19 04:33:54 +00:00
Dessalines
5d837780f5
Add diesel_async, get rid of blocking function ( #2510 )
...
* Moving settings to Database.
- Moves many settings into the database. Fixes #2285
- Adds a local_site and instance table. Fixes #2365 . Fixes #2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.
* Fix weird clippy issue.
* Removing extra lines.
* Some fixes from suggestions.
* Fixing apub tests.
* Using instance creation helper function.
* Move forms to their own line.
* Trying to fix local_site_data, still broken.
* Testing out async
* Testing out async 2
* Fixing federation tests.
* Trying to fix check features 1.
* Starting on adding diesel async. 1/4th done.
* Added async to views and schema.
* Adding some more async
* Compiling now.
* Added diesel async. Fixes #2465
* Running clippy --fix
* Trying to fix cargo test on drone.
* Trying new muslrust.
* Trying a custom dns
* Trying a custom dns 2
* Trying a custom dns 3
* Trying a custom dns 4
* Trying a custom dns 5
* Trying a custom dns 6
* Trying a custom dns 7
* Addressing PR comments.
* Adding check_apub to all verify functions.
* Reverting back drone.
* Fixing merge
* Fix docker images.
* Adding missing discussion_languages.
* Trying to fix federation tests.
* Fix site setup user creation.
* Fix clippy
* Fix clippy 2
* Test api faster
* Try to fix 1
* Try to fix 2
* What are these lines about
* Trying to fix 3
* Moving federation test back to top.
* Remove logging cat.
2022-11-09 10:05:00 +00:00
Dessalines
c9f1407429
Diesel 2.0.0 upgrade ( #2452 )
...
* Initial commit to bump diesel to 2.0.0-rc.0 and see what happens
* Add chrono feature from diesel
* db_schema crate is close to building?
* Upgrade diesel-derive-newtype
* Mostly modifying references to connections to be mutable ones; also used
new way to do migrations as suggested by the migration guide; a lot more
compiles now, though I can't figure out this tricky ToSql issue at the
moment
* Running clippy --fix
* Trying to fix drone clippy 1
* Fix clippy
* Upgrade clux-musl
* Trying to fix drone clippy 2
* Trying to fix drone clippy 3
* Trying to fix drone clippy 5
* Adding diesel table aliases, removing sql view hack. Fixes #2101
Co-authored-by: Steven Chu <stevenc1@gmail.com>
Co-authored-by: Nutomic <me@nutomic.com>
2022-09-26 14:09:32 +00:00
Dessalines
eacd4e44cd
Adding job to drop phantom ccnew indexes. Fixes #2431 ( #2432 )
2022-09-07 12:12:51 +00:00
Nutomic
a2a594b763
Extract Activitypub logic into separate library ( #2288 )
...
* Create example for apub lib
* some rewriting of apub lib
* Add LocalInstance struct for apub lib to avoid using Lemmy Settings
* Move ActorType trait to lemmy_apub, because its not needed in library
* Use reqwest_retry instead of custom impl, dont specify timeout on every send()
* Some improvements to example
* Moved inbox handling to library
* bug fixes
* Move context and serde helpers into library
* wip: example changes
* Add lemmy_utils feature to build only LemmyError
* Rename to activitypub_federation
* Remove lemmy_utils dep from activitypub_federation using generic error type
* Finish activitypub example
* Cleanup and fix tests
* Reorganize library files
* Remove ApubObject.to_tombstone()
* Extract activitypub library into separate git repository
2022-06-02 16:33:41 +02:00
Nutomic
3aa3d75a1e
Add cargo feature for building lemmy_api_common with mininum deps ( #2243 )
2022-05-03 17:44:13 +00:00
Dessalines
4cf0da7b60
Clippy fixes.
2022-03-30 10:58:03 -04:00
Dessalines
3ad172e8ed
Adding a ban expires update job. Fixes #2177
2022-03-30 09:56:23 -04:00
Riley
3fea5645f8
Add tracing ( #1942 )
2021-11-23 12:16:47 +00:00
Dessalines
f035af3738
Fixing clippy
2021-11-09 17:31:28 -05:00
Felix Ableitner
f24999027e
Merge crates db_schema and db_queries
2021-10-20 12:28:08 +02:00
Dessalines
53a2b6d013
Trying a background_jobs fix. #1820 ( #1822 )
2021-10-08 14:36:53 +00:00
Dessalines
0a91a28b27
Reindex tables non-concurrently on startup. Fixes #1726 . #1723 ( #1736 )
2021-08-26 11:49:16 +00:00
Dessalines
f54209c451
Running clippy --fix ( #1647 )
2021-07-05 16:07:26 +00:00
Felix Ableitner
7f56281c26
Forbid usage of unwrap
2021-03-01 19:24:34 +01:00
Dessalines
0fd0279543
Adding some recurring lemmy tasks. ( #1386 )
...
* Adding some recurring lemmy tasks.
- Add active users by day, week, month, and half year to site and
community. Fixes #1195
- Periodically re-index the aggregates tables that use hot_rank.
Fixes #1384
- Clear out old activities (> 6 months). Fixes #1133
* Some cleanup, recalculating actives every hour.
2021-01-29 11:38:27 -05:00
