Commit Graph

207 Commits (1e69d62a6ab6e1ad088661d598111e08bfd73efc)

Author SHA1 Message Date
Dessalines a3bf2f1cf1
Auto resolve reports on removing a comment or post. Fixes #4390 (#4402)
* Automatically resolve report when post/comment is removed (#3850)

* Automatically resolve report when post/comment is removed

* also handle apub removes

* Removing auto-resolve report triggers.

* Dont allow creating reports for deleted / removed items.

* Running pgformat.

* Fixing test.

* Addressing PR comments.

* Forgot comment report.

---------

Co-authored-by: Nutomic <me@nutomic.com>
2024-02-15 08:52:04 -05:00
Nutomic 0f414a95d5
Local only community (#4350)
* Add support for local only community (fixes #1576)

* add filters and tests to db views

* dont federate local only community

* test get apub community http

* tests

* more checks

* wip

* api test

* fix tests

* change community.local_only column to visibility enum
(for private communities)

* sql fmt

* rename vars

* clippy

* fix tests

* update lib

* review

* fix js client version

* update client
2024-01-25 11:04:25 -05:00
Nutomic dadf8f28f9
Send purges to federated instances (fixes #4119) (#4398)
* Send purges to federated instances (fixes #4119)

* clippy

* review

* remove unused function

* clippy
2024-01-25 09:24:09 -05:00
Nutomic e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00
Nutomic 9240a653c0
Fix bug with Mastodon undo follow activities (#4364) 2024-01-11 18:56:19 -05:00
Nutomic e3b715002b
Handle federated reports from Mastodon, Kbin (#4323)
* Test Kbin/Mbin federation

* Handle reports from Mastodon/Kbin (fixes #4217)

* prettier

* revert

* add mastodon activity

* ci

* revert

* ci
2024-01-05 11:03:13 -05:00
Nutomic 4ca63c5641
Move apub context to join-lemmy.org (fixes #4234) (#4302)
* Move apub context to join-lemmy.org (fixes #4234)

* Dont store federation context in sent_activity table

* include basic activitypub context

* update lib

* ci
2024-01-05 15:42:46 +01:00
Nutomic abe8b18ea8
Also send reports to user's home instance (fixes #4286) (#4305) 2024-01-03 10:31:51 -05:00
phiresky 6790b54d4d
make activity channel infallible (#4295) 2023-12-18 13:17:10 -05:00
Nutomic 25450ea090
Fix following local communities stuck on pending (fixes #4160) (#4161)
* Fix following local communities stuck on pending (fixes #4160)

* fmt

* remove import

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-11-17 04:55:26 -05:00
Nutomic a00313e680
Merge `/site_inbox` into `/inbox`, remove unique constraint for inboxes (#4138)
* Merge /site_inbox into /inbox (fixes #4137)

Get rid of different inboxes, only use /inbox

Remove shared_inbox_url db columns

add code migration

move to db migration, fixes

machete

fix sql

drop inbox url unique constraints

Dont create auth cookie in backend (#4136)

dont change individual inboxes to shared inbox

Dont send comment reply to user who has community blocked. Fixes #3684 (#4096)

* Dont send comment reply to user who has community blocked. Fixes #3684

* Adding source instance block check.

* Adding api test.

* Addressing PR comments.

* move site inbox rewrite to db

* fix test

* clippy

* clippy 2

* fix test
2023-11-16 14:22:40 +01:00
Dessalines 98ed0e51cc
Only check auth secure on release mode. (#4127)
* Only check auth secure on release mode.

* Fixing wrong js-client.

* Adding is_debug_mode var.
2023-11-07 11:03:13 +01:00
Dessalines 64b00ee850
Dont allow bots to vote. Fixes #3940 (#4100)
* Dont allow bots to vote. Fixes #3940

* Removing pointless function.
2023-10-25 10:14:12 -04:00
Nutomic 9ef28eb53b
Remove unused static (#4061) 2023-10-19 11:57:39 -04:00
Nutomic 6d27bfed08
Handle invalid ban expires values (fixes #4045) (#4046)
* Handle invalid ban expires values (fixes #4045)

* Adding a few missing expire time checks. Fixing up time conversions. (#4051)

* Adding a few missing expire time checks. Fixing up time conversions.

* Increase settings export wait time.

* get rid of RemoveCommunity.expires

* fmt

* tests

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-17 13:25:35 -04:00
Nutomic 256ee61908
Ignore activities in remote communities without local followers (#4006)
* Ignore activities in remote communities without local followers (fixes #3568)

* x

* comments

* prettier

* fix api test

* fix test

* cleanup

* fix remaining test

* clippy

* decrease delay
2023-10-16 06:03:49 -04:00
Nutomic 9e099726e6
Cleanup checks for community actions (fixes #2858, fixes #2868) (#4028)
* Cleanup checks for community actions (fixes #2858, fixes #2868)

* allow restoring deleted community

* review changes

* remove unneeded sql

* remove joins

* change mod log check
2023-10-13 09:48:18 -04:00
Nutomic 291ff19718
Only sanitize strings when generating RSS feeds and emails (fixes #4003) (#4024)
* Only sanitize strings when generating RSS feeds and emails (fixes #4003)

* clippy

* fix test
2023-10-11 16:48:19 +02:00
Nutomic 50589115e0
Fix federation of admin actions (fixes #3980) (#3988)
* Fix federation of admin actions (fixes #3980)

* clippy

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-09-25 21:39:18 -04:00
phiresky 2c283d90ca Revert "Revert "re-remove debug comments""
This reverts commit 6d519ef376.
2023-09-21 11:45:33 +00:00
phiresky 6d519ef376 Revert "re-remove debug comments"
This reverts commit ae37ca0137.
2023-09-21 11:07:36 +00:00
phiresky ae37ca0137 re-remove debug comments 2023-09-20 15:33:09 +00:00
phiresky 3d649e1d3e remove synchronous federation flag 2023-09-20 14:22:17 +00:00
phiresky 1a0c866b51 logging temporary 2023-09-20 13:37:18 +00:00
phiresky a59a94492d more debug log 2023-09-20 12:30:38 +00:00
phiresky 78a8a7b8b7 clippy 2023-09-20 11:41:34 +00:00
phiresky 2e7d2d1956 wait for post api test function, better announce activity id 2023-09-20 11:23:02 +00:00
phiresky 27141553b6 fix after revert 2023-09-18 14:28:49 +00:00
phiresky 08401fc85f Revert "remove synchronous federation"
This reverts commit 2767ab4a6f.
2023-09-18 14:25:35 +00:00
phiresky 375d9a2a3c
Persistent, performant, reliable federation queue (#3605)
* persistent activity queue

* fixes

* fixes

* make federation workers function callable from outside

* log federation instances

* dead instance detection not needed here

* taplo fmt

* split federate bin/lib

* minor fix

* better logging

* log

* create struct to hold cancellable task for readability

* use boxfuture for readability

* reset submodule

* fix

* fix lint

* swap

* remove json column, use separate array columns instead

* some review comments

* make worker a struct for readability

* minor readability

* add local filter to community follower view

* remove separate lemmy_federate entry point

* fix remaining duration

* address review comments mostly

* fix lint

* upgrade actitypub-fed to simpler interface

* fix sql format

* increase delays a bit

* fixes after merge

* remove selectable

* fix instance selectable

* add comment

* start federation based on latest id at the time

* rename federate process args

* dead instances in one query

* filter follow+report activities by local

* remove synchronous federation

remove activity sender queue

* lint

* fix federation tests by waiting for results to change

* fix fed test

* fix comment report

* wait some more

* Apply suggestions from code review

Co-authored-by: SorteKanin <sortekanin@gmail.com>

* fix most remaining tests

* wait until private messages

* fix community tests

* fix community tests

* move arg parse

* use instance_id instead of domain in federation_queue_state table

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
2023-09-09 12:25:03 -04:00
Dessalines 9785b20843
Adding a scaled sort, to boost smaller communities. (#3907)
* Adding a scaled sort, to boost smaller communities.

- Previously referred to as *best* .
- Fixes #3622

* Fixing scheduled task update.

* Converting hot_rank integers to floats.

* Altering hot_rank psql function to default to zero after a week.

* Setting scaled_rank to zero, where hot_rank is zero.

* Adding image_upload table.
2023-09-06 13:43:27 -04:00
Nutomic 71d61138bc
Replace ammonia lib with manual html escaping (fixes #3774) (#3938)
* Replace ammonia lib with manual html escaping (fixes #3774)

* prettier

* clippy

* remove sanitize unit test

* fix tests

* fix schema
2023-09-06 10:56:26 -04:00
Nutomic a0ea8dbc00
Auto resolve reports trigger (#3871)
* Revert "Automatically resolve report when post/comment is removed (#3850)"

This reverts commit f7f6766650.

* Automatically resolve reports using db trigger

* lint

* use mod log tables

* fix migration

* fix ci

* fix clippy
2023-09-05 05:35:10 -04:00
Nutomic 366d9d1e2e
Get rid of remaining Perform/SendActivity traits (fixes #3670) (#3926)
* Get rid of remaining Perform/SendActivity traits (fixes #3670)

* fix api tests

* ci
2023-09-05 05:33:46 -04:00
Dessalines 56e26fc3d4
Adding a post_view mode. Fixes #3730 (#3731)
* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Fixing column order.

* Fix default Ok returns.

* Removing return Err(... where feasible.
2023-08-31 09:01:08 -04:00
Nutomic 7fd14b3d2a
Make remove content optional during account deletion (fixes #1617) (#3817)
* Make remove content optional during account deletion (fixes #1617)

* simplify purge params by passing context

* update js client

* add delete content

* update woodpecker config

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-08-28 12:23:45 +02:00
phiresky 514f2222e0
Fix time zone handling (#3496)
* convert naive time to utc time

* compounding fixes

* cargo fmt

* fix the rest

* fix down migration

* fix migrations

* fix after merge

* clippy fix

* ap-fed 0.5.0

---------

Co-authored-by: Nutomic <me@nutomic.com>
2023-08-24 11:27:00 -04:00
Freakazoid182 ee7b35a04a
Add person name to PersonIsBannedFromSite error (#3786) (#3855)
* Add person name to PersonIsBannedFromSite error (#3786)

* Log PersonId instead of Person name (#3850)

* Log actor_id for PersonIsBannedFromSite (#3850)

---------

Co-authored-by: Freek van Zee <freek.van.zee@mediamonks.com>
2023-08-16 11:36:58 +02:00
Nutomic f7f6766650
Automatically resolve report when post/comment is removed (#3850)
* Automatically resolve report when post/comment is removed

* also handle apub removes
2023-08-08 11:19:55 -04:00
dullbananas 969f8b2ce9
Replace TypedBuilder with Default in update forms (#3814)
* Update comment.rs

* Update community.rs

* Update local_site.rs

* Update local_site_rate_limit.rs

* Update local_user.rs

* Update person.rs

* Update comment.rs

* Update community.rs

* Update local_site.rs

* Update local_site_rate_limit.rs

* Update local_user.rs

* Update post.rs

* Update private_message.rs

* Update site.rs

* Update post.rs

* Update person.rs

* Update private_message.rs

* Update comment.rs

* Update create.rs

* Update leave_admin.rs

* Update update.rs

* Update remove.rs

* Update add_admin.rs

* Update verify_email.rs

* Update mod.rs

* Update mod.rs

* Update undo_delete.rs

* Update undo_delete.rs

* Update utils.rs

* Update feature.rs

* Update delete.rs

* Update lock.rs

* Update create.rs

* Update approve.rs

* Update update.rs

* Update lock_page.rs

* Update block_user.rs

* Update delete.rs

* Update undo_block_user.rs

* Update collection_remove.rs

* Update post.rs

* Update hide.rs

* Update person.rs

* Update remove.rs

* Update post_view.rs

* Update create.rs

* Update remove.rs

* Update collection_add.rs

* Update community.rs

* Update update.rs

* Update post_aggregates.rs

* Update update.rs

* Update comment.rs

* Update code_migrations.rs

* Update registration_application_view.rs

* Update update.rs

* Update ban_person.rs

* Update community.rs

* Update delete.rs

* Update delete.rs

* Update delete.rs

* Update person_aggregates.rs

* Update save_settings.rs

* Update distinguish.rs

* Update mark_read.rs

* Update site_aggregates.rs

* Update create.rs

* Fix

* rerun ci

* Update comment.rs

* rerun ci

* Update create.rs

* Update create.rs

* Update post_view.rs

* rerun ci

* Update undo_delete.rs

* rerun ci
2023-08-08 11:41:41 +02:00
Dessalines 70fae9d68d
Fixing broken SQL migration formatting. (#3800)
* Fixing SQL format.

* Fixing clippy lints.

* Fixing clippy lint 2.
2023-08-03 11:07:35 +02:00
Nutomic 27be1efb74
Rewrite remaining federation actions, get rid of PerformCrud trait (#3794)
* Rewrite ban actions

* Rewrite delete/remove actions

* Rewrite remove/delete community

* Rewrite report actions

* Rewrite feature/lock post

* Rewrite update community actions

* Rewrite remaining federation actions

* Get rid of PerformCrud trait

* clippy
2023-08-02 12:52:41 -04:00
Louis GERARD 7bc64ab91a
Remove follow community traits (#3737)
* chore(FollowCommunity): remove Perform and Send Activity traits

* chore(FollowCommunity): avoid fetching community and person from db
2023-08-02 11:32:16 +02:00
Nutomic d82194cfe9
Rewrite some federation actions to remove Perform/SendActivity (ref #3670) (#3758) 2023-08-01 15:53:36 +02:00
Nutomic 37998b3398
Rewrite some API handlers to remove Perform trait (#3735)
* Rewrite some API handlers to remove Perform trait

* Convert CreateComment

* ci
2023-07-28 10:39:38 -04:00
RocketDerp 21a87ebaf2
Federation tests replication round1 - demonstrate absent replication of comment deletes (#3657)
* more robust test of unlike a comment, confirm replication to instance downstream from community home

* more robust 'delete a comment' test, confirm replication

* Far more robust "Report a comment" test. Many comments about situation, this is currently failing because gamma does not get the report

* typo and actually have Gamma comment check use gamma, not alpha

* prepare-drone-federation-test.sh has some more echo output and note about the LEMMY_DATABASE_URL format (#3651)

* Add http cache for webfingers (#3317)

* Add http cache for webfingers

* Remove the outgoing cache middleware & adjust the cache headers directive

* Use 1h & 3day cache header

* Update routes and adjust the cache headers location

* revert apub caching

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Felix Ableitner <me@nutomic.com>

* Rewrite activity lists to fix delete federation (fixes #3625)

* Revert "typo and actually have Gamma comment check use gamma, not alpha"

This reverts commit 7dfb6ee0f4.

* Revert "Far more robust "Report a comment" test. Many comments about situation, this is currently failing because gamma does not get the report"

This reverts commit 7bd3b20ae0.

* prettier TypeScript

* revised comments, as ResolveObject isn't using routine replication

* fmt

* fix api tests

* remove comment

---------

Co-authored-by: cetra3 <cetra3@hotmail.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Felix Ableitner <me@nutomic.com>
2023-07-27 06:17:40 -04:00
Nutomic 3471f3533c
Sanitize html (#3708)
* HTML sanitization in apub code

* Sanitize API inputs

* fmt

* Dont allow html a, img tags

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-26 14:01:15 -04:00
Nutomic 5d23ef960e
Remove SendActivity and Perform traits, rely on channel (#3596)
* Remove SendActivity and Perform traits, rely on channel

These traits arent necessary anymore now that websocket is removed.
Removing them allows us to use normal actix http handler methods
which are much more flexible, and allow using different middlewares
as well as setting response attributes.

* compiling and create post federating

* clippy

* rename methods, join outgoing activities task

* fix api tests

* no unwrap

* conditional compile

* add back getrandom

* make crates optional

* fmt
2023-07-19 09:49:41 -04:00
Nutomic e9e76549a8
Split activity table into sent and received parts (fixes #3103) (#3583)
* Split activity table into sent and received parts (fixes #3103)

The received activities are only stored in order to avoid processing
the same incoming activity multiple times. For this purpose it is
completely unnecessary to store the data. So we can split the
table into sent_activity and received_activity parts, where
only sent_activity table needs to store activity data. This should
reduce storage use significantly.

Also reduces activity storage duration to three months, we can reduce
this further if necessary.

Additionally the id columns of activity tables are removed because
they are completely unused and risk overflowing (fixes #3560).

* address review

* move insert_received_activity() methods to verify handlers

* remove unnecessary conflict line

* clippy

* use on conflict, add tests
2023-07-14 11:17:06 -04:00
Nutomic 7d8cb93b53
Check for dead federated instances (fixes #2221) (#3427)
* Check for dead federated instances (fixes #2221)

* move to apub crate, use timestamp

* make it compile

* clippy

* use moka to cache blocklists, dead instances, restore orig scheduled tasks

* remove leftover last_alive var

* error handling

* wip

* fix alive check for instances without nodeinfo, add coalesce

* clippy

* move federation blocklist cache to #3486

* unused deps
2023-07-13 16:12:01 +02:00