Commit Graph

213 Commits (495dd3f4f63c50367ecf28fe7494fa50f70fa5f3)

Author SHA1 Message Date
Nutomic e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00
Nutomic 1be7dbde33
Count chars, not bytes for max title length (fixes #4366) (#4367)
* Count chars, not bytes for max title length (fixes #4366)

* fix api test
2024-01-15 09:33:39 -05:00
Dessalines 1856e7c0ca
Increasing max items for user settings import. (#4352)
- Fixes #4307
2024-01-05 10:43:30 +01:00
Nutomic 7d9b59c467
Reduce default db pool size to 30, remove db timeout (ref #4282) (#4301)
* Reduce default db pool size to 30 (ref #4282)

* remove db timeout
2024-01-04 12:16:51 -05:00
ALEX11BR a5289dd4cf
Use `pretty_assertions` for assertions (#4347)
* Use `pretty_assertions` for assertions

* fixed ordering of `use`s

* ci

---------

Co-authored-by: Felix Ableitner <me@nutomic.com>
2024-01-04 04:47:18 -05:00
Andrew Voynov f223eb94d5
Fixed `matrix_user_id` regex (#4312)
* fix(matrix_user_id): fixed regex

* test(matrix_user_id): added test with underscore
2023-12-21 12:13:24 +01:00
dullbananas 3e2393993e
Update mod.rs (#4240)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-12-12 13:06:17 -05:00
Nutomic e317947b75
Dont allow blocking local instance (fixes #4241) (#4242)
* Dont allow blocking local instance (fixes #4241)

* use localuserview
2023-12-11 14:35:53 -05:00
Nutomic a790a24c4d
Revert debug auth (#4232)
* Revert "Some changes to help debug auth problems on lemmy.ml (#4220)"

This reverts commit 16ac893e15.

* Rename auth cookie back to jwt
2023-12-05 11:22:08 -05:00
Nutomic 16ac893e15
Some changes to help debug auth problems on lemmy.ml (#4220)
* Some changes to help debug auth problems on lemmy.ml

* fix

* clippy
2023-12-01 09:18:29 -05:00
Dessalines 170b3ec45f
Removing cookie secure check. (#4213) 2023-11-30 04:44:18 -05:00
Bhoomtawath Plinsut 6fa3b59d25
Fix:#4197: validate post title must be shorter than 200 letters (#4198)
* validate post title must be shorter than 200 letters

* use range contains
2023-11-27 10:46:03 +01:00
İsmail Karslı 2070381e81
Add upload_timeout to PictrsConfig (#4159)
* Add upload timeout to PictrsConfig

* Bad space 🤔

* Update PictrsConfig upload timeout to include units.
2023-11-17 04:01:32 -05:00
Nutomic 1d23df37d8
Rename cache_remote_images setting to cache_external_link_previews (ref #4035) (#4148) 2023-11-13 09:19:09 -05:00
Dessalines e00b0c9da2
Dont send comment reply to user who has community blocked. Fixes #3684 (#4096)
* Dont send comment reply to user who has community blocked. Fixes #3684

* Adding source instance block check.

* Adding api test.

* Addressing PR comments.
2023-11-09 12:03:25 +01:00
Nutomic 08739e2925
Move usage of env::var to lemmy_utils, simplify db init (ref #4095) (#4108) 2023-10-25 11:34:38 -04:00
Dessalines 1b751a8cac
Adding cors_origin to settings. Fixes #3665 (#4095)
* Adding cors_origin to settings. Fixes #3665

* Fix result to option.

* Forgot to update config defaults.

* Setting a cors origin doku default.

* Adding comments for CORS.
2023-10-25 16:46:34 +02:00
Dessalines 64b00ee850
Dont allow bots to vote. Fixes #3940 (#4100)
* Dont allow bots to vote. Fixes #3940

* Removing pointless function.
2023-10-25 10:14:12 -04:00
Nutomic 45bed71c36
Include prometheus in default build, remove build feature (fixes #3558) (#4071)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-25 06:54:58 -04:00
Nutomic d55bd2f2bc
Allow Arabic and Cyrillic usernames/community names (fixes #1764) (#4083)
* Allow Arabic and Cyrillic usernames/community names (fixes #1764)

* update comment
2023-10-24 16:25:52 -04:00
HackerNCoder 722d9efc13
Use starts_with for forbidden unicode (Fix #3888) (#4079)
* Use starts_with for forbidden unicode (Fix #3888)

* Require 3 visible chars in display name

* Run cargo fmt and scripts/lint

* Undo invisibly_starts_with_at

* Remove 3 min chars.count() check for display name
2023-10-24 16:57:40 +02:00
Dessalines 09a67472e7
Allow torrent magnet links as post urls. Fixes #3916 (#4092) 2023-10-24 16:48:00 +02:00
Dessalines 053dcf8c10
Fixing matrix id regex. Fixes #3431 (#4091) 2023-10-24 16:47:02 +02:00
Nutomic 6235ff45b5
Return status 401 on logout with invalid auth (fixes #4081) (#4082)
* Return status 401 on logout with invalid auth (fixes #4081)

* format
2023-10-23 12:17:26 -04:00
dullbananas a14657d124
Refactor rate limiter and improve rate limit bucket cleanup (#3937)
* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update scheduled_tasks.rs

* Shrink `RateLimitBucket`

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* rerun ci

* Update rate_limiter.rs

* Undo changes to  fields

* Manually undo changes to RateLimitBucket fields

* fmt

* Bucket cleanup loop in rate_limit/mod.rs

* Remove rate limit bucket cleanup from scheduled_tasks.rs

* Remove ;

* Remove UNINITIALIZED_TOKEN_AMOUNT

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* stuff

* MapLevel trait

* fix merge

* Prevent negative numbers in buckets

* Clean up MapLevel::check

* MapLevel::remove_full_buckets

* stuff

* Use remove_full_buckets to avoid allocations

* stuff

* remove tx

* Remove RateLimitConfig

* Rename settings_updated_channel to rate_limit_cell

* Remove global rate limit cell

* impl Default for RateLimitCell

* bucket_configs doc comment to explain EnumMap

* improve test_rate_limiter

* rename default to with_test_config

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2023-10-19 15:31:51 +02:00
Nutomic 6d27bfed08
Handle invalid ban expires values (fixes #4045) (#4046)
* Handle invalid ban expires values (fixes #4045)

* Adding a few missing expire time checks. Fixing up time conversions. (#4051)

* Adding a few missing expire time checks. Fixing up time conversions.

* Increase settings export wait time.

* get rid of RemoveCommunity.expires

* fmt

* tests

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-17 13:25:35 -04:00
Nutomic 3a19af5215
Allow marking multiple posts as read in single api call (fixes #3963) (#4048)
* Allow marking multiple posts as read in single api call (fixes #3963)

* cleanup

* limit array length

* fix test

* review

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-17 12:35:51 -04:00
Nutomic 256ee61908
Ignore activities in remote communities without local followers (#4006)
* Ignore activities in remote communities without local followers (fixes #3568)

* x

* comments

* prettier

* fix api test

* fix test

* cleanup

* fix remaining test

* clippy

* decrease delay
2023-10-16 06:03:49 -04:00
Nutomic 291ff19718
Only sanitize strings when generating RSS feeds and emails (fixes #4003) (#4024)
* Only sanitize strings when generating RSS feeds and emails (fixes #4003)

* clippy

* fix test
2023-10-11 16:48:19 +02:00
Nutomic 6d7b38f4de
Implement user data import/export (#3976)
* Implement endpoints for user data import/export

* add test

* exclude avatar/banner

* increase import url count, add rate limit

* also export/import saved posts

* rate limit

* rename

* saved posts also exist

* rename routes

* fix test

* error handling

* clippy

* limit parallelism

* clippy

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-11 16:47:22 +02:00
Nutomic dc327652a5
Add db table for login tokens which allows for invalidation (#3818)
* wip

* stuff

* fmt

* fmt 2

* fmt 3

* fix default feature

* use Authorization header

* store ip and user agent for each login

* add list_logins endpoint

* serde(skip) for token

* fix api tests

* A few suggestions for login_token (#3991)

* A few suggestions.

* Fixing SQL format.

* review

* review

* rename cookie

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-09 12:46:12 +02:00
Nutomic b7d570cf35
Make input length checks consistent with HTML maxlength attribute (#4009)
* Make input length checks consistent with HTML maxlength attr (fixes #3688)

* ci

* Extricating min and max length checks (#4018)

* revert string change

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-05 16:39:07 -04:00
Nutomic 0115613f90
Specify password length limits in error (fixes #3284) (#4007)
* Specify password length limits in error (fixes #3284)

* convert to comment
2023-10-05 07:40:22 -04:00
Nutomic 6950dd90e5
Return HTTP status 503 to all requests during startup (fixes #3780) (#4005)
* Return HTTP status 503 to all requests during startup (fixes #3780)

* add message

* fmt

* remove err

* dont run startup server if http disabled

* ci
2023-10-04 18:19:58 -04:00
Nutomic 626c7ebc85
Handle empty reason for registration application denial (fixes #3485) (#4008)
* Handle empty reason for registration application denial (fixes #3485)

* clippy

* clippy
2023-10-04 15:20:22 +02:00
Nutomic 50b7322ff3
Mention max length for site name in config (fixes #3006) (#4010)
* Mention max length for site name in config (fixes #3006)

* ci
2023-10-04 15:09:25 +02:00
Nutomic 9bb99bec5e
Increase default database connections (fixes #3394) (#3994) 2023-09-28 09:56:39 -04:00
Nutomic 22608ae983
Rework the way 2FA is enabled/disabled (fixes #3309) (#3959)
* Rework the way 2FA is enabled/disabled (fixes #3309)

* postgres format

* change algo to sha1 for better compat

* review comments

* review

* clippy

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-09-20 10:49:54 -04:00
Nutomic 50f81cf157
User can block instances (fixes #2397) (#3869)
* User can block instances (fixes #2397)

* update comments

* review comments

* use route

* update

* add api test

* update tests

* fix

* fix test

* ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-09-20 11:56:13 +02:00
phiresky 375d9a2a3c
Persistent, performant, reliable federation queue (#3605)
* persistent activity queue

* fixes

* fixes

* make federation workers function callable from outside

* log federation instances

* dead instance detection not needed here

* taplo fmt

* split federate bin/lib

* minor fix

* better logging

* log

* create struct to hold cancellable task for readability

* use boxfuture for readability

* reset submodule

* fix

* fix lint

* swap

* remove json column, use separate array columns instead

* some review comments

* make worker a struct for readability

* minor readability

* add local filter to community follower view

* remove separate lemmy_federate entry point

* fix remaining duration

* address review comments mostly

* fix lint

* upgrade actitypub-fed to simpler interface

* fix sql format

* increase delays a bit

* fixes after merge

* remove selectable

* fix instance selectable

* add comment

* start federation based on latest id at the time

* rename federate process args

* dead instances in one query

* filter follow+report activities by local

* remove synchronous federation

remove activity sender queue

* lint

* fix federation tests by waiting for results to change

* fix fed test

* fix comment report

* wait some more

* Apply suggestions from code review

Co-authored-by: SorteKanin <sortekanin@gmail.com>

* fix most remaining tests

* wait until private messages

* fix community tests

* fix community tests

* move arg parse

* use instance_id instead of domain in federation_queue_state table

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
2023-09-09 12:25:03 -04:00
Nick Webster a57658d99c
Adding a new config flag to disable pictrs caching for thumbnails (#3897)
* add logic to prevent downloading remote pictrs images

* apply formatting

* Do not attempt a pictrs fetch if the remote image is also on a pictrs instance

* Do not attempt a pictrs fetch if the remote image is also on a pictrs instance and cache_federated_images is false

* Generalising the no caching option to handle all remote images

* rustfmt

* Return None if the URL is not an image

* Updating defaults.hjson

* fixing typo

* Fixing typo

* Skip cloning the Url unless we need to

* using a HEAD request for checking the content type, saving bandwidth/improving perf

* Removing early returns

* Switching back to GET requests for Content-Type because pictrs does not handle HEAD requests

* Simplifying logic and using metadata_image instead of url if we do not get a pictrs thumbnail

* Removing unused import

* Return None as a thumbnail if caching is disabled

* formatting

---------

Co-authored-by: Djones4822 <david.jones4822@gmail.com>
2023-08-31 10:36:39 -04:00
Dessalines 56e26fc3d4
Adding a post_view mode. Fixes #3730 (#3731)
* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Fixing column order.

* Fix default Ok returns.

* Removing return Err(... where feasible.
2023-08-31 09:01:08 -04:00
Nutomic b2aee565f3
Allow passing auth via header or cookie (#3725)
* Allow passing auth via header or cookie

* revert submodule

* taplo

* fix build

* working

* convert apub api methods

* also set cache-control header

* opt

* clippy

* deduplicate code, ignore invalid auth

* clippy

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-08-29 10:47:57 -04:00
phiresky 514f2222e0
Fix time zone handling (#3496)
* convert naive time to utc time

* compounding fixes

* cargo fmt

* fix the rest

* fix down migration

* fix migrations

* fix after merge

* clippy fix

* ap-fed 0.5.0

---------

Co-authored-by: Nutomic <me@nutomic.com>
2023-08-24 11:27:00 -04:00
Freakazoid182 ee7b35a04a
Add person name to PersonIsBannedFromSite error (#3786) (#3855)
* Add person name to PersonIsBannedFromSite error (#3786)

* Log PersonId instead of Person name (#3850)

* Log actor_id for PersonIsBannedFromSite (#3850)

---------

Co-authored-by: Freek van Zee <freek.van.zee@mediamonks.com>
2023-08-16 11:36:58 +02:00
Piotr Juszczyk 2ad3450004
Allow filtering posts and comments by whether they were liked/disliked - fixes #3401 (#3523)
* Allow filtering posts and comments by whether they were liked/disliked

* Switch to 2 args - liked_only, disliked_only - taking bools

* Make liked_only and disliked_only Option<bool>

* Fix unrelated is_profile_view compilation error
2023-08-08 11:40:28 +02:00
Freakazoid182 9a1f9aad45
detailed error message for blocked domains (#3698) (#3701)
* detailed error message for blocked domains (#3698)

* Pass the domain as an error param

Not formatting the error message to support i18n

---------

Co-authored-by: Freek van Zee <freek.van.zee@mediamonks.com>
2023-07-25 13:26:54 -04:00
cetra3 1f21bdb2f9
Add http cache for webfingers (#3317)
* Add http cache for webfingers

* Remove the outgoing cache middleware & adjust the cache headers directive

* Use 1h & 3day cache header

* Update routes and adjust the cache headers location

* revert apub caching

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Felix Ableitner <me@nutomic.com>
2023-07-19 06:09:04 -04:00
cetra3 9256895635
Cache & Optimize Woodpecker CI (#3450)
* Try using drone cache plugin

* Try another path

* Include volume

* Fix formatting

* Include fmt

* Exclude cargo dir from prettier

* Don't override cargo

* Just do check

* Add cache key

* Use different cache plugin

* Add clippy

* Try minio

* Add quotes

* Try adding secrets

* Try again

* Again

* Use correct secret formation

* Add back clippy

* Use secret for the root bucket name

* Try drone cache instead

* Add region

* Add path-style option

* Include cargo clippy

* Include everything again

* Fix formatting

* Don't run clippy twice

* Add `allow` statements for tests to pass

* Adjust endpoint to be a secret

* Fix prettier

* Merge & fix tests

* Try to restart the woodpecker test

* Change the ENV var name

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-17 11:04:14 -04:00
Manuel Bleichner c0b7865896
Fix wrong SMTP port when TLS is being used (fixes #3574) (#3607)
Co-authored-by: Manuel Bleichner <manuel.bleichner@bvu.de>
2023-07-13 13:10:14 +02:00