Commit Graph

188 Commits (37754189524e761626917295958cc156b1d21481)

Author SHA1 Message Date
Dessalines 4e12e25c59
Adding admin purging of DB items and pictures. #904 #1331 (#1809)
* First pass at adding admin purge. #904 #1331

* Breaking out purge into 4 tables for the 4 purgeable types.

* Using CommunitySafe instead in view

* Fix db_schema features flags.

* Attempting to pass API key.

* Adding pictrs image purging

- Added pictrs_config block, for API_KEY
- Clear out image columns after purging

* Remove the remove_images field from a few of the purge API calls.

* Fix some suggestions by @nutomic.

* Add separate pictrs reqwest client.

* Update defaults.hjson

Co-authored-by: Nutomic <me@nutomic.com>
2022-06-13 19:15:04 +00:00
Nutomic a2a594b763
Extract Activitypub logic into separate library (#2288)
* Create example for apub lib

* some rewriting of apub lib

* Add LocalInstance struct for apub lib to avoid using Lemmy Settings

* Move ActorType trait to lemmy_apub, because its not needed in library

* Use reqwest_retry instead of custom impl, dont specify timeout on every send()

* Some improvements to example

* Moved inbox handling to library

* bug fixes

* Move context and serde helpers into library

* wip: example changes

* Add lemmy_utils feature to build only LemmyError

* Rename to activitypub_federation

* Remove lemmy_utils dep from activitypub_federation using generic error type

* Finish activitypub example

* Cleanup and fix tests

* Reorganize library files

* Remove ApubObject.to_tombstone()

* Extract activitypub library into separate git repository
2022-06-02 16:33:41 +02:00
Felix Ableitner 80684de7da Make opentelemetry dependency optional 2022-05-11 14:29:01 +02:00
Nutomic 3aa3d75a1e
Add cargo feature for building lemmy_api_common with mininum deps (#2243) 2022-05-03 17:44:13 +00:00
Nutomic 7058cfa1cf
Implement restricted community (only mods can post) (fixes #187) (#2235)
* Implement restricted community (only mods can post) (fixes #187)

* review fixes

* fix tests
2022-04-28 20:32:32 +00:00
Nutomic 3951a16447
Split apart api files (#2216) 2022-04-13 18:12:25 +00:00
Dessalines 2985e88a49
Fixing generate unique changeme (#2205)
* Fixing generate unique changeme

* Fixing generate unique changeme 2

* Changing link to .invalid domain.

Co-authored-by: Nutomic <me@nutomic.com>
2022-04-13 11:13:29 +00:00
Dessalines 4cf0da7b60 Clippy fixes. 2022-03-30 10:58:03 -04:00
Dessalines 3ad172e8ed Adding a ban expires update job. Fixes #2177 2022-03-30 09:56:23 -04:00
Dessalines e16821116e
Lowering search rate limit. Fixes #2153 (#2154)
* Lowering search rate limit. Fixes #2153

* Adding a search rate limit.

* Forgot to add the websocket search rate limit

* Fix wrong op
2022-03-29 15:46:03 +00:00
Dessalines f2a0841586
Rate limit websocket joins. (#2165)
* Rate limit websocket joins.

* Removing async on mutex lock fn.

* Removing redundant ip

* Return early if check fails.
2022-03-27 00:29:05 +00:00
Dessalines 1e9f609cdb
Consolidate and lower reqwest timeouts. Fixes #2150 (#2151) 2022-03-24 16:33:23 +00:00
Nutomic ef1e164cc5
Make activity queue worker count configurable, log stats (#2113) 2022-03-03 18:54:33 +00:00
dayinjing 7f9b55e793
Hide community v2 (#2055)
* Initial working of hiding communities and adding a db entry for mod log

* Return mod log for hidden communities

* Clean up hidding communities PR

* use lower case like other migration files

* Formatting fix

* pass in admin id to list, make match logic the same in post_view as community_view. Dont force non null for reason

* Clean PR review stuff

* Change person_id to mod_person_id on hide community table

* Make bools optional, add a space for formating

Co-authored-by: Thor Odinson <odinson@asgard.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2022-02-18 02:30:47 +00:00
Dessalines 81aa39ba68
Adding a reqwest timeout. Fixes #2089 (#2097) 2022-02-15 12:53:57 +00:00
Nutomic dd865c5af5
Implement instance actor (#1798)
* Implement instance actor

* wip: make site bans federate

* finish implementation and unit tests for federated bans

* start adding api tests

* fix api test

* remve site from GetCommunityResponse

* only federate site bans originating from user's home instance

* dont expose site.private_key in api
2022-02-07 19:23:12 +00:00
Aode (lion) 2e5746b74f Add console feature flag, not default
The most recent tracing-core changed the number of variants on metadata::Kind,
which broke console-api 0.1.1, and no fix has been applied yet. To avoid build
failures, disable console-subscriber for now
2022-02-04 11:31:38 -06:00
Dessalines 2ac6e1a463
Adding a test comment. (#2081) 2022-02-01 18:19:07 +00:00
Dessalines e36ad9d984
Removing the site creator, adding leave_admin. Fixes #1808 (#2052)
* Removing the site creator, adding leave_admin. Fixes #1808

* Making sure there's at least one admin. Fixing unit tests
2022-01-26 17:57:16 +00:00
Riley 41b90bb162
Add console-subscriber (#2003)
* Add console-subscriber

* Add larg event buffer capacity

* Add console port to lemmy prod container

* Expose pict-rs console port

* Update pictrs
2022-01-07 09:53:45 -05:00
Riley 3032a2745b
Opentelemetry (#1992)
* Add otlp export

* Add jaeger to dev docker

* More instrument

* Update deps

* Merge from main.

* Add comment about viewing traces

* Fixing reqwest-tracing dep.

* Update pre-commit regex

* Move imports to top

Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2022-01-06 19:10:20 +00:00
Dessalines bd31475dba Adding a banned endpoint for admins. Removing it from GetSite. Fixes #1806 2022-01-04 09:50:30 -05:00
Dessalines c883a49a40
First pass at invite-only migration. (#1949)
* First pass at invite-only migration.

* Implement email verification (fixes #219)

* remove unwrap

* Adding views and functionality to registration application. #209

* Add private instance site column, and back end checks.

* Adding some message fields to LoginResponse

* Adding private instance to site setup.

* A few additions:

- Add a DeleteAccount response.
- RegistrationApplicationView now has the safe LocalUserSettings.
- Adding VerifyEmail to websocket API, added a proper response type.

* Adding and reorganizing some email helpers.

* A few fixes for private sites:

- Added a check_registration_application function.
- Only send a verification email if its been changed.
- VerifyEmail now returns LoginResponse.
- Deleting the old tokens after a successful email verify.
- If port is missing on email config, display a better error message.

* Version 0.15.0-rc.3

* Adding published to email_verification table.

* Adding fixes from comments.

* Version 0.15.0-rc.4

* Adding modlog private site check.

* Version 0.15.0-rc.6

Co-authored-by: Felix Ableitner <me@nutomic.com>
2021-12-15 14:49:59 -05:00
Riley 0ed3856ef1
Remove awc (#1979)
* Remove unused awc dep

* Replace awc pict-rs client with reqwest

* Reinstate 30 second pict-rs timeout
2021-12-08 10:56:43 -05:00
Aode (lion) 97ebf2f6f3 Consolidate reqwest clients, use reqwest-middleware for tracing 2021-12-06 16:54:34 -06:00
Riley 35cbae61bc
Don't drop error context when adding a message to errors (#1958)
* Respond directly with LemmyError

Instrument Perform implementations for more precise traces
Use ApiError to format JSON errors when messages are present
Keep SpanTrace output in LemmyError Display impl

* Hide SpanTrace debug output from LemmyError

* Don't log when entering spans, only when leaving

* Update actix-web

* Update actix-rt

* Add newline after error info in LemmyError Display impl

* Propogate span information to blocking operations

* Instrument apub functions

* Use skip_all for more instrument attributes, don't skip 'self' in some api actions

* Make message a static string

* Send proper JSON over websocket

* Add 'message' to LemmyError display if present

* Use a quieter root span builder, don't pretty-print logs

* Keep passwords and emails out of logs

* Re-enable logging Login

* Instrument feeds

* Emit our own errors

* Move error log after status code recording

* Make Sensitive generic over the inner type

* Remove line that logged secrets
2021-12-06 09:54:47 -05:00
Dessalines e765b42d46
Adding a captcha rate limit. Fixes #1755 (#1941)
* Adding a captcha rate limit. Fixes #1755

* Changing to post rate limit.
2021-11-25 13:04:19 +00:00
Dessalines d2e28e5f38
Adding a GetComment endpoint. Fixes #1919 (#1944) 2021-11-23 15:53:48 +00:00
Dessalines 86b2901e8c
Adding MarkPostAsRead to API. Fixes #1784 (#1946)
* Adding MarkPostAsRead to API. Fixes #1784

* Adding error
2021-11-23 14:15:43 +00:00
Riley 1579ee566f
background-jobs 0.11 (#1943) 2021-11-23 12:20:01 +00:00
Riley 3fea5645f8
Add tracing (#1942) 2021-11-23 12:16:47 +00:00
Dessalines 05d43150bc Making public key required. Fixes #1934 2021-11-22 10:10:18 -05:00
layla 7fe7062c47 Implement rate limits on comments 2021-11-18 13:40:16 +00:00
Dessalines f035af3738 Fixing clippy 2021-11-09 17:31:28 -05:00
Felix Ableitner 614490d29b Fix problem that prevented viewing of pleroma user profiles 2021-10-29 12:45:53 +02:00
Felix Ableitner f24999027e Merge crates db_schema and db_queries 2021-10-20 12:28:08 +02:00
Dessalines 97aa7268ae
Adding GetUnreadCount to the API. Fixes #1794 (#1842)
* Adding GetUnreadCount to the API. Fixes #1794

* Reordering filters to fix unread replies.
2021-10-16 10:43:41 +00:00
Nutomic 454d3987a5
Generate config docs from code (#1786)
* Generate config docs from code

* update library

* move settings comments from config.hjson to code

* updte doku to 0.10

* update doku
2021-10-08 11:07:24 -04:00
Dessalines 53a2b6d013
Trying a background_jobs fix. #1820 (#1822) 2021-10-08 14:36:53 +00:00
Nutomic b96ce81f89
Move code to apub library (#1795)
* Remove dependency of apub_lib on LemmyContext

* Move ApubObject trait to library

* Reorganize files in apub lib

* Move ActorType, signatures, activity_queue to apub library
2021-10-06 16:20:05 -04:00
Dessalines 9c50cbf431
Adding a user agent. Fixes #1769 (#1800)
* Adding a user agent. Fixes #1769

* Fix client duplicate creation.
2021-09-29 16:05:38 -04:00
Dessalines 8708ad1b44 Moving settings and secrets to context. 2021-09-27 11:25:26 +02:00
Dessalines 0a91a28b27
Reindex tables non-concurrently on startup. Fixes #1726 . #1723 (#1736) 2021-08-26 11:49:16 +00:00
Felix Ableitner 251e0d3b82 Move resolving of activitypub objects to separate api endpoint (fixes #1584) 2021-08-20 12:35:50 +02:00
Dessalines 2016afc9db
User / community blocking. Fixes #426 (#1604)
* A first pass at user / community blocking. #426

* Adding unit tests for person and community block.

* Moving migration

* Fixing creator_blocked for comment queries, added tests.

* Don't let a person block themselves

* Fix post creator_blocked

* Adding creator_blocked to PersonMentionView

* Moving blocked and follows to MyUserInfo

* Rename to local_user_view

* Add moderates to MyUserInfo

* Adding BlockCommunityResponse

* Fixing name, and check_person_block

* Fixing tests.

* Using type in Blockable trait.

* Changing recipient to target, adding unfollow to block action.
2021-08-19 20:54:15 +00:00
Dessalines 6af75492a9
Swap out iframely (#1706)
* Replace Iframely. Fixes #1681

* Add post_link_tags to nginx

* Adding post_link_tags route

* Cleaning up post_link_tags

* Changing PostLink to SiteMetadata, adding it to the API.

* Fixing issue when local has no openssl certs.

* Fixing an issue with pictrs errors

* Revert "Fixing issue when local has no openssl certs."

This reverts commit dbf7d1b1ee03846e5ef7b7156e618424f1150e1d.

* Add ca-certs to dockerfile for volume mount.

* Cleaning up fetch_pictrs request

* Changing to fetch_site_data
2021-08-19 14:12:49 +00:00
Dessalines 7b8cbbba85
Simplify config using macros (#1686)
Co-authored-by: Felix Ableitner <me@nutomic.com>
2021-08-04 21:13:51 +00:00
Felix Ableitner c1db86925f Merge apub, apub_receive crates (fixes #1621) 2021-07-17 18:20:44 +02:00
Nutomic c7de1fcf24
Apub inbox rewrite (#1652)
* start to implement apub inbox routing lib

* got something that almost works

* it compiles!

* implemented some more

* move library code to separate crate (most of it)

* convert private message handlers

* convert all comment receivers (except undo comment)

* convert post receiver

* add verify trait

* convert community receivers

* add cc field for all activities which i forgot before

* convert inbox functions, add missing checks

* convert undo like/dislike receivers

* convert undo_delete and undo_remove receivers

* move block/unblock activities

* convert remaining activity receivers

* reimplement http signature verification and other checks

* also use actor type for routing, VerifyActivity and SendActivity traits

* cleanup and restructure apub_receive code

* wip: try to fix activity routing

* implement a (very bad) derive macro for activityhandler

* working activity routing!

* rework pm verify(), fix tests and confirm manually

also remove inbox username check which was broken

* rework following verify(), fix tests and test manually

* fix post/comment create/update, rework voting

* Rewrite remove/delete post/comment, fix tests, test manually

* Rework and fix (un)block user, announce, update post

* some code cleanup

* rework delete/remove activity receivers (still quite messy)

* rewrite, test and fix add/remove mod, update community handlers

* add docs for ActivityHandler derive macro

* dont try to compile macro comments
2021-07-17 12:08:46 -04:00
Dessalines 9840f5c1c1
Upgrading deps, running clippy fix on nightly 1.55.0 (#1638)
* Upgrading deps, running clippy fix on nightly 1.55.0

* Running clippy --fix

* Trying to fix transform

* Trying to fix another transform

* Upgrading http-signature-normalization-actix

* Fix app_data

* Fix pictrs uploading

* Fix unwrap
2021-07-06 13:26:46 +00:00
Dessalines f54209c451
Running clippy --fix (#1647) 2021-07-05 16:07:26 +00:00
Dessalines c3d64f996e
Removing community.creator column. Fixes #1504 (#1541)
* Removing community.creator column. Fixes #1504

* Fixing unit tests.

* Fixing federation tests.
2021-04-08 11:29:08 +00:00
Dessalines 1a70477fc7
Better DB default fields. (#1560)
* Better DB default fields.

* Fixing clippy
2021-04-08 11:19:58 +00:00
Dessalines 727fa610d4
Merge pull request #1552 from LemmyNet/feature/1550-optional-config-values
Make some of the database config values optional (fixes #1550)
2021-04-07 13:16:28 -04:00
Felix Ableitner ccdf117c8f Make some of the database config values optional (fixes #1550) 2021-04-06 14:57:37 +02:00
Dessalines f33f763278 Moving ChangePassword to its own API action. Fixes #1471 2021-04-01 17:39:01 -04:00
Felix Ableitner 722cdb78ba Split lemmy_apub crate into two parts apub and apub_receive 2021-03-30 22:29:16 +02:00
Dessalines 4da6e42fc1 Fix websocket location to /api/vX 2021-03-29 17:48:37 -04:00
Dessalines 4c8f2e976e Merge remote-tracking branch 'yerba/split-api-crate' into test_merge_api_crates_reorg 2021-03-29 16:24:50 -04:00
Felix Ableitner 01fc1228d5 Fix API and clippy warnings 2021-03-29 20:55:08 +02:00
Felix Ableitner 249fcc5066 Split api crate into api_structs and api 2021-03-25 20:19:40 +01:00
Dessalines c884510173 Creating default DB forms. Fixes #1511 2021-03-20 16:59:07 -04:00
Dessalines 1745b64ceb Moving matrix_user_id to person table. #1438 2021-03-20 15:21:51 -04:00
Dessalines 9cb4dad4b4 A first pass. 2021-03-10 23:43:11 -05:00
Dessalines ddf4a667b1 ~80% done 2021-03-10 17:33:55 -05:00
Felix Ableitner 7f56281c26 Forbid usage of unwrap 2021-03-01 19:24:34 +01:00
Dessalines 462c4a2954
Rewrite settings implementation. Fixes #1270 (#1433)
* A first attempt at using deser-hjson. Fixes #1270

* Trying to fix tests, try 1

* Trying to fix tests, try 2

* A few fixes to deser_hjson

- Removing unwrap_or_defaults, using impl functions.
- Reorganized settings

* Make clippy happy

* hjson list strings must be quoted.

* Adding support for env vars.

* Moving to structs and defaults file.

* Moving settings default and struct.
2021-03-01 17:24:11 +00:00
Felix Ableitner 3bdd78f341 Rename `lemmy_structs` to `lemmy_api_structs` 2021-03-01 14:08:41 +01:00
Felix Ableitner 3141ad31de Remove categories (fixes #1429) 2021-02-25 13:22:37 +01:00
Felix Ableitner 999d9f4d6c Move routes into separate crate to speed up compilation 2021-02-09 19:34:36 +01:00
Dessalines 14465b91b1 Fixing inbox url code migration. Fixes #1414 2021-02-05 12:06:32 -05:00
Dessalines d5d99fa3b9
Moving docs to join.lemmy.ml . Fixes #1396 (#1410)
* Moving docs to join.lemmy.ml . Fixes #1396

* Removing submodule fetch from drone.
2021-02-05 12:30:49 +00:00
nutomic 1a4e35eb50 Store activitypub endpoints in database (#162)
Address review comments

Store Activitypub urls in database (fixes #808)

Co-authored-by: Felix Ableitner <me@nutomic.com>
Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy/pulls/162
Co-Authored-By: nutomic <nutomic@noreply.yerbamate.ml>
Co-Committed-By: nutomic <nutomic@noreply.yerbamate.ml>
2021-02-04 16:34:58 +00:00
Dessalines 51465bc0d7 Nodeinfo devs think halfyear is one word. 2021-01-29 14:24:10 -05:00
Dessalines 3f23e0e6b9 Adding camelCase to node-info users. 2021-01-29 13:48:59 -05:00
Dessalines 0fd0279543
Adding some recurring lemmy tasks. (#1386)
* Adding some recurring lemmy tasks.

- Add active users by day, week, month, and half year to site and
  community. Fixes #1195
- Periodically re-index the aggregates tables that use hot_rank.
  Fixes #1384
- Clear out old activities (> 6 months). Fixes #1133

* Some cleanup, recalculating actives every hour.
2021-01-29 11:38:27 -05:00
Nutomic 3d578f9df2
Use Url type for ap_id fields in database (fixes #1364) (#1371) 2021-01-27 11:42:23 -05:00
Dessalines 1de8a4606a A few API v2 changes based on nutomic's suggestions.
- Changed `edit_id` s to their type (comment_id)
- Moved websocket actions to their own file in structs and api.
- Got rid of UserViewDangerous, added UserSafeSettings.
  - GetSite now returns UserSafeSettings for `my_user`.
- Got rid of `admin` field in `Register`.
2021-01-18 16:57:31 -05:00
Dessalines 4fdcb57753 Report only local counts in site_view.
- Move open_registrations under top level.
- Fixes #1340
2021-01-14 15:22:07 -05:00
Dessalines 1e0c32f7a3 Merge branch 'main' into move_views_to_diesel 2021-01-05 23:55:02 -05:00
Felix Ableitner 95e30f0e08 Split up lemmy_db_views, put lemmy_rate_limit into lemmy_utils 2020-12-22 00:34:54 +01:00
Felix Ableitner d5efebbf47 Split lemmy_db into lemmy_db_queries, lemmy_db_aggregates and lemmy_db_views 2020-12-21 17:39:11 +01:00
Dessalines 1a0d1f64f0 Merge remote-tracking branch 'origin/split-db-workspace' into move_views_to_diesel_split 2020-12-21 09:28:20 -05:00
Felix Ableitner 5231666465 Move remaining structs from lemmy_db::source to lemmy_db_schema 2020-12-21 14:38:34 +01:00
Felix Ableitner a7e231b35b Move community to lemmy_db_schema 2020-12-21 13:28:12 +01:00
Dessalines 929f1d02b5 Fixing integration tests. 2020-12-20 22:27:27 -05:00
Dessalines 5af8257e19 Changing unit tests to api v2. 2020-12-20 16:16:57 -05:00
Felix Ableitner f842bbff8d Move user to lemmy_db_schema, create traits for impls 2020-12-18 19:38:32 +01:00
Felix Ableitner 114f3cbfb5 Move comment, post definitions into lemmy_db_schema 2020-12-18 18:27:25 +01:00
Felix Ableitner 089d812dc8 Split lemmy_db into separate workspaces 2020-12-18 17:17:44 +01:00
dessalines 44b72ccbd6 Merge pull request 'Create empty inbox collections for actors (ref #1322)' (#151) from actor-inbox into main
Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy/pulls/151
2020-12-17 14:51:23 +00:00
Felix Ableitner 4bf0ec94c8 Create empty inbox collections for actors (ref #1322) 2020-12-17 14:22:51 +01:00
Dessalines 57c2f2ef1c Getting rid of terrible boxedjoin types. 2020-12-16 13:59:43 -05:00
Dessalines 711db4c790 Removing old user_mention_view. 2020-12-16 11:09:21 -05:00
Felix Ableitner cbd02f2a87 Use correct content-type headers for apub inbox (ref #1220) 2020-12-16 16:30:44 +01:00
Dessalines 471abf7f29 Removing old comment_view. 2020-12-15 14:39:18 -05:00
Dessalines f456f5da46 Re-organizing source tables into a different folder. 2020-12-13 12:04:42 -05:00
Dessalines 35bf50ab15 Removing old postview. 2020-12-11 10:27:33 -05:00
Dessalines 2400a078d7 Merge branch 'main' into move_views_to_diesel 2020-12-05 08:10:25 -06:00
Dessalines b92e7eb781 Updating cargo deps, fixing image if_some deprecation. 2020-12-04 09:00:15 -05:00