Commit Graph

39 Commits (d6b3d8230053abac050a31f2016d4f06dcc4401e)

Author SHA1 Message Date
phiresky 3d649e1d3e remove synchronous federation flag 2023-09-20 14:22:17 +00:00
phiresky 27141553b6 fix after revert 2023-09-18 14:28:49 +00:00
phiresky c86173577d fix export location 2023-09-15 15:51:31 +00:00
phiresky 375d9a2a3c
Persistent, performant, reliable federation queue (#3605)
* persistent activity queue

* fixes

* fixes

* make federation workers function callable from outside

* log federation instances

* dead instance detection not needed here

* taplo fmt

* split federate bin/lib

* minor fix

* better logging

* log

* create struct to hold cancellable task for readability

* use boxfuture for readability

* reset submodule

* fix

* fix lint

* swap

* remove json column, use separate array columns instead

* some review comments

* make worker a struct for readability

* minor readability

* add local filter to community follower view

* remove separate lemmy_federate entry point

* fix remaining duration

* address review comments mostly

* fix lint

* upgrade actitypub-fed to simpler interface

* fix sql format

* increase delays a bit

* fixes after merge

* remove selectable

* fix instance selectable

* add comment

* start federation based on latest id at the time

* rename federate process args

* dead instances in one query

* filter follow+report activities by local

* remove synchronous federation

remove activity sender queue

* lint

* fix federation tests by waiting for results to change

* fix fed test

* fix comment report

* wait some more

* Apply suggestions from code review

Co-authored-by: SorteKanin <sortekanin@gmail.com>

* fix most remaining tests

* wait until private messages

* fix community tests

* fix community tests

* move arg parse

* use instance_id instead of domain in federation_queue_state table

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
2023-09-09 12:25:03 -04:00
Nutomic 27be1efb74
Rewrite remaining federation actions, get rid of PerformCrud trait (#3794)
* Rewrite ban actions

* Rewrite delete/remove actions

* Rewrite remove/delete community

* Rewrite report actions

* Rewrite feature/lock post

* Rewrite update community actions

* Rewrite remaining federation actions

* Get rid of PerformCrud trait

* clippy
2023-08-02 12:52:41 -04:00
RocketDerp 05a7fced65
Enhanced testing of comments. Validate reply notifications, mentions (#3686)
* shared.ts first test of getReplies

* comment testing now validates reply notifications and mentions, some code comment cleanup in other functions

* comments revised

* first use of getUnreadCount in testing

* test notification of new comment replies, clarify usage of getReplies

* killall moved earlier in bash script

* api-test jest run does not need directory prefix, make consistent with other jest runs

* do not put my testing system password into script

* fix, killall exits script when no process found

* killall now moved to parent script to release locks before database create

* need to run killall a second time, before database drop

* first use of getReplies getPosts saveUserSettings

* accidental duplication of functions, removed

* try to sync shared library with main

* Nutomic feedback: Better to rename the var instead of putting a comment which can easily get outdated.

* Correct logic to meet join-lemmy requirement, don't have closed signups. Allows Open and Applications. (#3761)

Co-authored-by: Josh Bernardini <josh.bernardini@cologix.com>

* Fix fetch instance software version from nodeinfo (#3772)

Fixes #3771

* remove unused code, revert killall change

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: figure-0e <133478007+figure-0e@users.noreply.github.com>
Co-authored-by: Josh Bernardini <josh.bernardini@cologix.com>
Co-authored-by: Denis Dzyubenko <denis@ddenis.info>
Co-authored-by: Felix Ableitner <me@nutomic.com>
2023-08-01 09:14:40 -04:00
RocketDerp 2adf7d5008
prepare-drone-federation-test.sh has some more echo output and note about the LEMMY_DATABASE_URL format (#3651) 2023-07-19 10:28:10 +02:00
Charles Hall 4e5798852f
make shebangs posix compliant (#2974)
Previously, these scripts wouldn't work on exotic systems such as NixOS.

```
fd '\.sh$' -t f --exec sed -i 's@#!/bin/bash@#!/usr/bin/env bash@'
```
2023-06-08 15:38:26 -04:00
Dessalines 5d837780f5
Add diesel_async, get rid of blocking function (#2510)
* Moving settings to Database.

- Moves many settings into the database. Fixes #2285
- Adds a local_site and instance table. Fixes #2365 . Fixes #2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.

* Fix weird clippy issue.

* Removing extra lines.

* Some fixes from suggestions.

* Fixing apub tests.

* Using instance creation helper function.

* Move forms to their own line.

* Trying to fix local_site_data, still broken.

* Testing out async

* Testing out async 2

* Fixing federation tests.

* Trying to fix check features 1.

* Starting on adding diesel async. 1/4th done.

* Added async to views and schema.

* Adding some more async

* Compiling now.

* Added diesel async. Fixes #2465

* Running clippy --fix

* Trying to fix cargo test on drone.

* Trying new muslrust.

* Trying a custom dns

* Trying a custom dns 2

* Trying a custom dns 3

* Trying a custom dns 4

* Trying a custom dns 5

* Trying a custom dns 6

* Trying a custom dns 7

* Addressing PR comments.

* Adding check_apub to all verify functions.

* Reverting back drone.

* Fixing merge

* Fix docker images.

* Adding missing discussion_languages.

* Trying to fix federation tests.

* Fix site setup user creation.

* Fix clippy

* Fix clippy 2

* Test api faster

* Try to fix 1

* Try to fix 2

* What are these lines about

* Trying to fix 3

* Moving federation test back to top.

* Remove logging cat.
2022-11-09 10:05:00 +00:00
Nutomic fcaf7a084c
Upgrade activitypub_federation to 0.2.0, add setting federation.debug (#2300) 2022-06-08 11:45:39 -04:00
Felix Ableitner f24999027e Merge crates db_schema and db_queries 2021-10-20 12:28:08 +02:00
Nutomic b96ce81f89
Move code to apub library (#1795)
* Remove dependency of apub_lib on LemmyContext

* Move ApubObject trait to library

* Reorganize files in apub lib

* Move ActorType, signatures, activity_queue to apub library
2021-10-06 16:20:05 -04:00
Felix Ableitner c1db86925f Merge apub, apub_receive crates (fixes #1621) 2021-07-17 18:20:44 +02:00
Nutomic c7de1fcf24
Apub inbox rewrite (#1652)
* start to implement apub inbox routing lib

* got something that almost works

* it compiles!

* implemented some more

* move library code to separate crate (most of it)

* convert private message handlers

* convert all comment receivers (except undo comment)

* convert post receiver

* add verify trait

* convert community receivers

* add cc field for all activities which i forgot before

* convert inbox functions, add missing checks

* convert undo like/dislike receivers

* convert undo_delete and undo_remove receivers

* move block/unblock activities

* convert remaining activity receivers

* reimplement http signature verification and other checks

* also use actor type for routing, VerifyActivity and SendActivity traits

* cleanup and restructure apub_receive code

* wip: try to fix activity routing

* implement a (very bad) derive macro for activityhandler

* working activity routing!

* rework pm verify(), fix tests and confirm manually

also remove inbox username check which was broken

* rework following verify(), fix tests and test manually

* fix post/comment create/update, rework voting

* Rewrite remove/delete post/comment, fix tests, test manually

* Rework and fix (un)block user, announce, update post

* some code cleanup

* rework delete/remove activity receivers (still quite messy)

* rewrite, test and fix add/remove mod, update community handlers

* add docs for ActivityHandler derive macro

* dont try to compile macro comments
2021-07-17 12:08:46 -04:00
Dessalines 2309088b03 Renaming to api/v3 2021-03-29 11:26:15 -04:00
Dessalines 462c4a2954
Rewrite settings implementation. Fixes #1270 (#1433)
* A first attempt at using deser-hjson. Fixes #1270

* Trying to fix tests, try 1

* Trying to fix tests, try 2

* A few fixes to deser_hjson

- Removing unwrap_or_defaults, using impl functions.
- Reorganized settings

* Make clippy happy

* hjson list strings must be quoted.

* Adding support for env vars.

* Moving to structs and defaults file.

* Moving settings default and struct.
2021-03-01 17:24:11 +00:00
Dessalines 929f1d02b5 Fixing integration tests. 2020-12-20 22:27:27 -05:00
Felix Ableitner e492cce206 Allow running docker-less federation tests locally 2020-12-15 14:58:11 +01:00
Felix Ableitner d5955b60c0 silence lemmy output in federation logs 2020-12-14 18:01:12 +01:00
Felix Ableitner f33577b317 send activities sync for tests 2020-12-14 17:44:27 +01:00
Felix Ableitner 543be801ab disable debug log 2020-12-11 20:56:14 +01:00
Felix Ableitner ab6b28ee60 use dash 2020-12-11 20:39:54 +01:00
Felix Ableitner 0c89e9c2d6 use hosts file 2020-12-11 18:22:16 +01:00
Felix Ableitner 30a1a69850 setup db 2020-12-11 18:09:47 +01:00
Felix Ableitner 9c7f2cb0c3 fix bin path 2020-12-11 17:33:55 +01:00
Felix Ableitner b61bfcefa7 find the binary 2020-12-11 17:27:18 +01:00
Felix Ableitner 7c12b1026c faster release build 2020-12-11 17:01:05 +01:00
Felix Ableitner 5b376de5f5 need to use release bin 2020-12-11 16:25:20 +01:00
Felix Ableitner df2ec0aa38 try running federation tests in same alpine version 2020-12-11 14:13:30 +01:00
Felix Ableitner 4fd6b5f5e1 fix script location 2020-12-10 21:36:50 +01:00
Felix Ableitner 405e7eff27 try to fix federation test 2020-12-10 21:14:05 +01:00
Felix Ableitner 7af4a60ec4 change dir to read config 2020-12-10 20:19:17 +01:00
Felix Ableitner 7b2c59bd98 use bash, install psql 2020-12-10 20:06:09 +01:00
Felix Ableitner 580e397525 dont use alpine for federation test 2020-12-10 19:50:41 +01:00
Felix Ableitner 7717deda0e more debug 2020-12-10 18:57:09 +01:00
Felix Ableitner cdcbef088d more debugging 2020-12-10 18:50:15 +01:00
Felix Ableitner 94d6ceb4df install bash and curl 2020-12-10 18:29:13 +01:00
Felix Ableitner 200913f631 try with sh 2020-12-10 18:26:50 +01:00
Felix Ableitner f76f742ba7 implement federation test in bash 2020-12-10 18:03:11 +01:00