Compare commits

...

29 Commits

Author SHA1 Message Date
IAmTomahawkx
bf8aa9651e Merge branch 'main' into fix/january-ip-redirects 2026-05-06 22:09:40 -07:00
Tom
356491e934 fix: january ip redirects & domain resolver (#738)
* fix: properly block private ip ranges
I'm a dunce and forgot that domains do in fact resolve to ips.

* fix: reimplement max redirects

* fix: remove my debug error

* fix: actually check redirect urls
kind of the whole point of this thing.
2026-05-06 21:33:52 -07:00
IAmTomahawkx
1098e970f2 fix: actually check redirect urls
kind of the whole point of this thing.
2026-05-06 21:32:34 -07:00
IAmTomahawkx
7fb536cad4 fix: remove my debug error 2026-05-06 18:03:27 -07:00
IAmTomahawkx
1feec707c1 fix: reimplement max redirects 2026-05-06 18:01:44 -07:00
IAmTomahawkx
f0a6ed78fe fix: properly block private ip ranges
I'm a dunce and forgot that domains do in fact resolve to ips.
2026-05-06 17:59:55 -07:00
İspik
21d82018cf feat: add legal links to root payload (#733)
Signed-off-by: ispik <ispik@ispik.dev>
2026-05-06 16:35:20 -07:00
Tom
6b41db984b feat: blacklist private ip ranges and add january domain blocklist (#731)
Signed-off-by: IAmTomahawkx <iamtomahawkx@gmail.com>
2026-05-06 16:25:31 -07:00
İspik
5378cd22b4 fix: use correct response for NoEffect errors (#732)
Signed-off-by: ispik <ispik@ispik.dev>
2026-05-05 11:57:42 -07:00
İspik
841985d3b9 feat: add role icon support (#724)
Signed-off-by: ispik <ispik@ispik.dev>
2026-05-02 17:04:28 -07:00
İspik
279f5d5fd7 fix: add new_user_hours to configuration limits (#729)
Signed-off-by: ispik <ispik@ispik.dev>
2026-04-26 22:47:52 -07:00
jarvarvarvis
e93769786c feat: automatically sanitise usernames on create/update (#689)
* feat: automatically sanitise usernames on create/update

Signed-off-by: higgs01 <6546697+higgs01@users.noreply.github.com>

* test: add tests for validation and sanitasion

Signed-off-by: higgs01 <6546697+higgs01@users.noreply.github.com>

* fix: Return only the sanitised string in sanitise_username (#424)

* fix: Use as_str for role.id in insert_role
* Run rustfmt for code changes

Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>

* feat: Make minimum username length configurable in Revolt.toml (#424)

* fix: Remove redundant call to is_match with blocked username regex

Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>

* Update crates/core/database/src/models/users/model.rs

Co-authored-by: Tom <iamtomahawkx@gmail.com>
Signed-off-by: jarvarvarvis <53998846+jarvarvarvis@users.noreply.github.com>

* Update crates/core/database/src/models/users/model.rs

Co-authored-by: Tom <iamtomahawkx@gmail.com>
Signed-off-by: jarvarvarvis <53998846+jarvarvarvis@users.noreply.github.com>

* Update crates/core/database/src/models/users/model.rs

Co-authored-by: Tom <iamtomahawkx@gmail.com>
Signed-off-by: jarvarvarvis <53998846+jarvarvarvis@users.noreply.github.com>

* Update crates/core/database/src/models/users/model.rs

Co-authored-by: Tom <iamtomahawkx@gmail.com>
Signed-off-by: jarvarvarvis <53998846+jarvarvarvis@users.noreply.github.com>

* fix: Implement suggested changes and clean up last 4 commits

Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>

* fix: Disallow stoat as username, update create_user test

Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>

* fix: Use sanitised username to find updated discriminator

Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>

* feat: Sanitise revolt.chat in username

Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>

* fix: Implement discussed changes

Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>

* test: Fix create_user test

Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>

* fix: don't overflow the stack
not entirely sure why this fixes it and I don't like it. But work it does.

Signed-off-by: IAmTomahawkx <iamtomahawkx@gmail.com>

* fix: revert odd file mode change

Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>

---------

Signed-off-by: higgs01 <6546697+higgs01@users.noreply.github.com>
Signed-off-by: jarvarvarvis <jarvistrigo@gmail.com>
Signed-off-by: jarvarvarvis <53998846+jarvarvarvis@users.noreply.github.com>
Signed-off-by: IAmTomahawkx <iamtomahawkx@gmail.com>
Co-authored-by: higgs01 <6546697+higgs01@users.noreply.github.com>
Co-authored-by: Tom <iamtomahawkx@gmail.com>
2026-04-23 21:02:28 -07:00
İspik
ed4fd5ebfe fix: update message length validation to remove upper limit (#723)
Signed-off-by: ispik <ispik@ispik.dev>
2026-04-23 20:39:13 -07:00
Angelo Kontaxis
89171e9bd0 fix: dont send notification in fcm (#721)
* fix: dont send notification in fcm

Signed-off-by: Zomatree <me@zomatree.live>

* fix: add notification type to fcm

Signed-off-by: Zomatree <me@zomatree.live>

* fix: switch to structured notification data

Signed-off-by: Zomatree <me@zomatree.live>

---------

Signed-off-by: Zomatree <me@zomatree.live>
2026-04-22 23:48:26 -07:00
Infiland
057f2bb8b3 fix: add reconnection policy to Redis subscriber to prevent ghost state (#708)
* fix: add reconnection policy to Redis subscriber to prevent ghost state

- Add ReconnectPolicy::new_exponential(0, 100, 30_000, 2) to the subscriber builder, unlimited retries with exponential backoff (100ms min, 30s max)
- Add on_reconnect handler that signals the listener loop to force a subscription reset, re-subscribing to all topics on the new connection
- Add warn-level logging to on_error for all Redis subscriber errors (previously only Canceled was handled, others were silently ignored)

Signed-off-by: Infiland <ljubica.citydesign@gmail.com>

* Update websocket.rs

Signed-off-by: Infiland <88491175+Infiland@users.noreply.github.com>

* Auto-manage subscriptions on reconnect

Call subscriber.manage_subscriptions() so the subscriber will automatically re-subscribe tracked channels after a Redis reconnect. Remove the manual reconnect channel and on_reconnect handler along with the select branch that forced SubscriptionStateChange::Reset.

Signed-off-by: Infiland <88491175+Infiland@users.noreply.github.com>

---------

Signed-off-by: Infiland <ljubica.citydesign@gmail.com>
Signed-off-by: Infiland <88491175+Infiland@users.noreply.github.com>
2026-04-17 20:15:54 -07:00
Angelo Kontaxis
3675ff1a1f chore: migrate all local dependancies to workspace dependancies (#710)
* chore: start moving all deps to workspace deps

Signed-off-by: Zomatree <me@zomatree.live>

* chore: migrate all deps to workspace deps

Signed-off-by: Zomatree <me@zomatree.live>

* chore: add more dep groups

Signed-off-by: Zomatree <me@zomatree.live>

* fix: add migration to update existing files to be animated (#705)

* fix: add migration to update existing files to be animated

Signed-off-by: Zomatree <me@zomatree.live>

* Revert "fix: add migration to update existing files to be animated"

This reverts commit 4e1f1c116c.

Signed-off-by: Zomatree <me@zomatree.live>

* fix: calculate animated for existing files when fetched

Signed-off-by: Zomatree <me@zomatree.live>

---------

Signed-off-by: Zomatree <me@zomatree.live>

* fix: mise start + missing docker image (#564)

* fix: mise start + missing docker image

Signed-off-by: Damocles078 <hellodamocles078@gmail.com>

* fix: bump livekit version

Signed-off-by: Damocles <106018783+Damocles078@users.noreply.github.com>

---------

Signed-off-by: Damocles078 <hellodamocles078@gmail.com>
Signed-off-by: Tom <iamtomahawkx@gmail.com>
Signed-off-by: Damocles <106018783+Damocles078@users.noreply.github.com>
Co-authored-by: Tom <iamtomahawkx@gmail.com>

* docs: update donation link (#709)

Signed-off-by: Zomatree <me@zomatree.live>

* fix: remove usage of deprecated functions

Signed-off-by: Zomatree <me@zomatree.live>

---------

Signed-off-by: Zomatree <me@zomatree.live>
Signed-off-by: Damocles078 <hellodamocles078@gmail.com>
Signed-off-by: Tom <iamtomahawkx@gmail.com>
Signed-off-by: Damocles <106018783+Damocles078@users.noreply.github.com>
Co-authored-by: Damocles <106018783+Damocles078@users.noreply.github.com>
Co-authored-by: Tom <iamtomahawkx@gmail.com>
Co-authored-by: Paul Makles <me@insrt.uk>
2026-04-17 19:02:18 -07:00
infi
144e939c6b docs: add run in yaak button to endpoints page (#711) 2026-04-12 00:46:05 +01:00
stoat-release[bot]
61fd13629f chore(main): release 0.12.1 (#700)
Co-authored-by: stoat-release[bot] <245062572+stoat-release[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-04-11 12:31:11 +01:00
Paul Makles
47336a7940 docs: pr template (#713) 2026-04-10 20:44:29 +01:00
Paul Makles
4ea8f90c0d docs: update donation link (#709) 2026-04-03 00:28:49 +01:00
Damocles
fb8fe16557 fix: mise start + missing docker image (#564)
* fix: mise start + missing docker image

Signed-off-by: Damocles078 <hellodamocles078@gmail.com>

* fix: bump livekit version

Signed-off-by: Damocles <106018783+Damocles078@users.noreply.github.com>

---------

Signed-off-by: Damocles078 <hellodamocles078@gmail.com>
Signed-off-by: Tom <iamtomahawkx@gmail.com>
Signed-off-by: Damocles <106018783+Damocles078@users.noreply.github.com>
Co-authored-by: Tom <iamtomahawkx@gmail.com>
2026-04-01 19:51:14 -07:00
Angelo Kontaxis
f2c056a151 fix: add migration to update existing files to be animated (#705)
* fix: add migration to update existing files to be animated

Signed-off-by: Zomatree <me@zomatree.live>

* Revert "fix: add migration to update existing files to be animated"

This reverts commit 4e1f1c116c.

Signed-off-by: Zomatree <me@zomatree.live>

* fix: calculate animated for existing files when fetched

Signed-off-by: Zomatree <me@zomatree.live>

---------

Signed-off-by: Zomatree <me@zomatree.live>
2026-04-01 19:48:22 -07:00
Tom
f30b729ca9 fix: don't send self dm notifications (#706)
* fix: don't send self dm notifications

Signed-off-by: IAmTomahawkx <iamtomahawkx@gmail.com>

* fix: test failure due to wrong assertion (#707)

Signed-off-by: IAmTomahawkx <iamtomahawkx@gmail.com>

* fix: RA auto import moment

Signed-off-by: IAmTomahawkx <iamtomahawkx@gmail.com>

---------

Signed-off-by: IAmTomahawkx <iamtomahawkx@gmail.com>
2026-03-31 15:25:36 -07:00
Tom
f81e3291bd fix: test failure due to wrong assertion (#707) 2026-03-31 11:06:39 +01:00
Tom
1e80916b65 Fix/release please fix 4 (#703)
* fix: release-please-3

* fix: release please bs v4
2026-03-28 23:05:01 -07:00
Tom
8814bf4c23 fix: release-please-3 (#702) 2026-03-28 22:53:12 -07:00
Tom
5466ada9ae Fix: release please bs 2 electric boogaloo (#701)
* fix: update release (me from this) please

* fix: add dep to release please

* Revert "fix: update release (me from this) please"

This reverts commit 0618c492c7.
2026-03-28 22:42:13 -07:00
Tom
f0e513ccae fix: update release (me from this) please (#699) 2026-03-28 22:35:51 -07:00
stoat-release[bot]
4d4b0dd864 chore(main): release 0.12.0 (#602)
Co-authored-by: stoat-release[bot] <245062572+stoat-release[bot]@users.noreply.github.com>
2026-03-28 22:22:37 -07:00
61 changed files with 3491 additions and 2629 deletions

24
.github/pull_request_template.md vendored Normal file
View File

@@ -0,0 +1,24 @@
<!-- Describe your changes -->
Fixes # (issue)
## How was this PR tested?
<!-- What did you do to test your changes? -->
- [ ] Test A
- [ ] Test B
## Checklist:
- [ ] I have carefully read [the contributing guidelines](https://developers.stoat.chat/developing/contrib/)
- [ ] I have performed a self-review of my own code
- [ ] I have made corresponding changes to the documentation if applicable
- [ ] I have no unrelated changes in the PR
- [ ] I have confirmed that any new dependencies are strictly necessary
- [ ] I have written tests for new code (if applicable)
- [ ] I have followed naming conventions/patterns in the surrounding code
## Please declare, if any, LLM usage involved in creating this PR
...

View File

@@ -16,4 +16,13 @@ idiomatic_version_file_enable_tools = ["rust"]
[tasks.start]
description = "Run all services"
depends = ["docker:start", "build"]
wait_for = ["docker:start", "build"]
run = [{ task = "service:*" }]
[env]
BUILDER = "cargo"
DOCKER_NETWORK_NAME = "stoatchat_default"
DATABASE_PORT = "27017"
RABBIT_PORT = "5672"
REDIS_PORT = "6379"
_.file = { path = ".env", tools = true }

View File

@@ -2,4 +2,4 @@
#MISE description="Build project"
set -e
cargo build "$@"
${BUILDER} build "$@"

View File

@@ -3,3 +3,11 @@
set -e
docker compose up -d
docker run \
--network=${DOCKER_NETWORK_NAME} \
--name wait \
--rm dokku/wait -c \
rabbit:${RABBIT_PORT},\
database:${DATABASE_PORT},\
redis:${REDIS_PORT}

View File

@@ -1,3 +1,3 @@
{
".": "0.11.5"
".": "0.12.1"
}

View File

@@ -1,5 +1,52 @@
# Changelog
## [0.12.1](https://github.com/stoatchat/stoatchat/compare/v0.12.0...v0.12.1) (2026-04-10)
### Bug Fixes
* add migration to update existing files to be animated ([#705](https://github.com/stoatchat/stoatchat/issues/705)) ([f2c056a](https://github.com/stoatchat/stoatchat/commit/f2c056a1515be493b195f3f5db5886c2ddf36700))
* don't send self dm notifications ([#706](https://github.com/stoatchat/stoatchat/issues/706)) ([f30b729](https://github.com/stoatchat/stoatchat/commit/f30b729ca90d0be6853c57ab4935694e5e59ae56))
* mise start + missing docker image ([#564](https://github.com/stoatchat/stoatchat/issues/564)) ([fb8fe16](https://github.com/stoatchat/stoatchat/commit/fb8fe1655776791421284a6a093e86f0320c258a))
* test failure due to wrong assertion ([#707](https://github.com/stoatchat/stoatchat/issues/707)) ([f81e329](https://github.com/stoatchat/stoatchat/commit/f81e3291bdd57af9ceedb2987b111acc7051d69c))
## [0.12.0](https://github.com/stoatchat/stoatchat/compare/v0.11.5...v0.12.0) (2026-03-28)
### Features
* add bug report template for issue tracking ([#627](https://github.com/stoatchat/stoatchat/issues/627)) ([f777e28](https://github.com/stoatchat/stoatchat/commit/f777e2863c6ca50057c8b5d0a5be14915d287724))
* Add slowmode functionality to text channels ([#680](https://github.com/stoatchat/stoatchat/issues/680)) ([6107f24](https://github.com/stoatchat/stoatchat/commit/6107f242fd3ebaff71a15f9a16330ffbcb4f2d7b))
* Allow restricting server creation to specific users ([#685](https://github.com/stoatchat/stoatchat/issues/685)) ([edfa97d](https://github.com/stoatchat/stoatchat/commit/edfa97db108c9c81828547f98a1db5315cb5ba4a))
* compute thumbhash for images ([#596](https://github.com/stoatchat/stoatchat/issues/596)) ([c2d4369](https://github.com/stoatchat/stoatchat/commit/c2d4369e160f32d79bce0a0b0f14677f89de3669))
* Detect animation in image files for fetch_preview ([#574](https://github.com/stoatchat/stoatchat/issues/574)) ([3fa0abf](https://github.com/stoatchat/stoatchat/commit/3fa0abf47f5f42ddd8ee041fe4c44fbc5ba800c1))
* expose global and user limits in root API response ([#644](https://github.com/stoatchat/stoatchat/issues/644)) ([0b522eb](https://github.com/stoatchat/stoatchat/commit/0b522ebddc17f2e3f792ff5e2347793e9849fa23))
* implement time based message sweep on user ban ([#670](https://github.com/stoatchat/stoatchat/issues/670)) ([98c7b1b](https://github.com/stoatchat/stoatchat/commit/98c7b1b5a5b9fdac5c0ab83be10f0e23114dbfc9))
* load config from env vars ([#576](https://github.com/stoatchat/stoatchat/issues/576)) ([5191bd1](https://github.com/stoatchat/stoatchat/commit/5191bd16b2a905b8409838e34eb0baca96f08580))
* parse message push notification content and replace internal formatting ([#693](https://github.com/stoatchat/stoatchat/issues/693)) ([d1e72ce](https://github.com/stoatchat/stoatchat/commit/d1e72cee42c54e16f4e49af569897528b10a28ca))
* Transfer ownership ([#396](https://github.com/stoatchat/stoatchat/issues/396)) ([735d644](https://github.com/stoatchat/stoatchat/commit/735d644e043793cb86e74aab5b88bb4b8bc17ba2))
* update livekit ([#698](https://github.com/stoatchat/stoatchat/issues/698)) ([f181edc](https://github.com/stoatchat/stoatchat/commit/f181edc8f2ff3ce4b6d48938dfc73931ecfa2279))
### Bug Fixes
* add flag for disabling events instead of commenting them out ([#695](https://github.com/stoatchat/stoatchat/issues/695)) ([a5cd08a](https://github.com/stoatchat/stoatchat/commit/a5cd08a655dece4269f3ac84fa2387ae356709a5))
* add masquerade permission to default direct message settings ([#665](https://github.com/stoatchat/stoatchat/issues/665)) ([ab52569](https://github.com/stoatchat/stoatchat/commit/ab525699bd6663333f0e9fed6d2455e482e6a09f))
* Check for appropriate permission for removing other users avatar ([#657](https://github.com/stoatchat/stoatchat/issues/657)) ([d56135e](https://github.com/stoatchat/stoatchat/commit/d56135e0cbc713884c9378832952f7ad490fa315))
* default video resolution is a non-existent size ([#601](https://github.com/stoatchat/stoatchat/issues/601)) ([0698e11](https://github.com/stoatchat/stoatchat/commit/0698e115e8d003d615e468c4fb9654e6bbc9107f)), closes [#588](https://github.com/stoatchat/stoatchat/issues/588)
* **docs:** Update GitHub links ([#647](https://github.com/stoatchat/stoatchat/issues/647)) ([b830631](https://github.com/stoatchat/stoatchat/commit/b830631bd25a546844b7bdd30386084bb365e4de))
* don't use a bitop for OR ([#676](https://github.com/stoatchat/stoatchat/issues/676)) ([5701b5c](https://github.com/stoatchat/stoatchat/commit/5701b5c18c513f796af365169ceaea372a22638c))
* Fix typo for p256dh in vapid notification flow ([#622](https://github.com/stoatchat/stoatchat/issues/622)) ([a80ad1c](https://github.com/stoatchat/stoatchat/commit/a80ad1cbe58b8af5e45751e51d94d93c1cea1c9f))
* improve generated openapi.json ([#584](https://github.com/stoatchat/stoatchat/issues/584)) ([52ed510](https://github.com/stoatchat/stoatchat/commit/52ed5100c2446e0b261085639e123e7e124cab2c))
* no node state set on channel creation ([#653](https://github.com/stoatchat/stoatchat/issues/653)) ([24d0d2b](https://github.com/stoatchat/stoatchat/commit/24d0d2b7266f6f8a692d0a52704acfecf517674c))
* only show first line on commit messages ([#696](https://github.com/stoatchat/stoatchat/issues/696)) ([91783b9](https://github.com/stoatchat/stoatchat/commit/91783b906697fc85305dee683f7c15dda55f0c50))
* pass &str to Reference ([#697](https://github.com/stoatchat/stoatchat/issues/697)) ([ccda6f5](https://github.com/stoatchat/stoatchat/commit/ccda6f5c53ee043705f7ff6b5f6c393f020781de))
* redis_url vs redis_uri in config ([#666](https://github.com/stoatchat/stoatchat/issues/666)) ([b0b728f](https://github.com/stoatchat/stoatchat/commit/b0b728fb0dbc9ee28360301de1c3ea501bbbff1d))
* replace some links and Revolt mentions to current Stoat ([#515](https://github.com/stoatchat/stoatchat/issues/515)) ([d629e89](https://github.com/stoatchat/stoatchat/commit/d629e89304be2f0011e189293b278f07d346aa7d))
* send push notifications for DM and group messages ([#660](https://github.com/stoatchat/stoatchat/issues/660)) ([52c0d2f](https://github.com/stoatchat/stoatchat/commit/52c0d2f266b76d8975bba2d5e75c62bb30149c45))
* store server id in redis and in room metadata to be able to delete voice state in all scenarios ([#656](https://github.com/stoatchat/stoatchat/issues/656)) ([49c6289](https://github.com/stoatchat/stoatchat/commit/49c628958070e4f0a5edc764d3b48158589219d9))
* uname is missing from crond ([#675](https://github.com/stoatchat/stoatchat/issues/675)) ([dc4438b](https://github.com/stoatchat/stoatchat/commit/dc4438bc3c7b2cad8d442b3cd438afb9ed566a5e))
## [0.11.5](https://github.com/stoatchat/stoatchat/compare/v0.11.4...v0.11.5) (2026-02-17)

4034
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,5 @@
[workspace]
resolver = "2"
resolver = "3"
members = [
"crates/delta",
@@ -20,24 +20,129 @@ lto = true
[workspace.dependencies]
# Async
async-trait = "0.1.89"
tokio = { version = "1.49.0", features = ["macros", "rt"] }
tokio = "1.49.0"
async-channel = "2.3.1"
futures = "0.3.32"
async-std = "1.8.0"
async-tungstenite = "0.17.0"
futures-locks = "0.7.1"
async-lock = "2.8.0"
async-recursion = "1.0.4"
# Error Handling
anyhow = "1.0.100"
thiserror = "2.0.18"
sentry = "0.31.5"
sentry-anyhow = "0.38.1"
# Other Utilities
uuid = { version = "1.19.0", features = ["v4"] }
# Data Validation
regex = "1.12.3"
validator = "0.16"
# Data Types
uuid = "1.19.0"
ulid = "1.2.1"
nanoid = "0.4.0"
typenum = "1.17.0"
num_enum = "0.6.1"
bitfield = "0.13.2"
# Time
chrono = "0.4.15"
iso8601-timestamp = "0.2.10"
# Data Collections
lru = "0.16.3"
indexmap = "2.13.1"
dashmap = "5.2.0"
moka = "0.12.8"
lru_time_cache = "0.11.11"
deadqueue = "0.2.4"
# Web scraping
scraper = "0.20.0"
encoding_rs = "0.8.34"
# Mail
lettre = "0.10.0-alpha.4"
# HTTP Requests
reqwest = "0.13.2"
isahc = "1.7"
# Notifications
fcm_v1 = "0.3.0"
web-push = "0.10.0"
revolt_a2 = "0.10"
# Parsing
logos = "0.15"
# SVG rendering
usvg = "0.44.0"
resvg = "0.44.0"
tiny-skia = "0.11.4"
# Logging
log = "0.4.29"
pretty_env_logger = "0.4.0"
# Redis
redis-kiss = "0.1.4"
fred = "8.0.1"
# Serialisation
bincode = "1.3.3"
serde_json = "1.0.79"
rmp-serde = "1.0.0"
serde = { version = "1", features = ["derive"] }
strum_macros = "0.26.4"
# MongoDB
bson = { version = "2.1.0" }
mongodb = { version = "3.1.0" }
# S3
aws-config = "1.5.5"
aws-sdk-s3 = "1.46.0"
# Axum (HTTP server)
axum-macros = "0.4.1"
axum_typed_multipart = "0.12.1"
axum = { version = "0.7.5", features = ["multipart"] }
tower-http = { version = "0.5.2", features = ["cors", "trace"] }
axum = "0.7.5"
axum-extra = "0.9"
tower-http = "0.5.2"
# Rocket (HTTP server)
rocket = "0.5.1"
rocket_empty = "0.1.1"
revolt_rocket_okapi = "0.10.0"
rocket_cors = { git = "https://github.com/lawliet89/rocket_cors", rev = "072d90359b23e9b291df6b672c07c93de9c46011" }
rocket_authifier = "1.0.16"
rocket_prometheus = "0.10.0-rc.3"
# Spec Generation
utoipa = "4.2.3"
revolt_okapi = "0.9.1"
schemars = "0.8.8"
utoipa-scalar = "0.1.0"
# Image Processing
jxl-oxide = { version = "0.12.5", features = ["image"] }
image = "0.25.9"
jxl-oxide = "0.12.5"
sha2 = "0.10.8"
kamadak-exif = "0.5.4"
webp = "0.3.0"
image = "0.25.2" # avif encode requires dav1d system library: features = ["avif-native"]
thumbhash = "0.1.0"
# File processing
revolt_clamav-client = "0.1.5"
simdutf8 = "0.1.4"
# Content type processing
infer = "0.16.0"
ffprobe = "0.4.0"
imagesize = "0.13.0"
# OpenTelemetry
tracing = "0.1.44"
@@ -47,4 +152,50 @@ tracing-subscriber = { version = "0.3.22", features = [
opentelemetry = { version = "0.31.0", features = ["logs"] }
opentelemetry_sdk = { version = "0.31.0", features = ["logs"] }
opentelemetry-otlp = { version = "0.31.0", features = ["logs"] }
opentelemetry-appender-tracing = { version = "0.31.1" }
opentelemetry-appender-tracing = "0.31.1"
# Authifier
authifier = "1.0.16"
# RabbitMQ
amqprs = "1.7.0"
# Voice
livekit-api = "0.4.4"
livekit-protocol = "0.7.4"
livekit-runtime = "0.4.0"
# Other Utilities
once_cell = "1.9.0"
config = "0.13.3"
cached = "0.44.0"
rand = "0.8.5"
base64 = "0.21.3"
decancer = "3.3.3"
linkify = "0.8.1"
url-escape = "0.1.1"
revolt_optional_struct = "0.2.0"
unicode-segmentation = "1.10.1"
querystring = "1.1.0"
tempfile = "3.12.0"
aes-gcm = "0.10.3"
auto_ops = "0.3.0"
url = "2.2.2"
impl_ops = "0.1.1"
lazy_static = "1.5.0"
mime = "0.3.17"
# Build Dependencies
vergen = "7.5.0"
# Local packages
revolt-coalesced = { version = "0.12.0", path = "crates/core/coalesced" }
revolt-config = { version = "0.12.0", path = "crates/core/config" }
revolt-database = { version = "0.12.0", path = "crates/core/database" }
revolt-files = { version = "0.12.0", path = "crates/core/files" }
revolt-models = { version = "0.12.0", path = "crates/core/models" }
revolt-parser = { version = "0.12.0", path = "crates/core/parser" }
revolt-permissions = { version = "0.12.0", path = "crates/core/permissions" }
revolt-presence = { version = "0.12.0", path = "crates/core/presence" }
revolt-ratelimits = { version = "0.12.0", path = "crates/core/ratelimits" }
revolt-result = { version = "0.12.0", path = "crates/core/result" }

View File

@@ -76,6 +76,14 @@ mise install
mise build
```
> [!TIP]
> You can override `BUILDER` in your `.env` file to run cargo with mold if you installed it:
>
> ```bash
> # .env
> BUILDER = "mold --run cargo"
> ```
A default configuration `Revolt.toml` is present in this project that is suited for development.
If you'd like to change anything, create a `Revolt.overrides.toml` file and specify relevant variables.
@@ -112,7 +120,7 @@ If you'd like to change anything, create a `Revolt.overrides.toml` file and spec
> - "14672:15672"
> ```
>
> And corresponding Revolt configuration:
> With the corresponding Revolt configuration:
>
> ```toml
> # Revolt.overrides.toml
@@ -124,32 +132,25 @@ If you'd like to change anything, create a `Revolt.overrides.toml` file and spec
> [rabbit]
> port = 14072
> ```
>
> And mise configuration
>
> ```bash
> #.env
> DATABASE_PORT = "14017"
> RABBIT_PORT = "14072"
> REDIS_PORT = "14079"
> ```
Then continue:
```bash
# start other necessary services
docker compose up -d
cp livekit.example.yml livekit.yml
# run the API server
cargo run --bin revolt-delta
# run the events server
cargo run --bin revolt-bonfire
# run the file server
cargo run --bin revolt-autumn
# run the proxy server
cargo run --bin revolt-january
# run the tenor proxy
cargo run --bin revolt-gifbox
# run the push daemon (not usually needed in regular development)
cargo run --bin revolt-pushd
# hint:
# mold -run <cargo build, cargo run, etc...>
# mold -run ./scripts/start.sh
mise start
```
You can start a web client by doing the following:
You can start a web client by doing the following in another terminal:
```bash
# if you do not have yarn yet and have a modern Node.js:
@@ -163,6 +164,9 @@ cd stoat-web
When signing up, go to http://localhost:14080 to find confirmation/password reset emails.
To stop all services, hit (CTRL + c) in the terminal you ran `mise start` and run `mise docker:stop`
## Deployment Guide
### Cutting new crate releases
@@ -198,7 +202,7 @@ If you have bumped the crate versions, proceed to [GitHub releases](https://gith
First, start the required services:
```sh
docker compose -f docker-compose.db.yml up -d
docker compose up -d
```
Now run tests for whichever database:

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-bonfire"
version = "0.11.5"
version = "0.12.1"
license = "AGPL-3.0-or-later"
edition = "2021"
publish = false
@@ -9,42 +9,38 @@ publish = false
[dependencies]
# util
log = "*"
sentry = "0.31.5"
lru = "0.7.6"
ulid = "0.5.0"
once_cell = "1.9.0"
redis-kiss = "0.1.4"
lru_time_cache = "0.11.11"
async-channel = "2.3.1"
log = { workspace = true }
sentry = { workspace = true }
lru = { workspace = true }
ulid = { workspace = true }
once_cell = { workspace = true }
redis-kiss = { workspace = true }
lru_time_cache = { workspace = true }
async-channel = { workspace = true }
# parsing
querystring = "1.1.0"
regex = "1.11.1"
querystring = { workspace = true }
regex = { workspace = true }
# serde
bincode = "1.3.3"
serde_json = "1.0.79"
rmp-serde = "1.0.0"
serde = "1.0.136"
bincode = { workspace = true }
serde_json = { workspace = true }
rmp-serde = { workspace = true }
serde = { workspace = true }
# async
futures = "0.3.21"
async-tungstenite = { version = "0.17.0", features = ["async-std-runtime"] }
async-std = { version = "1.8.0", features = [
"tokio1",
"tokio02",
"attributes",
] }
futures = { workspace = true }
async-tungstenite = { workspace = true, features = ["async-std-runtime"] }
async-std = { workspace = true }
# core
authifier = { version = "1.0.16" }
revolt-result = { path = "../core/result" }
revolt-models = { path = "../core/models" }
revolt-config = { path = "../core/config" }
revolt-database = { path = "../core/database", features = ["voice"] }
revolt-permissions = { path = "../core/permissions" }
revolt-presence = { path = "../core/presence", features = ["redis-is-patched"] }
authifier = { workspace = true }
revolt-result = { workspace = true }
revolt-models = { workspace = true }
revolt-config = { workspace = true }
revolt-database = { workspace = true, features = ["voice"] }
revolt-permissions = { workspace = true }
revolt-presence = { workspace = true, features = ["redis-is-patched"] }
# redis
fred = { version = "8.0.1", features = ["subscriber-client"] }
fred = { workspace = true, features = ["subscriber-client"] }

View File

@@ -1,7 +1,5 @@
use std::{
collections::{HashMap, HashSet},
sync::Arc,
time::Duration,
collections::{HashMap, HashSet}, num::NonZeroUsize, sync::Arc, time::Duration
};
use async_std::sync::{Mutex, RwLock};
@@ -57,7 +55,7 @@ impl Default for Cache {
members: Default::default(),
servers: Default::default(),
seen_events: LruCache::new(20),
seen_events: LruCache::new(NonZeroUsize::new(20).unwrap()),
}
}
}

View File

@@ -5,7 +5,7 @@ use authifier::AuthifierEvent;
use fred::{
error::RedisErrorKind,
interfaces::{ClientLike, EventInterface, PubsubInterface},
types::RedisConfig,
types::{ReconnectPolicy, RedisConfig},
};
use futures::{
channel::oneshot,
@@ -225,9 +225,9 @@ async fn listener(
.unwrap_or(REDIS_URI.to_string());
let redis_config = RedisConfig::from_url(&url).unwrap();
let subscriber = match report_internal_error!(
fred::types::Builder::from_config(redis_config).build_subscriber_client()
) {
let mut builder = fred::types::Builder::from_config(redis_config);
builder.set_policy(ReconnectPolicy::new_exponential(8, 100, 30_000, 2));
let subscriber = match report_internal_error!(builder.build_subscriber_client()) {
Ok(subscriber) => subscriber,
Err(_) => return,
};
@@ -236,16 +236,21 @@ async fn listener(
return;
}
// Let Fred automatically re-subscribe to tracked channels on reconnect.
subscriber.manage_subscriptions();
// Handle Redis connection dropping
let (clean_up_s, clean_up_r) = async_channel::bounded(1);
let clean_up_s = Arc::new(Mutex::new(clean_up_s));
subscriber.on_error(move |err| {
warn!("Redis subscriber error: {:?}", err);
if let RedisErrorKind::Canceled = err.kind() {
let clean_up_s = clean_up_s.clone();
spawn(async move {
clean_up_s.lock().await.send(()).await.ok();
});
}
// Transient errors (IO, timeout) are handled by the reconnect policy.
Ok(())
});
@@ -522,4 +527,4 @@ async fn worker(
}
}
}
}
}

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-coalesced"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "MIT"
authors = ["Paul Makles <me@insrt.uk>", "Zomatree <me@zomatree.live>"]
@@ -15,12 +15,12 @@ cache = ["dep:lru"]
default = ["tokio"]
[dependencies]
tokio = { version = "1.47.0", features = ["sync"], optional = true }
indexmap = { version = "2.13.0", optional = true }
lru = { version = "0.16.3", optional = true }
tokio = { workspace = true, features = ["sync"], optional = true }
indexmap = { workspace = true, optional = true }
lru = { workspace = true, optional = true }
[dev-dependencies]
tokio = { version = "1.47.0", features = [
tokio = { workspace = true, features = [
"rt",
"rt-multi-thread",
"macros",

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-config"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "MIT"
authors = ["Paul Makles <me@insrt.uk>"]
@@ -18,24 +18,24 @@ default = ["test", "sentry"]
[dependencies]
# Utility
config = "0.13.3"
cached = "0.44.0"
once_cell = "1.18.0"
config = { workspace = true }
cached = { workspace = true }
once_cell = { workspace = true }
# Serde
serde = { version = "1", features = ["derive"] }
serde = { workspace = true }
# Async
futures-locks = "0.7.1"
async-std = { version = "1.8.0", features = ["attributes"], optional = true }
futures-locks = { workspace = true }
async-std = { workspace = true, features = ["attributes"], optional = true }
# Logging
log = "0.4.14"
pretty_env_logger = "0.4.0"
log = { workspace = true }
pretty_env_logger = { workspace = true }
# Sentry
sentry = { version = "0.31.5", optional = true }
sentry-anyhow = { version = "0.38.1", optional = true }
sentry = { workspace = true, optional = true }
sentry-anyhow = { workspace = true, optional = true }
# Core
revolt-result = { version = "0.11.5", path = "../result", optional = true }
revolt-result = { workspace = true, optional = true }

View File

@@ -78,6 +78,8 @@ call_ring_duration = 30
[api.livekit.nodes]
[api.users]
# Minimum allowed length of usernames
min_username_length = 2
[pushd]
# this changes the names of the queues to not overlap
@@ -130,6 +132,8 @@ pkcs8 = ""
key_id = ""
team_id = ""
[january]
blocked_domains = []
[files]
# Encryption key for stored files
@@ -313,6 +317,12 @@ emojis = 500_000
# default: 5
process_message_delay_limit = 5
[features.legal_links]
# URLs for legal documents
terms_of_service = ""
privacy_policy = ""
guidelines = ""
[sentry]
# Configuration for Sentry error reporting
api = ""

View File

@@ -231,6 +231,7 @@ pub struct LiveKitNode {
#[derive(Deserialize, Debug, Clone)]
pub struct ApiUsers {
pub early_adopter_cutoff: Option<u64>,
pub min_username_length: usize,
}
#[derive(Deserialize, Debug, Clone)]
@@ -301,6 +302,11 @@ impl Pushd {
}
}
#[derive(Deserialize, Debug, Clone)]
pub struct January {
pub blocked_domains: Vec<String>,
}
#[derive(Deserialize, Debug, Clone)]
pub struct FilesLimit {
pub min_file_size: usize,
@@ -376,6 +382,16 @@ pub struct FeaturesLimitsCollection {
pub roles: HashMap<String, FeaturesLimits>,
}
#[derive(Deserialize, Debug, Clone)]
pub struct LegalLinks {
/// Terms of Service URL
pub terms_of_service: String,
/// Privacy Policy URL
pub privacy_policy: String,
/// Guidelines URL
pub guidelines: String,
}
#[derive(Deserialize, Debug, Clone)]
pub struct FeaturesAdvanced {
#[serde(default)]
@@ -393,6 +409,7 @@ impl Default for FeaturesAdvanced {
#[derive(Deserialize, Debug, Clone)]
pub struct Features {
pub limits: FeaturesLimitsCollection,
pub legal_links: LegalLinks,
pub webhooks_enabled: bool,
pub mass_mentions_send_notifications: bool,
pub mass_mentions_enabled: bool,
@@ -420,6 +437,7 @@ pub struct Settings {
pub hosts: Hosts,
pub api: Api,
pub pushd: Pushd,
pub january: January,
pub files: Files,
pub features: Features,
pub sentry: Sentry,

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-database"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "AGPL-3.0-or-later"
authors = ["Paul Makles <me@insrt.uk>"]
@@ -32,80 +32,71 @@ default = ["mongodb", "async-std-runtime", "tasks"]
[dependencies]
# Core
revolt-config = { version = "0.11.5", path = "../config", features = [
"report-macros",
] }
revolt-result = { version = "0.11.5", path = "../result" }
revolt-models = { version = "0.11.5", path = "../models", features = [
"validator",
] }
revolt-presence = { version = "0.11.5", path = "../presence" }
revolt-permissions = { version = "0.11.5", path = "../permissions", features = [
"serde",
"bson",
] }
revolt-parser = { version = "0.11.5", path = "../parser" }
revolt-config = { workspace = true, features = ["report-macros"] }
revolt-result = { workspace = true }
revolt-models = { workspace = true, features = ["validator"] }
revolt-presence = { workspace = true }
revolt-permissions = { workspace = true, features = ["serde", "bson"] }
revolt-parser = { workspace = true }
# Utility
log = "0.4"
lru = "0.11.0"
rand = "0.8.5"
ulid = "1.0.0"
nanoid = "0.4.0"
base64 = "0.21.3"
once_cell = "1.17"
indexmap = "1.9.1"
decancer = "1.6.2"
deadqueue = "0.2.4"
linkify = { optional = true, version = "0.8.1" }
url-escape = { optional = true, version = "0.1.1" }
validator = { version = "0.16", features = ["derive"] }
isahc = { optional = true, version = "1.7", features = ["json"] }
log = { workspace = true }
lru = { workspace = true }
rand = { workspace = true }
ulid = { workspace = true }
nanoid = { workspace = true }
base64 = { workspace = true }
once_cell = { workspace = true }
indexmap = { workspace = true }
decancer = { workspace = true }
deadqueue = { workspace = true }
linkify = { workspace = true, optional = true }
url-escape = { workspace = true, optional = true }
validator = { workspace = true, features = ["derive"] }
isahc = { workspace = true, features = ["json"], optional = true }
# Serialisation
serde_json = "1"
revolt_optional_struct = "0.2.0"
serde = { version = "1", features = ["derive"] }
iso8601-timestamp = { version = "0.2.10", features = ["serde", "bson"] }
serde_json = { workspace = true }
revolt_optional_struct = { workspace = true }
serde = { workspace = true }
iso8601-timestamp = { workspace = true, features = ["serde", "bson"] }
# Events
redis-kiss = { version = "0.1.4" }
redis-kiss = { workspace = true }
# Database
bson = { optional = true, version = "2.1.0" }
mongodb = { optional = true, version = "3.1.0" }
bson = { workspace = true, optional = true }
mongodb = { workspace = true, optional = true }
# Database Migration
unicode-segmentation = "1.10.1"
regex = "1"
unicode-segmentation = { workspace = true }
regex = { workspace = true }
# Async Language Features
futures = "0.3.19"
async-lock = "2.8.0"
async-trait = "0.1.51"
async-recursion = "1.0.4"
futures = { workspace = true }
async-lock = { workspace = true }
async-trait = { workspace = true }
async-recursion = { workspace = true }
# Async
async-std = { version = "1.8.0", features = ["attributes"], optional = true }
async-std = { workspace = true, features = ["attributes"], optional = true }
# Axum Impl
axum = { version = "0.7.5", optional = true }
axum = { workspace = true, optional = true }
# Rocket Impl
schemars = { version = "0.8.8", optional = true }
rocket = { version = "0.5.1", default-features = false, features = [
"json",
], optional = true }
revolt_okapi = { version = "0.9.1", optional = true }
revolt_rocket_okapi = { version = "0.10.0", optional = true }
schemars = { workspace = true, optional = true }
rocket = { workspace = true, features = ["json"], optional = true }
revolt_okapi = { workspace = true, optional = true }
revolt_rocket_okapi = { workspace = true, optional = true }
# Authifier
authifier = { version = "1.0.16" }
authifier = { workspace = true }
# RabbitMQ
amqprs = { version = "1.7.0" }
amqprs = { workspace = true }
# Voice
livekit-api = { version = "0.4.4", optional = true }
livekit-protocol = { version = "0.4.0", optional = true }
livekit-runtime = { version = "0.3.1", features = ["tokio"], optional = true }
livekit-api = { workspace = true, optional = true }
livekit-protocol = { workspace = true, optional = true }
livekit-runtime = { workspace = true, features = ["tokio"], optional = true }

View File

@@ -46,8 +46,7 @@ auto_derived!(
width: isize,
height: isize,
thumbhash: Option<Vec<u8>>,
#[serde(default)]
animated: bool,
animated: Option<bool>,
},
/// File is a video with specific dimensions
Video { width: isize, height: isize },

View File

@@ -17,6 +17,12 @@ pub trait AbstractAttachmentHashes: Sync + Send {
/// Update an attachment hash nonce value.
async fn set_attachment_hash_nonce(&self, hash: &str, nonce: &str) -> Result<()>;
/// Updates the attachments animated metadata value.
///
/// The primary use for this is to update the metadata for existing uploaded files, this
/// can only be used for images.
async fn set_attachment_hash_animated(&self, hash: &str, animated: bool) -> Result<()>;
/// Delete attachment hash by id.
async fn delete_attachment_hash(&self, id: &str) -> Result<()>;
}

View File

@@ -48,6 +48,29 @@ impl AbstractAttachmentHashes for MongoDb {
.map_err(|_| create_database_error!("update_one", COL))
}
/// Updates the attachments animated metadata value.
///
/// The primary use for this is to update the metadata for existing uploaded files, this
/// can only be used for images.
async fn set_attachment_hash_animated(&self, hash: &str, animated: bool) -> Result<()> {
self.col::<FileHash>(COL)
.update_one(
doc! {
"_id": hash,
"metadata.type": "Image",
"metadata.animated": { "$exists": false },
},
doc! {
"$set": {
"metadata.animated": animated
}
},
)
.await
.map(|_| ())
.map_err(|_| create_database_error!("update_one", COL))
}
/// Delete attachment hash by id.
async fn delete_attachment_hash(&self, id: &str) -> Result<()> {
query!(self, delete_one_by_id, COL, id).map(|_| ())

View File

@@ -1,7 +1,6 @@
use revolt_result::Result;
use crate::FileHash;
use crate::ReferenceDb;
use crate::{FileHash, Metadata, ReferenceDb};
use super::AbstractAttachmentHashes;
@@ -39,6 +38,29 @@ impl AbstractAttachmentHashes for ReferenceDb {
}
}
/// Updates the attachments animated metadata value.
///
/// The primary use for this is to update the metadata for existing uploaded files, this
/// can only be used for images.
async fn set_attachment_hash_animated(&self, hash: &str, animated: bool) -> Result<()> {
let mut hashes = self.file_hashes.lock().await;
if let Some(FileHash {
metadata:
Metadata::Image {
animated: Some(animated_metadata),
..
},
..
}) = hashes.get_mut(hash)
{
*animated_metadata = animated;
Ok(())
} else {
Err(create_error!(NotFound))
}
}
/// Delete attachment hash by id.
async fn delete_attachment_hash(&self, id: &str) -> Result<()> {
let mut file_hashes = self.file_hashes.lock().await;

View File

@@ -70,6 +70,7 @@ auto_derived!(
LegacyGroupIcon,
ChannelIcon,
ServerIcon,
RoleIcon,
}
/// Information about what the file was used for
@@ -239,4 +240,23 @@ impl File {
)
.await
}
/// Use a file for a role icon
pub async fn use_role_icon(
db: &Database,
id: &str,
parent: &str,
uploader_id: &str,
) -> Result<File> {
db.find_and_use_attachment(
id,
"icons",
FileUsedFor {
id: parent.to_owned(),
object_type: FileUsedForType::RoleIcon,
},
uploader_id.to_owned(),
)
.await
}
}

View File

@@ -705,7 +705,7 @@ impl Message {
Some(
PushNotification::from(
self.clone().into_model(user, member),
Some(author),
Some(author.clone()),
channel.to_owned().into(),
)
.await,
@@ -713,7 +713,11 @@ impl Message {
self.clone(),
match channel {
Channel::DirectMessage { recipients, .. }
| Channel::Group { recipients, .. } => recipients.clone(),
| Channel::Group { recipients, .. } => recipients
.iter()
.filter(|uid| *uid != author.id())
.cloned()
.collect(),
Channel::TextChannel { .. } => {
self.mentions.clone().unwrap_or_default()
}

View File

@@ -249,7 +249,7 @@ impl AbstractMessages for ReferenceDb {
let mut messages = self.messages.lock().await;
if let Some(message) = messages.get_mut(id) {
if let Some(users) = message.reactions.get_mut(emoji) {
users.remove(&user.to_string());
users.swap_remove(&user.to_string());
}
Ok(())
@@ -262,7 +262,7 @@ impl AbstractMessages for ReferenceDb {
async fn clear_reaction(&self, id: &str, emoji: &str) -> Result<()> {
let mut messages = self.messages.lock().await;
if let Some(message) = messages.get_mut(id) {
message.reactions.remove(emoji);
message.reactions.swap_remove(emoji);
Ok(())
} else {
Err(create_error!(NotFound))

View File

@@ -86,6 +86,9 @@ auto_derived_partial!(
/// Ranking of this role
#[serde(default)]
pub rank: i64,
/// Custom icon attachment
#[serde(skip_serializing_if = "Option::is_none")]
pub icon: Option<File>,
},
"PartialRole"
);
@@ -129,6 +132,7 @@ auto_derived!(
/// Optional fields on server object
pub enum FieldsRole {
Colour,
Icon,
}
);
@@ -305,6 +309,7 @@ impl Role {
colour: self.colour,
hoist: Some(self.hoist),
rank: Some(self.rank),
icon: self.icon,
}
}
@@ -318,6 +323,7 @@ impl Role {
colour: None,
hoist: false,
permissions: Default::default(),
icon: None,
};
db.insert_role(&server.id, &role).await?;
@@ -367,6 +373,7 @@ impl Role {
pub fn remove_field(&mut self, field: &FieldsRole) {
match field {
FieldsRole::Colour => self.colour = None,
FieldsRole::Icon => self.icon = None,
}
}

View File

@@ -77,7 +77,7 @@ impl AbstractServers for MongoDb {
},
doc! {
"$set": {
"roles.".to_owned() + &role.id: to_document(role)
"roles.".to_owned() + role.id.as_str(): to_document(role)
.map_err(|_| create_database_error!("to_document", "role"))?
}
},
@@ -172,6 +172,7 @@ impl IntoDocumentPath for FieldsRole {
fn as_path(&self) -> Option<&'static str> {
Some(match self {
FieldsRole::Colour => "colour",
FieldsRole::Icon => "icon",
})
}
}

View File

@@ -7,6 +7,7 @@ use futures::future::join_all;
use iso8601_timestamp::Timestamp;
use once_cell::sync::Lazy;
use rand::seq::SliceRandom;
use regex::{Regex, RegexBuilder};
use revolt_config::{config, FeaturesLimits};
use revolt_models::v0::{self, UserBadges, UserFlags};
use revolt_presence::filter_online;
@@ -163,6 +164,13 @@ pub static DISCRIMINATOR_SEARCH_SPACE: Lazy<HashSet<String>> = Lazy::new(|| {
set.into_iter().collect()
});
static BLOCKED_USERNAME_PATTERNS: Lazy<Regex> = Lazy::new(|| {
RegexBuilder::new("`{3}|(discord|rvlt|guilded|stt)\\.gg|(revolt|stoat)\\.chat|https?:\\/\\/")
.case_insensitive(true)
.build()
.unwrap()
});
#[allow(clippy::derivable_impls)]
impl Default for User {
fn default() -> Self {
@@ -198,11 +206,13 @@ impl User {
I: Into<Option<String>>,
D: Into<Option<PartialUser>>,
{
let username = User::validate_username(username)?;
let new_username = User::sanitise_username(&username).await?;
User::validate_username(&new_username)?;
let mut user = User {
id: account_id.into().unwrap_or_else(|| Ulid::new().to_string()),
discriminator: User::find_discriminator(db, &username, None).await?,
username,
discriminator: User::find_discriminator(db, &new_username, None).await?,
username: new_username.clone(),
last_acknowledged_policy_change: Timestamp::now_utc(),
..Default::default()
};
@@ -278,39 +288,40 @@ impl User {
}
}
/// Sanitise and validate a username can be used
pub fn validate_username(username: String) -> Result<String> {
// Copy the username for validation
/// Validate a username
///
/// This will check if the username is a blocked name or contains a blocked pattern.
fn validate_username(username: &str) -> Result<()> {
let username_lowercase = username.to_lowercase();
// Block homoglyphs
if decancer::cure(&username_lowercase).into_str() != username_lowercase {
const BLOCKED_USERNAMES: &[&str] = &["admin", "revolt", "stoat"];
if BLOCKED_USERNAMES.contains(&username_lowercase.as_str())
|| BLOCKED_USERNAME_PATTERNS.is_match(username)
{
return Err(create_error!(InvalidUsername));
}
// Ensure the username itself isn't blocked
const BLOCKED_USERNAMES: &[&str] = &["admin", "revolt"];
Ok(())
}
for username in BLOCKED_USERNAMES {
if username_lowercase == *username {
return Err(create_error!(InvalidUsername));
}
}
/// Sanitise a username
///
/// This will clean up Unicode homoglyphs and pad to the min username length with underscores.
async fn sanitise_username(username: &str) -> Result<String> {
let options = decancer::Options::default().retain_capitalization();
let mut username = decancer::cure(username, options)
.map_err(|_| create_error!(InvalidUsername))?
.to_string();
// Ensure none of the following substrings show up in the username
const BLOCKED_SUBSTRINGS: &[&str] = &[
"```",
"discord.gg",
"rvlt.gg",
"guilded.gg",
"https://",
"http://",
];
for substr in BLOCKED_SUBSTRINGS {
if username_lowercase.contains(substr) {
return Err(create_error!(InvalidUsername));
}
let config = revolt_config::config().await;
let username_length_diff = config
.api
.users
.min_username_length
.saturating_sub(username.len());
if username_length_diff > 0 {
username.push_str(&"_".repeat(username_length_diff))
}
Ok(username)
@@ -416,12 +427,14 @@ impl User {
/// Update a user's username
pub async fn update_username(&mut self, db: &Database, username: String) -> Result<()> {
let username = User::validate_username(username)?;
if self.username.to_lowercase() == username.to_lowercase() {
let new_username = User::sanitise_username(&username).await?;
User::validate_username(&new_username)?;
if self.username.to_lowercase() == new_username.to_lowercase() {
self.update(
db,
PartialUser {
username: Some(username),
username: Some(new_username),
..Default::default()
},
vec![],
@@ -434,12 +447,12 @@ impl User {
discriminator: Some(
User::find_discriminator(
db,
&username,
&new_username,
Some((self.discriminator.to_string(), self.id.clone())),
)
.await?,
),
username: Some(username),
username: Some(new_username),
..Default::default()
},
vec![],
@@ -825,3 +838,109 @@ impl User {
badges
}
}
#[cfg(test)]
mod tests {
use crate::User;
#[test]
fn username_validation_blocked_names() {
let username_admin = "Admin";
let username_revolt = "Revolt";
let username_stoat = "Stoat";
let username_allowed = "Allowed";
assert!(User::validate_username(username_admin).is_err());
assert!(User::validate_username(username_revolt).is_err());
assert!(User::validate_username(username_stoat).is_err());
assert!(User::validate_username(username_allowed).is_ok());
}
#[test]
fn username_validation_blocked_patterns() {
let username_grave = "```_test";
let username_discord = "discord.gg_test";
let username_rvlt = "rvlt.gg_test";
let username_guilded = "guilded.gg_test";
let username_stt = "stt.gg_test";
let username_revolt = "revolt.chat_test";
let username_stoat = "stoat.chat_test";
let username_http = "http://_test";
let username_https = "https://_test";
assert!(User::validate_username(username_grave).is_err());
assert!(User::validate_username(username_discord).is_err());
assert!(User::validate_username(username_rvlt).is_err());
assert!(User::validate_username(username_guilded).is_err());
assert!(User::validate_username(username_stt).is_err());
assert!(User::validate_username(username_revolt).is_err());
assert!(User::validate_username(username_stoat).is_err());
assert!(User::validate_username(username_http).is_err());
assert!(User::validate_username(username_https).is_err());
}
#[async_std::test]
async fn username_sanitisation_clean() {
let username_clean = "Test";
let username_clean_sanitised = User::sanitise_username(username_clean).await;
assert!(username_clean_sanitised.is_ok());
assert_eq!(username_clean, username_clean_sanitised.unwrap());
}
#[async_std::test]
async fn username_sanitisation_homoglyphs() {
let username_homoglyphs = "𝔽𝕌Ňℕy";
let username_homoglyphs_sanitised =
User::sanitise_username(username_homoglyphs).await.unwrap();
assert_ne!(username_homoglyphs, username_homoglyphs_sanitised);
assert_eq!("funny", username_homoglyphs_sanitised);
}
#[async_std::test]
async fn username_sanitisation_padding() {
let username_padding = "a";
let username = User::sanitise_username(username_padding).await.unwrap();
assert_eq!("a_", username);
}
#[async_std::test]
async fn create_user() {
use revolt_result::Result;
database_test!(|db| async move {
let mut created_clean = User::create(&db, "Test".to_string(), None, None)
.await
.unwrap();
assert_eq!("Test", created_clean.username);
created_clean
.update_username(&db, "Test2".to_string())
.await
.unwrap();
assert_eq!("Test2", created_clean.username);
let created_invalid_result: Result<_> =
User::create(&db, "stoat.chat".to_string(), None, None).await;
assert!(created_invalid_result.is_err());
let mut updated_invalid = User::create(&db, "Test".to_string(), None, None)
.await
.unwrap();
let updated_invalid_update_result = updated_invalid
.update_username(&db, "http://test".to_string())
.await;
assert!(updated_invalid_update_result.is_err());
});
}
}

View File

@@ -190,7 +190,7 @@ impl From<crate::Channel> for Channel {
role_permissions,
nsfw,
voice,
slowmode
slowmode,
} => Channel::TextChannel {
id,
server,
@@ -202,7 +202,7 @@ impl From<crate::Channel> for Channel {
role_permissions,
nsfw,
voice: voice.map(|voice| voice.into()),
slowmode
slowmode,
},
}
}
@@ -256,7 +256,7 @@ impl From<Channel> for crate::Channel {
role_permissions,
nsfw,
voice,
slowmode
slowmode,
} => crate::Channel::TextChannel {
id,
server,
@@ -268,7 +268,7 @@ impl From<Channel> for crate::Channel {
role_permissions,
nsfw,
voice: voice.map(|voice| voice.into()),
slowmode
slowmode,
},
}
}
@@ -307,7 +307,7 @@ impl From<PartialChannel> for crate::PartialChannel {
default_permissions: value.default_permissions,
last_message_id: value.last_message_id,
voice: value.voice.map(|voice| voice.into()),
slowmode: value.slowmode
slowmode: value.slowmode,
}
}
}
@@ -926,6 +926,7 @@ impl From<crate::Role> for Role {
colour: value.colour,
hoist: value.hoist,
rank: value.rank,
icon: value.icon.map(|f| f.into()),
}
}
}
@@ -939,6 +940,7 @@ impl From<Role> for crate::Role {
colour: value.colour,
hoist: value.hoist,
rank: value.rank,
icon: value.icon.map(|f| f.into()),
}
}
}
@@ -952,6 +954,7 @@ impl From<crate::PartialRole> for PartialRole {
colour: value.colour,
hoist: value.hoist,
rank: value.rank,
icon: value.icon.map(|f| f.into()),
}
}
}
@@ -965,6 +968,7 @@ impl From<PartialRole> for crate::PartialRole {
colour: value.colour,
hoist: value.hoist,
rank: value.rank,
icon: value.icon.map(|f| f.into()),
}
}
}
@@ -973,6 +977,7 @@ impl From<crate::FieldsRole> for FieldsRole {
fn from(value: crate::FieldsRole) -> Self {
match value {
crate::FieldsRole::Colour => FieldsRole::Colour,
crate::FieldsRole::Icon => FieldsRole::Icon,
}
}
}
@@ -981,6 +986,7 @@ impl From<FieldsRole> for crate::FieldsRole {
fn from(value: FieldsRole) -> Self {
match value {
FieldsRole::Colour => crate::FieldsRole::Colour,
FieldsRole::Icon => crate::FieldsRole::Icon,
}
}
}

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-files"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "AGPL-3.0-or-later"
authors = ["Paul Makles <me@insrt.uk>"]
@@ -16,33 +16,33 @@ tracing = { workspace = true }
tokio = { workspace = true }
async-trait = { workspace = true }
ffprobe = "0.4.0"
imagesize = "0.13.0"
tempfile = "3.12.0"
ffprobe = { workspace = true }
imagesize = { workspace = true }
tempfile = { workspace = true }
base64 = "0.22.1"
aes-gcm = "0.10.3"
typenum = "1.17.0"
base64 = { workspace = true }
aes-gcm = { workspace = true }
typenum = { workspace = true }
aws-config = "1.5.5"
aws-sdk-s3 = { version = "1.46.0", features = ["behavior-version-latest"] }
aws-config = { workspace = true }
aws-sdk-s3 = { workspace = true, features = ["behavior-version-latest"] }
revolt-config = { version = "0.11.5", path = "../config", features = [
revolt-config = { workspace = true, features = [
"report-macros",
] }
revolt-result = { version = "0.11.5", path = "../result" }
revolt-result = { workspace = true }
# image processing
jxl-oxide = { workspace = true }
jxl-oxide = { workspace = true, features = ["image"] }
image = { workspace = true }
# svg rendering
usvg = "0.44.0"
resvg = "0.44.0"
tiny-skia = "0.11.4"
usvg = { workspace = true }
resvg = { workspace = true }
tiny-skia = { workspace = true }
# encoding
webp = "0.3.0"
webp = { workspace = true }
[dev-dependencies]
uuid = { workspace = true }
uuid = { workspace = true, features = ["v4"] }

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-models"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "MIT"
authors = ["Paul Makles <me@insrt.uk>"]
@@ -21,26 +21,26 @@ default = ["serde", "partials", "rocket"]
[dependencies]
# Core
revolt-config = { version = "0.11.5", path = "../config" }
revolt-permissions = { version = "0.11.5", path = "../permissions" }
revolt-config = { workspace = true }
revolt-permissions = { workspace = true }
# Utility
regex = "1.11"
indexmap = "1.9.3"
once_cell = "1.17.1"
num_enum = "0.6.1"
regex = { workspace = true }
indexmap = { workspace = true }
once_cell = { workspace = true }
num_enum = { workspace = true }
# Rocket
rocket = { optional = true, version = "0.5.0-rc.2", default-features = false }
rocket = { workspace = true, optional = true }
# Serialisation
revolt_optional_struct = { version = "0.2.0", optional = true }
serde = { version = "1", features = ["derive"], optional = true }
iso8601-timestamp = { version = "0.2.11", features = ["schema", "bson"] }
revolt_optional_struct = { workspace = true, optional = true }
serde = { workspace = true, optional = true }
iso8601-timestamp = { workspace = true, features = ["schema", "bson"] }
# Spec Generation
schemars = { version = "0.8.8", optional = true, features = ["indexmap1"] }
utoipa = { version = "4.2.3", optional = true }
schemars = { workspace = true, features = ["indexmap2"], optional = true }
utoipa = { workspace = true, optional = true }
# Validation
validator = { version = "0.16.0", optional = true, features = ["derive"] }
validator = { workspace = true, features = ["derive"], optional = true }

View File

@@ -51,7 +51,7 @@ auto_derived!(
width: usize,
height: usize,
thumbhash: Option<Vec<u8>>,
animated: bool,
animated: Option<bool>,
},
/// File is a video with specific dimensions
Video { width: usize, height: usize },

View File

@@ -261,7 +261,7 @@ auto_derived!(
pub nonce: Option<String>,
/// Message content to send
#[cfg_attr(feature = "validator", validate(length(min = 0, max = 2000)))]
#[cfg_attr(feature = "validator", validate(length(min = 0)))]
pub content: Option<String>,
/// Attachments to include in message
pub attachments: Option<Vec<String>>,
@@ -345,7 +345,7 @@ auto_derived!(
#[cfg_attr(feature = "validator", derive(Validate))]
pub struct DataEditMessage {
/// New message content
#[cfg_attr(feature = "validator", validate(length(min = 1, max = 2000)))]
#[cfg_attr(feature = "validator", validate(length(min = 1)))]
pub content: Option<String>,
/// Embeds to include in the message
#[cfg_attr(feature = "validator", validate(length(min = 0, max = 10)))]
@@ -391,6 +391,7 @@ auto_derived!(
);
/// Message Author Abstraction
#[derive(Clone)]
pub enum MessageAuthor<'a> {
User(&'a User),
Webhook(&'a Webhook),

View File

@@ -106,6 +106,9 @@ auto_derived_partial!(
/// Ranking of this role
#[cfg_attr(feature = "serde", serde(default))]
pub rank: i64,
/// Role icon
#[cfg_attr(feature = "serde", serde(skip_serializing_if = "Option::is_none"))]
pub icon: Option<File>,
},
"PartialRole"
);
@@ -123,6 +126,7 @@ auto_derived!(
/// Optional fields on server object
pub enum FieldsRole {
Colour,
Icon,
}
/// Channel category
@@ -278,6 +282,11 @@ auto_derived!(
///
/// **Removed** - no effect, use the edit server role positions route
pub rank: Option<i64>,
/// Role icon
///
/// Provide an Autumn attachment Id.
#[cfg_attr(feature = "validator", validate(length(min = 1, max = 128)))]
pub icon: Option<String>,
/// Fields to remove from role object
#[cfg_attr(feature = "serde", serde(default))]
pub remove: Vec<FieldsRole>,

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-parser"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "MIT"
authors = ["Zomatree <me@zomatree.live>", "Paul Makles <me@insrt.uk>"]
@@ -8,4 +8,4 @@ description = "Revolt Backend: Message Parser"
repository = "https://github.com/stoatchat/stoatchat"
[dependencies]
logos = { version = "0.15" }
logos = { workspace = true }

View File

@@ -262,7 +262,7 @@ mod tests {
)
.collect::<Vec<_>>();
assert_eq!(output.len(), 6);
assert_eq!(output.len(), 7);
assert_eq!(output[0], MessageToken::CodeblockMarker(1));
assert_eq!(
output[1],

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-permissions"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "MIT"
authors = ["Paul Makles <me@insrt.uk>"]
@@ -18,23 +18,23 @@ try-from-primitive = ["dep:num_enum"]
[dev-dependencies]
# Async
async-std = { version = "1.8.0", features = ["attributes"] }
async-std = { workspace = true, features = ["attributes"] }
[dependencies]
# Core
revolt-result = { version = "0.11.5", path = "../result" }
revolt-result = { workspace = true }
# Utility
auto_ops = "0.3.0"
once_cell = "1.17"
num_enum = { version = "0.6.1", optional = true }
auto_ops = { workspace = true }
once_cell = { workspace = true }
num_enum = { workspace = true, optional = true }
# Async
async-trait = "0.1.51"
async-trait = { workspace = true }
# Serialisation
serde = { version = "1", features = ["derive"], optional = true }
bson = { version = "2.1.0", optional = true }
serde = { workspace = true, optional = true }
bson = { workspace = true, optional = true }
# Spec Generation
schemars = { version = "0.8.8", optional = true }
schemars = { workspace = true, optional = true }

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-presence"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "AGPL-3.0-or-later"
authors = ["Paul Makles <me@insrt.uk>"]
@@ -14,16 +14,16 @@ redis-is-patched = []
[dev-dependencies]
# Async
async-std = { version = "1.8.0", features = ["attributes"] }
async-std = { workspace = true, features = ["attributes"] }
# Config for loading Redis URI
revolt-config = { version = "0.11.5", path = "../config" }
revolt-config = { workspace = true }
[dependencies]
# Utility
log = "0.4.17"
rand = "0.8.5"
once_cell = "1.17.1"
log = { workspace = true }
rand = { workspace = true }
once_cell = { workspace = true }
# Redis
redis-kiss = "0.1.4"
redis-kiss = { workspace = true }

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-ratelimits"
version = "0.11.5"
version = "0.12.1"
edition = "2024"
license = "MIT"
authors = ["Zomatree <me@zomatree.live>", "Paul Makles <me@insrt.uk>"]
@@ -18,17 +18,17 @@ axum = ["dep:axum", "revolt-database/axum-impl"]
default = ["rocket", "axum"]
[dependencies]
revolt-database = { version = "0.11.5", path = "../database" }
revolt-result = { version = "0.11.5", path = "../result" }
revolt-config = { version = "0.11.5", path = "../config" }
revolt-database = { workspace = true }
revolt-result = { workspace = true }
revolt-config = { workspace = true }
rocket = { version = "0.5.1", optional = true }
revolt_rocket_okapi = { version = "0.10.0", optional = true }
rocket = { workspace = true, optional = true }
revolt_rocket_okapi = { workspace = true, optional = true }
axum = { version = "0.7.5", optional = true, features = ["macros"] }
axum = { workspace = true, optional = true, features = ["macros"] }
serde = { version = "1", features = ["derive"] }
authifier = { version = "1.0.16" }
dashmap = "5.2.0"
async-trait = "0.1.81"
log = "0.4"
serde = { workspace = true }
authifier = { workspace = true }
dashmap = { workspace = true }
async-trait = { workspace = true }
log = { workspace = true }

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-result"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "MIT"
authors = ["Paul Makles <me@insrt.uk>"]
@@ -22,21 +22,21 @@ default = ["serde", "sentry"]
[dependencies]
# Serialisation
serde_json = { version = "1", optional = true }
serde = { version = "1", features = ["derive"], optional = true }
serde_json = { workspace = true, optional = true }
serde = { workspace = true, optional = true }
# Spec Generation
schemars = { version = "0.8.8", optional = true }
utoipa = { version = "4.2.3", optional = true }
schemars = { workspace = true, optional = true }
utoipa = { workspace = true, optional = true }
# Rocket
rocket = { optional = true, version = "0.5.0-rc.2", default-features = false }
revolt_rocket_okapi = { version = "0.10.0", optional = true }
revolt_okapi = { version = "0.9.1", optional = true }
rocket = { workspace = true, optional = true }
revolt_rocket_okapi = { workspace = true, optional = true }
revolt_okapi = { workspace = true, optional = true }
# utilities
log = "0.4"
log = { workspace = true }
# Axum
axum = { version = "0.7.5", optional = true }
axum = { workspace = true, optional = true }
sentry = { version = "0.31.5", optional = true }
sentry = { workspace = true, optional = true }

View File

@@ -36,9 +36,7 @@ impl IntoResponse for Error {
ErrorType::NotInGroup => StatusCode::NOT_FOUND,
ErrorType::AlreadyPinned => StatusCode::BAD_REQUEST,
ErrorType::NotPinned => StatusCode::BAD_REQUEST,
ErrorType::InSlowmode {
retry_after: _,
} => StatusCode::TOO_MANY_REQUESTS,
ErrorType::InSlowmode { retry_after: _ } => StatusCode::TOO_MANY_REQUESTS,
ErrorType::CantCreateServers => StatusCode::FORBIDDEN,
ErrorType::UnknownServer => StatusCode::NOT_FOUND,
@@ -78,7 +76,7 @@ impl IntoResponse for Error {
ErrorType::DuplicateNonce => StatusCode::CONFLICT,
ErrorType::VosoUnavailable => StatusCode::BAD_REQUEST,
ErrorType::NotFound => StatusCode::NOT_FOUND,
ErrorType::NoEffect => StatusCode::OK,
ErrorType::NoEffect => StatusCode::BAD_REQUEST,
ErrorType::FailedValidation { .. } => StatusCode::BAD_REQUEST,
ErrorType::LiveKitUnavailable => StatusCode::BAD_REQUEST,
ErrorType::NotConnected => StatusCode::BAD_REQUEST,

View File

@@ -42,9 +42,7 @@ impl<'r> Responder<'r, 'static> for Error {
ErrorType::NotInGroup => Status::NotFound,
ErrorType::AlreadyPinned => Status::BadRequest,
ErrorType::NotPinned => Status::BadRequest,
ErrorType::InSlowmode {
retry_after: _,
} => Status::TooManyRequests,
ErrorType::InSlowmode { retry_after: _ } => Status::TooManyRequests,
ErrorType::InvalidFlagValue => Status::BadRequest,
ErrorType::CantCreateServers => Status::Forbidden,
@@ -84,7 +82,7 @@ impl<'r> Responder<'r, 'static> for Error {
ErrorType::NotAuthenticated => Status::Unauthorized,
ErrorType::DuplicateNonce => Status::Conflict,
ErrorType::NotFound => Status::NotFound,
ErrorType::NoEffect => Status::Ok,
ErrorType::NoEffect => Status::BadRequest,
ErrorType::FailedValidation { .. } => Status::BadRequest,
ErrorType::LiveKitUnavailable => Status::BadRequest,
ErrorType::NotAVoiceChannel => Status::BadRequest,

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-crond"
version = "0.11.5"
version = "0.12.1"
license = "AGPL-3.0-or-later"
authors = ["Paul Makles <me@insrt.uk>"]
edition = "2021"
@@ -11,13 +11,13 @@ publish = false
[dependencies]
# Utility
log = "0.4"
log = { workspace = true }
# Async
tokio = { version = "1" }
tokio = { workspace = true }
# Core
revolt-database = { version = "0.11.5", path = "../../core/database" }
revolt-result = { version = "0.11.5", path = "../../core/result" }
revolt-config = { version = "0.11.5", path = "../../core/config" }
revolt-files = { version = "0.11.5", path = "../../core/files" }
revolt-database = { workspace = true }
revolt-result = { workspace = true }
revolt-config = { workspace = true }
revolt-files = { workspace = true }

View File

@@ -1,47 +1,40 @@
[package]
name = "revolt-pushd"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "AGPL-3.0-or-later"
publish = false
[dependencies]
revolt-result = { version = "0.11.5", path = "../../core/result" }
revolt-config = { version = "0.11.5", path = "../../core/config", features = [
"report-macros",
"anyhow",
] }
revolt-database = { version = "0.11.5", path = "../../core/database" }
revolt-models = { version = "0.11.5", path = "../../core/models", features = [
"validator",
] }
revolt-presence = { version = "0.11.5", path = "../../core/presence", features = [
"redis-is-patched",
] }
revolt-parser = { version = "0.11.5", path = "../../core/parser" }
revolt-result = { workspace = true }
revolt-config = { workspace = true, features = ["report-macros", "anyhow"] }
revolt-database = { workspace = true }
revolt-models = { workspace = true, features = ["validator"] }
revolt-presence = { workspace = true, features = ["redis-is-patched"] }
revolt-parser = { workspace = true }
anyhow = { version = "1.0.98" }
anyhow = { workspace = true }
amqprs = { version = "1.7.0" }
fcm_v1 = "0.3.0"
web-push = "0.10.0"
isahc = { optional = true, version = "1.7", features = ["json"] }
revolt_a2 = { version = "0.10", default-features = false, features = ["ring"] }
redis-kiss = "0.1.4"
tokio = "1.39.2"
async-trait = "0.1.81"
ulid = "1.0.0"
amqprs = { workspace = true }
fcm_v1 = { workspace = true }
web-push = { workspace = true }
isahc = { workspace = true, features = ["json"], optional = true }
revolt_a2 = { workspace = true, features = ["ring"] }
redis-kiss = { workspace = true }
tokio = { workspace = true }
async-trait = { workspace = true }
ulid = { workspace = true }
authifier = "1.0.16"
authifier = { workspace = true }
log = "0.4.11"
pretty_env_logger = "0.4.0"
log = { workspace = true }
pretty_env_logger = { workspace = true }
regex = "1.12.3"
regex = { workspace = true }
#serialization
serde_json = "1"
revolt_optional_struct = "0.2.0"
serde = { version = "1", features = ["derive"] }
iso8601-timestamp = { version = "0.2.10", features = ["serde", "bson"] }
base64 = "0.22.1"
serde_json = { workspace = true }
revolt_optional_struct = { workspace = true }
serde = { workspace = true }
iso8601-timestamp = { workspace = true, features = ["serde", "bson"] }
base64 = { workspace = true }

View File

@@ -5,9 +5,8 @@ use amqprs::{channel::Channel as AmqpChannel, consumer::AsyncConsumer, BasicProp
use anyhow::{anyhow, bail, Result};
use async_trait::async_trait;
use fcm_v1::{
android::{AndroidConfig, AndroidMessagePriority},
auth::{Authenticator, ServiceAccountKey},
message::{Message, Notification},
message::Message,
Client, Error as FcmError,
};
use revolt_config::config;
@@ -15,6 +14,102 @@ use revolt_database::{events::rabbit::*, Database};
use revolt_models::v0::{Channel, PushNotification};
use serde_json::Value;
/// Custom notification data
#[derive(Debug, Clone, PartialEq)]
pub enum NotificationData {
FRReceived {
id: String,
username: String,
},
FRAccepted {
id: String,
username: String,
},
Generic {
title: String,
body: String,
image: Option<String>,
},
Message {
title: String,
body: String,
image: String,
tag: String,
},
DmCallStartEnd {
initiator_id: String,
channel_id: String,
started_at: String,
ended: bool,
duration: usize,
},
}
impl NotificationData {
pub fn get_type(&self) -> &str {
match self {
NotificationData::FRReceived { .. } => "push.fr.receive",
NotificationData::FRAccepted { .. } => "push.fr.accept",
NotificationData::Generic { .. } => "push.generic",
NotificationData::Message { .. } => "push.message",
NotificationData::DmCallStartEnd { .. } => "push.dm.call",
}
}
pub fn into_payload(self) -> HashMap<String, Value> {
let mut data = HashMap::new();
data.insert(
"type".to_string(),
Value::String(self.get_type().to_string()),
);
match self {
NotificationData::FRReceived { id, username } => {
data.insert("id".to_string(), Value::String(id));
data.insert("username".to_string(), Value::String(username));
}
NotificationData::FRAccepted { id, username } => {
data.insert("id".to_string(), Value::String(id));
data.insert("username".to_string(), Value::String(username));
}
NotificationData::Generic { title, body, image } => {
data.insert("title".to_string(), Value::String(title));
data.insert("body".to_string(), Value::String(body));
if let Some(image) = image {
data.insert("image".to_string(), Value::String(image));
}
}
NotificationData::Message {
title,
body,
image,
tag,
} => {
data.insert("title".to_string(), Value::String(title));
data.insert("body".to_string(), Value::String(body));
data.insert("image".to_string(), Value::String(image));
data.insert("tag".to_string(), Value::String(tag));
}
NotificationData::DmCallStartEnd {
initiator_id,
channel_id,
started_at,
ended,
duration,
} => {
data.insert("initiator_id".to_string(), Value::String(initiator_id));
data.insert("channel_id".to_string(), Value::String(channel_id));
data.insert("started_at".to_string(), Value::String(started_at));
data.insert("ended".to_string(), Value::Bool(ended));
data.insert("duration".to_string(), Value::Number(duration.into()));
}
}
data
}
}
pub struct FcmOutboundConsumer {
db: Database,
client: Client,
@@ -93,17 +188,14 @@ impl FcmOutboundConsumer {
.clone()
.ok_or_else(|| anyhow!("missing name"))?;
let mut data = HashMap::new();
data.insert(
"type".to_string(),
Value::String("push.fr.receive".to_string()),
);
data.insert("id".to_string(), Value::String(alert.from_user.id));
data.insert("username".to_string(), Value::String(name));
let data = NotificationData::FRReceived {
id: alert.from_user.id,
username: name,
};
let msg = Message {
token: Some(payload.token),
data: Some(data),
data: Some(data.into_payload()),
..Default::default()
};
@@ -121,30 +213,29 @@ impl FcmOutboundConsumer {
.clone()
.ok_or_else(|| anyhow!("missing name"))?;
let mut data: HashMap<String, Value> = HashMap::new();
data.insert(
"type".to_string(),
Value::String("push.fr.accept".to_string()),
);
data.insert("id".to_string(), Value::String(alert.accepted_user.id));
data.insert("username".to_string(), Value::String(name));
let data = NotificationData::FRAccepted {
id: alert.accepted_user.id,
username: name,
};
let msg = Message {
token: Some(payload.token),
data: Some(data),
data: Some(data.into_payload()),
..Default::default()
};
resp = self.client.send(&msg).await;
}
PayloadKind::Generic(alert) => {
let data = NotificationData::Generic {
title: alert.title,
body: alert.body,
image: alert.icon,
};
let msg = Message {
token: Some(payload.token),
notification: Some(Notification {
title: Some(alert.title),
body: Some(alert.body),
image: alert.icon,
}),
data: Some(data.into_payload()),
..Default::default()
};
@@ -152,19 +243,16 @@ impl FcmOutboundConsumer {
}
PayloadKind::MessageNotification(alert) => {
let title = self.format_title(&alert);
let data = NotificationData::Message {
title: self.format_title(&alert),
body: alert.body,
image: alert.icon,
tag: alert.tag,
};
let msg = Message {
token: Some(payload.token),
notification: Some(Notification {
title: Some(title),
body: Some(alert.body),
image: Some(alert.icon),
}),
android: Some(AndroidConfig {
collapse_key: Some(alert.tag),
..Default::default()
}),
data: Some(data.into_payload()),
..Default::default()
};
@@ -172,30 +260,17 @@ impl FcmOutboundConsumer {
}
PayloadKind::DmCallStartEnd(alert) => {
let mut data: HashMap<String, Value> = HashMap::new();
data.insert(
"initiator_id".to_string(),
Value::String(alert.initiator_id),
);
data.insert("channel_id".to_string(), Value::String(alert.channel_id));
data.insert(
"started_at".to_string(),
Value::String(alert.started_at.unwrap_or_else(|| "".to_string())),
);
data.insert("ended".to_string(), Value::Bool(alert.ended));
let data = NotificationData::DmCallStartEnd {
initiator_id: alert.initiator_id,
channel_id: alert.channel_id,
started_at: alert.started_at.unwrap_or_else(|| "".to_string()),
ended: alert.ended,
duration: config().await.api.livekit.call_ring_duration,
};
let msg = Message {
token: Some(payload.token),
notification: None,
data: Some(data),
android: Some(AndroidConfig {
priority: Some(AndroidMessagePriority::High),
ttl: Some(format!(
"{}s",
config().await.api.livekit.call_ring_duration
)),
..Default::default()
}),
data: Some(data.into_payload()),
..Default::default()
};

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-voice-ingress"
version = "0.11.5"
version = "0.12.1"
license = "AGPL-3.0-or-later"
edition = "2021"
publish = false
@@ -9,41 +9,41 @@ publish = false
[dependencies]
# util
log = "*"
sentry = "0.31.5"
lru = "0.7.6"
ulid = "0.5.0"
redis-kiss = "0.1.4"
chrono = "0.4.15"
log = { workspace = true }
sentry = { workspace = true }
lru = { workspace = true }
ulid = { workspace = true }
redis-kiss = { workspace = true }
chrono = { workspace = true }
# Serde
serde_json = "1.0.79"
rmp-serde = "1.0.0"
serde = "1.0.136"
serde_json = { workspace = true }
rmp-serde = { workspace = true }
serde = { workspace = true }
# Http
rocket = { version = "0.5.0-rc.2", features = ["json"] }
rocket_empty = "0.1.1"
rocket = { workspace = true, features = ["json"] }
rocket_empty = { workspace = true }
# Async
futures = "0.3.21"
async-std = { version = "1.8.0", features = [
futures = { workspace = true }
async-std = { workspace = true, features = [
"tokio1",
"tokio02",
"attributes",
] }
# Core
revolt-result = { path = "../../core/result" }
revolt-models = { path = "../../core/models" }
revolt-config = { path = "../../core/config" }
revolt-database = { path = "../../core/database", features = ["voice"] }
revolt-permissions = { path = "../../core/permissions" }
revolt-result = { workspace = true, features = ["rocket"] }
revolt-models = { workspace = true }
revolt-config = { workspace = true }
revolt-database = { workspace = true, features = ["voice"] }
revolt-permissions = { workspace = true }
# Voice
livekit-api = "0.4.4"
livekit-protocol = "0.4.0"
livekit-runtime = { version = "0.3.1", features = ["tokio"] }
livekit-api = { workspace = true }
livekit-protocol = { workspace = true }
livekit-runtime = { workspace = true, features = ["tokio"] }
# RabbitMQ
amqprs = { version = "1.7.0" }
amqprs = { workspace = true }

View File

@@ -1,6 +1,6 @@
[package]
name = "revolt-delta"
version = "0.11.5"
version = "0.12.1"
license = "AGPL-3.0-or-later"
authors = ["Paul Makles <paulmakles@gmail.com>"]
edition = "2018"
@@ -10,84 +10,83 @@ publish = false
[dependencies]
# Test
rand = "0.8.5"
redis-kiss = "0.1.4"
rand = { workspace = true }
redis-kiss = { workspace = true }
# Utility
lru = "0.7.0"
url = "2.2.2"
log = "0.4.11"
dashmap = "5.2.0"
linkify = "0.6.0"
once_cell = "1.17.1"
env_logger = "0.7.1"
lru = { workspace = true }
url = { workspace = true }
log = { workspace = true }
dashmap = { workspace = true }
linkify = { workspace = true }
once_cell = { workspace = true }
# Lang. Utilities
regex = "1"
num_enum = "0.5.1"
impl_ops = "0.1.1"
bitfield = "0.13.2"
regex = { workspace = true }
num_enum = { workspace = true }
impl_ops = { workspace = true }
bitfield = { workspace = true }
# ID / key generation
ulid = "0.4.1"
nanoid = "0.4.0"
ulid = { workspace = true }
nanoid = { workspace = true }
# serde
serde_json = "1.0.57"
serde = { version = "1.0.115", features = ["derive"] }
validator = { version = "0.16", features = ["derive"] }
iso8601-timestamp = { version = "0.2.11", features = [] }
serde_json = { workspace = true }
serde = { workspace = true }
validator = { workspace = true, features = ["derive"] }
iso8601-timestamp = { workspace = true }
# async
futures = "0.3.8"
chrono = "0.4.15"
async-channel = "1.6.1"
reqwest = { version = "0.11.4", features = ["json"] }
async-std = { version = "1.8.0", features = [
futures = { workspace = true }
chrono = { workspace = true }
async-channel = { workspace = true }
reqwest = { workspace = true, features = ["json"] }
async-std = { workspace = true, features = [
"tokio1",
"tokio02",
"attributes",
] }
# internal util
lettre = "0.10.0-alpha.4"
lettre = { workspace = true }
# web
rocket = { version = "0.5.1", default-features = false, features = ["json"] }
rocket_cors = { git = "https://github.com/lawliet89/rocket_cors", rev = "072d90359b23e9b291df6b672c07c93de9c46011" }
rocket_empty = { version = "0.1.1", features = ["schema"] }
rocket_authifier = { version = "1.0.16" }
rocket_prometheus = "0.10.0-rc.3"
rocket = { workspace = true, features = ["json"] }
rocket_cors = { workspace = true }
rocket_empty = { workspace = true, features = ["schema"] }
rocket_authifier = { workspace = true }
rocket_prometheus = { workspace = true }
# spec generation
schemars = "0.8.8"
revolt_rocket_okapi = { version = "0.10.0", features = ["swagger"] }
schemars = { workspace = true }
revolt_rocket_okapi = { workspace = true, features = ["swagger"] }
# rabbit
amqprs = { version = "1.7.0" }
amqprs = { workspace = true }
# core
authifier = "1.0.16"
revolt-config = { path = "../core/config" }
revolt-database = { path = "../core/database", features = [
authifier = { workspace = true }
revolt-config = { workspace = true }
revolt-database = { workspace = true, features = [
"rocket-impl",
"redis-is-patched",
"voice",
] }
revolt-models = { path = "../core/models", features = [
revolt-models = { workspace = true, features = [
"schemas",
"validator",
"rocket",
] }
revolt-presence = { path = "../core/presence" }
revolt-result = { path = "../core/result", features = ["rocket", "okapi"] }
revolt-permissions = { path = "../core/permissions", features = ["schemas"] }
revolt-ratelimits = { path = "../core/ratelimits", features = ["rocket"] }
revolt-presence = { workspace = true }
revolt-result = { workspace = true, features = ["rocket", "okapi"] }
revolt-permissions = { workspace = true, features = ["schemas"] }
revolt-ratelimits = { workspace = true, features = ["rocket"] }
# voice
livekit-api = "0.4.4"
livekit-protocol = "0.4.0"
livekit-api = { workspace = true }
livekit-protocol = { workspace = true }
[build-dependencies]
vergen = "7.5.0"
vergen = { workspace = true }

View File

@@ -1,4 +1,5 @@
use chrono::Utc;
use std::time::Duration;
use revolt_database::{
util::{permissions::DatabasePermissionQuery, reference::Reference},
Database, Message, User,
@@ -36,10 +37,9 @@ pub async fn bulk_delete_messages(
if ulid::Ulid::from_string(id)
.map_err(|_| create_error!(InvalidOperation))?
.datetime()
.signed_duration_since(Utc::now())
.num_days()
.abs()
> 7
.elapsed()
.expect("Time went backwards")
> Duration::from_hours(7 * 24) // 7 days
{
return Err(create_error!(InvalidOperation));
}

View File

@@ -1,4 +1,5 @@
use chrono::{Duration, Utc};
use std::time::Duration;
use redis_kiss::{get_connection, redis, AsyncCommands};
use revolt_database::util::permissions::DatabasePermissionQuery;
use revolt_database::{
@@ -111,8 +112,12 @@ pub async fn message_send(
// Disallow mentions for new users (TRUST-0: <12 hours age) in public servers
let allow_mentions = if let Some(server) = query.server_ref() {
if server.discoverable {
(Utc::now() - ulid::Ulid::from_string(&user.id).unwrap().datetime())
>= Duration::hours(12)
(ulid::Ulid::from_string(&user.id)
.unwrap()
.datetime()
.elapsed()
.expect("Time went backwards"))
>= Duration::from_hours(12)
} else {
true
}

View File

@@ -57,6 +57,8 @@ pub struct RevoltFeatures {
pub livekit: VoiceFeature,
/// Limits
pub limits: LimitsConfig,
/// Legal links
pub legal_links: LegalLinks,
}
/// # Limits For Users
@@ -70,6 +72,17 @@ pub struct LimitsConfig {
pub default: UserLimits,
}
/// # Legal links
#[derive(Serialize, JsonSchema, Debug)]
pub struct LegalLinks {
/// Terms of Service URL
pub terms_of_service: String,
/// Privacy Policy URL
pub privacy_policy: String,
/// Guidelines URL
pub guidelines: String,
}
/// # Global limits
#[derive(Serialize, JsonSchema, Debug)]
pub struct GlobalLimits {
@@ -92,6 +105,8 @@ pub struct GlobalLimits {
/// restrict server creation to these users.
/// if blank, all users can create servers
pub restrict_server_creation: Vec<String>,
/// New user hours
new_user_hours: i64,
}
/// # User Limits
@@ -231,10 +246,16 @@ pub async fn root() -> Result<Json<RevoltConfig>> {
.limits
.global
.restrict_server_creation,
new_user_hours: config.features.limits.global.new_user_hours as i64,
},
new_user: UserLimits::from_feature_limits(config.features.limits.new_user),
default: UserLimits::from_feature_limits(config.features.limits.default),
},
legal_links: LegalLinks {
terms_of_service: config.features.legal_links.terms_of_service,
privacy_policy: config.features.legal_links.privacy_policy,
guidelines: config.features.legal_links.guidelines,
},
},
ws: config.hosts.events,
app: config.hosts.app,

View File

@@ -1,7 +1,7 @@
use revolt_database::{
util::{permissions::DatabasePermissionQuery, reference::Reference},
voice::{sync_voice_permissions, VoiceClient},
Database, PartialRole, User
Database, File, PartialRole, User,
};
use revolt_models::v0;
use revolt_permissions::{calculate_server_permissions, ChannelPermission};
@@ -47,14 +47,27 @@ pub async fn edit(
name,
colour,
hoist,
icon,
remove,
..
} = data;
if remove.contains(&v0::FieldsRole::Icon) {
if let Some(existing_icon) = &role.icon {
db.mark_attachment_as_deleted(&existing_icon.id).await?;
}
}
let mut final_icon = None;
if let Some(icon_id) = icon {
final_icon = Some(File::use_role_icon(db, &icon_id, &role_id, &user.id).await?);
}
let partial = PartialRole {
name,
colour,
hoist,
icon: final_icon,
..Default::default()
};
@@ -69,8 +82,9 @@ pub async fn edit(
for channel_id in &server.channels {
let channel = Reference::from_unchecked(channel_id).as_channel(db).await?;
sync_voice_permissions(db, voice_client, &channel, Some(&server), Some(&role_id)).await?;
};
sync_voice_permissions(db, voice_client, &channel, Some(&server), Some(&role_id))
.await?;
}
Ok(Json(role.into()))
} else {

View File

@@ -1,68 +1,63 @@
[package]
name = "revolt-autumn"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "AGPL-3.0-or-later"
publish = false
[dependencies]
# ID generation
ulid = "1.1.3"
nanoid = "0.4.0"
ulid = { workspace = true }
nanoid = { workspace = true }
# Media processing
webp = "0.3.0"
sha2 = "0.10.8"
jxl-oxide = "0.8.1"
kamadak-exif = "0.5.4"
webp = { workspace = true }
sha2 = { workspace = true }
jxl-oxide = { workspace = true }
kamadak-exif = { workspace = true }
# revolt_little_exif = "0.5.1"
image = { version = "0.25.2" } # avif encode requires dav1d system library: features = ["avif-native"]
thumbhash = "0.1.0"
image = { workspace = true }
thumbhash = { workspace = true }
# File processing
revolt_clamav-client = { version = "0.1.5" }
simdutf8 = { version = "0.1.4", features = ["aarch64_neon"] }
revolt_clamav-client = { workspace = true }
simdutf8 = { workspace = true, features = ["aarch64_neon"] }
# Content type processing
infer = "0.16.0"
ffprobe = "0.4.0"
imagesize = "0.13.0"
infer = { workspace = true }
ffprobe = { workspace = true }
imagesize = { workspace = true }
# Utility
lazy_static = "1.5.0"
moka = { version = "0.12.8", features = ["future"] }
lazy_static = { workspace = true }
moka = { workspace = true, features = ["future"] }
# Serialisation
strum_macros = "0.26.4"
serde_json = "1.0.68"
serde = { version = "1.0", features = ["derive"] }
strum_macros = { workspace = true }
serde_json = { workspace = true }
serde = { workspace = true }
# Async runtime
tokio = { version = "1.0", features = ["full"] }
tokio = { workspace = true, features = ["full"] }
# Logging
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
# Core crates
revolt-files = { version = "0.11.5", path = "../../core/files" }
revolt-config = { version = "0.11.5", path = "../../core/config" }
revolt-database = { version = "0.11.5", path = "../../core/database", features = [
"axum-impl",
] }
revolt-result = { version = "0.11.5", path = "../../core/result", features = [
"utoipa",
"axum",
] }
revolt-ratelimits = { version = "0.11.5", path = "../../core/ratelimits", features = ["axum"] }
revolt-files = { workspace = true }
revolt-config = { workspace = true }
revolt-database = { workspace = true, features = ["axum-impl"] }
revolt-result = { workspace = true, features = ["utoipa", "axum"] }
revolt-ratelimits = { workspace = true, features = ["axum"] }
# Axum / web server
tempfile = "3.12.0"
axum-macros = "0.4.1"
axum_typed_multipart = "0.12.1"
axum = { version = "0.7.5", features = ["multipart"] }
tower-http = { version = "0.5.2", features = ["cors"] }
tempfile = { workspace = true }
axum-macros = { workspace = true }
axum_typed_multipart = { workspace = true }
axum = { workspace = true, features = ["multipart"] }
tower-http = { workspace = true, features = ["cors"] }
# OpenAPI & documentation generation
utoipa-scalar = { version = "0.1.0", features = ["axum"] }
utoipa = { version = "4.2.3", features = ["axum_extras", "ulid"] }
utoipa-scalar = { workspace = true, features = ["axum"] }
utoipa = { workspace = true, features = ["axum_extras", "ulid"] }

View File

@@ -1,5 +1,5 @@
use std::{
io::{Cursor, Read},
io::{Cursor, Read, Write},
time::Duration,
};
@@ -15,9 +15,10 @@ use lazy_static::lazy_static;
use revolt_config::{config, report_internal_error};
use revolt_database::{iso8601_timestamp::Timestamp, Database, FileHash, Metadata, User};
use revolt_files::{
create_thumbnail, decode_image, fetch_from_s3, upload_to_s3, AUTHENTICATION_TAG_SIZE_BYTES,
create_thumbnail, decode_image, fetch_from_s3, is_animated, upload_to_s3,
AUTHENTICATION_TAG_SIZE_BYTES,
};
use revolt_result::{create_error, Error, Result};
use revolt_result::{create_error, Error, Result, ToRevoltError};
use serde::{Deserialize, Serialize};
use sha2::Digest;
use tempfile::NamedTempFile;
@@ -25,7 +26,9 @@ use tokio::time::Instant;
use tower_http::cors::{AllowHeaders, Any, CorsLayer};
use utoipa::ToSchema;
use crate::{exif::strip_metadata, metadata::generate_metadata, mime_type::determine_mime_type, AppState};
use crate::{
exif::strip_metadata, metadata::generate_metadata, mime_type::determine_mime_type, AppState,
};
/// Build the API router
pub async fn router() -> Router<AppState> {
@@ -380,7 +383,30 @@ async fn fetch_preview(
let hash = file.as_hash(&db).await?;
let is_animated = matches!(hash.metadata, Metadata::Image { animated: true, .. });
let mut data = None;
// If animated is unset, check the file contents to see if it is animated and update the filehash
let is_animated = match &hash.metadata {
Metadata::Image {
animated: Some(value),
..
} => *value,
Metadata::Image { animated: None, .. } => {
let file_data = retrieve_file_by_hash(&hash).await?;
let mut named_file = NamedTempFile::new().to_internal_error()?;
named_file.write(&file_data).to_internal_error()?;
data = Some(file_data);
// If it fails for some reason, set it to not be animated
let animated = is_animated(&named_file, &hash.content_type).unwrap_or(false);
db.set_attachment_hash_animated(&hash.id, animated).await?;
animated
}
_ => false,
};
// Only process image files and don't process GIFs if not avatar or icon
if !matches!(hash.metadata, Metadata::Image { .. })
@@ -392,7 +418,11 @@ async fn fetch_preview(
}
// Original image data
let data = retrieve_file_by_hash(&hash).await?;
let data = if let Some(data) = data {
data
} else {
retrieve_file_by_hash(&hash).await?
};
// Read image and create thumbnail
let data = create_thumbnail(

View File

@@ -37,7 +37,7 @@ pub fn generate_metadata(f: &NamedTempFile, mime_type: &str) -> Metadata {
thumbhash::rgba_to_thumb_hash(width as usize, height as usize, &rgba)
})
.ok(),
animated: is_animated(f, mime_type).unwrap_or(false),
animated: is_animated(f, mime_type).or(Some(false)),
})
.unwrap_or_default()
} else if mime_type.starts_with("video/") {

View File

@@ -1,49 +1,40 @@
[package]
name = "revolt-gifbox"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "AGPL-3.0-or-later"
publish = false
[dependencies]
# Serialisation
serde = { version = "1.0", features = ["derive", "rc"] }
serde_json = "1.0.68"
serde = { workspace = true, features = ["rc"] }
serde_json = { workspace = true }
# Async runtime
tokio = { version = "1.0", features = ["full"] }
tokio = { workspace = true, features = [] }
# Web requests
reqwest = { version = "0.12", features = ["json"] }
reqwest = { workspace = true, features = ["json", "query"] }
# Core crates
revolt-config = { version = "0.11.5", path = "../../core/config" }
revolt-models = { version = "0.11.5", path = "../../core/models" }
revolt-result = { version = "0.11.5", path = "../../core/result", features = [
"utoipa",
"axum",
] }
revolt-coalesced = { version = "0.11.5", path = "../../core/coalesced", features = [
"queue",
] }
revolt-database = { version = "0.11.5", path = "../../core/database", features = [
"axum-impl",
] }
revolt-ratelimits = { version = "0.11.5", path = "../../core/ratelimits", features = [
"axum",
] }
revolt-config = { workspace = true }
revolt-models = { workspace = true }
revolt-result = { workspace = true, features = ["utoipa", "axum"] }
revolt-coalesced = { workspace = true, features = ["queue"] }
revolt-database = { workspace = true, features = ["axum-impl"] }
revolt-ratelimits = { workspace = true, features = ["axum"] }
# Axum / web server
axum = { version = "0.7.5" }
axum-extra = { version = "0.9", features = ["typed-header"] }
tower-http = { version = "0.5.2", features = ["cors"] }
axum = { workspace = true }
axum-extra = { workspace = true, features = ["typed-header"] }
tower-http = { workspace = true, features = ["cors"] }
# OpenAPI & documentation generation
utoipa-scalar = { version = "0.1.0", features = ["axum"] }
utoipa = { version = "4.2.3", features = ["axum_extras", "ulid"] }
utoipa-scalar = { workspace = true, features = ["axum"] }
utoipa = { workspace = true, features = ["axum_extras", "ulid"] }
# Logging
tracing = "0.1"
tracing = { workspace = true }
# Utils
lru_time_cache = "0.11.11"
lru_time_cache = { workspace = true }

View File

@@ -1,50 +1,49 @@
[package]
name = "revolt-january"
version = "0.11.5"
version = "0.12.1"
edition = "2021"
license = "AGPL-3.0-or-later"
publish = false
[dependencies]
# Utility
mime = "0.3.17"
regex = "1.11.0"
tempfile = "3.13.0"
lazy_static = "1.5.0"
moka = { version = "0.12.8", features = ["future"] }
mime = { workspace = true }
regex = { workspace = true }
tempfile = { workspace = true }
lazy_static = { workspace = true }
moka = { workspace = true, features = ["future"] }
# Web scraping
scraper = "0.20.0"
encoding_rs = "0.8.34"
scraper = { workspace = true }
encoding_rs = { workspace = true }
# Serialisation
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0.68"
serde = { workspace = true }
serde_json = { workspace = true }
# Async runtime
async-recursion = "1.1.1"
tokio = { version = "1.0", features = ["full"] }
async-recursion = { workspace = true }
tokio = { workspace = true, features = [] }
# Web requests
reqwest = { version = "0.12", features = ["json"] }
reqwest = { workspace = true, features = ["json"] }
pdk-ip-filter-lib = "1.8.0"
url = { workspace = true }
# Logging
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
tracing = { workspace = true }
tracing-subscriber = { workspace = true, features = ["env-filter"] }
# Core crates
revolt-config = { version = "0.11.5", path = "../../core/config" }
revolt-models = { version = "0.11.5", path = "../../core/models" }
revolt-result = { version = "0.11.5", path = "../../core/result", features = [
"utoipa",
"axum",
] }
revolt-files = { version = "0.11.5", path = "../../core/files" }
revolt-config = { workspace = true }
revolt-models = { workspace = true }
revolt-result = { workspace = true, features = ["utoipa", "axum"] }
revolt-files = { workspace = true }
# Axum / web server
axum = { version = "0.7.5" }
axum-extra = { version = "0.9", features = ["typed-header"] }
axum = { workspace = true }
axum-extra = { workspace = true, features = ["typed-header"] }
# OpenAPI & documentation generation
utoipa-scalar = { version = "0.1.0", features = ["axum"] }
utoipa = { version = "4.2.3", features = ["axum_extras", "ulid"] }
utoipa-scalar = { workspace = true, features = ["axum"] }
utoipa = { workspace = true, features = ["axum_extras", "ulid"] }

View File

@@ -1,42 +1,37 @@
use encoding_rs::{Encoding, UTF_8_INIT};
use lazy_static::lazy_static;
use mime::Mime;
use pdk_ip_filter_lib::IpFilter;
use regex::Regex;
use reqwest::{
header::{self, CONTENT_TYPE},
redirect, Client, Response,
};
use revolt_config::report_internal_error;
use revolt_config::{config, report_internal_error};
use revolt_files::{create_thumbnail, decode_image, image_size_vec, is_valid_image, video_size};
use revolt_models::v0::{Embed, Image, ImageSize, Video};
use revolt_result::{create_error, Error, Result};
use revolt_result::{create_error, Error, Result, ToRevoltError};
use std::{
io::{Cursor, Write},
str::FromStr,
time::Duration,
};
use url::{Host, Url};
lazy_static! {
/// Request client
static ref CLIENT: Client = reqwest::Client::builder()
.timeout(Duration::from_secs(10)) // TODO config
.connect_timeout(Duration::from_secs(5)) // TODO config
.redirect(redirect::Policy::custom(|attempt| {
if attempt.previous().len() > 5 { // TODO config
attempt.error("too many redirects")
} else if attempt.url().host_str() == Some("jan.revolt.chat") { // TODO config
attempt.stop()
} else {
attempt.follow()
}
}))
.redirect(redirect::Policy::none())
.build()
.expect("reqwest Client");
/// Spoof User Agent as Discord
static ref RE_USER_AGENT_SPOOFING_AS_DISCORD: Regex = Regex::new("^(?:(?:https?:)?//)?(?:(?:vx|fx)?twitter|(?:fixv|fixup)?x|(?:old\\.|new\\.|www\\.)reddit).com").expect("valid regex");
static ref RE_USER_AGENT_SPOOFING_AS_DISCORD: Regex = Regex::new("^(?:(?:vx|fx)?twitter|(?:fixv|fixup)?x|(?:old\\.|new\\.|www\\.)reddit).com").expect("valid regex");
/// Regex for matching new Reddit URLs
static ref RE_URL_NEW_REDDIT: Regex = Regex::new("^(?:(?:https?:)?//)?(?:(?:new\\.|www\\.)?reddit).com").expect("valid regex");
static ref RE_URL_NEW_REDDIT: Regex = Regex::new("^(?:(?:new\\.|www\\.)?reddit).com").expect("valid regex");
/// Cache for proxy results
static ref PROXY_CACHE: moka::future::Cache<String, Result<(String, Vec<u8>)>> = moka::future::Cache::builder()
@@ -59,6 +54,17 @@ lazy_static! {
.max_capacity(10_000) // Cache up to 10k embeds
.time_to_live(Duration::from_secs(60)) // For up to 1 minute
.build();
static ref IP_BLOCKLIST: IpFilter = IpFilter::block(&[
"10.0.0.0/8",
"192.168.0.0/16",
"127.0.0.0/8",
"172.16.0.0/12",
"169.254.0.0/16",
"::1",
"fc00::/7"
]
).unwrap();
}
/// Information about a successful request
@@ -73,7 +79,7 @@ impl Request {
if let Some(hit) = PROXY_CACHE.get(url).await {
hit
} else {
let Request { response, mime } = Request::new(url).await?;
let Request { response, mime } = Request::new_from_str(url).await?;
if matches!(mime.type_(), mime::IMAGE | mime::VIDEO) {
let bytes = report_internal_error!(response.bytes().await);
@@ -135,7 +141,7 @@ impl Request {
let request = if let Some(request) = request {
request
} else {
let request = Request::new(url).await?;
let request = Request::new_from_str(url).await?;
if matches!(request.mime.type_(), mime::IMAGE) {
request
} else {
@@ -173,7 +179,7 @@ impl Request {
let response = if let Some(Request { response, .. }) = request {
response
} else {
let Request { response, mime } = Request::new(url).await?;
let Request { response, mime } = Request::new_from_str(url).await?;
if matches!(mime.type_(), mime::VIDEO) {
response
} else {
@@ -212,7 +218,7 @@ impl Request {
if let Some(hit) = EMBED_CACHE.get(&url).await {
Ok(hit)
} else {
let request = Request::new(&url).await?;
let request = Request::new_from_str(&url).await?;
let embed = match (request.mime.type_(), request.mime.subtype()) {
(_, mime::HTML) => {
let content_type = request
@@ -255,15 +261,22 @@ impl Request {
}
/// Send a new request to a service
pub async fn new(url: &str) -> Result<Request> {
let response = CLIENT
pub async fn new(url: Url) -> Result<Request> {
let mut url = url;
let url_host_str = url.host_str().ok_or(create_error!(ProxyError))?.to_string();
Request::url_is_blacklisted(&url).await?;
let mut redirect_count = 0;
loop {
let response = CLIENT
.get(url)
.header(
"User-Agent",
if RE_USER_AGENT_SPOOFING_AS_DISCORD.is_match(url) {
if RE_USER_AGENT_SPOOFING_AS_DISCORD.is_match(&url_host_str) {
"Mozilla/5.0 (compatible; Discordbot/2.0; +https://discordapp.com)"
} else {
"Mozilla/5.0 (compatible; January/2.0; +https://github.com/revoltchat/backend)"
"Mozilla/5.0 (compatible; January/2.0; +https://github.com/stoatchat/stoatchat)"
},
)
.header("Accept-Language", "en-US,en;q=0.5")
@@ -271,31 +284,105 @@ impl Request {
.await
.map_err(|_| create_error!(ProxyError))?;
if !response.status().is_success() {
tracing::error!("{:?}", response);
return Err(create_error!(ProxyError));
if response.status().is_redirection() {
redirect_count += 1;
if redirect_count > 5 {
return Err(create_error!(ProxyError));
}
if let Some(location) = response.headers().get("location") {
let location = location.to_str().map_err(|_| create_error!(ProxyError))?;
url = Url::from_str(location).to_internal_error()?;
if !Request::url_is_blacklisted(&url).await? {
continue;
}
} else {
return Err(create_error!(ProxyError));
}
}
if !response.status().is_success() {
tracing::error!("{:?}", response);
return Err(create_error!(ProxyError));
}
let content_type = response
.headers()
.get(CONTENT_TYPE)
.ok_or(create_error!(ProxyError))?
.to_str()
.map_err(|_| create_error!(ProxyError))?;
let mime: mime::Mime = content_type
.parse()
.map_err(|_| create_error!(ProxyError))?;
return Ok(Request { response, mime });
}
}
let content_type = response
.headers()
.get(CONTENT_TYPE)
.ok_or(create_error!(ProxyError))?
.to_str()
.map_err(|_| create_error!(ProxyError))?;
let mime: mime::Mime = content_type
.parse()
.map_err(|_| create_error!(ProxyError))?;
Ok(Request { response, mime })
pub async fn new_from_str(url: &str) -> Result<Request> {
let proper_url = Url::parse(url).map_err(|_| create_error!(ProxyError))?;
Request::new(proper_url).await
}
/// Check if something exists
pub async fn exists(url: &str) -> bool {
pub async fn exists(url: Url) -> bool {
if let Ok(response) = CLIENT.head(url).send().await {
response.status().is_success()
} else {
false
}
}
pub async fn exists_from_str(url: &str) -> Result<bool> {
let proper_url = Url::parse(url).map_err(|_| create_error!(ProxyError))?;
Ok(Request::exists(proper_url).await)
}
pub async fn url_is_blacklisted(url: &Url) -> Result<bool> {
if let Some(host) = url.host() {
match host {
Host::Ipv4(ipv4) => {
let url_str = ipv4.to_string();
if !IP_BLOCKLIST.is_allowed(&url_str) {
return Err(create_error!(InvalidOperation));
}
}
Host::Domain(domain) => {
let mut domain = domain.to_string();
let config = config().await;
// First step: TLDs and blocked domains
if !domain.contains(".") // lazily block TLDs
|| config.january.blocked_domains.iter().any(|x| x == &domain)
{
return Err(create_error!(InvalidOperation));
}
if !domain.contains(":") {
domain += ":80";
}
// Second step: resolve the IP and check the blocklist
if let Ok(mut resolved_ip) = tokio::net::lookup_host(domain.clone()).await {
if let Some(resolved_ip) = resolved_ip.next() {
if !IP_BLOCKLIST.is_allowed(&resolved_ip.ip().to_string()) {
return Err(create_error!(InvalidOperation));
}
} else {
return Err(create_error!(InvalidOperation));
}
} else {
return Err(create_error!(ProxyError));
}
}
_ => (),
}
};
Ok(false)
}
}

View File

@@ -236,11 +236,12 @@ pub async fn populate_special(original_url: String, metadata: &mut WebsiteMetada
metadata.site_name.take();
// Verify the video exists
if !crate::requests::Request::exists(&format!(
if !crate::requests::Request::exists_from_str(&format!(
"http://img.youtube.com/vi/{}/sddefault.jpg",
id
))
.await
.unwrap_or(false)
{
return;
}

View File

@@ -4,6 +4,8 @@ sidebar_position: 1
# Endpoints
[![Run in Yaak](https://external.stoatusercontent.com/proxy?url=https://yaak.app/static/button.svg)](https://yaak.app/button/run?name=Stoat+API&url=https%3A%2F%2Fstoat.chat%2Fapi%2Fopenapi.json)
**We are moving stuff around currently following the rebrand, guidance will follow soon!**
You can connect to the API on the following URLs:

View File

@@ -23,7 +23,7 @@ You can contribute translations through [**Weblate**](https://translate.stoat.ch
### 3. Donate
Stoat is not backed by a big company, is not currently monetised (for example, via a subscription service) and does not serve you advertisements; as such, Stoat currently relies entirely on donations.
You can learn more about donating [here](https://wiki.revolt.chat/notes/project/financial-support/) - if you want to make a larger donation, please consult me first.
You can learn more about donating [here](https://ko-fi.com/stoatchat) - if you want to make a larger donation, please consult me first.
### 4. Join the project

View File

@@ -30,76 +30,21 @@
"path": "crates/core/config/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/core/config/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
},
{
"type": "toml",
"path": "crates/core/database/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/core/database/Cargo.toml",
"jsonpath": "$.dependencies['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/core/database/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
},
{
"type": "toml",
"path": "crates/core/database/Cargo.toml",
"jsonpath": "$.dependencies['revolt-models'].version"
},
{
"type": "toml",
"path": "crates/core/database/Cargo.toml",
"jsonpath": "$.dependencies['revolt-presence'].version"
},
{
"type": "toml",
"path": "crates/core/database/Cargo.toml",
"jsonpath": "$.dependencies['revolt-permissions'].version"
},
{
"type": "toml",
"path": "crates/core/database/Cargo.toml",
"jsonpath": "$.dependencies['revolt-parser'].version"
},
{
"type": "toml",
"path": "crates/core/files/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/core/files/Cargo.toml",
"jsonpath": "$.dependencies['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/core/files/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
},
{
"type": "toml",
"path": "crates/core/models/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/core/models/Cargo.toml",
"jsonpath": "$.dependencies['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/core/models/Cargo.toml",
"jsonpath": "$.dependencies['revolt-permissions'].version"
},
{
"type": "toml",
"path": "crates/core/parser/Cargo.toml",
@@ -110,41 +55,16 @@
"path": "crates/core/permissions/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/core/permissions/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
},
{
"type": "toml",
"path": "crates/core/presence/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/core/presence/Cargo.toml",
"jsonpath": "$['dev-dependencies']['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/core/ratelimits/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/core/ratelimits/Cargo.toml",
"jsonpath": "$.dependencies['revolt-database'].version"
},
{
"type": "toml",
"path": "crates/core/ratelimits/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
},
{
"type": "toml",
"path": "crates/core/ratelimits/Cargo.toml",
"jsonpath": "$.dependencies['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/core/result/Cargo.toml",
@@ -155,56 +75,11 @@
"path": "crates/daemons/crond/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/daemons/crond/Cargo.toml",
"jsonpath": "$.dependencies['revolt-database'].version"
},
{
"type": "toml",
"path": "crates/daemons/crond/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
},
{
"type": "toml",
"path": "crates/daemons/crond/Cargo.toml",
"jsonpath": "$.dependencies['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/daemons/crond/Cargo.toml",
"jsonpath": "$.dependencies['revolt-files'].version"
},
{
"type": "toml",
"path": "crates/daemons/pushd/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/daemons/pushd/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
},
{
"type": "toml",
"path": "crates/daemons/pushd/Cargo.toml",
"jsonpath": "$.dependencies['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/daemons/pushd/Cargo.toml",
"jsonpath": "$.dependencies['revolt-database'].version"
},
{
"type": "toml",
"path": "crates/daemons/pushd/Cargo.toml",
"jsonpath": "$.dependencies['revolt-models'].version"
},
{
"type": "toml",
"path": "crates/daemons/pushd/Cargo.toml",
"jsonpath": "$.dependencies['revolt-presence'].version"
},
{
"type": "toml",
"path": "crates/daemons/voice-ingress/Cargo.toml",
@@ -215,66 +90,11 @@
"path": "crates/services/autumn/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/services/autumn/Cargo.toml",
"jsonpath": "$.dependencies['revolt-files'].version"
},
{
"type": "toml",
"path": "crates/services/autumn/Cargo.toml",
"jsonpath": "$.dependencies['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/services/autumn/Cargo.toml",
"jsonpath": "$.dependencies['revolt-database'].version"
},
{
"type": "toml",
"path": "crates/services/autumn/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
},
{
"type": "toml",
"path": "crates/services/autumn/Cargo.toml",
"jsonpath": "$.dependencies['revolt-ratelimits'].version"
},
{
"type": "toml",
"path": "crates/services/gifbox/Cargo.toml",
"jsonpath": "$.package.version"
},
{
"type": "toml",
"path": "crates/services/gifbox/Cargo.toml",
"jsonpath": "$.dependencies['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/services/gifbox/Cargo.toml",
"jsonpath": "$.dependencies['revolt-models'].version"
},
{
"type": "toml",
"path": "crates/services/gifbox/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
},
{
"type": "toml",
"path": "crates/services/gifbox/Cargo.toml",
"jsonpath": "$.dependencies['revolt-coalesced'].version"
},
{
"type": "toml",
"path": "crates/services/gifbox/Cargo.toml",
"jsonpath": "$.dependencies['revolt-database'].version"
},
{
"type": "toml",
"path": "crates/services/gifbox/Cargo.toml",
"jsonpath": "$.dependencies['revolt-ratelimits'].version"
},
{
"type": "toml",
"path": "crates/services/january/Cargo.toml",
@@ -282,23 +102,53 @@
},
{
"type": "toml",
"path": "crates/services/january/Cargo.toml",
"jsonpath": "$.dependencies['revolt-config'].version"
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-coalesced'].version"
},
{
"type": "toml",
"path": "crates/services/january/Cargo.toml",
"jsonpath": "$.dependencies['revolt-models'].version"
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-config'].version"
},
{
"type": "toml",
"path": "crates/services/january/Cargo.toml",
"jsonpath": "$.dependencies['revolt-result'].version"
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-database'].version"
},
{
"type": "toml",
"path": "crates/services/january/Cargo.toml",
"jsonpath": "$.dependencies['revolt-files'].version"
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-files'].version"
},
{
"type": "toml",
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-models'].version"
},
{
"type": "toml",
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-parser'].version"
},
{
"type": "toml",
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-permissions'].version"
},
{
"type": "toml",
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-presence'].version"
},
{
"type": "toml",
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-ratelimits'].version"
},
{
"type": "toml",
"path": "Cargo.toml",
"jsonpath": "$.workspace.dependencies['revolt-result'].version"
}
]
}

View File

@@ -1 +1 @@
0.11.5
0.12.1