Commit began by changing the duplicated SITE_NAME conditional in the
header to use a `has_logo` param, much like the recent `has_sidebar`.
From there, a need to include site-specific assets in assetcache was
revealed, adding the `asset_siteimg(...)` macro. This was applied
quite broadly, though not comprehensively.
Also: change to docker-compose.yml is to see if it'll fix the Github
tests failing. Lumped in here because it's too dumb to make a commit
for by itself.
h/t to @official-techsupport for finding and help fixing this bug.
When given certain pathological input, `sanitize` would time out
(notably only on posts, rather than comments, perhaps due to the
longer maximum length of input). For example, using as input the
result of:
with open("test.txt", "w") as f:
for i in range(26):
f.write(f":{chr(ord('a') + i)}: ")
f.write('x' * 20_000)
We believe this to be because of some combination of the greedy
quantifiers and the negative lookahead before the match. The regex
was rewritten to (in theory) have much more linear performance.
This bug was discovered when lottery.check_if_end_lottery_task was
failing due to a stack trace thru end_lottery session < badge_grant
< send_repeatable_notifications < sanitize L208. In particular, when
`flask cron` (helpers/cron.py) executes, it does not set g.v, whereas
this code previously assumed that g.v : (None + User) and did not
check for its presence.
These are all minor and uncontroversial enough it just felt gross
making multiple commits.
- Adds marseymummified.
- Changes the new `raise ValueError(...)` in badge_grant to
an `assert`.
- Expands assetcache to a convenient grab bag of JS files.
First, the apparent errors with >= 9 and 99 in the Marseys and
referrals code after the refactor are not actually bugs: they were
bug fixes mixed in with the refactor to fix an off-by-one.
Next, instead of failing silently on the `not user` branch in
badge_grant, we throw a ValueError. This retains the current
behavior where users get 500s to report while also enforcing the
assertion near the edge of the function.
Despite being very fun, this fixes the recently discovered bug where
placing '#' or '!' within the 'pat:' suffix of a patted emoji causes
the enclosing <span> to not be given the proper CSS `display` or
`position`, leading to the hand being sized relative to the comment
bounding box rather than the emoji box.
This should be backward compatible. The only posts it wont fix are
existing ones with the giant hands. Main example being:
https://rdrama.net/h/slackernews/post/76302/
grant_lottery_tickets_to_user incorrectly deposited the full ticket
value into manager account, not just the net value.
Additionally, royalty rate has been set to zero because Chapose won
the first lottery and grifting 8% that could instead go into the
prize pool seems unwarranted given that.
Prior to this comment, the every-1d cron.py command was broken due
to lack of proper stats import. However, while refactoring this, it
was convenient to move other recurring tasks that had been stuffed in
odd places--not least `stats(...)`--into the new cron system. This
entailed a number of refactorings of other things.
1. Move stats(...) from static.py to helpers/stats.py.
2. Move hole inactivity purge task from stats(...) to routes/subs.py.
3. Move bot award timer checks from stats(...) to helpers/awards.py.
4. Unify award timer logic formerly in routes/front.py into the new
helpers/awards.py.
h/t @official-techsupport again for finding another optimization.
We are now cumulatively at about 70% speedup over original.
It remains one of the hottest paths of the codebase in relative
terms, but its absolute performance demands have decreased enough
to buy us potentially substantial time on it.
h/t @official-techsupport for digging into the regex performance and
coming up with one that greatly reduces backtracking. We see an
approximately 2x speedup under typical loads, which proves to be a
major overall savings in performance. Previously, censor_slurs was,
second to ORM DB accesses, by far the most time-consuming function
in the codebase under typical loads. It's still not ideal, but it is
much better.
Future options to improve this critical path further would be:
1) Precompute a slur-replaced HTML, rather than recomputing
each pageload. Storage is cheap.
2) Tokenize the HTML and replace plaintext words using O(1)
exact-match lookups to a dict.
The reddit mentions system contained much duplicated code and was
grafted onto the post thumbnail pipeline to achieve semi-regular
invocation. Instead, we now run it through the new cron system,
and the duplicate code has been refactored out.
Adding an empty __init__.py, the imports-only cli.py, and setting
FLASK_APP in the environment are enough to get the `flask` command
to work. This will enable future changes, including database
migrations.
The proximate reason for the fix is to add a `flask cron` command
to run scheduled tasks within the application from cron. Specifically,
the lottery should be run from cron.
It was observed in prod that the lottery prize as tracked by the DB
had diverged from the amount held in the Lottershe manager account.
This appears to be the result of grant_lottery_tickets_to_user
adding the # of _tickets_ rather than the value of those tickets to
the manager.
- Increment cache version on popover badges.
- Add comments+submission_listing.js to assetcache to support ^.
- Append new words to wordle list.
- Cache bust assorted assets for recent PRs.
In general, we don't do a great job of length validating body_html
fields. Lots of ways to get 500 errors by providing too long of
input. Really ought to find a way to fix it in the classes/comment.py
and classes/submission.py classes. In the interim, the recent gifts
messages change is salient because the notification can 500 out
mid-way through performing coin transactions.
Recommended to find a better way of truncating or safely bubbling
the exception up. Truncating probably not best long-term solution
because it could hypothetically permit strings that would otherwise
be considered unsanitized.
A rare case where users receive 0 lotto tickets from a treasure chest
occurs when they received 10 or 11 coins from a chest pre-conversion
to lotto tickets. Rather than change ticket_count to the ceil of
dividing coins by ticket cost, it seems less distortionary to instead
imperceptibly raise the minimum to avoid this case.
Due to presently hitting perpetual 429s after a mishap with lottery
polling on production, among past events where admins have gotten
rate-limited for doing otherwise normal admin behavior, the
flask_limiter.Limiter now has a request filter to whitelist JL2+.
Despite running on every request, I don't anticipate this undermining
the DoS prevention power of the Limiter.
It is yet unknown whether there are edge cases where running
get_logged_in_user in a different spot in the request pipeline might
e.g. subtly break the logged-in counters. This is not expected at
present, however.
Users now have a toggleable can_gamble setting which disables their
ability to use all chance-based gains on the site: viz. slots,
blackjack, the lottery, and treasure chests.
This only applies on invocation of commands that start gambling
games, so it should cause no bugs when toggled with e.g. active
blackjack games.
This was added for the benefit of users with actual problems with
gambling, be they past addiction or religious conviction. All future
gambling features are humbly requested to respect it.
Adds award to enable viewing profile visitors for non-mops and
non-patrons. This commit should encompass all frontend, backend, and
database changes necessary. Perhaps usable as a model for other
user upgrade flag awards.
The users online count recently added to wrappers.py:get_logged_in_user
uses g.timestamp for its calculations. This is primarily set in
__main__.py:before_request. However, chat has requests which do not
trigger @app.before_request. To resolve this, we now set g.timestamp
in the auth_required wrapper before calling get_logged_in_user().
I think this is safe in general; there's no particular harm to setting
the timestamp _more_ frequently.
Originally prompted by https://rdrama.net/post/18459/-/1984609 which
noticed that streamable.com/e/ links as posts would have another e/
added to them. This was in spite of logic in posts.py api_is_repost
and submit_post designed to specifically counteract this.
Proximal cause was a copypasta'd url.replace(...) chain which
caused the mistake before the streamable-specific logic had a chance
to avoid making it.
Solution: remove the streamable replacement from the chained statement
and create `helpers.normalize_url(url)` to get rid of the copypasta.
Per https://rdrama.net/post/70341/-/1976650 added more gTLDs that
are actually desired by site users.
Also, hard wrapped the `TLDS` and `allowed_tags` tuple-lists at a
100char hard ruler for my sanity.
Removes temporary changes for the Birthgay 2022 event, ending
approximately at 2022-05-23 0600Z. Changes:
- Double XP disabled.
- Lootboxes disabled.
- Three event awards given real descriptions and colors.
- Partyhats: restored to cakeday-only condition and tooltip.
- Header UI: balloons removed, text visibility outline removed,
journoid banner reverted, marseyjam as header icon.
- Birthgay banners: removed from template, moved to new storage dir.
Confetti, though remaining as a permanent award, has been temporarily
removed pending a less performance-intensive implementation and one
that doesn't require maintaining a third-party package in the codebase.
Comments of the style e.g. "@TLSM / @TLSM2" would mistakenly be
`sanitize`d to have identical links only on "@TLSM", the latter
instance having a dangling 2 on the end. It seems this is purely an
issue with text formatting; alerts.py @ NOTIF_USERS had no such
issues. The root cause appears to be partly an optimization and
partly the use of str.replace without a count limit.
The shop discount from Big Spender badges was incorrectly implemented:
intended behavior was for 2pp per badge. However, the values in
const.py were set as if a user could only have one Spender badge at
once, while user.py @ discount was implemented as if they were
additive. Since users factually do have multiple big spender badges,
the values in const were adjusted to reflect this fact.
Also, lootbox Roller badges now each offer a 1pp discount.
users.py @ discount was improved to check against the `discounts`
keys for appropriate badges, rather than a hardcoded list.
Aevann1
Snakes
gooseman
Marco Rebhan
