rDrama/files/helpers/reddit_mentions.py

100 lines
3.3 KiB
Python
Raw Normal View History

import requests
[DO NOT MERGE] import detanglation (#442) * move Base definition to files.classes.__init__.py * fix ImportError * move userpage listing to users.py * don't import the app from classes * consts: set default values to avoid crashes consts: warn if the secret key is the default config value * card view: sneed (user db schema) * cloudflare: use DEFAULT_CONFIG_VALUE * const: set default values * decouple media.py from __main__ * pass database to avoid imports * import cleanup and import request not in const, but in the requests mega import * move asset_submissions site check to __init__ * asset submissions feature flag * flag * g.is_tor * don't import request where it's not needed * i think this is fine * mail: move to own routes and helper * wrappers * required wrappers move * unfuck wrappers a bit * move snappy quotes and marseys to stateful consts * marsify * :pepodrool: * fix missing import * import cache * ...and settings.py * and static.py * static needs cache * route * lmao all of the jinja shit was in feeds.py amazing * classes should only import what they need from flask * import Response * hdjbjdhbhjf * ... * dfdfdfdf * make get a non-required import * isort imports (mostly) * but actually * configs * reload config on import * fgfgfgfg * config * config * initialize snappy and test * cookie of doom debug * edfjnkf * xikscdfd * debug config * set session cookie domain, i think this fixes the can't login bug * sdfbgnhvfdsghbnjfbdvvfghnn * hrsfxgf * dump the entire config on a request * kyskyskyskyskyskyskyskyskys * duifhdskfjdfd * dfdfdfdfdfdfdfdfdfdfdfdf * dfdfdfdf * imoprt all of the consts beacuse fuck it * 😭 * dfdfdfdfdfdfsdasdf * print the entire session * rffdfdfjkfksj * fgbhffh * not the secret keys * minor bug fixes * be helpful in the warning * gfgfgfg * move warning lower * isort main imports (i hope this doesn't fuck something up) * test * session cookie domain redux * dfdfdfd * try only importing Flask * formkeys fix * y * :pepodrool: * route helper * remove before flight * dfdfdfdfdf * isort classes * isort helpers * move check_for_alts to routehelpers and also sort imports and get rid of unused ones * that previous commit but actkally * readd the cache in a dozen places they were implicitly imported * use g.is_tor instead of request.headers. bla bla bla * upgrade streamers to their own route file * get rid of unused imports in __main__ * fgfgf * don't pull in the entire ORM where we don't need it * features * explicit imports for the get helper * explicit imports for the get helper redux * testing allroutes * remove unused import * decouple flask from classes * syntax fix also remember these have side fx for some reason (why?) * move side effects out of the class * posts * testing on devrama * settings * reloading * settingssdsdsds * streamer features * site settings * testing settings on devrama * import * fix modlog * remove debug stuff * revert commit 67275b21ab6e2f2520819e84d10bfc1c746a15b6 * archiveorg to _archiveorg * skhudkfkjfd * fix cron for PCM * fix bugs that snekky wants me to * Fix call to realbody passing db, standardize kwarg * test * import check_for_alts from the right place * cloudflare * testing on devrama * fix cron i think * shadow properly * tasks * Remove print which will surely be annoying in prod. * v and create new session * use files.classes * make errors import little and fix rare 500 in /allow_nsfw * Revert "use files.classes" This reverts commit 98c10b876cf86ce058b7fb955cf1ec0bfb9996c6. * pass v to media functions rather than using g * fix * dfdfdfdfd * cleanup, py type checking is dumb so don't use it where it causes issues * Fix some merge bugs, add DEFAULT_RATELIMIT to main. * Fix imports on sqlalchemy expressions. * `from random import random` is an error. * Fix replies db param. * errors: fix missing import * fix rare 500: only send to GIFT_NOTIF_ID if it exists, and send them the right text * Fix signup formkey. * fix 2 500s * propagate db to submissions * fix replies * dfdfdfdf * Fix verifiedcolor. * is_manual * can't use getters outside of an app context * don't attempt to do gumroad on sites where it's not enabled * don't attempt to do gumraod on sites's where it's unnecessary * Revert "don't attempt to do gumroad on sites where it's not enabled" This reverts commit 6f8a6331878655492dfaf1907b27f8be513c14d3. * fix 500 * validate media type Co-authored-by: TLSM <duolsm@outlook.com>
2022-11-15 09:19:08 +00:00
from flask import g
2024-02-16 20:42:42 +00:00
from files.helpers.config.const import *
[DO NOT MERGE] import detanglation (#442) * move Base definition to files.classes.__init__.py * fix ImportError * move userpage listing to users.py * don't import the app from classes * consts: set default values to avoid crashes consts: warn if the secret key is the default config value * card view: sneed (user db schema) * cloudflare: use DEFAULT_CONFIG_VALUE * const: set default values * decouple media.py from __main__ * pass database to avoid imports * import cleanup and import request not in const, but in the requests mega import * move asset_submissions site check to __init__ * asset submissions feature flag * flag * g.is_tor * don't import request where it's not needed * i think this is fine * mail: move to own routes and helper * wrappers * required wrappers move * unfuck wrappers a bit * move snappy quotes and marseys to stateful consts * marsify * :pepodrool: * fix missing import * import cache * ...and settings.py * and static.py * static needs cache * route * lmao all of the jinja shit was in feeds.py amazing * classes should only import what they need from flask * import Response * hdjbjdhbhjf * ... * dfdfdfdf * make get a non-required import * isort imports (mostly) * but actually * configs * reload config on import * fgfgfgfg * config * config * initialize snappy and test * cookie of doom debug * edfjnkf * xikscdfd * debug config * set session cookie domain, i think this fixes the can't login bug * sdfbgnhvfdsghbnjfbdvvfghnn * hrsfxgf * dump the entire config on a request * kyskyskyskyskyskyskyskyskys * duifhdskfjdfd * dfdfdfdfdfdfdfdfdfdfdfdf * dfdfdfdf * imoprt all of the consts beacuse fuck it * 😭 * dfdfdfdfdfdfsdasdf * print the entire session * rffdfdfjkfksj * fgbhffh * not the secret keys * minor bug fixes * be helpful in the warning * gfgfgfg * move warning lower * isort main imports (i hope this doesn't fuck something up) * test * session cookie domain redux * dfdfdfd * try only importing Flask * formkeys fix * y * :pepodrool: * route helper * remove before flight * dfdfdfdfdf * isort classes * isort helpers * move check_for_alts to routehelpers and also sort imports and get rid of unused ones * that previous commit but actkally * readd the cache in a dozen places they were implicitly imported * use g.is_tor instead of request.headers. bla bla bla * upgrade streamers to their own route file * get rid of unused imports in __main__ * fgfgf * don't pull in the entire ORM where we don't need it * features * explicit imports for the get helper * explicit imports for the get helper redux * testing allroutes * remove unused import * decouple flask from classes * syntax fix also remember these have side fx for some reason (why?) * move side effects out of the class * posts * testing on devrama * settings * reloading * settingssdsdsds * streamer features * site settings * testing settings on devrama * import * fix modlog * remove debug stuff * revert commit 67275b21ab6e2f2520819e84d10bfc1c746a15b6 * archiveorg to _archiveorg * skhudkfkjfd * fix cron for PCM * fix bugs that snekky wants me to * Fix call to realbody passing db, standardize kwarg * test * import check_for_alts from the right place * cloudflare * testing on devrama * fix cron i think * shadow properly * tasks * Remove print which will surely be annoying in prod. * v and create new session * use files.classes * make errors import little and fix rare 500 in /allow_nsfw * Revert "use files.classes" This reverts commit 98c10b876cf86ce058b7fb955cf1ec0bfb9996c6. * pass v to media functions rather than using g * fix * dfdfdfdfd * cleanup, py type checking is dumb so don't use it where it causes issues * Fix some merge bugs, add DEFAULT_RATELIMIT to main. * Fix imports on sqlalchemy expressions. * `from random import random` is an error. * Fix replies db param. * errors: fix missing import * fix rare 500: only send to GIFT_NOTIF_ID if it exists, and send them the right text * Fix signup formkey. * fix 2 500s * propagate db to submissions * fix replies * dfdfdfdf * Fix verifiedcolor. * is_manual * can't use getters outside of an app context * don't attempt to do gumroad on sites where it's not enabled * don't attempt to do gumraod on sites's where it's unnecessary * Revert "don't attempt to do gumroad on sites where it's not enabled" This reverts commit 6f8a6331878655492dfaf1907b27f8be513c14d3. * fix 500 * validate media type Co-authored-by: TLSM <duolsm@outlook.com>
2022-11-15 09:19:08 +00:00
from files.classes.comment import Comment
2023-02-07 03:31:49 +00:00
from files.helpers.sanitize import *
2023-02-24 17:58:09 +00:00
from files.helpers.alerts import push_notif
2022-12-04 19:48:28 +00:00
from files.classes.notifications import Notification
2024-02-16 20:42:42 +00:00
def reddit_mentions_task():
site_mentions = get_mentions(OFFSITE_NOTIF_QUERIES)
notify_mentions(site_mentions)
2024-02-16 20:42:42 +00:00
if REDDIT_NOTIFS_USERS:
for query, send_user in REDDIT_NOTIFS_USERS.items():
user_mentions = get_mentions(cache, [query], reddit_notifs_users=True)
2023-11-14 19:33:08 +00:00
if user_mentions:
notify_mentions(user_mentions, send_to=send_user, mention_str='mention of you')
2024-02-16 20:42:42 +00:00
def get_mentions(queries, reddit_notifs_users=False):
mentions = []
2023-09-12 20:28:10 +00:00
for kind in ('submission', 'comment'):
q = " or ".join(queries)
url = f'https://api.pullpush.io/reddit/search/{kind}?q={q}'
2023-11-14 19:33:08 +00:00
try: data = requests.get(url, headers=HEADERS, timeout=5).json()['data']
except: return []
for thing in data:
2023-09-11 18:16:16 +00:00
if not thing.get('permalink'):
2023-09-11 18:12:10 +00:00
continue
2023-09-14 20:19:06 +00:00
if thing.get('subreddit_subscribers') and thing['subreddit_subscribers'] < 2: continue
2023-09-18 11:47:06 +00:00
if thing.get('subreddit_type') == 'user': continue
if thing['subreddit'] in {'IAmA', 'PokemonGoRaids', 'SubSimulatorGPT2', 'SubSimGPT2Interactive'}: continue
if 'bot' in thing['author'].lower(): continue
if 'AutoModerator' == thing['author']: continue
if kind == 'comment':
body = thing["body"].replace('>', '> ')
text = f'<blockquote><p>{body}</p></blockquote>'
else:
title = thing["title"].replace('>', '> ')
2022-07-17 17:14:01 +00:00
# Special case: a spambot says 'WPD' a lot unrelated to us.
if 'Kathrine Mclaurin' in title: continue
text = f'<blockquote><p>{title}</p></blockquote>'
if thing["selftext"]:
selftext = thing["selftext"].replace('>', '> ')[:5000]
text += f'<br><blockquote><p>{selftext}</p></blockquote>'
mentions.append({
'permalink': thing['permalink'],
'author': thing['author'],
'created_utc': thing['created_utc'],
'text': text,
})
return mentions
def notify_mentions(mentions, send_to=None, mention_str='site mention'):
2023-09-12 20:26:24 +00:00
mentions = sorted(mentions, key=lambda x: int(x['created_utc']), reverse=True)
for m in mentions:
2022-07-10 13:02:24 +00:00
author = m['author']
permalink = m['permalink']
2024-02-16 20:42:42 +00:00
text = (
f'<p>New {mention_str} by <a href="https://old.reddit.com/user/{author}" '
f'rel="nofollow noopener" target="_blank">/u/{author}</a></p>'
f'<p><a href="https://old.reddit.com{permalink}?context=89" '
'rel="nofollow noopener" target="_blank">'
f'https://old.reddit.com{permalink}?context=89</a></p>'
2024-02-16 20:42:42 +00:00
f'{m["text"]}'
)
2024-02-16 20:42:42 +00:00
text = sanitize(text, blackjack="reddit mention", golden=False)
g.db.flush()
try:
existing_comment = g.db.query(Comment.id).filter_by(
2024-02-16 20:42:42 +00:00
author_id=AUTOJANNY_ID,
parent_post=None,
2024-02-16 20:42:42 +00:00
body_html=text).one_or_none()
if existing_comment: break
except:
pass
new_comment = Comment(
2024-02-16 20:42:42 +00:00
author_id=AUTOJANNY_ID,
parent_post=None,
2024-02-16 20:42:42 +00:00
body_html=text,
distinguished=True,
created_utc=int(m['created_utc']),
)
2023-03-16 06:27:58 +00:00
g.db.add(new_comment)
g.db.flush()
new_comment.top_comment_id = new_comment.id
if send_to:
notif = Notification(comment_id=new_comment.id, user_id=send_to)
2023-03-16 06:27:58 +00:00
g.db.add(notif)
2023-08-23 10:42:25 +00:00
push_notif({send_to}, f'New mention of you on reddit by /u/{author}', '', f'{SITE_FULL}/notification/{new_comment.id}')