rDrama/files/helpers/assetcache.py

64 lines
1.6 KiB
Python
Raw Permalink Normal View History

import os
import zlib
from collections import defaultdict
[DO NOT MERGE] import detanglation (#442) * move Base definition to files.classes.__init__.py * fix ImportError * move userpage listing to users.py * don't import the app from classes * consts: set default values to avoid crashes consts: warn if the secret key is the default config value * card view: sneed (user db schema) * cloudflare: use DEFAULT_CONFIG_VALUE * const: set default values * decouple media.py from __main__ * pass database to avoid imports * import cleanup and import request not in const, but in the requests mega import * move asset_submissions site check to __init__ * asset submissions feature flag * flag * g.is_tor * don't import request where it's not needed * i think this is fine * mail: move to own routes and helper * wrappers * required wrappers move * unfuck wrappers a bit * move snappy quotes and marseys to stateful consts * marsify * :pepodrool: * fix missing import * import cache * ...and settings.py * and static.py * static needs cache * route * lmao all of the jinja shit was in feeds.py amazing * classes should only import what they need from flask * import Response * hdjbjdhbhjf * ... * dfdfdfdf * make get a non-required import * isort imports (mostly) * but actually * configs * reload config on import * fgfgfgfg * config * config * initialize snappy and test * cookie of doom debug * edfjnkf * xikscdfd * debug config * set session cookie domain, i think this fixes the can't login bug * sdfbgnhvfdsghbnjfbdvvfghnn * hrsfxgf * dump the entire config on a request * kyskyskyskyskyskyskyskyskys * duifhdskfjdfd * dfdfdfdfdfdfdfdfdfdfdfdf * dfdfdfdf * imoprt all of the consts beacuse fuck it * 😭 * dfdfdfdfdfdfsdasdf * print the entire session * rffdfdfjkfksj * fgbhffh * not the secret keys * minor bug fixes * be helpful in the warning * gfgfgfg * move warning lower * isort main imports (i hope this doesn't fuck something up) * test * session cookie domain redux * dfdfdfd * try only importing Flask * formkeys fix * y * :pepodrool: * route helper * remove before flight * dfdfdfdfdf * isort classes * isort helpers * move check_for_alts to routehelpers and also sort imports and get rid of unused ones * that previous commit but actkally * readd the cache in a dozen places they were implicitly imported * use g.is_tor instead of request.headers. bla bla bla * upgrade streamers to their own route file * get rid of unused imports in __main__ * fgfgf * don't pull in the entire ORM where we don't need it * features * explicit imports for the get helper * explicit imports for the get helper redux * testing allroutes * remove unused import * decouple flask from classes * syntax fix also remember these have side fx for some reason (why?) * move side effects out of the class * posts * testing on devrama * settings * reloading * settingssdsdsds * streamer features * site settings * testing settings on devrama * import * fix modlog * remove debug stuff * revert commit 67275b21ab6e2f2520819e84d10bfc1c746a15b6 * archiveorg to _archiveorg * skhudkfkjfd * fix cron for PCM * fix bugs that snekky wants me to * Fix call to realbody passing db, standardize kwarg * test * import check_for_alts from the right place * cloudflare * testing on devrama * fix cron i think * shadow properly * tasks * Remove print which will surely be annoying in prod. * v and create new session * use files.classes * make errors import little and fix rare 500 in /allow_nsfw * Revert "use files.classes" This reverts commit 98c10b876cf86ce058b7fb955cf1ec0bfb9996c6. * pass v to media functions rather than using g * fix * dfdfdfdfd * cleanup, py type checking is dumb so don't use it where it causes issues * Fix some merge bugs, add DEFAULT_RATELIMIT to main. * Fix imports on sqlalchemy expressions. * `from random import random` is an error. * Fix replies db param. * errors: fix missing import * fix rare 500: only send to GIFT_NOTIF_ID if it exists, and send them the right text * Fix signup formkey. * fix 2 500s * propagate db to submissions * fix replies * dfdfdfdf * Fix verifiedcolor. * is_manual * can't use getters outside of an app context * don't attempt to do gumroad on sites where it's not enabled * don't attempt to do gumraod on sites's where it's unnecessary * Revert "don't attempt to do gumroad on sites where it's not enabled" This reverts commit 6f8a6331878655492dfaf1907b27f8be513c14d3. * fix 500 * validate media type Co-authored-by: TLSM <duolsm@outlook.com>
2022-11-15 09:19:08 +00:00
2022-09-27 06:02:02 +00:00
import gevent
import gevent_inotifyx as inotify
from files.helpers.config.const import *
2022-11-29 22:01:25 +00:00
ASSET_DIR = 'files/assets'
ASSET_URL = '/assets/'
ASSET_CACHE = defaultdict(lambda: None)
2023-09-28 23:58:09 +00:00
ASSET_SUBDIRS = [
'/css', '/js',
'/events/shared/js',
'/events/fistmas/css', '/events/fistmas/js',
'/events/homoween/css', '/events/homoween/js',
]
new_dirs = []
for directory in ASSET_SUBDIRS:
new_dirs += [x[0].split(ASSET_DIR)[1] for x in os.walk(f'{ASSET_DIR}{directory}')]
ASSET_SUBDIRS = new_dirs
2022-11-29 22:01:25 +00:00
def assetcache_build(asset_dir, subdirs):
for subdir in subdirs:
for root, dirs, files in os.walk(asset_dir + subdir):
for fname in files:
fpath = root + '/' + fname
relpath = fpath[len(asset_dir) + 1:].replace('\\', '/')
with open(fpath, 'rb') as f:
fhash = zlib.crc32(f.read())
ASSET_CACHE[relpath] = '%x' % fhash
def assetcache_hash(asset_path):
return ASSET_CACHE[asset_path]
def assetcache_path(asset_path):
cachehash = assetcache_hash(asset_path)
url = ASSET_URL + asset_path
if cachehash:
2023-09-28 23:58:09 +00:00
url += '?x=' + cachehash
return url
2022-09-27 06:02:02 +00:00
def assetcache_watch_directories(asset_dir, subdirs):
fd = inotify.init()
try:
for sd in subdirs:
inotify.add_watch(fd, asset_dir + sd, inotify.IN_CLOSE_WRITE)
while True:
for event in inotify.get_events(fd, 0):
print("Rebuilding assetcache: " + event.name, flush=True)
assetcache_build(asset_dir, subdirs)
break
gevent.sleep(0.5)
finally:
os.close(fd)
assetcache_build(ASSET_DIR, ASSET_SUBDIRS)
2022-09-27 06:02:02 +00:00
gevent.spawn(assetcache_watch_directories, ASSET_DIR, ASSET_SUBDIRS)