Update readme
parent
04ea9c9e5e
commit
70bad811bb
12
readme.md
12
readme.md
|
@ -1,3 +1,11 @@
|
||||||
|
# Reddit Ingest
|
||||||
|
|
||||||
|
Distributed server/client setup for ingesting all of reddit.
|
||||||
|
|
||||||
|
- Scales to multiple clients.
|
||||||
|
- Supports reddit authentication.
|
||||||
|
- Tolerant to clients losing state/going offline.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# ...install PostgreSQL...
|
# ...install PostgreSQL...
|
||||||
pip install -r requirements.txt
|
pip install -r requirements.txt
|
||||||
|
@ -7,8 +15,10 @@ mv fetcher_config.example.yaml fetcher_config.yaml
|
||||||
mkdir logs
|
mkdir logs
|
||||||
bash compile_proto.sh
|
bash compile_proto.sh
|
||||||
|
|
||||||
# Then run both of these files:
|
# Run one instance of batcher.py:
|
||||||
python batcher.py
|
python batcher.py
|
||||||
|
|
||||||
|
# And several instances of fetcher.py:
|
||||||
python fetcher.py
|
python fetcher.py
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue