mirror of
https://github.com/webrecorder/browsertrix-crawler.git
synced 2025-12-24 19:10:15 +00:00
e02058f001
* ad blocking via request interception, extending block rules system, adding new AdBlockRules * Load list of hosts to block from https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts added as json on image build * Enabled via --blockAds and setting a custom message via --adBlockMessage * new test to check for ad blocking * Add test-crawls dir to .gitignore and .dockerignore
8 lines
78 B
Plaintext
8 lines
78 B
Plaintext
*.pyc
|
|
__pycache__
|
|
*.egg-info/
|
|
collections/
|
|
node_modules/
|
|
crawls/
|
|
test-crawls/
|