Ilya Kreymer a767721f5e crawl state: add getPendingList() to return pending state from either… (#205) ...

* crawl state: add getPendingList() to return pending state from either memory or redis crawl state, fix stats logging with redis state. Return pending list as json object
logging: check if data object is an error, log fields from error. Convert missing console.* to new logger
* evaluate failuire: log with error, not fatal

2023-01-23 10:43:12 -08:00

37 KiB

Raw Blame History

View Raw