Verified Commit b1142312 authored by Markus Shepherd's avatar Markus Shepherd 🙈
Browse files

always keep around image_file and rules_file, if any

parent ad0efcf7
......@@ -48,7 +48,7 @@ def merge_config(
kwargs["out_path"] = out_path or DATA_DIR / "scraped" / f"{spider}_{item}.jl"
kwargs.setdefault(
"fieldnames_exclude",
("image_file", "rules_file", "published_at", "updated_at", "scraped_at"),
("published_at", "updated_at", "scraped_at"),
)
kwargs.setdefault("sort_keys", True)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment