VirusTotal Batching not working for hash lookups
I am running the search below to populate the virustotal_hash_cache lookup. I have tried to configure different values for batch size, like 4, 50, etc, but it always uses one request per hash.
| `vt_macro_hash_filter`
| eval hash=lower(hash)| lookup virustotal_hash_cache vt_hashes AS hash OUTPUT vt_classification, vt_query_time
| eval vt_expired=if(
(((now()-vt_query_time)<(`vt_cache_unknown_maxage`)) AND vt_classification="unknown_hash") OR
(((now()-vt_query_time)<(`vt_cache_maxage`)) AND vt_classification!="unknown_hash"), 0, 1)
| search NOT vt_classification=* OR vt_expired=1| head 4 | virustotal hash=hash
| search vt_resource=*
| table vt_resource, vt_query_time, vt_classification, vt_scan_date, vt_permalink, vt_positives, vt_total, vt_threat_av, vt_threat_id, vt_hashes
| eval _key=vt_resource
| outputlookup append=t key_field=_key virustotal_hash_cache
In Splunk, I see the following error, don't know if it's related:
02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: AuthenticationError at "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/binding.py", line 304 : Request failed: Session is not logged in.
02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/searchcommands/search_command.py", line 741, in _process_protocol_v2
02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/virustotal.py", line 541, in prepare
02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/client.py", line 1275, in iter
02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/client.py", line 1438, in iter
02-26-2020 11:12:28.614 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/client.py", line 1668, in get
02-26-2020 11:12:28.614 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/client.py", line 766, in get
02-26-2020 11:12:28.614 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/binding.py", line 304, in wrapper
Environment: RHEL7, Splunk 7.3.3 Enterprise, on-premises, distributed, Python 2.7.15, TA-VirusTotal 2.1.0 is installed on search head only.
Can you help with this issue?
Thanks, Alex.