Skip to content

VirusTotal Batching not working for hash lookups

I am running the search below to populate the virustotal_hash_cache lookup. I have tried to configure different values for batch size, like 4, 50, etc, but it always uses one request per hash.

| `vt_macro_hash_filter`
| eval hash=lower(hash)| lookup virustotal_hash_cache vt_hashes AS hash OUTPUT vt_classification, vt_query_time
| eval vt_expired=if(
(((now()-vt_query_time)<(`vt_cache_unknown_maxage`)) AND vt_classification="unknown_hash") OR
(((now()-vt_query_time)<(`vt_cache_maxage`)) AND vt_classification!="unknown_hash"), 0, 1)
| search NOT vt_classification=* OR vt_expired=1| head 4 | virustotal hash=hash
| search vt_resource=*
| table vt_resource, vt_query_time, vt_classification, vt_scan_date, vt_permalink, vt_positives, vt_total, vt_threat_av, vt_threat_id, vt_hashes
| eval _key=vt_resource
| outputlookup append=t key_field=_key virustotal_hash_cache

In Splunk, I see the following error, don't know if it's related:

02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: AuthenticationError at "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/binding.py", line 304 : Request failed: Session is not logged in.

02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/searchcommands/search_command.py", line 741, in _process_protocol_v2

02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/virustotal.py", line 541, in prepare

02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/client.py", line 1275, in iter

02-26-2020 11:12:28.613 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/client.py", line 1438, in iter

02-26-2020 11:12:28.614 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/client.py", line 1668, in get

02-26-2020 11:12:28.614 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/client.py", line 766, in get

02-26-2020 11:12:28.614 +0100 ERROR ChunkedExternProcessor - stderr: File "/opt/splunk/etc/apps/TA-VirusTotal/bin/../lib/splunklib/binding.py", line 304, in wrapper

Environment: RHEL7, Splunk 7.3.3 Enterprise, on-premises, distributed, Python 2.7.15, TA-VirusTotal 2.1.0 is installed on search head only.

Can you help with this issue?

Thanks, Alex.

Edited by Wiktor Kaczor
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information