All Apps and Add-ons

Why does search work when in streaming mode, but fails when the job switches to map/reduce?

lob5ter
Engager

Greetings,

I hope you can help with the following.
This query works when in streaming mode, but fails when the job switches to map/reduce:

index=hdpidx event.original="*\"EventId\":\"4776\"*"
| rex field=event.original "TargetUserName\":\"(?.+?)\""
| ldapfilter search="(sAMAccountNAme=$foo$)" attrs="name"
| table foo, name

It will begin returning results, but once it switches to map/red we see the following error repeated for multiple hadoop worker nodes:

[hdpprov] [w001.cluster] External search command 'ldapfilter' returned error code 1. Script output = "error_message=TypeError at "/tmp/splunk/splunk.hdpidx/splunk/var/run/searchpeers/splunk.hdpidx-1585082701/apps/SA-ldapsearch/bin/packages/splunklib/binding.py", line 478 : int() argument must be a string or a number, not 'NoneType' ". 

[hdpprov] [w003.cluster] External search command 'ldapfilter' returned error code 1. Script output = "error_message=TypeError at "/tmp/splunk/splunk.hdpidx/splunk/var/run/searchpeers/splunk.hdpidx-1585082701/apps/SA-ldapsearch/bin/packages/splunklib/binding.py", line 478 : int() argument must be a string or a number, not 'NoneType' ". 

... error repeated for each of the worker nodes participating in the job ...

I do not believe this is an issue with ldapfilter, but with an interaction between $var$ expansion and hunk.

If I add a head 10 directly after the initial search (before the rex), the search runs successfully - IMHO pointing to the issue only occurring during map-reduce.

I have verified worker node network access the AD servers.

Could this be an escaping issue where $foo$ is being replaced with a (non-existent) environment variable? Hence the NoneType?

Any thoughts would be much appreciated!

0 Karma

lob5ter
Engager

For anyone with the same problem - I've solved this a different way. We're taking a full copy of our AD entries using ldapsearch on a daily basis and using the data in a lookup instead. Added benefit is that we're not querying AD per event, downside it's a big lookup and it's ~a day old.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...