topic Problems with subsearch. in Splunk Search

Problems with subsearch.

lylereger — Tue, 29 Sep 2020 12:22:17 GMT

I am having trouble getting a subsearch to work and was hoping someone might be able to help. I am trying to compare enterprise security hashes to another log/index I have that also has those same searches referenced. The index of that other log is "otherlog" in my sample. BridgeMsg is just a macro/field for a part of the syslog string.

It is my understanding that inner joins are done first, so I am getting the event hash from ES, then I want to see if that result is in my otherlog. Obviously, I am missing something fundamental here though. I am also getting this error message and have no idea why:

Error in 'dedup' command: The field '(' is specified multiple times. It should only be specified once.

Any ideas?
(Edited to clarify as per comment 1)

Re: Problems with subsearch.

hunters_splunk — Thu, 12 Jan 2017 15:29:54 GMT

Hi lylereger,

I think the rex syntax in your search is incorrect. A field name should be supplied to assign the captured group to.

... | rex field=bridgeMsg "hash:\s+[\d.]+\s+(?<new_field>(\S+))" | ...

For details, see documentation: http://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/Rex
Not sure what new field you want to create with the regex capture, but please correct this syntax and try again.

Hope this helps. Thanks!
Hunter

Re: Problems with subsearch.

lylereger — Thu, 12 Jan 2017 15:43:40 GMT

That line does indeed have a field, I was bitten by cut and paste it seems. Hope it shows up in the tags. event_hash is the field.

rex field=bridgeMsg "hash:\s+[\d.]+\s+(?<event_hash>(\S+))"

Re: Problems with subsearch.

somesoni2 — Thu, 12 Jan 2017 19:28:31 GMT

You want use the result of subsearch as filter, so it should be used with filtering command such as | search or | where (or in the base search itself. Try this

index=otherlog sourcetype=syslog | rex field=bridgeMsg "hash:\s+[\d.]+\s+(?<event_hash>(\S+))" | fields event_hash | dedup event_hash | search [ search index=notable | search NOT `suppression` | `get_event_hash` | dedup event_hash ]

OR (filter in base search and use event_hash value as text filter)

index=otherlog sourcetype=syslog [ search index=notable | search NOT `suppression` | `get_event_hash` | dedup event_hash | table event_hash | rename event_hash as search ]| rex field=bridgeMsg "hash:\s+[\d.]+\s+(?<event_hash>(\S+))" | fields event_hash | dedup event_hash | search

Re: Problems with subsearch.

lylereger — Mon, 16 Jan 2017 22:58:31 GMT

I think I am closer to the answer now, still need to do a lot of verification. For my needs, I think the second solution was a bit closer. Where I wound up:

index=notable  source="*"  
| search NOT `suppression` 
| `get_event_hash` 
| dedup event_hash 
| fields * | table * 
| where  [search index=snaplec sourcetype=syslog-og | rex field=bridgeMsg hash:\s+[\d.]+\s+(?<event_hash>(\S+))"
    | fields event_hash 
    | dedup event_hash ]  
| table event_hash _time search_name
| rename event_hash as Notable, _time as Time, search_name as “Rule Name”