topic Lookup using temporary dataset? in Splunk Search

Lookup using temporary dataset?

Mick_OBrien — Mon, 27 Feb 2023 14:43:47 GMT

I'm trying to add a lookup to enrich results returned from a 'simple' search. The search command I'm using [and I have limited to one key/value pair] is: -

index=ee_commercialbankingeforms_pcf "*LEVEL=WARN*" | rex "^\S+\s(?<microService>\S+).*MESSAGE=(?<message>.+)" | bucket _time span=day | stats count by microService, message | lookup [ {JIRASummary: "No JWT found on UserPrincipal and no custom JWT claims configured. No nested JWT will be sent in downstream requests!", JIRA: "CBE-968"} ] JIRASummary AS message OUTPUT JIRA

...but I keep seeing following error...

Error in 'SearchParser': Missing a search command before '{'. Error at position '192' of search query 'search index=ee_commercialbankingeforms_pcf "*LEVE...{snipped} {errorcontext = lookup [ {JIRASummar}'.

Can someone explain the error that I see?

Regards

Mick

Re: Lookup using temporary dataset

richgalloway — Mon, 27 Feb 2023 13:56:12 GMT

The query contains a subsearch, which is executed before the main search. The subsearch must be valid SPL, which is not the case here.

{JIRASummary: "No JWT found on UserPrincipal and no custom JWT claims configured. No nested JWT will be sent in downstream requests!", JIRA: "CBE-968"}

is not SPL. What exactly are you trying to do there?

A subsearch in this location doesn't make much sense. The lookup command expects the name of a CSV file or defined lookup, which I've never seen come out of a subsearch. Not that it can't be done, but the subseasrch would have to return a valid lookup argument.

Re: Lookup using temporary dataset?

Mick_OBrien — Mon, 27 Feb 2023 15:13:10 GMT

From...

lookup command examples - Splunk Documentation

..I see this example...

... | lookup users uid OUTPUTNEW username, department

What I was trying to do was include a temporary dataset...

Datasets - Splunk Documentation

i.e. trying to use same sample data as example then something along the lines of...

... | lookup [ {uid: "1066", username: "Claudia Gasrcia", department: "Engineering" }, {...}, {...} ] uid OUTPUTNEW username, department

...or generally...

search ... | lookup [temporary_dataset] key OUTPUTNEW <lookup_table_fields>

Is this use of lookup and temporary datasets possible?

Re: Lookup using temporary dataset?

richgalloway — Mon, 27 Feb 2023 15:20:48 GMT

The cited documentation is for SPL2, which cannot be used in a standard search (SPL).

Re: Lookup using temporary dataset?

Mick_OBrien — Tue, 28 Feb 2023 08:00:52 GMT

Sorry - I don't know what you mean by SPL and SPL2?

Is there another way of enriching output results using loopup?

Re: Lookup using temporary dataset?

richgalloway — Tue, 28 Feb 2023 15:10:29 GMT

SPL2 is a revised query language used by Splunk in some of their newer products. It is not supported in Splunk Enterprise, however.

The standard method for enriching data is by using a lookup table to find information from an event and insert related information from the lookup table. The general format is

| lookup <lookup> <input field(s)> OUTPUT <output field(s)>

where <lookup> is either a CSV file, a lookup definition, or a KVStore collection
and <input field(s)> is one or more field names from the current results
and <output field(s)> is one or more column names from <lookup>.

See the Search Reference manual for specifics.