topic Re: How to use values from outputlookup file in Splunk Search

How to use values from outputlookup file?

Mike6960 — Wed, 22 Mar 2023 15:35:24 GMT

I created a outputlookup file with just one column

...My search | table D_ID

| outputlookup Total.csv

I want to use the data in a new search like a subsearch but results are 0 while I am certain the events exists

Is there also a max limit when using inputlookup ?

...My search [| inputlookup Total.csv]

Re: How to use values from outputlookup file

ITWhisperer — Mon, 20 Mar 2023 23:06:16 GMT

Try

...My search [| inputlookup Total.csv|format]

Re: How to use values from outputlookup file

PickleRick — Mon, 20 Mar 2023 23:08:38 GMT

Do the events you're searching have a field called D_ID?

Re: How to use values from outputlookup file

gcusello — Tue, 21 Mar 2023 07:26:12 GMT

Hi @Mike6960 ,

did you created the Lookup and the Lookup Definition before outputlookup?

Ciao.

Giuseppe

Re: How to use values from outputlookup file

Mike6960 — Tue, 21 Mar 2023 15:31:36 GMT

@gcusello I first created the outpoutlookup then I tried the inputlookup

Re: How to use values from outputlookup file

Mike6960 — Tue, 21 Mar 2023 15:31:55 GMT

Yes

Re: How to use values from outputlookup file

Mike6960 — Tue, 21 Mar 2023 15:32:57 GMT

@ITWhisperer when adding the Format the search keeps on the status 'parsing job'

Re: How to use values from outputlookup file

gcusello — Tue, 21 Mar 2023 15:48:07 GMT

Hi @Mike6960 ,

did you created also Lookup Definition ? I'm not speaking of lookup.

Ciao.

Giuseppe

Re: How to use values from outputlookup file

Mike6960 — Tue, 21 Mar 2023 15:53:16 GMT

@gcusello ...I suspect I am doing somthing wrong? I thought if I create a outputlookup file I could use these results in a inputlookup ?

Re: How to use values from outputlookup file

gcusello — Tue, 21 Mar 2023 17:26:02 GMT

Hi @Mike6960 ,

I ask you for the third time: did you created the Lookup Definition in [ Settings > Lookups > Lookup Definitoions ]?

You cannot use a lookup without Lookup Definition.

Ciao.

Giuseppe

Re: How to use values from outputlookup file

Mike6960 — Tue, 21 Mar 2023 17:33:03 GMT

@gcusello No I didn't, I thought this wasn't necessary because I use inputlookup and not lookup. I went through a lot of Splunk docs and questions but I can't find a answer. As far as I now understand is that lookup and inputlookup are two different things. By further evaluation I suspect that the max limit of 50000 is the problem. The outputlookup contains more then 100.000 results per day. I guess the number of results is just to much for Splunk to handle if you want to use results from another search in a new search

Re: How to use values from outputlookup file

PickleRick — Tue, 21 Mar 2023 17:53:39 GMT

OK, so you wanted to output more than 50k results into a lookup table then use with an inputlookup in a subsearch to generate a set of conditions?The default result limit for a subsearch is 10k so it's not gonna work. Anyway, a search with 10k conditions can hardly be called very effective 😉

Re: How to use values from outputlookup file

gcusello — Tue, 21 Mar 2023 18:13:40 GMT

Hi @Mike6960

this is surely a problem

try to save results in a summary index instead a lookup

Ciao.

Giuseppe

Re: How to use values from outputlookup file

Mike6960 — Tue, 21 Mar 2023 18:39:28 GMT

@PickleRick well, I don't know if would call a set of ID's conditions. I was just trying to use those ID's as input. Kind of like a join. But now I know this is not possible

Re: How to use values from outputlookup file

PickleRick — Tue, 21 Mar 2023 19:23:30 GMT

A set of ids contained within a lookup is just a lookup but when you do

[ | inputlookup whatever.csv ]

in your search, it's getting rendered as

(( field1=value1 field2=value2 [...])  OR (field1=valuea field2=valueb [...]) OR [...] )

so it effectively becomes a set of conditions for your main search.

But yes, with too many rows in your lookup you're hitting limits.

Re: How to use values from outputlookup file

ITWhisperer — Wed, 22 Mar 2023 06:15:44 GMT

It sounds like you have too many rows in your lookup file which is breaching the limits on search sizes.