How to use values from outputlookup file?

[Mike6960]

I created a outputlookup  file with just one column

...My search | table D_ID 

| outputlookup Total.csv

I want to use the data in a new search like a subsearch but results are 0 while I am certain the events exists

Is there also a max limit when using inputlookup ?

...My search [| inputlookup Total.csv]

[gcusello]

Hi @Mike6960 ,

did you created the Lookup and the Lookup Definition before outputlookup?

Ciao.

Giuseppe

[Mike6960]

@gcusello I first created the outpoutlookup then I tried the inputlookup

[gcusello]

Hi @Mike6960 ,

did you created also Lookup Definition ? I'm not speaking of lookup.

Ciao.

Giuseppe

[Mike6960]

@gcusello  ...I suspect I am doing somthing wrong? I thought if I create a outputlookup file I could use these results in a inputlookup ?

[gcusello]

Hi @Mike6960 ,

I ask you for the third time: did you created the Lookup Definition in [ Settings > Lookups > Lookup Definitoions ]?

You cannot use a lookup without Lookup Definition.

Ciao.

Giuseppe

[Mike6960]

@gcusello No I didn't, I thought this wasn't necessary because I use inputlookup and not lookup. I went through a lot of Splunk docs and questions but I can't find a answer. As far as I now understand is that lookup and inputlookup are two different things. By further evaluation I suspect that the max limit of 50000 is the problem. The outputlookup contains more then 100.000 results per day. I guess the number of results is just to much for Splunk to handle if you want to use results from another search in a new search 

[gcusello]

Hi @Mike6960 

this is surely a problem

try to save results in a summary index instead a lookup

Ciao.

Giuseppe

[PickleRick]

OK, so you wanted to output more than 50k results into a lookup table then use with an inputlookup in a subsearch to generate a set of conditions?The default result limit for a subsearch is 10k so it's not gonna work. Anyway, a search with 10k conditions can hardly be called very effective 😉

[Mike6960]

@PickleRick well, I don't know if would call a set of ID's conditions. I was just trying to use those ID's as input. Kind of like a join. But now I know this is not possible 

[PickleRick]

A set of ids contained within a lookup is just a lookup but when you do

[ | inputlookup whatever.csv ]

in your search, it's getting rendered as

(( field1=value1 field2=value2 [...])  OR (field1=valuea field2=valueb [...]) OR [...] )

so it effectively becomes a set of conditions for your main search.

But yes, with too many rows in your lookup you're hitting limits.

[PickleRick]

Do the events you're searching have a field called D_ID?

[Mike6960]

Yes

[ITWhisperer]

Try

...My search [| inputlookup Total.csv|format]

[Mike6960]

@ITWhisperer  when adding the Format the search keeps on the status 'parsing job' 

[ITWhisperer]

It sounds like you have too many rows in your lookup file which is breaching the limits on search sizes.