How can I specify specific lines within my lookup file to search against?

[the_wolverine]

I have a large resultset, lookupb.csv which consists of about 4 million lines, that I'm searching against that I need to break up in order to bypass the 10k result limit in subsearch.

Example:

| inputlookup lookupa.csv | search NOT [ |inputlookup lookupb.csv | head 10000 | fields fieldname ]

Obviously, I have a way to get 10000 first lines, and maybe 10000 last lines, but what about in-between?

[the_wolverine]

That worked! Make it an answer so I can accept it.

[somesoni2 · Accepted solution]

One option could be to have serial no in the lookup file itself or add serial no in subsearch and search based on it.

|inputlookup lookupb.csv | eval sno=1 | accum sno | where sno >= yourlowerlimit AND sno <= yourupperlimit

Also, consider option to increase the limit value in limits.conf so that there are less no of breakups.