Good Afternoon,
I have a simple search. Normally this would be:
sourcetype=j_s_i Session_ID=000002b89784b98e91bd OR Session_ID=1c1b68e349f3b98f3570 OR Sesssion_ID=1c1b68e349f3b98f4d3f | stats count by Status
However, in this case I have 60K Session_ID's that I need to input into a single search query.
I have looked at inputcsv, lookup, inputlookup and outputlookup, and to be honest I just dont get it.
Some guidance on how best to proceed please?
Your best option (that comes to mind for me at least) would probably to use this CSV as a lookup. Past 8000 search terms or so you're not getting a performance boost out of adding more terms anyway (and there's even a hard limit of 10500 results that you can emit from a subsearch), so you might as well run a query against all data and then filter it using your lookup.
Your lookup would be something like:
Session_ID,exists
000002b89784b98e91bd,1
1c1b68e349f3b98f3570,1
1c1b68e349f3b98f4d3f,1
And then your search would look something like this:
sourcetype=j_s_i | lookup session_ids Session_ID OUTPUT exists | search exists=1 | stats count by Status
Your best option (that comes to mind for me at least) would probably to use this CSV as a lookup. Past 8000 search terms or so you're not getting a performance boost out of adding more terms anyway (and there's even a hard limit of 10500 results that you can emit from a subsearch), so you might as well run a query against all data and then filter it using your lookup.
Your lookup would be something like:
Session_ID,exists
000002b89784b98e91bd,1
1c1b68e349f3b98f3570,1
1c1b68e349f3b98f4d3f,1
And then your search would look something like this:
sourcetype=j_s_i | lookup session_ids Session_ID OUTPUT exists | search exists=1 | stats count by Status
Hi @rberkheimer
Do you have exactly 60,000 Session_ID's in your data, or are there more than that and you're only looking for data on those 60,000?
Hi!
It is actually 58190 Session_IDs.
In my sourcetype=j_s_i, there are many details on each Session_ID, one of those being Status=Success or Status="some error code".
This log file for one day has 117K Session_IDs in them, and I only want to know the "| stats count by Status" for these 58190.