topic Re: Query with Thousands IDs in searche whit OR in Splunk Search

Query with Thousands IDs in searche whit OR

Dmitriy — Thu, 05 Aug 2021 05:06:03 GMT

Hello, i need help.

I have 6500 IIN (like id) and put this id to lookup then tried search: index=alfa [|inputlookup IIN_oleg.csv |rename IIN as search | fields search]

They given result only for one firs IIN in lookup.

If i search whit out lookup just 10 IIN whit "OR" the give me 10 result

Re: Query with Thousands IDs in searche whit OR

kamlesh_vaghela — Thu, 05 Aug 2021 05:11:16 GMT

@Dmitriy

Can you please share the field name and sample data from index alfa ?

Re: Query with Thousands IDs in searche whit OR

kamlesh_vaghela — Thu, 05 Aug 2021 05:13:55 GMT

@Dmitriy

If you have same field name then you can try like this.

index = alfa | lookup IIN_oleg.csv IIN output IIN as IIN_1 | where isnotnull(IIN_1)

Re: Query with Thousands IDs in searche whit OR

Dmitriy — Thu, 05 Aug 2021 05:16:52 GMT

lookup data

data in index

Re: Query with Thousands IDs in searche whit OR

kamlesh_vaghela — Thu, 05 Aug 2021 05:19:31 GMT

Thanks @Dmitriy

Are you able to extract name and value fields from events?

index=alfa name="IIN" | stats count by value

Is this query working for you?

If Not can you please share sample _raw event, so I can help you on extraction also.

Re: Query with Thousands IDs in searche whit OR

Dmitriy — Thu, 05 Aug 2021 05:26:41 GMT

No, not give any result 😔

Re: Query with Thousands IDs in searche whit OR

Dmitriy — Thu, 05 Aug 2021 05:31:55 GMT

index=alfa name="IIN" | stats count by value

result 0

Re: Query with Thousands IDs in searche whit OR

kamlesh_vaghela — Thu, 05 Aug 2021 05:34:41 GMT

@Dmitriy

Can you please try this?

index=alfa | rex field=_raw "parameter\sname=\\\\\"(?<name>.*)\\\\\"\svalue=\\\\\"(?<value>\d+)\\\\\"\/>" | where name="IIN" | stats count by value

Re: Query with Thousands IDs in searche whit OR

Dmitriy — Thu, 05 Aug 2021 05:40:38 GMT

we dont have field like IIN in index data

index=alfa 
| rex field=_raw "parameter\sname=\\\\\"(?<name>.*)\\\\\"\svalue=\\\\\"(?<value>\d+)\\\\\"\/&gt;"
| where name="IIN" 
| stats count by value

Result 0

Re: Query with Thousands IDs in searche whit OR

kamlesh_vaghela — Thu, 05 Aug 2021 05:44:31 GMT

@Dmitriy

What is the value in field name ?

Re: Query with Thousands IDs in searche whit OR

Dmitriy — Thu, 05 Aug 2021 06:04:52 GMT

the IIN dont have field name in index

Re: Query with Thousands IDs in searche whit OR

kamlesh_vaghela — Thu, 05 Aug 2021 06:27:44 GMT

Then how you want to map with IIN_oleg.csv?

Re: Query with Thousands IDs in searche whit OR

kamlesh_vaghela — Thu, 05 Aug 2021 06:34:02 GMT

@Dmitriy

One more thing I observed in my instance. Incase if may OR conditions the search it self is getting break.. So can you please run your search and check the job inspect? Is that any error?

Re: Query with Thousands IDs in searche whit OR

Dmitriy — Thu, 05 Aug 2021 06:47:15 GMT

start search index=alfa [|inputlookup IIN_oleg.csv |rename IIN as search | fields search]

search.log

08-05-2021 12:36:55.531 ERROR LookupProviderFactory - Must specify one or more lookup fields.

08-05-2021 12:36:54.743 INFO  SearchParser - PARSING: search index=alfa [|inputlookup IIN_oleg.csv  |rename IIN as search | fields search] |

08-05-2021 12:36:55.531 ERROR LookupProviderFactory - Must specify one or more lookup fields.
08-05-2021 12:36:55.531 ERROR AutoLookupDriver - Could not load lookup='LOOKUP-cisco_asa_ids_lookup' reason='Error in 'lookup' command: Must specify one or more lookup fields.'
08-05-2021 12:36:55.531 ERROR LookupProviderFactory - Must specify one or more lookup fields.
08-05-2021 12:36:55.531 ERROR AutoLookupDriver - Could not load lookup='LOOKUP-cisco_asa_intrusion_severity_lookup' reason='Error in 'lookup' command: Must specify one or more lookup fields.'

Re: Query with Thousands IDs in searche whit OR

Dmitriy — Thu, 05 Aug 2021 06:50:21 GMT

i can not find in logs why the thake only firs value in lookup. I think the problem in query

Re: Query with Thousands IDs in searche whit OR

kamlesh_vaghela — Thu, 05 Aug 2021 06:58:05 GMT

Try this,

index=alfa [|inputlookup IIN_oleg.csv |rename IIN as search | table search | format "(" "(" "AND" ")" "OR" ")"]

Re: Query with Thousands IDs in searche whit OR

Dmitriy — Thu, 05 Aug 2021 07:05:07 GMT

YEEHAA !!! Its work now, they give me correct result. Thank you very much.

Re: Query with Thousands IDs in searche whit OR

kamlesh_vaghela — Thu, 05 Aug 2021 07:07:02 GMT

cool

Glad to help you 🙂

!! Happy Splunking !!