topic search on multiple indexes in Splunk Search

search on multiple indexes

avitallange — Mon, 15 Jul 2013 10:42:38 GMT

Hi,

I have two indexes: index1, index2.

index1 has a field 'Message' which index2 doesn't have.
There are duplicated messages that I'd like to dedup by |dedup Message.

I'd like to find the records with text "TextToFind" across the 2 indexes but not to get multiple records for the duplicated 'Message' field.

index=index1 TextToFind returns 94 results (appear in field Message)
index=index2 TextToFind returns 8 results (appear in field Content)
(index=index1) OR (index=index2) TextToFind returns 102 results

(index=index1) OR (index=index2) TextToFind | dedup Message returns 2 results from index1 only although the TextToFind appears on index2 as well (in field Content).

Could you please fix my query?

Re: search on multiple indexes

Ayn — Mon, 15 Jul 2013 12:28:46 GMT

If your first search works but your second search returns 0 results, that's because there is no field called Message in your search.

Re: search on multiple indexes

samhughe — Mon, 15 Jul 2013 13:31:52 GMT

Is the text you are looking for in index1's message field or is it in a different field?
if so you could use the eval and coalesce functions (http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/CommonEvalFunctions) to put them into a new, single field and then dedup. If the text you are looking for isn't in index1's Message field not sure of your best option other than some of the join/subsearch type functions

Re: search on multiple indexes

avitallange — Mon, 15 Jul 2013 15:04:44 GMT

I have a field called Message, I see in the left pane with many other fields. You are right, I got results but I still can't query what I want. I updated my question above.
Your help is welcome!

Re: search on multiple indexes

avitallange — Mon, 15 Jul 2013 15:06:52 GMT

Yes, the test I am looking for is in index1's Message field.
It also appears on index2's Content field.
I updated my question above.
Your help is welcome!

Re: search on multiple indexes

samhughe — Mon, 15 Jul 2013 15:08:10 GMT

Based on your additional comments, try:
(index=index1) OR (index=index2) TextToFind | eval newField=coalesce(Message,Content) | dedup newField

Assuming there are no duplicates of 'Content' you want to keep.

Or probably the better option is to use the keepempty=true parameter for dedup: http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/Dedup

Re: search on multiple indexes

Ayn — Mon, 15 Jul 2013 15:24:39 GMT

See hamhughe's answer which I think is spot on.

Re: search on multiple indexes

avitallange — Mon, 15 Jul 2013 18:55:07 GMT

It worked with the keepempty=true, thank you!