Archive

How to efficiently search for a specific message in my data without aggregating millions of useless logs?

Explorer

I'm running a search on the same index and sourcetype with a few different messages, but one particular message has spaces and the words within it are pretty generic. For example, "Find analytic value". From reading online, it looks like Splunk would look for any logs with "find" "analytic" and "value" and then look for Message="Find analytic value". Is this accurate? If so, is there a way to get this to be more specific before aggregating millions of useless logs? The amount of logs generated with this message should be small.

0 Karma

Esteemed Legend

So long as that string does not begin with a major breaker (see docs on segmenters.conf), you can do this:

index=foo TERM(Find analytic value)

Give it a try and see:
https://docs.splunk.com/Documentation/Splunk/latest/Search/UseCASEandTERMtomatchphrases
https://docs.splunk.com/Documentation/Splunk/latest/Admin/Segmentersconf

0 Karma

Explorer

To my understanding of major breakers the message doesn't start with one, but I couldn't get that or any close search TERM('Find Analytic Value') or TERM("Find Analytic Value") to work. CASE(Find Analytic Value) combined with a subsequent search on the exact Message definitely speeds it up, but I think getting TERM to work would be even better. The first link you sent me does mention that if it's logged as x=y then TERM(y) won't work, but I'm not sure if they mean the literal log says x=y or x could be Message and y "Find Analytic Value".

0 Karma

SplunkTrust
SplunkTrust

That is accurate.

Two points. First, a "generic" term would not be a barrier. On the other hand, "common" terms could become a barrier. One would think that the term "analytic" would be relatively sparse, so probably not an issue.

Second, If this information is going to be accessed repeatedly, then this search might profitably be accelerated, or turned into an accelerated data model.

0 Karma

Explorer

What are the benefits/costs of an accelerated search/data model?

0 Karma