Getting Data In

Why is eventtype not tagging 100% of events?

a109120
New Member

In an attempt to explain this right...

We have set up multiple eventtypes to different occurrences.

For example:

eventtype=major
eventtype=warning

major works just fine.. When running a simple search :

sourcetype="example" eventtype=warning

The matched results return a result that is not 100% of events. So, for example, the search returns 200 events, but when selecting eventtype in the interesting fields column, it shows that that warning only shows up for 90% (180) of the events. The search is still returning events that meet the requirements for the eventtype=warning, but it is not tagging them as such.

The goal here is to generate alerts based off of these eventtypes to make it much easier to manage. My concern is if the eventtype field is not applying to all occurrences that an alert may not have triggered.

Looking into the events that are not getting the eventtype field, i notice they are rather long, and the portion of the log that would fulfill the requirements for the eventtype field are over 100 lines down in the log. Is there a props.conf or maybe an eventtype.conf setting that can be modified? I'm wondering if it is not looking all the way through the logs to apply the field.

Thanks for any help

0 Karma

greich
Communicator

I have the same issue. What I found out so far is that on some very large events, the eventtype is correctly identified by the indexers and returned to the search heads, but if the marker (search string, term) appears to "deep" into the event, it is not matched on the search head, and of course neither is any tag associated with it.
Typically on stacktraces where the event is 2-400 lines or more.
Likely some limit on search heads with respect to pattern matching. Will update if I dig something up.

0 Karma
Get Updates on the Splunk Community!

Tech Talk | Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

Tech Talk | 3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...

Thank You for Celebrating CX Day with Splunk!

Yesterday the entire team at Splunk + Cisco joined the global celebration of CX Day - celebrating our ...