Getting Data In

Why is eventtype not tagging 100% of events?

a109120
New Member

In an attempt to explain this right...

We have set up multiple eventtypes to different occurrences.

For example:

eventtype=major
eventtype=warning

major works just fine.. When running a simple search :

sourcetype="example" eventtype=warning

The matched results return a result that is not 100% of events. So, for example, the search returns 200 events, but when selecting eventtype in the interesting fields column, it shows that that warning only shows up for 90% (180) of the events. The search is still returning events that meet the requirements for the eventtype=warning, but it is not tagging them as such.

The goal here is to generate alerts based off of these eventtypes to make it much easier to manage. My concern is if the eventtype field is not applying to all occurrences that an alert may not have triggered.

Looking into the events that are not getting the eventtype field, i notice they are rather long, and the portion of the log that would fulfill the requirements for the eventtype field are over 100 lines down in the log. Is there a props.conf or maybe an eventtype.conf setting that can be modified? I'm wondering if it is not looking all the way through the logs to apply the field.

Thanks for any help

0 Karma

greich
Communicator

I have the same issue. What I found out so far is that on some very large events, the eventtype is correctly identified by the indexers and returned to the search heads, but if the marker (search string, term) appears to "deep" into the event, it is not matched on the search head, and of course neither is any tag associated with it.
Typically on stacktraces where the event is 2-400 lines or more.
Likely some limit on search heads with respect to pattern matching. Will update if I dig something up.

0 Karma
Get Updates on the Splunk Community!

3 Ways to Make OpenTelemetry Even Better

My role as an Observability Specialist at Splunk provides me with the opportunity to work with customers of ...

What's New in Splunk Cloud Platform 9.2.2406?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2406 with many ...

Enterprise Security Content Update (ESCU) | New Releases

In August, the Splunk Threat Research Team had 3 releases of new security content via the Enterprise Security ...