Getting Data In

Why is eventtype not tagging 100% of events?

a109120
New Member

In an attempt to explain this right...

We have set up multiple eventtypes to different occurrences.

For example:

eventtype=major
eventtype=warning

major works just fine.. When running a simple search :

sourcetype="example" eventtype=warning

The matched results return a result that is not 100% of events. So, for example, the search returns 200 events, but when selecting eventtype in the interesting fields column, it shows that that warning only shows up for 90% (180) of the events. The search is still returning events that meet the requirements for the eventtype=warning, but it is not tagging them as such.

The goal here is to generate alerts based off of these eventtypes to make it much easier to manage. My concern is if the eventtype field is not applying to all occurrences that an alert may not have triggered.

Looking into the events that are not getting the eventtype field, i notice they are rather long, and the portion of the log that would fulfill the requirements for the eventtype field are over 100 lines down in the log. Is there a props.conf or maybe an eventtype.conf setting that can be modified? I'm wondering if it is not looking all the way through the logs to apply the field.

Thanks for any help

0 Karma

greich
Communicator

I have the same issue. What I found out so far is that on some very large events, the eventtype is correctly identified by the indexers and returned to the search heads, but if the marker (search string, term) appears to "deep" into the event, it is not matched on the search head, and of course neither is any tag associated with it.
Typically on stacktraces where the event is 2-400 lines or more.
Likely some limit on search heads with respect to pattern matching. Will update if I dig something up.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...