All Apps and Add-ons

Trackme Status Message Not Clearing

mark_wymer
Path Finder

Hi,

I've got an issue with one of my Data Sources where TrackMe falsely detected a Data Sampling anomaly and, consequently, set the state to Red and the Status Message to:

Alert: data source status is red, monitoring conditions are not met due to anomalies detected in the data sampling and format recognition, review the data sampling window to investigate. This alert means that trackMe detected an issue in the format of the events compared to the format that was previsouly identified for this source.

I used the 'Clear state and run sampling' from the Data Sampling tab and that is now Green. The Status Flipping tab also shows the object_state as Green but the 'state' ( in the summary at the top) is still Red and the Status message is still the same (as above). However, I've just noticed that the Timeline in the Status Message tab also shows as 'Green'???

How can I drill down into why TrackMe is showing both  a Red and Green status for the same Data Source?

Many thanks,
Mark.

Labels (1)
0 Karma

mark_wymer
Path Finder

Hi,

Ignore the above. The message and state has now cleared / changed to Green. It cleared after I closed the Actions window and reopened it ( possible bug? ).

So, I guess the solution for anyone else hitting this or a similar issue - close the Action window and re-open it.

Cheers,
Mark 

0 Karma

guilmxm
SplunkTrust
SplunkTrust

Hi @mark_wymer !

When the data sampling detects an anomaly in a data source, it is very unlikely to be a false positive, at least from a technical point view, this means the component detected an issue in the data quality such as a change in the format, multiple format which was not detected during the discovery etc.

Review the documentation:
https://trackme.readthedocs.io/en/latest/userguide.html#data-sampling-and-event-formats-recognition

The UI shows what the summary of the data sampling state is, and you can access to the latest sample events that were at the root of the issue:

guilmxm_0-1612253877156.png

guilmxm_1-1612253907977.png


Then, either you can fix the issue, or you can create a custom model that matches your data source context with no issues, and you can as well disable the data sampling feature if the quality issue is not something you are able to get fixed.

Wen the data sampling status is cleared there is a some delay potentially until the data sources tracker runs again, so either you click on Refresh after you cleared the data sample state, or you click on Run the tracker now, or you wait a bit until these are executed automatically and the status will be cleared.

So in short after you cleared the data sampling you would have clicked in the refresh button to have the data source global state refreshed against the new status of the data sampling without closing the current screen, I'll check if both actions can be triggered in the same time when you process the clear state and run sampling action.

Thus, do no ignore the data sampling alert, if it triggered it was for a reason and it is likely this will happen again unless you disable the data sampling for it, or fix the issue, or create a model etc

The very last version, 1.2.31, includes a new component called Smart Status which give an extended understanding of the root cause and runs the models automatically to provide an answer:
https://trackme.readthedocs.io/en/latest/userguide.html#smart-status

Let me know if this is clear and if you have any question, always more than happy to have users inputs.

Guilhem













0 Karma

mark_wymer
Path Finder

Hi @guilmxm 

Thanks for getting back to me, I appreciate that you were busy getting the next update ready.

I have now upgraded to your latest version and I still have the same anomaly. If it helps with troubleshooting, I've taken some snapshots:
Data_Sampling_Anomaly.JPGSmart_Status_Results.JPG

 

Please feel free to PM be if you prefer.

Thanks, Mark.

0 Karma

guilmxm
SplunkTrust
SplunkTrust

Hi @mark

Perfect, so as you can see this is not a false positive, TrackMe is detecting a quality issue in your data, multiple format match means that the sampling detects a given format as shown in the screenshot, in addition events not matching that same format.

A proper sourcetype should be matching the same type of events respecting the same type of data under the same format ideally, this is what ensures the data sampling feature.

So it's a true positive.

If you click on "Show latest sample" this opens a search UI, which shows the events that were sampled and the format that was detected, if you checkout these you will see that there are events not matching the other format.

So you can:

- Click on the disable button on the right which will disable the sampling feature for this data source if the quality issue cannot be fixed
- Potentially you can create a custom model for that data source, if this makes sense and that depends on the data

If you disable the feature, then run the Tracker now in the main screen and the data sources will go back to the state it would without the data sampling.

Guilhem









0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...