Splunk Search

Changing a real-time search to date range search

matthewcanty
Communicator

Hi all. I have been working on a query to alert us to when a date changes for a specific item, the item is represented in my query as GroupId. Dates can change any number of times, and I want to receive an alert whenever they do.

I have come up with the following real-time search. I am concerned about it only having the data available from when the search starts.

index=daluat PublishedGtpRule GroupId | dedup 100 GroupId sortby -_time | stats first(Date) As CurrentDate list(Date) As PreviousDates dc(Date) As Count by GroupId | where Count>1

I'll try my best to explain what I think would work best.

  1. Search for distinct GroupIds over a time period
  2. For each GroupId, search for the previous occurrence of it - all time
  3. Result if date has changed

Preferably I'd like to see what the new and previous dates are too.

This way my alerting doesn't have to be real-time. I can simply run it every minute, and look back over the last 2 or 3 minutes and allow the subsearch to go back through all time to find the last occurrence of a GroupId.

FYI the number of GroupIds per minute is a low number, 10 at most.

0 Karma
1 Solution

matthewcanty
Communicator

I have solved my problem. If anyone has any cleaner suggestions I would be more than happy to hear from you. I hope this helps someone one day.

I run the following search on a schedule every 10 minutes with a time range from -1y to now.

index=daluat Published GroupId 
| DEDUP 2 GroupId 
| STATS DC(Date) AS dist_dates FIRST(Date) AS NewDate LAST(Date) AS OldDate LATEST(_time) AS latest_time BY GroupId
| EVAL relative_time_threshold=relative_time(now(),"-10m") 
| WHERE latest_time>relative_time_threshold AND dist_dates>1 
| CONVERT CTIME(latest_time) 
| FIELDS GroupId NewDate OldDate

View solution in original post

0 Karma

matthewcanty
Communicator

I have solved my problem. If anyone has any cleaner suggestions I would be more than happy to hear from you. I hope this helps someone one day.

I run the following search on a schedule every 10 minutes with a time range from -1y to now.

index=daluat Published GroupId 
| DEDUP 2 GroupId 
| STATS DC(Date) AS dist_dates FIRST(Date) AS NewDate LAST(Date) AS OldDate LATEST(_time) AS latest_time BY GroupId
| EVAL relative_time_threshold=relative_time(now(),"-10m") 
| WHERE latest_time>relative_time_threshold AND dist_dates>1 
| CONVERT CTIME(latest_time) 
| FIELDS GroupId NewDate OldDate
0 Karma

matthewcanty
Communicator

It's not real time anymore. It's running once every 10 minutes.

0 Karma

linu1988
Champion

if it is a realtime then shouldn't it be -1y to rt?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...