Hello, I would like to try using Splunk to calculate the difference in numbers from one sample to the next. Here is some theoretical log entries.
The data indexed will look like this:
yesterday_timeStamp clusterId=abc, topicName=mytopic, partition=0, lastOffset=100
yesterday_timeStamp clusterId=abc, topicName=mytopic, partition=1, lastOffset=200
today_timeStamp clusterId=abc, topicName=mytopic, partition=0, lastOffset=355
today_timeStamp clusterId=abc, topicName=mytopic, partition=1, lastOffset=401
The number of events in the last 24 hours would be partition 0(355-200=155), partition 1(401-200=201) Sum of partitions for topic(mytopic) = 155+201=356 There will be many topicName(s) and possible different numbers of partition(s) per topicName. Can I use splunk to calculate the numbers of events per partition and topic since yesterday?
... View more