Splunk Search

Dedup on data model aftet tstats shows the older event, not the newest

thisissplunk
Builder

I have a tstats query that pulls its data from an accelerated data model. I need to grab only the most up to date host event with the latest IP value. I cannot dedup in the data model root search itself as I need to keep track of _time to get point-in-time results as well.

Anyways, for the most current point-in-time IP value (right now), dedup is not working as intended. It's showing me the older value.

Query without dedup:

 

 

| tstats latest(_time) as _time  FROM datamodel="Host_Info" WHERE nodename="hostinfo" hostname=bobs by hostinfo.hostname hostinfo.ip

 

 

Results (two values for ip)

hostninfo.hostname hostinfo.ip _time

bobs10.10.10.102021-10-22 19:55:03
bobs33.33.33.332021-10-22 21:23:06

Query with dedup:

 

 

| tstats latest(_time) as _time  FROM datamodel="Host_Info" WHERE nodename="hostinfo" hostname=bobs by hostinfo.hostname hostinfo.ip | dedup hostname

 

 

Results (older value, not newer):

hostninfo.hostname hostinfo.ip _time

bobs10.10.10.102021-10-22 19:55:03

Why isn't dedup working correctly? If I dedup the actual indexed data, before it hits the datamodel, it works fine and shows me the latest hostname and IP.

 

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

https://docs.splunk.com/Documentation/Splunk/8.2.2/SearchReference/Dedup

 Events returned by dedup are based on search order. 
0 Karma

thisissplunk
Builder

Ok. So I'm left wondering why the data coming back from the accelerated data model is out of order.

0 Karma

thisissplunk
Builder

I haven't figured out why this is happening but the current workaround is to add a latest(hostname.ip) and removing hostname.ip from the by clause.

Not sure why latest() understands the timestamps but dedup doesn't. Maybe dedup works off of something else than _time?

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...