Splunk Search

Dedup on data model aftet tstats shows the older event, not the newest

thisissplunk
Builder

I have a tstats query that pulls its data from an accelerated data model. I need to grab only the most up to date host event with the latest IP value. I cannot dedup in the data model root search itself as I need to keep track of _time to get point-in-time results as well.

Anyways, for the most current point-in-time IP value (right now), dedup is not working as intended. It's showing me the older value.

Query without dedup:

 

 

| tstats latest(_time) as _time  FROM datamodel="Host_Info" WHERE nodename="hostinfo" hostname=bobs by hostinfo.hostname hostinfo.ip

 

 

Results (two values for ip)

hostninfo.hostname hostinfo.ip _time

bobs10.10.10.102021-10-22 19:55:03
bobs33.33.33.332021-10-22 21:23:06

Query with dedup:

 

 

| tstats latest(_time) as _time  FROM datamodel="Host_Info" WHERE nodename="hostinfo" hostname=bobs by hostinfo.hostname hostinfo.ip | dedup hostname

 

 

Results (older value, not newer):

hostninfo.hostname hostinfo.ip _time

bobs10.10.10.102021-10-22 19:55:03

Why isn't dedup working correctly? If I dedup the actual indexed data, before it hits the datamodel, it works fine and shows me the latest hostname and IP.

 

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

https://docs.splunk.com/Documentation/Splunk/8.2.2/SearchReference/Dedup

 Events returned by dedup are based on search order. 
0 Karma

thisissplunk
Builder

Ok. So I'm left wondering why the data coming back from the accelerated data model is out of order.

0 Karma

thisissplunk
Builder

I haven't figured out why this is happening but the current workaround is to add a latest(hostname.ip) and removing hostname.ip from the by clause.

Not sure why latest() understands the timestamps but dedup doesn't. Maybe dedup works off of something else than _time?

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...