Splunk Search

Deduping Results But Preserving Most Recent Value

cgalligan
Explorer

I'm running a query to pull data on some agents, which have each have a unique "aid". For example, my computer would have a unique aid, but if I check in once every hour the most recent up to data detail set is 60min ago. How can I dedup by aid while showing the most recent data?

Here is my query - the agentlocaltime and AgentVersion is what would vary based on results:

index=<AGENT_INFO> sourcetype=<SOURCETYPE> event_platform=<OS TYPE> event_simpleName=AgentOnline|eval agentlocal_time=strftime(AgentLocalTime,"%m/%d/%y %H:%M:%S")| table aip,aid,SystemProductName,SystemManufacturer,MoboManufacturer,ComputerName,BiosVersion,BiosReleaseDate,BiosManufacturer,AgentVersion,agentlocal_time,aid,aip
Tags (2)
0 Karma
1 Solution

DalJeanis
Legend

You actually have the exact right term.

your search that returns the records 
| dedup aid

https://docs.splunk.com/Documentation/SplunkCloud/6.6.3/SearchReference/Dedup

What dedup does is it saves the first record it encounters by the fields listed (in this case aid) and discards all others.

Since Splunk by its nature returns the most recent record first, that's all you need, unless you have run a command like stats that implicitly or sort that explicitly changes the order.

The dedup command can require the data to be sent to the search head, so you want a fields command before dedup to limit the data transmitted. For this purpose, we just copied your table command above dedup and changed it to fields.

 index=<AGENT_INFO> sourcetype=<SOURCETYPE> event_platform=<OS TYPE> event_simpleName=AgentOnline
| fields aip, aid, SystemProductName, SystemManufacturer, MoboManufacturer, ComputerName, BiosVersion, BiosReleaseDate, BiosManufacturer, AgentVersion, agentlocal_time    
| dedup aid
| eval agentlocal_time=strftime(AgentLocalTime,"%m/%d/%y %H:%M:%S")
| table aip, aid, SystemProductName, SystemManufacturer, MoboManufacturer, ComputerName, BiosVersion, BiosReleaseDate, BiosManufacturer, AgentVersion, agentlocal_time

View solution in original post

0 Karma

DalJeanis
Legend

You actually have the exact right term.

your search that returns the records 
| dedup aid

https://docs.splunk.com/Documentation/SplunkCloud/6.6.3/SearchReference/Dedup

What dedup does is it saves the first record it encounters by the fields listed (in this case aid) and discards all others.

Since Splunk by its nature returns the most recent record first, that's all you need, unless you have run a command like stats that implicitly or sort that explicitly changes the order.

The dedup command can require the data to be sent to the search head, so you want a fields command before dedup to limit the data transmitted. For this purpose, we just copied your table command above dedup and changed it to fields.

 index=<AGENT_INFO> sourcetype=<SOURCETYPE> event_platform=<OS TYPE> event_simpleName=AgentOnline
| fields aip, aid, SystemProductName, SystemManufacturer, MoboManufacturer, ComputerName, BiosVersion, BiosReleaseDate, BiosManufacturer, AgentVersion, agentlocal_time    
| dedup aid
| eval agentlocal_time=strftime(AgentLocalTime,"%m/%d/%y %H:%M:%S")
| table aip, aid, SystemProductName, SystemManufacturer, MoboManufacturer, ComputerName, BiosVersion, BiosReleaseDate, BiosManufacturer, AgentVersion, agentlocal_time
0 Karma

cgalligan
Explorer

thanks, I didn't realize that it was returning newest first

Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...