Splunk Search

How to use the dedup command for this search?

evinasco
Communicator

Hi Team,

I have the next source list indexed in Splunk

alt text

I need to let in only the last source by each factory owner

**source**                                                                                                    -     **_time**

/var/log/gestion/logs/equipments/alcatel_ports.1519230396.log   -      2018-03-22 17:41:44
/var/log/gestion/logs/equipments/cisco_ports.1519230441.log          -      2018-03-22 17:41:44
/var/log/gestion/logs/equipments/huawei_ports.1519230476.log     -     2018-03-22 17:41:44
/var/log/gestion/logs/equipments/juniper_ports.1519230550.log    -     2018-03-22 17:41:44

I have tried to use dedup command but for this data doesn't work ok. Does someone know how can I do that?

Regards

Tags (2)
0 Karma
1 Solution

adonio
Ultra Champion

hello there,

dedup does not work as it looks at the exact value for the field, and your values for each factory owner are different in the source
try and extract the factory owner from the source field and then stats on the latest
something like this:

... your search ... | rex field=source "\/equipments\/(?<factory_owner>\w+)"
| stats latest(factory_owner) as last_source_from_factory_owner by source 

see also here:
https://regex101.com/r/uWt0Mf/2
hope it helps

View solution in original post

DalJeanis
Legend

what do you mean by "let in"? Dedup has nothing whatsoever to do with ingestion of data.

0 Karma

adonio
Ultra Champion

hello there,

dedup does not work as it looks at the exact value for the field, and your values for each factory owner are different in the source
try and extract the factory owner from the source field and then stats on the latest
something like this:

... your search ... | rex field=source "\/equipments\/(?<factory_owner>\w+)"
| stats latest(factory_owner) as last_source_from_factory_owner by source 

see also here:
https://regex101.com/r/uWt0Mf/2
hope it helps

evinasco
Communicator

Ready

Thanks a lot

0 Karma
Get Updates on the Splunk Community!

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Deprecation of Splunk Observability Kubernetes “Classic Navigator” UI starting ...

Access to Splunk Observability Kubernetes “Classic Navigator” UI will no longer be available starting January ...

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...