Getting Data In

How do I only see the latest data from a dynamically changing CSV file?

ragedsparrow
SplunkTrust
SplunkTrust

I have a customer's CSV file that is dynamically updated. By that' what i mean is that the last row is sometimes deleted and readded with updated information:

DATE,HOUR,WORKFILE_COUNT,RATE_PER_WORKFILE,EST_NBR_WORKFILES_PER_HOUR,WORK_HOLD_FILE_COUNT,WORK_DONE_FILE_COUNT,UNPROCESSED_FILE_COUNT,BAD_FILE_COUNT
 09/13/15,15,1172,247.238,14560.9,0,0,0,13
 09/13/15,16,1190,247.011,14574.3,0,0,0,13
 09/13/15,17,1217,227.313,15837.2,0,0,0,13
 09/13/15,18,782,231.839,15528,0,0,0,13
 09/13/15,19,648,240.61,14962,0,0,0,13
 09/13/15,20,629,238.669,15083.6,0,0,0,13
 09/13/15,21,394,242.297,14857.8,0,0,0,13
 09/13/15,22,493,295.181,12195.9,0,0,0,13
 09/13/15,23,257,280.949,12813.7,0,0,0,13

May change to:

DATE,HOUR,WORKFILE_COUNT,RATE_PER_WORKFILE,EST_NBR_WORKFILES_PER_HOUR,WORK_HOLD_FILE_COUNT,WORK_DONE_FILE_COUNT,UNPROCESSED_FILE_COUNT,BAD_FILE_COUNT
 09/13/15,15,1172,247.238,14560.9,0,0,0,13
 09/13/15,16,1190,247.011,14574.3,0,0,0,13
 09/13/15,17,1217,227.313,15837.2,0,0,0,13
 09/13/15,18,782,231.839,15528,0,0,0,13
 09/13/15,19,648,240.61,14962,0,0,0,13
 09/13/15,20,629,238.669,15083.6,0,0,0,13
 09/13/15,21,394,242.297,14857.8,0,0,0,13
 09/13/15,22,493,295.181,12195.9,0,0,0,13
 09/13/15,23,257,291.949,12886.7,0,0,0,13

Splunk, is forwarding both of the lines

09/13/15,23,257,280.949,12813.7,0,0,0,13
09/13/15,23,257,291.949,12886.7,0,0,0,13

My timestamps key off the date and hour in the 1st and 2nd columns, and I only need the latest data for the timestamp, but I'm getting multiple events with the same timestamp due to the updating rows. Any suggestions?

Tags (1)
0 Karma
1 Solution

woodcock
Esteemed Legend

You can use:

... | sort - _indextime | dedup DATE HOUR WORKFILE_COUNT

View solution in original post

woodcock
Esteemed Legend

You can use:

... | sort - _indextime | dedup DATE HOUR WORKFILE_COUNT

ragedsparrow
SplunkTrust
SplunkTrust

Ha! I should have known that. Thank you for your answer. It's perfect. I appreciate it.

0 Karma
Get Updates on the Splunk Community!

Cannot push config from deployer to search head cluster

Hello,I recently upgraded our deployer/deployment server from 8.1.6 to version 9.0 and when I try to push ...

Clear text password in command line

mogod command line argument having clear text password like "--sslPEMKeyPassword=password"how to avoid clear ...

Traffic logs from Splunk Add-on for Cisco Meraki

Recently deployed this add-on, but it doesn't seem to bring back Traffic or URL logs like we did when using ...