How do I only see the latest data from a dynamically changing CSV file?

[ragedsparrow]

I have a customer's CSV file that is dynamically updated. By that' what i mean is that the last row is sometimes deleted and readded with updated information:

DATE,HOUR,WORKFILE_COUNT,RATE_PER_WORKFILE,EST_NBR_WORKFILES_PER_HOUR,WORK_HOLD_FILE_COUNT,WORK_DONE_FILE_COUNT,UNPROCESSED_FILE_COUNT,BAD_FILE_COUNT
 09/13/15,15,1172,247.238,14560.9,0,0,0,13
 09/13/15,16,1190,247.011,14574.3,0,0,0,13
 09/13/15,17,1217,227.313,15837.2,0,0,0,13
 09/13/15,18,782,231.839,15528,0,0,0,13
 09/13/15,19,648,240.61,14962,0,0,0,13
 09/13/15,20,629,238.669,15083.6,0,0,0,13
 09/13/15,21,394,242.297,14857.8,0,0,0,13
 09/13/15,22,493,295.181,12195.9,0,0,0,13
 09/13/15,23,257,280.949,12813.7,0,0,0,13

May change to:

DATE,HOUR,WORKFILE_COUNT,RATE_PER_WORKFILE,EST_NBR_WORKFILES_PER_HOUR,WORK_HOLD_FILE_COUNT,WORK_DONE_FILE_COUNT,UNPROCESSED_FILE_COUNT,BAD_FILE_COUNT
 09/13/15,15,1172,247.238,14560.9,0,0,0,13
 09/13/15,16,1190,247.011,14574.3,0,0,0,13
 09/13/15,17,1217,227.313,15837.2,0,0,0,13
 09/13/15,18,782,231.839,15528,0,0,0,13
 09/13/15,19,648,240.61,14962,0,0,0,13
 09/13/15,20,629,238.669,15083.6,0,0,0,13
 09/13/15,21,394,242.297,14857.8,0,0,0,13
 09/13/15,22,493,295.181,12195.9,0,0,0,13
 09/13/15,23,257,291.949,12886.7,0,0,0,13

Splunk, is forwarding both of the lines

09/13/15,23,257,280.949,12813.7,0,0,0,13
09/13/15,23,257,291.949,12886.7,0,0,0,13

My timestamps key off the date and hour in the 1st and 2nd columns, and I only need the latest data for the timestamp, but I'm getting multiple events with the same timestamp due to the updating rows. Any suggestions?

[woodcock · Accepted solution]

You can use:

... | sort - _indextime | dedup DATE HOUR WORKFILE_COUNT

[ragedsparrow]

Ha! I should have known that. Thank you for your answer. It's perfect. I appreciate it.