topic Re: Help Getting CSV File in when time field is in Seconds Past Midnight in Getting Data In

Help Getting CSV File in when time field is in Seconds Past Midnight

edgarrity — Wed, 06 Jul 2022 23:41:55 GMT

I need help with loading CSV files into Splunk with the event time recorded as seconds past midnight instead of HH:MM:SS time. Below is a sample of the data I need to load. How do I specify that the time column is the number of seconds past midnight when defining the Timestamp for the Source Type?

PickStartDate,BTVersion,TripNumber,Sequence,PassingTime,ArrivalTime,DepartureTime,FlagStop,ByPass,EarlyDeparture,event_line_number
2021-04-25,S1000216,1020,1,54900,54900.0,54900.0,0,0,,1
2021-04-25,S1000216,1020,2,54955,,,0,0,,2
2021-04-25,S1000216,1020,3,54999,,,0,0,,3

Re: Help Getting CSV File in when time field is in Seconds Past Midnight

VatsalJagani — Thu, 07 Jul 2022 05:36:47 GMT

@edgarrity - INGEST_EVAL would be one of the options I guess.

(you may have to combine multiple INGEST_EVAL and in combination with normal TRANSFORMS with REGEX to achieve the extraction and generate the right timestamp as you need.)

I hope this helps!!!

Re: Help Getting CSV File in when time field is in Seconds Past Midnight

PickleRick — Thu, 07 Jul 2022 05:53:39 GMT

Yep, INGEST_EVAL is the way to go.

Firstly parse out the day as usual so you'll get the beginning of the day in the _time field. Then parse out an indexed field containing number of seconds. Then add this number to your _time. And finally, if not needed for anything else, unset the extracted field (otherwise it will get indexed).