Getting Data In

use an event field as event time in Time column

youSayGo
Explorer

Hi, I have a CSV file containing events, like meta-data of user visiting a URL, that I import. The challenge I face is getting Splunk to use one of the fields in the data, event_time (shown in the screen shot, last line on the bottom), as the actual event time shown in the default display Time column. If I knew what I was doing this is probably super easy. I keep importing the same file and trying different timestamp methods when defining a new sourceType during the import. There is probably a simple way to do this using the sourceType fields on import or the props.conf, even without having to keep importing it? I have read user guide Modify Event Processing and Assign Source Types to Data, but hours later...here I am.  Thanks, Shane

the field event_time is what I would like to be in the TIme columnthe field event_time is what I would like to be in the TIme column

 

Labels (2)
0 Karma

youSayGo
Explorer

update, ok so I modified the soureType assigned to the data. In Timestamp Field, I added event_time, in Timestamp Prefix I added, %Y-%m-%dT%H:%M:%S.%f    . Then I hit Save, go back to the Search and refresh the page. First, all I have to do is Save then back to data and refresh, correct? The mods to the sourceType will automatically be applied to the data that has been Indexed correct? Either way, if that is ok sequence to apply the sourceType mods to the data shown in the Search, then it is still not taking the time in event_time. Just updating as I move through this. Thanks, Shane

0 Karma

youSayGo
Explorer

here is screen shot, after a full reindex of the data with timestamp prefix and format shown, still not detecting. This is just an import of a CSV file. 

Screen Shot 2021-06-24 at 4.57.47 PM.pngScreen Shot 2021-06-24 at 4.57.47 PM.png

0 Karma

youSayGo
Explorer

Hi, more info... I am using Splunk Enterprise Free. When doing the import and creating a new sourceType, in the section for Timestamps >> Advanced >> time stamp prefix, I did try entering the data field "event_time" in there. Although it did change the date/time shown in the Time column, I could not get it to match the actual value in the event_time. I am guessing that Splunk cannot process the format of the time value of event_time in the data, that being time shown in this format: 2021-06-21T10:52:56.462000. So if this is the case, then it seems I would need to figure out how to convert that to "strptime", maybe with a RegEx in the ? Maybe this is on track, or not? I am reading through the docs on Timestamp Recognition to see if I can figure this out. Maybe I am to use the props.conf, set the [<spec>] to source::<source>, where <source> is event_time, the field pulled from the data? I am not sure how to get Splunk to recognize the time in the event_time field though, which is like this: "event_time: 2021-06-21T10:52:56.462000"

 

Thanks, Shane

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

How much can you really learn in 3 minutes?

Observability can certainly be hard to understand – there's a lot of jargon and buzzwords and it seems to ...

Event Series: The Agentic SOC: Trust Before Autonomy

AI is fundamentally changing security operations, but true progress requires more than just automation—it ...

Free Professional Services for .conf26 Attendees

This year at .conf26, we are doing something a little different. We are bringing the best minds from ...