Getting Data In

How do I set _time with collect _raw?

yurykiselev
Path Finder

Hi!
I have to collect some JSON "as is" - not as key-value pair. How can I set event timestamp in this case?

... | eval _time=strptime(...) | table _time _raw | collect index="..."

... doesn't work: _time is ignored with _raw present and replaced with cirrent time. I could do:

... | eval data = _raw | table _time data | collect ... 

, but it generates key-value-event with "data={my_json_from_raw}"

0 Karma
1 Solution

VatsalJagani
SplunkTrust
SplunkTrust

Hi @yurykiselev,
If time is already in the _raw in a format that Splunk can understand easily then you will add addtime=false as a option with collect command and Splunk will automatically extracts time from _raw no need to extract separately.

If not the above scenario then you have to write props.conf for the source-type you are using.

  • If time format is weird then you must have specify time format into TIME_FORMAT. (This may help you in writing that format)

  • If time format is not available within first 128 characters of _raw event then you have to add MAX_TIMESTAMP_LOOKAHEAD.

  • You can also specify TIME_PREFIX. (props.conf may help you in all the configuration writing)

Hope this helps!!

View solution in original post

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

Hi @yurykiselev,
If time is already in the _raw in a format that Splunk can understand easily then you will add addtime=false as a option with collect command and Splunk will automatically extracts time from _raw no need to extract separately.

If not the above scenario then you have to write props.conf for the source-type you are using.

  • If time format is weird then you must have specify time format into TIME_FORMAT. (This may help you in writing that format)

  • If time format is not available within first 128 characters of _raw event then you have to add MAX_TIMESTAMP_LOOKAHEAD.

  • You can also specify TIME_PREFIX. (props.conf may help you in all the configuration writing)

Hope this helps!!

0 Karma

yurykiselev
Path Finder

Thank you! I added date at begin of data "%Y-%m-%d %H:%M:%S" - it's recignized without any props defineding.

VatsalJagani
SplunkTrust
SplunkTrust

Nice!!, This time format is identify by Splunk so good for you.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Enriching Signals with the OTel Lookup Processor

Introduction Raw telemetry is often incomplete. A log line might record an IP address, but what you really ...

Meet the .conf26 AI Agent Buildathon Winners

At the .conf26 AI Agent Buildathon, Splunk users turned their operational expertise into working AI agents ...

Painting a Clearer Picture: Unifying Your Investigations with Cisco AI Canvas

Painting a Clearer Picture: Unifying Your Investigations with Cisco AI Canvas  We’ve all been there. You’re ...