Splunk Search

index time with specific field is not working

fabiofox
Explorer

We used the rest receivers simple api to send a body with some fields to index as a urlencoded form.
Among these there is a field time field containing a timestamp. We configure the sourcetype as in figure

Screenshot 2021-09-16 at 15.27.33.png

The problem is that Splunk is indexing when it receives the data ( as if datetime was CURRENT or it found no fields with time information) .

An example of the data is

 

name=session_started&params=%7B%22request_id%22%3A+%220af2918a-0125-4573-9a27-bd1a6deef75d%22%2C+%22subject%22%3A+%22mmt-112%22%7D&time=2021-09-16T09%3A24%3A08.355865

 

we thought that the encoded data could be a problem so we changed the format of the body sent to splunk to json

 

{"name": "session_started", "params": "{\"request_id\": \"0af2918a-0125-4573-9a27-bd1a6deef75d\", \"subject\": \"mmt-112\"}", "time": "2021-09-16T09:24:08.355865"}

 

but the _time was again the time of recevieng.

We tried several tweaks but none of them had success:

  • we checked the format of the strptime ("% Y-% m-% dT% H:% M:% S.% 6N") and it is correct, e.g. "2021-08-31T18: 15: 20.268841"
  • we tried to explicitly set the timezone (our times are in UTC) but nothing has changed 
  • No error or warning in the internal log, even if we try to put a non-existent field instead of time.
  • When searching using that sourcetype, the field time is parsed correctly, so the system is reading correctly.

Any suggestion? What to do? What to try?

A big thanks to the Splunk gurus that will help us!

Labels (2)
0 Karma
1 Solution

Azeemering
Builder

I used your event string as a test:

{"name": "session_started", "params": "{\"request_id\": \"0af2918a-0125-4573-9a27-bd1a6deef75d\", \"subject\": \"mmt-112\"}", "time": "2021-09-16T09:24:08.355865"}

When I tried

Timestamp Format %Y-%m-%dT%H:%M:%3

with

Timestamp Prefix time":\s"

it already parsed the correct date and time in splunk

Azeemering_0-1632134565211.png

 

View solution in original post

fabiofox
Explorer

it works! thanks

0 Karma

Azeemering
Builder

I used your event string as a test:

{"name": "session_started", "params": "{\"request_id\": \"0af2918a-0125-4573-9a27-bd1a6deef75d\", \"subject\": \"mmt-112\"}", "time": "2021-09-16T09:24:08.355865"}

When I tried

Timestamp Format %Y-%m-%dT%H:%M:%3

with

Timestamp Prefix time":\s"

it already parsed the correct date and time in splunk

Azeemering_0-1632134565211.png

 

Get Updates on the Splunk Community!

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...