Splunk Search

index time with specific field is not working

fabiofox
Explorer

We used the rest receivers simple api to send a body with some fields to index as a urlencoded form.
Among these there is a field time field containing a timestamp. We configure the sourcetype as in figure

Screenshot 2021-09-16 at 15.27.33.png

The problem is that Splunk is indexing when it receives the data ( as if datetime was CURRENT or it found no fields with time information) .

An example of the data is

 

name=session_started&params=%7B%22request_id%22%3A+%220af2918a-0125-4573-9a27-bd1a6deef75d%22%2C+%22subject%22%3A+%22mmt-112%22%7D&time=2021-09-16T09%3A24%3A08.355865

 

we thought that the encoded data could be a problem so we changed the format of the body sent to splunk to json

 

{"name": "session_started", "params": "{\"request_id\": \"0af2918a-0125-4573-9a27-bd1a6deef75d\", \"subject\": \"mmt-112\"}", "time": "2021-09-16T09:24:08.355865"}

 

but the _time was again the time of recevieng.

We tried several tweaks but none of them had success:

  • we checked the format of the strptime ("% Y-% m-% dT% H:% M:% S.% 6N") and it is correct, e.g. "2021-08-31T18: 15: 20.268841"
  • we tried to explicitly set the timezone (our times are in UTC) but nothing has changed 
  • No error or warning in the internal log, even if we try to put a non-existent field instead of time.
  • When searching using that sourcetype, the field time is parsed correctly, so the system is reading correctly.

Any suggestion? What to do? What to try?

A big thanks to the Splunk gurus that will help us!

Labels (2)
0 Karma
1 Solution

Azeemering
Builder

I used your event string as a test:

{"name": "session_started", "params": "{\"request_id\": \"0af2918a-0125-4573-9a27-bd1a6deef75d\", \"subject\": \"mmt-112\"}", "time": "2021-09-16T09:24:08.355865"}

When I tried

Timestamp Format %Y-%m-%dT%H:%M:%3

with

Timestamp Prefix time":\s"

it already parsed the correct date and time in splunk

Azeemering_0-1632134565211.png

 

View solution in original post

fabiofox
Explorer

it works! thanks

0 Karma

Azeemering
Builder

I used your event string as a test:

{"name": "session_started", "params": "{\"request_id\": \"0af2918a-0125-4573-9a27-bd1a6deef75d\", \"subject\": \"mmt-112\"}", "time": "2021-09-16T09:24:08.355865"}

When I tried

Timestamp Format %Y-%m-%dT%H:%M:%3

with

Timestamp Prefix time":\s"

it already parsed the correct date and time in splunk

Azeemering_0-1632134565211.png

 

Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...