How do I reconfigure _time field?

[Hod152]

Hey,

It has been several days that I'm trying to solve the following issue.

I'm sending JSON data over tcp (9997), the data arrives completly but, the "_time" field that is saved is the actual indexing time instead of  "time" field, that I've added before.
I tried to configure props.conf as with JSON and with regex but both hasn't work.

What makes things more wired is that some of the logs that contains another timestamp field are indexed with this field. I was also trying to play arround this but nothing helped. 

I'm running an all-in-one deployment from a container.

Thanks,

------------

props.conf

[evtx]
 MAX_DAYS_AGO = 10951
#INDEXED_EXTRACTIONS = JSON
#KV_MODE = none
#TRANSFORMS-jsonextraction = json_extract
#TIME_PREFIX = time.:. 
#TIME_FORMAT = %Y-%m-%dT%H:%M:%S
#TIMESTAMP_FIELDS = time
CHARSET = UTF-8
INDEXED_EXTRACTIONS = json
KV_MODE = none
SHOULD_LINEMERGE = false
disabled = false
pulldown_type = 1
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6NZ
TIMESTAMP_FIELDS=time
LINE_BREAKER = ([\r\n]+)
category = Structured
description = ODATA SCPI MPL JSON

log sample - time is indexed  based on "Event.NewTime":

 { "cribl_breaker":"fallback",
"collector":"in_evtx_to_json",
"time":"2023-05-18T11:41:20.687637Z",
"EventData":{"NewTime":"2023-05-18T11:41:20.697000Z",
"PreviousTime":"2023-05-18T11:41:20.707011Z",
"ProcessId":"0x46c",
"ProcessName":"C:\\Windows\\System32\\svchost.exe",
"SubjectDomainName":"NT AUTHORITY",
"SubjectLogonId":"0x3e5",
"SubjectUserName":"LOCAL SERVICE",
"SubjectUserSid":"S-1-5-19"},
"System":{"Channel":"Security",
"Computer":"test123",
"Correlation":null,
"EventID":4616,
"EventRecordID":108506811,
"Execution":{"#attributes":{"ProcessID":4,
"ThreadID":1752}},
"Keywords":"0x8020000000000000",
"Level":0,
"Opcode":0,
"Provider":{"#attributes":{"Guid":"54849625-5478-4994-A5BA-3E3B0328C30D",
"Name":"Microsoft-Windows-Security-Auditing"}},
"Security":null,
"Task":12288,
"TimeCreated":{"#attributes":{"SystemTime":"2023-05-18T11:41:20.687637Z"}},
"Version":1}}

log samle - time is indexed based on arrival time:

{"cribl_breaker":"fallback",
"collector":"in_evtx_to_json",
"time":"2023-05-06T20:01:58.205343Z",
"EventData":{"HandleId":"0x6a30",
"ObjectServer":"Security",
"ProcessId":"0x4f4",
"ProcessName":"C:\\Windows\\System32\\svchost.exe",
"SubjectDomainName":"test.local",
"SubjectLogonId":"0x3e4",
"SubjectUserName":"test",
"SubjectUserSid":"S-1-5-20"},
"System":{"Channel":"Security",
"Computer":"test123",
"Correlation":null,
"EventID":4658,
"EventRecordID":107343779,
"Execution":{"#attributes":{"ProcessID":4,
"ThreadID":109608}},
"Keywords":"0x8020000000000000",
"Level":0,
"Opcode":0,
"Provider":{"#attributes":{"Guid":"54849625-5478-4994-A5BA-3E3B0328C30D",
"Name":"Microsoft-Windows-Security-Auditing"}},
"Security":null,
"Task":12800,
"TimeCreated":{"#attributes":{"SystemTime":"2023-05-06T20:01:58.205343Z"}},
"Version":0}}

[PickleRick]

First things first - which endpoint are you sending your data to?

By default, unless you specify ?auto_extract_timestamp=true in your HEC endpoint URL, HEC bypasses timestamp recognition from the event itself regardless of whatever you have in it and what props you set.

https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTinput#services.2Fcollector.2Fevent

[Hod152]

I'm sending it using cribl over the default port directly to splunk. It's an all in one deployment.

[PickleRick]

Have you read the Cribl HEC docs?

https://docs.cribl.io/stream/destinations-splunk-hec

[isoutamo]

Based on port this is not an HEC endpoint. I assume that as used port 9997 it’s using S2S protocol.

[PickleRick · Accepted solution]

True. I have no idea where I got this HEC from. Might have mixed it with some other thread.

Anyway, the cribl docs say that data for splunk destination is sent as "cooked and parsed" which means that if those docs are correct and the data is indeed sent parsed, the only thing that can happen in index-time on the receiving end is ingest actions.

(A side note - it's also relatively ineffective way to send events bandwidth-wise)

[Hod152]

Thank you, I'll work around 🙂

[Hod152]

I read the Splunk single instance doc which is the how I recive the events, the events are arriving to splunk but not indexed as I need.

[isoutamo]

Hi

If I check correctly then these props.conf works for me.

[ <FIRST> ]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=json
KV_MODE=none
description=First
category=Structured
disabled=false
pulldown_type=true
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6Q%Z
TIMESTAMP_FIELDS=time

[ <SECOND> ]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=json
KV_MODE=none
description=second
category=Structured
disabled=false
pulldown_type=true
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6Q%Z
TIMESTAMP_FIELDS=time

I check these with 9.0.5 and another version and both works as expected.

r. Ismo

[Hod152]

Thanks for the quick response!

I've tried your solution but, it still not working. My splunk version is 9.0.5