topic Re: Time format for log in Splunk Enterprise

Time format for log

justindett — Mon, 15 Feb 2021 12:05:37 GMT

Hi,

I am struggling with some logs in a specific directory. They just don't seem to be ingested into splunk.

If I put a normal .log file in with a standard time format it populates just fine.

But these logs have the following format:

O", "message": "Test logging" }
{ "time": "2020-12-07 09:46:52.7940", "threadId": "30", "level": "INFO", "message": "Test logging" }
{ "time": "2020-12-07 12:14:34.7402", "threadId": "53", "level": "INFO", "message": "Test logging" }
{ "time": "2020-12-07 13:48:24.8650", "threadId": "12", "level": "INFO", "message": "Test logging" }
{ "time": "2020-12-08 10:33:40.0607", "threadId": "68", "level": "INFO", "message": "Test logging" }
{ "time": "2020-12-08 11:53:56.7778", "threadId": "51", "level": "INFO", "message": "Test logging" }
{ "time": "2020-12-09 08:42:53.6465", "threadId": "133", "level": "INFO", "message": "Test logging" }
{ "time": "2020-12-09 10:35:44.0103", "threadId": "152", "level": "INFO", "message": "Test logging" }
{ "time": "2020-12-11 10:38:27.0194", "threadId": "113", "level": "INFO", "message": "Test logging" }
{ "time": "2020-12-11 12:18:25.0442", "threadId": "6", "level": "INFO", "message": "Test logging" }

And nothing comes into splunk at all. I have commented out all the timestamp options in the props.conf to force it to use default manner ,but still nothing at all.

Is it related to a setting that should be in the props.conf?

Any assistance would be appreciated.

Thanks

Re: Time format for log

manjunathmeti — Mon, 15 Feb 2021 12:14:58 GMT

hi @justindett,

You can use INDEXED_EXTRACTIONS to parse these logs with JSON events. Set below configs in props.conf on the forwarder.

[sourcetype_name] SHOULD_LINEMERGE = true INDEXED_EXTRACTIONS = json KV_MODE = none AUTO_KV_JSON = false TIMESTAMP_FIELDS = time

If this reply helps you, an upvote/like would be appreciated.

Re: Time format for log

justindett — Mon, 15 Feb 2021 12:22:16 GMT

Thanks, I'll give that a try.

Re: Time format for log

scelikok — Mon, 15 Feb 2021 12:50:43 GMT

Hi @justindett,

Did try searching these logs with "All Time"? I don't think a way that Splunk does not ingest, most probably ingesting with wrong timestamp. For exapmle, Jul 12nd, Aug 12nd, Sep 12nd and Nov 12nd ...

Maybe you should update your TIME_FORMAT in your props.conf will work. If you can share your setting I will try to help.

Re: Time format for log

justindett — Mon, 15 Feb 2021 13:04:46 GMT

Hi,

I selected all time and still nothing. The props.conf is as follows as per manjunathmeti

[sanport:dcm]
SHOULD_LINEMERGE = true
INDEXED_EXTRACTIONS = json
KV_MODE = none
AUTO_KV_JSON = false
TIMESTAMP_FIELDS = time