Hi,
I am struggling with some logs in a specific directory. They just don't seem to be ingested into splunk.
If I put a normal .log file in with a standard time format it populates just fine.
But these logs have the following format:
O", "message": "Test logging" } { "time": "2020-12-07 09:46:52.7940", "threadId": "30", "level": "INFO", "message": "Test logging" } { "time": "2020-12-07 12:14:34.7402", "threadId": "53", "level": "INFO", "message": "Test logging" } { "time": "2020-12-07 13:48:24.8650", "threadId": "12", "level": "INFO", "message": "Test logging" } { "time": "2020-12-08 10:33:40.0607", "threadId": "68", "level": "INFO", "message": "Test logging" } { "time": "2020-12-08 11:53:56.7778", "threadId": "51", "level": "INFO", "message": "Test logging" } { "time": "2020-12-09 08:42:53.6465", "threadId": "133", "level": "INFO", "message": "Test logging" } { "time": "2020-12-09 10:35:44.0103", "threadId": "152", "level": "INFO", "message": "Test logging" } { "time": "2020-12-11 10:38:27.0194", "threadId": "113", "level": "INFO", "message": "Test logging" } { "time": "2020-12-11 12:18:25.0442", "threadId": "6", "level": "INFO", "message": "Test logging" }
And nothing comes into splunk at all. I have commented out all the timestamp options in the props.conf to force it to use default manner ,but still nothing at all.
Is it related to a setting that should be in the props.conf?
Any assistance would be appreciated.
Thanks
Hi @justindett,
Did try searching these logs with "All Time"? I don't think a way that Splunk does not ingest, most probably ingesting with wrong timestamp. For exapmle, Jul 12nd, Aug 12nd, Sep 12nd and Nov 12nd ...
Maybe you should update your TIME_FORMAT in your props.conf will work. If you can share your setting I will try to help.
Hi,
I selected all time and still nothing. The props.conf is as follows as per manjunathmeti
[sanport:dcm] SHOULD_LINEMERGE = true INDEXED_EXTRACTIONS = json KV_MODE = none AUTO_KV_JSON = false TIMESTAMP_FIELDS = time
hi @justindett,
You can use INDEXED_EXTRACTIONS to parse these logs with JSON events. Set below configs in props.conf on the forwarder.
[sourcetype_name]
SHOULD_LINEMERGE = true
INDEXED_EXTRACTIONS = json
KV_MODE = none
AUTO_KV_JSON = false
TIMESTAMP_FIELDS = time
If this reply helps you, an upvote/like would be appreciated.
Thanks, I'll give that a try.