Splunk Parser Ignore Datetime and Host Fields

[jmsilva9500]

Hello,

I'm struggling with finding a parser in splunk for the following log:

May 20 12:22:21 127.0.0.1 {"rootId": "AXIxikL8ao-yaSvA", "requestId": "f6a873jkjjkjk:-8000:5738", "details": {"flag": false, "title": "task 1", "status": "Waiting", "group": "", "order": 0}, "operation": "Creation", "objectId": "AXIyCN5Oao-H5aYyaSvd", "startDate": 1589977341890, "objectType": "case_task", "base": true, "object": {"_routing": "AXIxikL8ao-H5aYyaSvA", "flag": false, "_type": "case_task", "title": "task 1", "createdAt": 1589977341516, "_parent": "AXIxikL8ao-H5aYyaSvA", "createdBy": "user", "_id": "AXIyCN5Oao-H5aYyaSvd", "id": "AXIyCN5Oao-H5aYyaSvd", "_version": 1, "order": 0, "status": "Waiting", "group": ""}}

The log itself is a valid json, and i can parse it well with the default _json parser. However, splunk inserts the datetime and hostname at the beginning of the log, which makes the parser stop working..

Is there any workaround for this?

Thanks!

[to4kawa]

default _json parser is INDEXED_EXTRACTION=json
This can't work on using SEDCMD

props.conf

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
INDEXED_EXTRACTION = none
KV_MODE = json
TRUNCATE = 0
SEDCMD-trim = s/.*?{/{/

[richgalloway]

The parser works, but it doesn't work. Please explain.
How is the data getting to Splunk?

[jmsilva9500]

The data is being sent by remote syslog. the _json parser works for the log that is sent to splunk, but at indexing time splunk inserts at the beginning of the log the datetime and the host values, which makes the log itself no longer a valid json, therefore the parser fails

[richgalloway]

I think it's syslog that is adding the time and host values. Splunk also adds timestamp and host, but it's in metadata, not the raw data.

[jmsilva9500]

Do you know how i can parse to log giving that the time and host values are in the beggining of it?

[richgalloway]

Try using SEDCMD in props.conf to strip out the timestamp and host info. Should be easy with "s/^.*\{//".