topic parsing_err="No data" JSON Works in Add Data in Getting Data In

parsing_err="No data" JSON Works in Add Data

lavster — Mon, 16 Aug 2021 06:31:57 GMT

Wondered if someone can assist me, we're trying to send some log files from AWS in JSON format, coming over as an event. ive copied the log into a text file, gone ADD DATA and initially it fails but then changing sourcetype to _json it formats it fine. However when trying to send the data in properly, i just get a parsing error, is there an easy way to identify whats causing this? the format is as follows.

{
"time": "1628855079519",
"host": "sgw-3451B77A",
"source": "share-114D5B31",
"sourcetype": "aws:storagegateway",
"sourceAddress": "xx.xx.xx.xx",
"accountDomain": "XXX",
"accountName": "server_name",
"type": "FileSystemAudit",
"version": "1.0",
"objectType": "File",
"bucket": "test-test-test",
"objectName": "/random-210813-1230.toSend",
"shareName": "test-test-test",
"operation": "ReadData",
"timestamp": "1333222111111",
"gateway": "aaa-XXXXXXA",
"status": "Success"
}

Re: parsing_err="No data" JSON Works in Add Data

m_pham — Mon, 16 Aug 2021 20:22:04 GMT

Did you set any of the big six configurations below to help Splunk parse the data more efficiently (props.conf)?

[aws:storagegateway]

TIME_PREFIX =
MAX_TIMESTAMP_LOOKAHEAD =
TIME_FORMAT =
SHOULD_LINEMERGE = false
LINE_BREAKER =
TRUNCATE = 10000

I recommend you set "KV_MODE = json" instead of the default "KV_MODE = auto" for this sourcetype on your search head/search head cluster to prevent any potential issues with the "maxchars" config in limits.conf.

Re: parsing_err="No data" JSON Works in Add Data

lavster — Tue, 17 Aug 2021 06:08:51 GMT

Thanks for getting back to me, i worked it out in the end.

As it was being sent through as an event, i had to wrap every KVP in "event":{} and that sorted it out. took quite a bit of work with curl.