Getting Data In

parsing_err="No data" JSON Works in Add Data

lavster
Path Finder

Wondered if someone can assist me, we're trying to send some log files from AWS in JSON format, coming over as an event. ive copied the log into a text file, gone ADD DATA and initially it fails but then changing sourcetype to _json it formats it fine. However when trying to send the data in properly, i just get a parsing error, is there an easy way to identify whats causing this? the format is as follows.

 

{
"time": "1628855079519",
"host": "sgw-3451B77A",
"source": "share-114D5B31",
"sourcetype": "aws:storagegateway",
"sourceAddress": "xx.xx.xx.xx",
"accountDomain": "XXX",
"accountName": "server_name",
"type": "FileSystemAudit",
"version": "1.0",
"objectType": "File",
"bucket": "test-test-test",
"objectName": "/random-210813-1230.toSend",
"shareName": "test-test-test",
"operation": "ReadData",
"timestamp": "1333222111111",
"gateway": "aaa-XXXXXXA",
"status": "Success"
}

Labels (3)
Tags (3)
0 Karma

m_pham
Splunk Employee
Splunk Employee

Did you set any of the big six configurations below to help Splunk parse the data more efficiently (props.conf)?

[aws:storagegateway]

TIME_PREFIX =
MAX_TIMESTAMP_LOOKAHEAD =
TIME_FORMAT =
SHOULD_LINEMERGE = false
LINE_BREAKER =
TRUNCATE = 10000

I recommend you set "KV_MODE = json" instead of the default "KV_MODE = auto" for this sourcetype on your search head/search head cluster to prevent any potential issues with the "maxchars" config in limits.conf.

0 Karma

lavster
Path Finder

Thanks for getting back to me, i worked it out in the end.

As it was being sent through as an event, i had to wrap every KVP in "event":{} and that sorted it out. took quite a bit of work with curl.

0 Karma
Get Updates on the Splunk Community!

Good Sourcetype Naming

When it comes to getting data in, one of the earliest decisions made is what to use as a sourcetype. Often, ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Splunk App for Anomaly Detection End of Life Announcement

Q: What is happening to the Splunk App for Anomaly Detection?A: Splunk is officially announcing the ...