Getting Data In

parsing_err="No data" JSON Works in Add Data

lavster
Path Finder

Wondered if someone can assist me, we're trying to send some log files from AWS in JSON format, coming over as an event. ive copied the log into a text file, gone ADD DATA and initially it fails but then changing sourcetype to _json it formats it fine. However when trying to send the data in properly, i just get a parsing error, is there an easy way to identify whats causing this? the format is as follows.

 

{
"time": "1628855079519",
"host": "sgw-3451B77A",
"source": "share-114D5B31",
"sourcetype": "aws:storagegateway",
"sourceAddress": "xx.xx.xx.xx",
"accountDomain": "XXX",
"accountName": "server_name",
"type": "FileSystemAudit",
"version": "1.0",
"objectType": "File",
"bucket": "test-test-test",
"objectName": "/random-210813-1230.toSend",
"shareName": "test-test-test",
"operation": "ReadData",
"timestamp": "1333222111111",
"gateway": "aaa-XXXXXXA",
"status": "Success"
}

Labels (3)
Tags (3)
0 Karma

m_pham
Splunk Employee
Splunk Employee

Did you set any of the big six configurations below to help Splunk parse the data more efficiently (props.conf)?

[aws:storagegateway]

TIME_PREFIX =
MAX_TIMESTAMP_LOOKAHEAD =
TIME_FORMAT =
SHOULD_LINEMERGE = false
LINE_BREAKER =
TRUNCATE = 10000

I recommend you set "KV_MODE = json" instead of the default "KV_MODE = auto" for this sourcetype on your search head/search head cluster to prevent any potential issues with the "maxchars" config in limits.conf.

0 Karma

lavster
Path Finder

Thanks for getting back to me, i worked it out in the end.

As it was being sent through as an event, i had to wrap every KVP in "event":{} and that sorted it out. took quite a bit of work with curl.

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer at Splunk .conf24 ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...