Getting Data In

parsing_err="No data" JSON Works in Add Data

lavster
Path Finder

Wondered if someone can assist me, we're trying to send some log files from AWS in JSON format, coming over as an event. ive copied the log into a text file, gone ADD DATA and initially it fails but then changing sourcetype to _json it formats it fine. However when trying to send the data in properly, i just get a parsing error, is there an easy way to identify whats causing this? the format is as follows.

 

{
"time": "1628855079519",
"host": "sgw-3451B77A",
"source": "share-114D5B31",
"sourcetype": "aws:storagegateway",
"sourceAddress": "xx.xx.xx.xx",
"accountDomain": "XXX",
"accountName": "server_name",
"type": "FileSystemAudit",
"version": "1.0",
"objectType": "File",
"bucket": "test-test-test",
"objectName": "/random-210813-1230.toSend",
"shareName": "test-test-test",
"operation": "ReadData",
"timestamp": "1333222111111",
"gateway": "aaa-XXXXXXA",
"status": "Success"
}

Labels (3)
Tags (3)
0 Karma

m_pham
Splunk Employee
Splunk Employee

Did you set any of the big six configurations below to help Splunk parse the data more efficiently (props.conf)?

[aws:storagegateway]

TIME_PREFIX =
MAX_TIMESTAMP_LOOKAHEAD =
TIME_FORMAT =
SHOULD_LINEMERGE = false
LINE_BREAKER =
TRUNCATE = 10000

I recommend you set "KV_MODE = json" instead of the default "KV_MODE = auto" for this sourcetype on your search head/search head cluster to prevent any potential issues with the "maxchars" config in limits.conf.

0 Karma

lavster
Path Finder

Thanks for getting back to me, i worked it out in the end.

As it was being sent through as an event, i had to wrap every KVP in "event":{} and that sorted it out. took quite a bit of work with curl.

0 Karma
Get Updates on the Splunk Community!

Splunk Enterprise Security 8.0.2 Availability: On cloud and On-premise!

A few months ago, we released Splunk Enterprise Security 8.0 for our cloud customers. Today, we are excited to ...

Logs to Metrics

Logs and Metrics Logs are generally unstructured text or structured events emitted by applications and written ...

Developer Spotlight with Paul Stout

Welcome to our very first developer spotlight release series where we'll feature some awesome Splunk ...