Getting Data In

Need help on json data

sekhar463
Path Finder

i have below stanza to ingest json data file and added in deployment server as below an in HF added props.conf file 
initially  i have uploaded using splunk UI but getting events in one line

[monitor:///var/log/Netapp_testobject.json]
disabled = false
index = Test_index
sourcetype = Test_sourcetype

[Test_sourcetype]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
LINE_BREAKER=([{}\,\s]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
EVENT_BREAKER=([{}\,\s]+)
INDEXED_EXTRACTIONS=json
KV_MODE=json
TRUNCATE=0

json data looks like below:

[
{
"Name": "test name",
"Description": "",
"DNSHostname": "test name",
"OperatingSystem": "NetApp Release 9.1",
"WhenCreated": "2/13/2018 08:24:22 AM",
"distinguishedName": "CN=test name,OU=NAS,OU=AVZ Special Purpose,DC=corp,DC=amvescap,DC=net"
},
{
"Name": "test name",
"Description": "London DR smb FSX vserver",
"DNSHostname": "test name",
"OperatingSystem": "NetApp Release 9.13.0P4",
"WhenCreated": "11/14/2023 08:43:36 AM",
"distinguishedName": "CN=test name,OU=NAS,OU=AVZ Special Purpose,DC=corp,DC=amvescap,DC=net"
}
]





Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Your LINE_BREAKER (and EVENT_BREAKER - they work very similarily but on different levels) makes no sense.

This parameter is not used to find whole event. It is supposed to find and match (the part captured within the capture group is discarded as not belonging to either the preceeding or following event) the text which is _between_ events. That's why by default it matches ([\r\n]+) - it finds all sequences of continuous end of line characters, splits the stream where those sequences happen, and removes those sequences from the ingestion pipeline.

In your case the situation is more complicated since you're trying to do a Bad Thing (tm) which is approach the structured data with simple regex manipulation. You could try to define your LINE_BREAKER as

^}(,[\r\n]+){

Which would mean that Splunk is to break the events in those places where you have only "}," alone in the line and immediately after that another "{" starts (possibly having several empty lines in between). But you're running into a risk of:

1) Incorrectly spliting your event in case you have a more complicated json structure

2) Laving the beginning and dangling square brackets as parts of the events (well, this one could be mitigated be editing the regex further but by expense of increasing risk number 1.

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

@sekhar463,

what's your issue?

Anyway, load this sourcetype both on the UF and on the Search Head.

Ciao.

Giuseppe

0 Karma

sekhar463
Path Finder

issue there is a json data want to ingest in splunk as i have attached sample data

but its getting all in one single event.

how we can get as separate events after which is enclosed with { }

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sekhar463,

usually json events are a single event, if you want to separate, you have to define the LINE_BREAKER, the TIME_FORMAT and the TIME_PREFIX for your sourcetype

[your_sourcetype]
LINE_BREAKER = \{
TIME_FORMAT = %m/%d/%Y %I:%M:%S %p
TIME_PREFIX = \"WhenCreated\": \"

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...