Re: Why is this linebreak is not working with JSON...

fisuser1 · ‎04-20-2018

Any ideas why this linebreak is not working with JSON data? I've even set the sourcetype to _json, but still no luck.

[prd_thousand_eyes_rest]
INDEXED_EXTRACTIONS = json
**BREAK_ONLY_BEFORE={|**
CHARSET=UTF-8
MAX_TIMESTAMP_LOOKAHEAD=300
SHOULD_LINEMERGE=false
category=Structured
disabled=false
pulldown_type=true

_raw
[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

jconger · ‎04-20-2018

There are 2 main problems here:

1) The _raw data is not JSON. JSON should look like

{"key1":"value1", "key2":"value2"}

Instead, you have

{|field1=value1||field2=value2|}

2) The _raw data looks like it has arrays of collections. Are you collecting this data programmatically, or reading it from a file? If reading from a file, you could use props.conf to strip out the square brackets using SEDCMD, set up your line breaker, and regex the keys and values. If you are getting this data programmatically, you could format it before streaming it over to Splunk.

richgalloway · ‎04-20-2018

BREAK_ONLY_BEFORE only applies when SHOULD_LINEMERGE=true.

---
If this reply helps you, Karma would be appreciated.

ssadanala1 · ‎04-20-2018

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)[\{|

fisuser1 · ‎04-20-2018

thanks, unfortunetly still seeing the same behavior. not breaking before "{|type" with in the json bracket.

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

Why is this linebreak is not working with JSON data?

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Announcing Modern Navigation: A New Era of Splunk User Experience

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

Join the Conversation