Getting Data In

Why is this linebreak is not working with JSON data?

Communicator

Any ideas why this linebreak is not working with JSON data? I've even set the sourcetype to _json, but still no luck.

[prd_thousand_eyes_rest]
INDEXED_EXTRACTIONS = json
**BREAK_ONLY_BEFORE={|**
CHARSET=UTF-8
MAX_TIMESTAMP_LOOKAHEAD=300
SHOULD_LINEMERGE=false
category=Structured
disabled=false
pulldown_type=true

_raw
[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]
0 Karma

Splunk Employee
Splunk Employee

There are 2 main problems here:

1) The _raw data is not JSON. JSON should look like

{"key1":"value1", "key2":"value2"}

Instead, you have

{|field1=value1||field2=value2|}

2) The _raw data looks like it has arrays of collections. Are you collecting this data programmatically, or reading it from a file? If reading from a file, you could use props.conf to strip out the square brackets using SEDCMD, set up your line breaker, and regex the keys and values. If you are getting this data programmatically, you could format it before streaming it over to Splunk.

SplunkTrust
SplunkTrust

BREAK_ONLY_BEFORE only applies when SHOULD_LINEMERGE=true.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Contributor

SHOULDLINEMERGE = false
LINE
BREAKER = ([\r\n]+)[\{|

0 Karma

Communicator

thanks, unfortunetly still seeing the same behavior. not breaking before "{|type" with in the json bracket.

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

[{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}{|type=HTTP Server||ruleName=Default HTTP Alert Rule||ResponseCode=200||testId=126574||testName=RAPIDReviewer - Prod||active=1||alertId=13123086||ruleExpression=Error Type is any||dateStart=2018-03-11 06:10:38||ruleId=387415||violationCount=3|}]

0 Karma