Getting Data In

Updated : JSON event break for multiple source type and single event in multiple lines.

AKG1_old1
Builder

Hi,

I am uploading json file having 2 different source type and single event is split multiple lines.

I am very close to solution but having issue with a small problem.

Problem: I am not be able to use these together in props.conf. only 1 is working at a time.

1.
SEDCMD-remove_bracket = s/\[|\]//g
SEDCMD-remove_coma = s/\},/}\n/g
2.
MUST_BREAK_AFTER = \}

JSON logs after removing extra charaters(" [ , ] , }, ") to convert it in correct Json format

  {
      "name" : "CounterpartyNotional",
      "type" : "RiskBreakdown",
      "duration" : 20848,
      "count" : 1,
      "average" : 20848.0
    }
 {
      "name" : "CounterpartyPreSettlement",
      "type" : "RiskBreakdown",
      "duration" : 15370,
      "count" : 1,
      "average" : 15370.0
    }
     {
      "name" : "TraderCurrency",
      "type" : "Formula",
      "duration" : 344,
      "count" : 1,
      "average" : 344.0
    } 

alt text

transform.conf

[reset_profiler_formula]
DEST_KEY = MetaData:Sourcetype
REGEX = \"type\" : \"RiskBreakdown\"
FORMAT = sourcetype::reset_profiler:risks

[reset_profiler_risks]
DEST_KEY = MetaData:Sourcetype
REGEX = \"type\" : \"Formula\"
FORMAT = sourcetype::reset_profiler:formula

prop.conf

[reset_profiler]
SEDCMD-remove_bracket = s/\[|\]//g
SEDCMD-remove_coma = s/\},/}\n/g
KV_MODE = json
NO_BINARY_CHECK = true
MUST_BREAK_AFTER = \}
TRANSFORMS-sourcetye_routing = reset_profiler_formula,reset_profiler_risks

[reset_profiler:formula]
KV_MODE= json

[reset_profiler:risks]
KV_MODE= json

Thanks

0 Karma

somesoni2
Revered Legend

Give this a try

props.conf (on indexer/heavy forwarder whichever comes first)

[reset_profiler]
SHOULD_LINEMERGE = false
LINE_BREAKER = (\r\n]+)(?=\[\s*\{)
DATETIME_CONFIG = CURRENT
TRANSFORMS-sourcetye_routing = reset_profiler_st_override

transforms.conf (on indexer/heavy forwarder whichever comes first)

[reset_profiler_st_override]
DEST_KEY = MetaData:Sourcetype
 REGEX =\"type\"\s*\:\s*\"([^\"]+)\"
 FORMAT = sourcetype::reset_profiler:$1

props.conf (on search head)

[reset_profiler:formula]
 KV_MODE= json

 [reset_profiler:risks]
 KV_MODE= json
0 Karma

AKG1_old1
Builder

@somesoni : Thanks for reply but now all events in one file taken as single event.

0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...