Getting Data In

Updated : JSON event break for multiple source type and single event in multiple lines.

AKG1_old1
Builder

Hi,

I am uploading json file having 2 different source type and single event is split multiple lines.

I am very close to solution but having issue with a small problem.

Problem: I am not be able to use these together in props.conf. only 1 is working at a time.

1.
SEDCMD-remove_bracket = s/\[|\]//g
SEDCMD-remove_coma = s/\},/}\n/g
2.
MUST_BREAK_AFTER = \}

JSON logs after removing extra charaters(" [ , ] , }, ") to convert it in correct Json format

  {
      "name" : "CounterpartyNotional",
      "type" : "RiskBreakdown",
      "duration" : 20848,
      "count" : 1,
      "average" : 20848.0
    }
 {
      "name" : "CounterpartyPreSettlement",
      "type" : "RiskBreakdown",
      "duration" : 15370,
      "count" : 1,
      "average" : 15370.0
    }
     {
      "name" : "TraderCurrency",
      "type" : "Formula",
      "duration" : 344,
      "count" : 1,
      "average" : 344.0
    } 

alt text

transform.conf

[reset_profiler_formula]
DEST_KEY = MetaData:Sourcetype
REGEX = \"type\" : \"RiskBreakdown\"
FORMAT = sourcetype::reset_profiler:risks

[reset_profiler_risks]
DEST_KEY = MetaData:Sourcetype
REGEX = \"type\" : \"Formula\"
FORMAT = sourcetype::reset_profiler:formula

prop.conf

[reset_profiler]
SEDCMD-remove_bracket = s/\[|\]//g
SEDCMD-remove_coma = s/\},/}\n/g
KV_MODE = json
NO_BINARY_CHECK = true
MUST_BREAK_AFTER = \}
TRANSFORMS-sourcetye_routing = reset_profiler_formula,reset_profiler_risks

[reset_profiler:formula]
KV_MODE= json

[reset_profiler:risks]
KV_MODE= json

Thanks

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Give this a try

props.conf (on indexer/heavy forwarder whichever comes first)

[reset_profiler]
SHOULD_LINEMERGE = false
LINE_BREAKER = (\r\n]+)(?=\[\s*\{)
DATETIME_CONFIG = CURRENT
TRANSFORMS-sourcetye_routing = reset_profiler_st_override

transforms.conf (on indexer/heavy forwarder whichever comes first)

[reset_profiler_st_override]
DEST_KEY = MetaData:Sourcetype
 REGEX =\"type\"\s*\:\s*\"([^\"]+)\"
 FORMAT = sourcetype::reset_profiler:$1

props.conf (on search head)

[reset_profiler:formula]
 KV_MODE= json

 [reset_profiler:risks]
 KV_MODE= json
0 Karma

AKG1_old1
Builder

@somesoni : Thanks for reply but now all events in one file taken as single event.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...