Getting Data In

Updated : JSON event break for multiple source type and single event in multiple lines.

AKG1_old1
Builder

Hi,

I am uploading json file having 2 different source type and single event is split multiple lines.

I am very close to solution but having issue with a small problem.

Problem: I am not be able to use these together in props.conf. only 1 is working at a time.

1.
SEDCMD-remove_bracket = s/\[|\]//g
SEDCMD-remove_coma = s/\},/}\n/g
2.
MUST_BREAK_AFTER = \}

JSON logs after removing extra charaters(" [ , ] , }, ") to convert it in correct Json format

  {
      "name" : "CounterpartyNotional",
      "type" : "RiskBreakdown",
      "duration" : 20848,
      "count" : 1,
      "average" : 20848.0
    }
 {
      "name" : "CounterpartyPreSettlement",
      "type" : "RiskBreakdown",
      "duration" : 15370,
      "count" : 1,
      "average" : 15370.0
    }
     {
      "name" : "TraderCurrency",
      "type" : "Formula",
      "duration" : 344,
      "count" : 1,
      "average" : 344.0
    } 

alt text

transform.conf

[reset_profiler_formula]
DEST_KEY = MetaData:Sourcetype
REGEX = \"type\" : \"RiskBreakdown\"
FORMAT = sourcetype::reset_profiler:risks

[reset_profiler_risks]
DEST_KEY = MetaData:Sourcetype
REGEX = \"type\" : \"Formula\"
FORMAT = sourcetype::reset_profiler:formula

prop.conf

[reset_profiler]
SEDCMD-remove_bracket = s/\[|\]//g
SEDCMD-remove_coma = s/\},/}\n/g
KV_MODE = json
NO_BINARY_CHECK = true
MUST_BREAK_AFTER = \}
TRANSFORMS-sourcetye_routing = reset_profiler_formula,reset_profiler_risks

[reset_profiler:formula]
KV_MODE= json

[reset_profiler:risks]
KV_MODE= json

Thanks

0 Karma

somesoni2
Revered Legend

Give this a try

props.conf (on indexer/heavy forwarder whichever comes first)

[reset_profiler]
SHOULD_LINEMERGE = false
LINE_BREAKER = (\r\n]+)(?=\[\s*\{)
DATETIME_CONFIG = CURRENT
TRANSFORMS-sourcetye_routing = reset_profiler_st_override

transforms.conf (on indexer/heavy forwarder whichever comes first)

[reset_profiler_st_override]
DEST_KEY = MetaData:Sourcetype
 REGEX =\"type\"\s*\:\s*\"([^\"]+)\"
 FORMAT = sourcetype::reset_profiler:$1

props.conf (on search head)

[reset_profiler:formula]
 KV_MODE= json

 [reset_profiler:risks]
 KV_MODE= json
0 Karma

AKG1_old1
Builder

@somesoni : Thanks for reply but now all events in one file taken as single event.

0 Karma
Get Updates on the Splunk Community!

SplunkTrust Application Period is Officially OPEN!

It's that time, folks! The application/nomination period for the 2025 SplunkTrust is officially open! If you ...

Splunk Answers Content Calendar, June Edition II

Get ready to dive into Splunk Dashboard panels this week! We'll be tackling common questions around ...

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...