Getting Data In

Updated : JSON event break for multiple source type and single event in multiple lines.

AKG1_old1
Builder

Hi,

I am uploading json file having 2 different source type and single event is split multiple lines.

I am very close to solution but having issue with a small problem.

Problem: I am not be able to use these together in props.conf. only 1 is working at a time.

1.
SEDCMD-remove_bracket = s/\[|\]//g
SEDCMD-remove_coma = s/\},/}\n/g
2.
MUST_BREAK_AFTER = \}

JSON logs after removing extra charaters(" [ , ] , }, ") to convert it in correct Json format

  {
      "name" : "CounterpartyNotional",
      "type" : "RiskBreakdown",
      "duration" : 20848,
      "count" : 1,
      "average" : 20848.0
    }
 {
      "name" : "CounterpartyPreSettlement",
      "type" : "RiskBreakdown",
      "duration" : 15370,
      "count" : 1,
      "average" : 15370.0
    }
     {
      "name" : "TraderCurrency",
      "type" : "Formula",
      "duration" : 344,
      "count" : 1,
      "average" : 344.0
    } 

alt text

transform.conf

[reset_profiler_formula]
DEST_KEY = MetaData:Sourcetype
REGEX = \"type\" : \"RiskBreakdown\"
FORMAT = sourcetype::reset_profiler:risks

[reset_profiler_risks]
DEST_KEY = MetaData:Sourcetype
REGEX = \"type\" : \"Formula\"
FORMAT = sourcetype::reset_profiler:formula

prop.conf

[reset_profiler]
SEDCMD-remove_bracket = s/\[|\]//g
SEDCMD-remove_coma = s/\},/}\n/g
KV_MODE = json
NO_BINARY_CHECK = true
MUST_BREAK_AFTER = \}
TRANSFORMS-sourcetye_routing = reset_profiler_formula,reset_profiler_risks

[reset_profiler:formula]
KV_MODE= json

[reset_profiler:risks]
KV_MODE= json

Thanks

0 Karma

somesoni2
Revered Legend

Give this a try

props.conf (on indexer/heavy forwarder whichever comes first)

[reset_profiler]
SHOULD_LINEMERGE = false
LINE_BREAKER = (\r\n]+)(?=\[\s*\{)
DATETIME_CONFIG = CURRENT
TRANSFORMS-sourcetye_routing = reset_profiler_st_override

transforms.conf (on indexer/heavy forwarder whichever comes first)

[reset_profiler_st_override]
DEST_KEY = MetaData:Sourcetype
 REGEX =\"type\"\s*\:\s*\"([^\"]+)\"
 FORMAT = sourcetype::reset_profiler:$1

props.conf (on search head)

[reset_profiler:formula]
 KV_MODE= json

 [reset_profiler:risks]
 KV_MODE= json
0 Karma

AKG1_old1
Builder

@somesoni : Thanks for reply but now all events in one file taken as single event.

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...