Splunk Search

How do I ignore part of an event?

wilhelmF
Path Finder
16:02:33.5336 Info {"message":"ddfsd Transfer execution ended","level":"Information","logType":"Default","timeStamp":"2018-02-03T16:02:33.5278396+00:00","fingerprint":"63cf8b04-f818-49f0-a133-1f13775ed599","windowsIdentity":"fdg-SRV01\\dsfas","machineName":"TRADING-SRV01","processName":"fdg Transfer","processVersion":"1.0.6601.32033","fileName":"dfas","jobId":"f8347685-63e8-4f64-a4da-0518758d7ea5","robotName":"WILHELM_FRITSCHE","totalExecutionTimeInSeconds":10,"totalExecutionTime":"00:00:10"}

I have an event like above and I want to remove the "16:02:33.5336 Info" part so that it's recognized as a Json event. Where do I need to do this? In props.conf?

Tags (2)
0 Karma
1 Solution

493669
Super Champion

Try this run anywhere search:

|makeresults|eval_raw="16:02:33.5336 Info {\"message\":\"ddfsd Transfer execution ended\",\"level\":\"Information\",\"logType\":\"Default\",\"timeStamp\":\"2018-02-03T16:02:33.5278396+00:00\",\"fingerprint\":\"63cf8b04-f818-49f0-a133-1f13775ed599\",\"windowsIdentity\":\"fdg-SRV01dsfas\",\"machineName\":\"TRADING-SRV01\"}"|rex field=_raw mode=sed "s/^\d+:\d+:\d+\.\d+\s\w+\s\{/{/g"|spath

use SEDCMD in props.conf to remove first part till 'Info' from log.

[sourcetypename]
SEDCMD-start=s/^\d+:\d+:\d+\.\d+\s\w+\s\{/{/

View solution in original post

cpetterborg
SplunkTrust
SplunkTrust

If you put the following in your props.conf file, then you should be able to allow the JSON formatting that you want by making the event JSON string format compliant (at least by removing the non-JSON formatting at the front of the string):

[your-source-type-here]
SEDCMD-json-fix=s/^[^{]*//
...other configs for the sourcetype...

It will remove all characters that come before the first curly brace, no matter the format. You will still have to set the configs to do the JSON parsing to get the nice JSON formatting in the search results (which there is more than one way, and I'm assuming that you have settled on one of the ways already).

493669
Super Champion

Try this run anywhere search:

|makeresults|eval_raw="16:02:33.5336 Info {\"message\":\"ddfsd Transfer execution ended\",\"level\":\"Information\",\"logType\":\"Default\",\"timeStamp\":\"2018-02-03T16:02:33.5278396+00:00\",\"fingerprint\":\"63cf8b04-f818-49f0-a133-1f13775ed599\",\"windowsIdentity\":\"fdg-SRV01dsfas\",\"machineName\":\"TRADING-SRV01\"}"|rex field=_raw mode=sed "s/^\d+:\d+:\d+\.\d+\s\w+\s\{/{/g"|spath

use SEDCMD in props.conf to remove first part till 'Info' from log.

[sourcetypename]
SEDCMD-start=s/^\d+:\d+:\d+\.\d+\s\w+\s\{/{/

gcusello
SplunkTrust
SplunkTrust

Hi wilhelmF,
If you want to mask this data you can see at https://docs.splunk.com/Documentation/Splunk/latest/Data/Anonymizedata
or to delete this part from your log you have to put in props.conf

SEDCMD-firstpart= s/^\d+:\d+:\d+\.\d+\s\w+\s\{/^\{/

Bye.
Giuseppe

Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...