How to drop specific lines from an event not the whole event

[Mfmahdi]

we are getting WAF log and the events are very big we need to drop some lines from the events that has no meaningful value not the whole event.

@gcusello 

thank you in advance.

[kiran_panchavat]

@Mfmahdi Example: 

SEDCMD:

[your_sourcetype]
SEDCMD-drop_unwanted_lines = <regex>

Truncate:

[your_sourcetype]
TRUNCATE = 10000

Adjust the value to your desired maximum event length.

SEDCMD-<class> = <sed script>
* Only used at index time.
* Commonly used to anonymize incoming data at index time, such as credit
card or social security numbers. For more information, search the online
documentation for "anonymize data."
* Used to specify a sed script which Splunk software applies to the _raw
field.
* A sed script is a space-separated list of sed commands. Currently the
following subset of sed commands is supported:
* replace (s) and character substitution (y).
* Syntax:
* replace - s/regex/replacement/flags
* regex is a perl regular expression (optionally containing capturing
groups).
* replacement is a string to replace the regex match. Use \n for back
references, where "n" is a single digit.
* flags can be either: g to replace all matches, or a number to
replace a specified match.
* substitute - y/string1/string2/
* substitutes the string1[i] with string2[i]
* No default.

[gcusello · Accepted solution]

Hi @Mfmahdi,

you could truncate your events defining the max lenght of each event using the TRUNCATE option in props.conf.

Otherwise you could define a regex to exclude from each event the part that you don't want.

You should use the SEDCMD command in props.conf

For more infos see at https://docs.splunk.com/Documentation/Splunk/9.2.0/admin/Propsconf

Ciao.

Giuseppe

[gcusello]

Hi @Mfmahdi ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉