Hi All,
I'm banging my head against a wall attempting to figure out why a SEDCMD inside of a props.conf on a UF isn't wanting to strip out the value I tell it to. We are wanting to strip out a hashed value from a log that is inside of a bracket (example below), as well as the brackets, with the SEDCMD. I am able to successfully test this command inside of the searchhead, but when I place it inside of the props.conf on the UF, I don't see it successfully implemented. I'm sure I'm missing something pretty simple. I've tried quite a few variations of this and no luck. Could anyone help me or possibly give me a hint as to what I could be doing wrong? Thank you all.
| rex mode=sed field=_raw "s/\[ecid: .+?\]//g"
[log4j]
SEDCMD-random=s/\[ecid: .+?\]//g
Sourcetype: log4j
[2020-06-24T10:02:08.590-04:00] [Server] [NOTIFICATION] [] [] [tid: 394025] [userId: <anonymous>] [ecid: 3956b675-4930-42d5-9e7d-94ca9013d2ea-0037ac42,0:26:74:38:2010:52:52:71:38] [APP: oraclediagent2] [partition-name: DOMAIN] [tenant-name: GLOBAL] [oracle.odi.runtime.MrepExtId: 38392028449]
Try this SEDCMD on your UF.
SEDCMD-ecid = s/(.*?)\[ecid: .+?\](.*)/\1\2/
Thanks for the reply Rich!
I recall in the past (6 or so months ago), I was able to place a SEDCMD in the props on a UF and saw the stripping of data. Did this change recently? By placing it in a props on the indexers, will this allow the data to be stripped BEFORE it enters the licensing phase? We are hoping to remove this large amount of unnecessary data before it hits this stage to limit ingestion.
Awesome. Thanks!
Yes, it was on the UF of our Syslog relay farm. It was a SEDCMD that obfuscated some sensitive data. Host -> Syslog -> Splunk Cloud
Try this SEDCMD on your UF.
SEDCMD-ecid = s/(.*?)\[ecid: .+?\](.*)/\1\2/
A little late on my reply, but it worked. Thanks Rich! I guess in some cases, we can SED on the UF.