Getting Data In

Strip leading junk to index json

ntripp_element
Explorer

I have some json coming in from syslog and there's a bunch of added junk that's inhibiting the field extraction.

I tried 

transforms.conf

[eliminate_header]
REGEX=s/^[^{]+//g
DEST_KEY=queue
FORMAT=nullQueue

and

props.conf

[syslog_cue]
TRANSFORMS-t1=eliminate_header

and a couple other things but can't quite seem to get the outcome I want. Testing the regex works, but isn't helping in the way I need it to. 

Tags (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Please provide examples of the original data and the desired output.  It's almost impossible to solve regex problems without that.

That said, the REGEX line shown is not valid in transforms.conf.  Sed commands are not supported so the regex contains unescaped delimiter characters (slashes).

Additionally, FORMAT=nullQueue discards entire events that match the REGEX, not just the text that matches.

You probably want to use SEDCMD in props.conf.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Please provide examples of the original data and the desired output.  It's almost impossible to solve regex problems without that.

That said, the REGEX line shown is not valid in transforms.conf.  Sed commands are not supported so the regex contains unescaped delimiter characters (slashes).

Additionally, FORMAT=nullQueue discards entire events that match the REGEX, not just the text that matches.

You probably want to use SEDCMD in props.conf.

---
If this reply helps you, Karma would be appreciated.
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Rounding off the Splunk Dashboard Contest

What does a contest-winning Splunk dashboard look like? In this case, it isn't in a browser tab at all. It ...

A Four Part Event Series: AI + Observability: AI Agents, LLMs, Apps, & Infrastructure

AI + Observability: AI Agents, LLMs, Apps, & Infrastructure The rapid evolution of artificial intelligence ...