Getting Data In

Is there a way to configure Splunk to parse a sourcetype with mixed data types?

daniel333
Builder

All,

I have a log file which is largely key value, with some random human readable language tossed in. Recent upgrades have resulted in XML getting mixed into the party.

Does Splunk have a mode to which might be able to address the mixture of data types in the same logs?

%time% app=SOMETHING something=value message="Some human readable thing"
<value> 12  </value>
<nameofprod>Name </nameofprod>
dump is status="complete"
0 Karma

pgreer_splunk
Splunk Employee
Splunk Employee

Question for you pertaining to your example data - would it be multiple 'events' in a similar fashion as:

%time% app=SOMETHING something=value message="Some human readable thing"
<value> 12  </value>
<nameofprod>Name </nameofprod>
dump is status="complete" 
%time% app=SOMETHING something=value message="Some human readable thing"
<value> 12  </value>
<nameofprod>Name </nameofprod>
dump is status="complete"
....

If so, you could (after the data is already in the index) use transaction to encapsulate an entire 'stanza' as one event, then use other search commands to include/exclude what you want.

0 Karma

somesoni2
Revered Legend

How complex your xml portion can get?? Is it always just single level values??

0 Karma

daniel333
Builder

It's pretty standard. I've been temped to just SEDCMD away the opening tag with a key=" instead then completely remove the closing tag to just a closing ". But this seems like a lot of manual work. Would be nice is there was a "mixed type" auto extraction option? But if not, maybe the SED option?

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...