topic Re: How to extract different sourcetypes from the same file in Splunk Search

How to extract different sourcetypes from the same file

splunkuzleuven — Fri, 07 Jun 2019 13:47:40 GMT

I'm wondering if/how I can do the following:

I have a JSON structured file that is being parsed perfectly as JSON, so I get all my fields.
However, there is a certain field, let's call it "message" that is a standard web Access log.

So currently, I'm reading the log file and parse it as a json, and that works perfectly, except that the value of "message" is just a textblob of course. How can I parse the "message" part as well, so that all fields are being extraced at indextime?
Can I somehow combine sourcetypes?
I want to have all fields related to the same event though so that searching is easier.

Is this possible? Or will I need to JSON my "message" part as well before writing it to a log file?

Re: How to extract different sourcetypes from the same file

martynoconnor — Sat, 08 Jun 2019 21:48:10 GMT

You can't parse part of an event as one sourcetype and another part as a different sourcetype. If it's necessary to read it as that particular sourcetype you should split the content off before it reaches Splunk and read it in separately under a different sourcetype. If that's not an option you could (though I really don't recommend it) extend the JSON sourcetype definition to include additional field extractions in props.conf

Re: How to extract different sourcetypes from the same file

VatsalJagani — Sun, 09 Jun 2019 05:19:45 GMT

@splunkuzleuven - If you already have control over writing log file then I would suggest that you write message part as JSON only. As doing props and transforms will be a performance downside in this case as it will require to go through the _raw event with regex.

Re: How to extract different sourcetypes from the same file

splunkuzleuven — Tue, 11 Jun 2019 07:48:38 GMT

Okay, it's what I was afraid of already. Thanks for the feedback.