Getting Data In

parsing logs from logstash

gcusello
SplunkTrust
SplunkTrust

at all,

I have to parse logs extracted from logstash.

I'm receiving logstash logs and they are in json format and almost all the fields I need are already parsed and available in json.

My issue is that the event rawdata is in a field called "message" and these fields aren't automatically extracted as I would.

I'd like to avoid to re-parse all datasources and create custom add-ons from all data sources.

Does anybody encounter this kind of integration and know a way to use standard Add-Ons to parse only the message field?

Thank you for your help.

Ciao.

Giuseppe

Labels (2)
Tags (3)

_JP
Contributor

Just some thoughts from a philosophical perspective...

Splunk loves to parse/extract/search data, and the overall architecture to me lets us treat comput and storage as a total commodity in the pursuit of "searching and making sense of our data."  So let Splunk do it's thing...if you have to do some extra parsing, do some extra parsing to get the problem solved.  Then optimize. That's sort of my same coding philosophy that I just pull forward into what I do in Splunk:  get it working, then get it working well.

So if you get it working...and it isn't the slowest thing in your environment, then let all of your distributed compute do its thing until the "cost" of your time to optimize outweighs the "cost" of the extra processing time spent running your query/extracts.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @_JP,

I am conceptually agree with you, but the customer already has logs on logstash and wants to use Enterprise Security, that uses CIM.

For this reason I have to ingest and parse logstash data, trying to persuade customer to pass to Universal Forwarders.

I asked to the Community if someone has already addressed this problem, to have some hint or attention point.

Anyway, working by myself, I already reconducted some data flows to standard add-ons.

Thank you for your answer.

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...