parsing logs from logstash

[gcusello]

at all,

I have to parse logs extracted from logstash.

I'm receiving logstash logs and they are in json format and almost all the fields I need are already parsed and available in json.

My issue is that the event rawdata is in a field called "message" and these fields aren't automatically extracted as I would.

I'd like to avoid to re-parse all datasources and create custom add-ons from all data sources.

Does anybody encounter this kind of integration and know a way to use standard Add-Ons to parse only the message field?

Thank you for your help.

Ciao.

Giuseppe

[_JP]

Just some thoughts from a philosophical perspective...

Splunk loves to parse/extract/search data, and the overall architecture to me lets us treat comput and storage as a total commodity in the pursuit of "searching and making sense of our data."  So let Splunk do it's thing...if you have to do some extra parsing, do some extra parsing to get the problem solved.  Then optimize. That's sort of my same coding philosophy that I just pull forward into what I do in Splunk:  get it working, then get it working well.

So if you get it working...and it isn't the slowest thing in your environment, then let all of your distributed compute do its thing until the "cost" of your time to optimize outweighs the "cost" of the extra processing time spent running your query/extracts.

[gcusello]

Hi @_JP,

I am conceptually agree with you, but the customer already has logs on logstash and wants to use Enterprise Security, that uses CIM.

For this reason I have to ingest and parse logstash data, trying to persuade customer to pass to Universal Forwarders.

I asked to the Community if someone has already addressed this problem, to have some hint or attention point.

Anyway, working by myself, I already reconducted some data flows to standard add-ons.

Thank you for your answer.

Ciao.

Giuseppe