Getting Data In

Why is the field extraction transformation not working in conjunction with data filtering transformation?

alanzchan
Path Finder

I'm trying to minimize the amount of data from Kubernetes JSON events that are being indexed into my Splunk instance. Rather than having the whole JSON which includes headers and mostly unimportant metadata, I want to only display the raw text payload in my Splunk event viewer. However, this filters out the metadata that I actually find useful so I created a field extraction transformation in an attempt to add the metadata I need to my events before doing my filtering logic.

I've tried the above, but it doesn't work when they are in combination.

Is it possible to add, for example key1=value1 in _meta and have this field extracted for all events even if the events would not have it in _raw?

0 Karma

jnudell_2
Builder

If you want to keep the metadata and only display the message field at search time, you could just update your props.conf to display only the message part of the data:
props.conf

[my_sourcetype]
EVAL-_raw = <your json path to message>

For the example in the other Answers question you referenced it would be (https://answers.splunk.com/answers/659626/modify-raw-but-keep-extracted-fields.html):

[my_sourcetype]
EVAL-_raw = 'fields.message'

0 Karma

alanzchan
Path Finder

Applied this in my props.conf, but unfortunately it doesn't work.

0 Karma

woodcock
Esteemed Legend

Why not just click on the All Fields, search for your field, click the checkbox, and then see that it is BOTH a Selected field shown in aggregate on the left side of your events and ALSO an in-line callout field at the bottom of each event. I think that this meets your desire without having to change anything.

0 Karma

FrankVl
Ultra Champion

If it is really just about how it shows in the event view in Splunk Web, then you could just add an | eval _raw = message to your search query (assuming you have that message extracted as a field called message). And if that is to your liking, then you could also implement that as a calculated field in your props.conf.

alanzchan
Path Finder

I would prefer not to do it during search time if possible, but this would achieve a somewhat desirable result.

0 Karma

FrankVl
Ultra Champion

You prefer not to do it during search time, as in: you don't like doing it manually as part of the query? Or would even moving it to a search time field calculation in props.conf (such that it happens automatically) not be preferred?

0 Karma

woodcock
Esteemed Legend

See my new answer at the bottom.

0 Karma

woodcock
Esteemed Legend

Sure you can, especially if it is a static value. See here:
https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureindex-timefieldextraction

So like this:

[my_sourcetype]
FORMAT = key1::value1
WRITE_META = true

alanzchan
Path Finder

The value in this case would not be static.

The problem here is that I only want the message of the log event to show in my event preview on my search head, not the whole JSON structure which means I then have to click [+] each time I want to see my log message.

Is there anyway that I can “bind” the metadata to their events without having the data in raw?

Hope that make sense, but my gut feeling that’s me there’s no way to do this.

0 Karma

woodcock
Esteemed Legend

wait, you would like to replace all of your event ( _raw ) with just the part that is in message and throw the rest away forever?

0 Karma

alanzchan
Path Finder

Let’s put it this way, is it possible to just display a portion of _raw and leave the rest out. I don’t want to discard the other information, but I don’t want it clogging up my single event where I have to click [+] three times just to see the text payload. I prefer not to do this during search time. Can I edit a Splunk event viewer configuration or something?

0 Karma

alanzchan
Path Finder

Please check out this question from another Splunk user for a better description of my problem: https://answers.splunk.com/answers/659626/modify-raw-but-keep-extracted-fields.html

0 Karma

woodcock
Esteemed Legend

Attention @mmodestino_splunk !!!

0 Karma

mattymo
Splunk Employee
Splunk Employee

HI @alanzchan!

How are you getting the data into Splunk? Are we talking Docker JSON driver logging here?

Our Splunk Connect for Kubernetes solution solves the problem of docker json driver cruft, and allows you to reshape and enrich the data inflight.

https://github.com/splunk/splunk-connect-for-kubernetes

- MattyMo
0 Karma

alanzchan
Path Finder

I’m using the Google Cloud Platform add-on for Splunk. Kubernetes logs are sent in a JSON format and my goal is to display only the text payload of an event without the other metadata and attributes (not that they’re not important, but for convenience sake). However, If I filter out for just the text payload, I would lost out on all of the metadata which I would like to avoid. Can I modify some Splunk configuration? Any advice?

For another description of my problem, my question is identical to this one: https://answers.splunk.com/answers/659626/modify-raw-but-keep-extracted-fields.html

0 Karma

mattymo
Splunk Employee
Splunk Employee

Can you not deploy our collector to your k8s cluster instead??

It solves for all this pain already, ensuring only the log payload shows up in your events, but the key metadata fields are preserved as index time fields.

It is also going to be much more reliable and scalable than having to pull with a TA, and likely will optimize your storage/index volumes.

- MattyMo
0 Karma

alanzchan
Path Finder

Thanks for you reply. I will work on this solution and provide an update shortly.

0 Karma

alanzchan
Path Finder

Hi @mmodestino_splunk,

Sorry for the super late reply. I'm testing Splunk Connect for Kubernetes right now, sending Kubernetes cluster container data to Splunk via HEC. The logs are still coming in as JSON format. Is there any way to process this data so it indexes as raw text?

Thanks,
Alan

0 Karma

nickhills
Ultra Champion

Can you share what you have so far?

If my comment helps, please give it a thumbs up!
0 Karma

alanzchan
Path Finder

[my_sourcetype]
TIME_PREFIX = "publish_time":
TIME_FORMAT = %s.%Q
TRANSFORMS-assign_sourcetype_access=access_log_to_st
TRANSFORMS-assign_sourcetype_application=application_log_to_st
TRANSFORMS-assign_index = change_index_by_application

TRANSFORMS-system_labels_transform = system_labels_transform

TRANSFORMS-change_format = change_format

Here is my props.conf.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...