Why is the field extraction transformation not working in conjunction with data filtering transformation?

[alanzchan]

I'm trying to minimize the amount of data from Kubernetes JSON events that are being indexed into my Splunk instance. Rather than having the whole JSON which includes headers and mostly unimportant metadata, I want to only display the raw text payload in my Splunk event viewer. However, this filters out the metadata that I actually find useful so I created a field extraction transformation in an attempt to add the metadata I need to my events before doing my filtering logic.

I've tried the above, but it doesn't work when they are in combination.

Is it possible to add, for example key1=value1 in _meta and have this field extracted for all events even if the events would not have it in _raw?

[jnudell_2]

If you want to keep the metadata and only display the message field at search time, you could just update your props.conf to display only the message part of the data:
props.conf

[my_sourcetype]

EVAL-_raw = <your json path to message>

For the example in the other Answers question you referenced it would be (https://answers.splunk.com/answers/659626/modify-raw-but-keep-extracted-fields.html):

[my_sourcetype]

EVAL-_raw = 'fields.message'

[alanzchan]

Applied this in my props.conf, but unfortunately it doesn't work.

[woodcock]

Why not just click on the All Fields, search for your field, click the checkbox, and then see that it is BOTH a Selected field shown in aggregate on the left side of your events and ALSO an in-line callout field at the bottom of each event. I think that this meets your desire without having to change anything.

[FrankVl]

If it is really just about how it shows in the event view in Splunk Web, then you could just add an | eval _raw = message to your search query (assuming you have that message extracted as a field called message). And if that is to your liking, then you could also implement that as a calculated field in your props.conf.

[alanzchan]

I would prefer not to do it during search time if possible, but this would achieve a somewhat desirable result.

[FrankVl]

You prefer not to do it during search time, as in: you don't like doing it manually as part of the query? Or would even moving it to a search time field calculation in props.conf (such that it happens automatically) not be preferred?

[woodcock]

See my new answer at the bottom.

[woodcock]

Sure you can, especially if it is a static value. See here:
https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureindex-timefieldextraction

So like this:

[my_sourcetype]
FORMAT = key1::value1
WRITE_META = true

[alanzchan]

The value in this case would not be static.

The problem here is that I only want the message of the log event to show in my event preview on my search head, not the whole JSON structure which means I then have to click [+] each time I want to see my log message.

Is there anyway that I can “bind” the metadata to their events without having the data in raw?

Hope that make sense, but my gut feeling that’s me there’s no way to do this.

[woodcock]

wait, you would like to replace all of your event ( _raw ) with just the part that is in message and throw the rest away forever?

[alanzchan]

Let’s put it this way, is it possible to just display a portion of _raw and leave the rest out. I don’t want to discard the other information, but I don’t want it clogging up my single event where I have to click [+] three times just to see the text payload. I prefer not to do this during search time. Can I edit a Splunk event viewer configuration or something?

[alanzchan]

Please check out this question from another Splunk user for a better description of my problem: https://answers.splunk.com/answers/659626/modify-raw-but-keep-extracted-fields.html

[woodcock]

Attention @mmodestino_splunk !!!

[mattymo]

HI @alanzchan!

How are you getting the data into Splunk? Are we talking Docker JSON driver logging here?

Our Splunk Connect for Kubernetes solution solves the problem of docker json driver cruft, and allows you to reshape and enrich the data inflight.

https://github.com/splunk/splunk-connect-for-kubernetes

[alanzchan]

I’m using the Google Cloud Platform add-on for Splunk. Kubernetes logs are sent in a JSON format and my goal is to display only the text payload of an event without the other metadata and attributes (not that they’re not important, but for convenience sake). However, If I filter out for just the text payload, I would lost out on all of the metadata which I would like to avoid. Can I modify some Splunk configuration? Any advice?

For another description of my problem, my question is identical to this one: https://answers.splunk.com/answers/659626/modify-raw-but-keep-extracted-fields.html

[mattymo]

Can you not deploy our collector to your k8s cluster instead??

It solves for all this pain already, ensuring only the log payload shows up in your events, but the key metadata fields are preserved as index time fields.

It is also going to be much more reliable and scalable than having to pull with a TA, and likely will optimize your storage/index volumes.

[alanzchan]

Thanks for you reply. I will work on this solution and provide an update shortly.

[alanzchan]

Hi @mmodestino_splunk,

Sorry for the super late reply. I'm testing Splunk Connect for Kubernetes right now, sending Kubernetes cluster container data to Splunk via HEC. The logs are still coming in as JSON format. Is there any way to process this data so it indexes as raw text?

Thanks,
Alan

[nickhills]

Can you share what you have so far?

[alanzchan]

[my_sourcetype]
TIME_PREFIX = "publish_time":
TIME_FORMAT = %s.%Q
TRANSFORMS-assign_sourcetype_access=access_log_to_st
TRANSFORMS-assign_sourcetype_application=application_log_to_st
TRANSFORMS-assign_index = change_index_by_application

TRANSFORMS-system_labels_transform = system_labels_transform

TRANSFORMS-change_format = change_format

Here is my props.conf.