Getting Data In

Modify _raw but keep extracted fields


Hi All,

Today I encapsulate system logs in a JSON structure in order to add metadata that I would like to add to Splunk:

Raw field:

      "message": 2018-05-21T04:36:52.685 WARN: This is a warning",

The actual log line is in the "message" field. Because this is JSON, Splunk parses it easily and extracts the fields nicely.

Now my issue is when I perform searches on this data, having the whole JSON structure in the _raw field is very cumbersome and prevents me from using the normal events viewer to browse the logs (because of all the extra information around the data i'm interested in)

My questions:
- Is there a way (via transforms/props) to replace _raw by the field "message" while keeping the fields saved at index time?
-If the above solution is not possible/not clean, is there a way for me to somehow send metadata on top of _raw when i send the data via TCP?

Thanks in advance,

0 Karma


If you want to keep the metadata, and only show the message in the search results you could try this in your props.conf:


EVAL-_raw = 'fields.message'

This will keep the other field information that was extracted, but only show the fields.message JSON data as the event.

0 Karma

Path Finder

I will suggest you to use " | fields - _raw" in your search to remove the raw data on the search time .

0 Karma

Path Finder

Hi Benoit,

I am also running across this problem. If you found a solution, please share!


0 Karma

Super Champion

if you want entire raw text in a message field then you can use calculated fields knowledge objects.
go to Fields » Calculated fields » Add new
Put necessary details and in eval Expression put message=_raw and save.

let me know if this helps!

0 Karma


I would be more interested in _raw=message.
Do you think it would be possible with calculated fields?

0 Karma


The extracted fields from your JSON data are not created at index-time but at search-time, so somehow changing _raw from its JSON format will also break the field extraction.

What kind of metadata are you missing in your current _raw that you'd want to include?

0 Karma


Good to know for the JSON extraction.

For the metadata, some of the fields I'm interested in are:
- The original file as source (lost as I'm sending data via TCP)
- The Site where the data is coming from (I thought about having one TCP input configured for each, but it's not very easy to maintain)

There might be extra fields.

0 Karma
Get Updates on the Splunk Community!

Starting With Observability: OpenTelemetry Best Practices

Tech Talk Starting With Observability: OpenTelemetry Best Practices Tuesday, October 17, 2023   |  11AM PST / ...

.conf23 | Get Your Cybersecurity Defense Analyst Certification in Vegas

We’re excited to announce a new Splunk certification exam being released at .conf23! If you’re going to Las ...

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW! Every day the list of sources Admins are responsible for gets bigger and bigger, often making ...