Splunk Search

Is it possible to add a field by source at index-time ?

magilbert1
Explorer

I would like to add a new field at index-time that will be visible in the list of events. In the same way as Host, source, sourcetype, etc ...
It can't be extracted from the log itself because the information does not appear in the _raw.

Example : [source :: C:\ABC\Log1.log]
Application = App1
[source :: C:\ABC\Log2.log]
Application = App2
[source :: C:\xyz\Log3.log]
Application = App3

The reason is to be able to quickly identify the origin of an event.
Considering that the source path is not enough for us.

I found two temporary solutions

  • To add the name of the app in from of the source path.
  • To add a calculated fields in the conf field. EVAL-APPLICATION = "App1"

Is someone have a better solution for me ?

Thanks

Tags (1)
0 Karma

cvssravan
Path Finder

To make the field visible in event list along with metadata, it doesn't necessarily have to be metadata field. If you can add it during the indexing time and make it appear in Interesting fields during search, you just have to mark it as a selected field and it will appear in your event list along with your metadata(i.e. host, sourcetype, source).

0 Karma

magilbert1
Explorer

Hi

You mean in my inputs.conf file?

Can you give me an example ?

thanks

0 Karma

magilbert1
Explorer

My client wants to see it in the Event list not just in statistic table for example.
He wants to see it just beside Host,sourcetype fields Ex : Host = abc Application = MY_APP

0 Karma

FrankVl
Ultra Champion

There is no need to add it at index time to have it visible in the field list on the left, as long as the field is extracted, you can mark it as a selected field and it will show up alongside host, sourcetype, etc.

magilbert1
Explorer

HI

Yeah I know that but the information I need appears nowhere in the logs. So I need to add it manually.
In my case : the application name.
I can't put it in selected field if I don't have the field indexed.

0 Karma

FrankVl
Ultra Champion

All the information you need to determine what app it is, is in the source field, right? So you can perfectly fine define a calculated field using a case statement (if it is not too many options) or set up an automated lookup.

Again: any extracted field can be part of selected fields, it doesn't have to be an indexed field.

magilbert1
Explorer

FrankVI

For some reason yesterday my fields was not visible in the fields list but now it works.
I can see the fields Application.

Everything is working as i wish.

Thanks you

0 Karma

FrankVl
Ultra Champion

What exactly is the reason for looking at adding this field at index-time?

You say you can't extract it from the log, since it is not in _raw. But if there is a clear mapping from source value to application, you could simply write search time configuration to set the application field based on the value of the source field. For example by setting up an automatic lookup that maps source values to application values.

Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...