Getting Data In

How to create index field?

rashid47010
Communicator

Hi

we want an indexed field called ‘actual_server’ to indicate the hostname of the forwarder that passed us the data.

My initial thought process is there are might be two options to achieve this

1- hostname available in the logs. which I think is not correct

2- write the system hostname in transforms.conf

I will create an app on CM and roll out this props.conf and transforms.conf against sourcetype=testlog

[testlog]
TRANSFORMS-netscreen = example
[example1]
WRITE_META=true
FORMAT = actual_server::FORWARDER1

and on search head

ields.conf

Add the following lines to fields.conf:

[actual_server]
INDEXED=true

Is this correct ?

 

Labels (1)
0 Karma

Action01
Loves-to-Learn

We have used this app as a solution to add the forwarder name: https://github.com/aholzel/TA-add_forwarder_name 

0 Karma

jotne
Builder

We like to know the name of the HF server the data are passing trough, so we have this app on all our  HF server.

prosps.conf

[source::...]
TRANSFORMS_set_hf_server_name = set_hf_server_name

transforms.conf

[set_hf_server_name]
INGEST_EVAL = splunk_hf := splunk_server


This uses the server name, so we do not need to set it.  All data will then be searchable using 
splunk_hf=<something>

We do also do the same for all collector servers and set splunk_collector (for Syslog/HEC/Azure etc)

0 Karma

PickleRick
SplunkTrust
SplunkTrust

1. If the field value is not included in the raw event, you should set

INDEXED_VALUE=false

in fields.conf

2. If you want to identify particular forwarder by inserting a static value, you might consider adding _meta at input level on the forwarder. The only caveat is that if you wanna add multiple meta fields on the UF it can quickly get ugly.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @rashid47010,

as you can read at https://docs.splunk.com/Documentation/Splunk/9.1.0/Data/Configureindex-timefieldextraction, it's correct.

Only one question: did you tried to se the hostname in the input stanza od the Forwarder?

Ciao.

Giuseppe

0 Karma

meetmshah
Contributor

Hello @rashid47010 Yes this should work.

Note - you have mentioned TRANSFORMS-netscreen = example and have created a stanza as example1 (there is "1" extra in the stanza name, you may want to correct them).

 

Let me know if ^^ doesn't work

0 Karma

rashid47010
Communicator

hi @meetmshah 

below are my props and transform. Still index field is not showing. I have fresh Splunk AIO instance. and I am uploading example log file named access.log

below is sample event

Jul 20 2023 09:37:08 www1 sshd[1654]: Failed password for happy from 2.229.4.58 port 2111 ssh2

Props.conf

 

[newfield]

TRANSFORMS-test = test_newfield

transforms.conf

[test_newfield]

REGEX = sshd\[(\d+)\]

FORMAT = request::"$1"

INGEST_EVAL = splunk_orig_fwd=host_test

WRITE_META = true

0 Karma

rashid47010
Communicator

Yes, you are right, i recreate the config but still it is not working. Only missing part is updating fields.conf file

let me try again shortly.

please note that i have test splunk AIO server and i am uploading sample access.log file.

 

 

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...