Splunk Search

How can I tag or mark _internal events from different environments?

sniderwj
Explorer

We have a requirement to collect data from testing enclaves (that have copies of production devices) to our primary Splunk environment. I have event data going to a separate index through a heavy forwarder. What I am concerned about is the Internal index. I will need to track if I have Splunk UFs on clients in the enclave. Is there a way to mark or tag the data coming through the Heavy Forwarder to indicate that it is coming from that testing environment?

Tags (1)
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

One of the options you have is to add a metadata field to the events, which will require you to update forwarder configuration whenever a host moves between environments. Here is an answer that describes the process.

An alternative approach is to create (and - the tricky part - maintain) a lookup file that maps host names to enclave at search time.

0 Karma

somesoni2
Revered Legend

Do the UFs on Testing enclaves follow any particular naming conventions (basically how can you differentiate a UF from Primary vs UF from testing enclave)? How are those UF's receiving configurations, deployment servers or direct?

0 Karma

sniderwj
Explorer

The flow of data from the Enclave is UF --> Heavy Forwarder --> Production Splunk --> Enclave Specific Index

They are copying devices into the enclave from production so we will have duplicate names (host names and domain names) between the enclaves and in production.

I guess the easiest place would be to hit the events at the HF layer but I'm unsure how I can do that.

0 Karma

niketn
Legend

Can you try the dbinspect command? Following is a sample which may suit your need.

| dbinspect index=_internal 
| stats sum(eventCount) as eventCount by splunk_server
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

.conf25 Global Broadcast: Don’t Miss a Moment

Hello Splunkers, .conf25 is only a click away.  Not able to make it to .conf25 in person? No worries, you can ...

Observe and Secure All Apps with Splunk

 Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

What's New in Splunk Observability - August 2025

What's New We are excited to announce the latest enhancements to Splunk Observability Cloud as well as what is ...