Getting Data In

How to check why sourcetype is not mapping correctly to child sourcetype?

donfernandez
Explorer

Has anyone experienced thier sourcetypes not mapping correctly during production deployment but in test/local it is mapping properly?
- How did you identify the culprit?
- I would like to try using btool but am not very familiar on how to write the syntax to show me the results I need

0 Karma
1 Solution

esix_splunk
Splunk Employee
Splunk Employee

When you say mapping correctly, do you mean aliasing, or do you mean using transforms at ingestion to rewrite the sourcetype before it is indexed? (based either on regex or by metadata perhaps..)

In the first case, sourcetype (ST) aliasing is done at search time and would thus be applied on your Search Head or Search Head Cluster.

The later, using props and transforms to change the data "in-flight", as in before its indexed to disk is a bit more tricky. You need to know where in the ingestion phase the data is cooked vs uncooked before index time. By this, I mean in complex deployments, there are typically intermediate forwarders before the indexers. Think a scenario like this:

UF ---->  HF -----> IDX

In this case, if you are ingesting data at the UF, and then have props on your IDX to remap the sourcetype, this wont work as the data is cooked at the HF layer. Those props / transforms need to go at the nearest HF to the UF, or ingestion point.

To this point, its working in your dev environment most likely because your dev instance is a single node all-in-one box.

For btool, good thought. This should be the very first place you check!

General syntax is quite simple..

./splunk btool config list

Here config is the config file you want to check, e.g., props, transforms, authorize, web, inputs, outputs etc. And list, well this does just that, it lists out the contents as Splunk sees them on disk.

In addition to this, you can specify specific stanzas in those config files if you know this..

./splunk btool indexes list _internal

This lists out the configuration for the _internal index as it is applied by Splunk from the configurations it reads. You can use this with props / transforms in order to drill down on specific sourcetypes and the configurations applied.

HAppy hunting!

View solution in original post

0 Karma

donfernandez
Explorer

We were able to find the root cause. Apparently the forwarder was configured to be a HF which was expecting the TA to be deployed in the HF as well.

So initially, it was communicated that we are working on a
UF -> IDX
turned out to be a
HF -> IDX
which makes the deployed TA in the IDX not able to map the sourcetype since it should have been deployed in HF.

Thanks for the great help @esix 🙂

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

General rule of thumb is that UF doesn’t do parsing. There are some exceptions to this with structured data sources like CSV and indexed extractions however.

So for you, these props and transforms do need to go on your indexers.

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

When you say mapping correctly, do you mean aliasing, or do you mean using transforms at ingestion to rewrite the sourcetype before it is indexed? (based either on regex or by metadata perhaps..)

In the first case, sourcetype (ST) aliasing is done at search time and would thus be applied on your Search Head or Search Head Cluster.

The later, using props and transforms to change the data "in-flight", as in before its indexed to disk is a bit more tricky. You need to know where in the ingestion phase the data is cooked vs uncooked before index time. By this, I mean in complex deployments, there are typically intermediate forwarders before the indexers. Think a scenario like this:

UF ---->  HF -----> IDX

In this case, if you are ingesting data at the UF, and then have props on your IDX to remap the sourcetype, this wont work as the data is cooked at the HF layer. Those props / transforms need to go at the nearest HF to the UF, or ingestion point.

To this point, its working in your dev environment most likely because your dev instance is a single node all-in-one box.

For btool, good thought. This should be the very first place you check!

General syntax is quite simple..

./splunk btool config list

Here config is the config file you want to check, e.g., props, transforms, authorize, web, inputs, outputs etc. And list, well this does just that, it lists out the contents as Splunk sees them on disk.

In addition to this, you can specify specific stanzas in those config files if you know this..

./splunk btool indexes list _internal

This lists out the configuration for the _internal index as it is applied by Splunk from the configurations it reads. You can use this with props / transforms in order to drill down on specific sourcetypes and the configurations applied.

HAppy hunting!

0 Karma

donfernandez
Explorer

hi esix.

It's thru transforms at ingestion..regarding the setup, we have something like UF(on-prem) --> IDX (cloud).. With this scenario, would it make any difference?

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...