All Apps and Add-ons
Highlighted

Which Pipeline Uses "CIM" to convert the fields in Splunk?

Engager

I am trying to understand which pipeline deals with CIM? We have four sets of pipelines - is it the indexing pipeline which uses CIM or Parsing Pipeline. Please let me know.

Thanks
nandha

0 Karma
Highlighted

Re: Which Pipeline Uses "CIM" to convert the fields in Splunk?

Champion

This is kind of a big question, but I will try to be clear.

First, the Common Information Model (CIM) is not a specific step or process inside of Splunk. The CIM is basically a standardized set of fields, tags, and eventtypes. Think of it as a schema where the different data sources follow the rules of the schema by mapping their custom fields to the schema defined fields, e.g., aliasing a sourcetype-specific field "UserName" to the CIM-standard field "user".

In the most common cases, mapping sourcetype-specific fields to CIM is done at search-time, not at index-time. So my answer doesn't really meet your expectations, because search-time field extractions comes after the input, parsing, and indexing pipelines.

Now, one can do index-time field extraction to CIM-compliant names (which would be the parsing pipeline), but I would say that is not the norm. Most CIM-compliant mapping is done in the props.conf, transforms.conf, tags.conf, and eventtypes.conf files on a search-head.

More reading: http://docs.splunk.com/Documentation/CIM/4.6.0/User/UsetheCIMtonormalizedataatsearchtime

View solution in original post

Highlighted

Re: Which Pipeline Uses "CIM" to convert the fields in Splunk?

Engager

convincing for me.. thanks

0 Karma
Highlighted

Re: Which Pipeline Uses "CIM" to convert the fields in Splunk?

Champion

Glad to be helpful.

Please accept the answer if it satisfies your question.

0 Karma