Splunk Search

Different type of parcing in a single index

ethanthomas
Path Finder

The requirement is, there is a single index . Data in three different format and there is an InputType coming in the raw data to identify . Below is just an exmaple 

InputType="mBP" |name|employee id|username|address|designation | manager name

InputType="eMQ" |designation|years of emp|office location|department 

InputType="qCP" |department| department head| employee count | designations 

The data format is different in these cases . So the question is , it is posisble to have different parcing based on the InputType ? What is the solution for this ? Do i need to create new 3 indexes ? or wth in same index , how the 3 different parcing can be done ? what are the conf file changes required? Help is appreciated with different  possible solutions on this . 

Labels (1)
Tags (1)
0 Karma

ethanthomas
Path Finder

I have changed the props and transforms file to assin the source type as InputType . But still the source type coming is the one value defined in the input .conf file . Any issues ?

0 Karma

manjunathmeti
Champion

Post .conf files and sample data.

0 Karma

manjunathmeti
Champion

hi @ethanthomas,

You can index the data in three separate source types per InputType in a single index. If you have separate input monitors for each InputType raw data then you can specify sourcetype in the monitor stanza in inputs.conf.

[monitor:///path/to/fileInputType1]
index=index
sourcetype=sourcetype1

[monitor:///path/to/fileInputType2]
index=index
sourcetype=sourcetype2

 But if the raw data is coming in the same file then you can use props.conf and transforms.conf to route data to particular source type based InputType values.

Check this page: https://docs.splunk.com/Documentation/Splunk/8.1.2/Data/Advancedsourcetypeoverrides

 

If this reply helps you, a like would be appreciated.
 

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...