Getting Data In

Is it possible to have your sourcetype be determined at index-time based on host?

Explorer

Title pretty self explanatory.
The files that I am indexing are having their host be determined by the directory in which they are located in. In my case, it is the system's hostname. For sourcetype, I would like to have it be the type of device (router, firewall, switch, etc). Is there a way to have the sourcetype dynamically be determined based off of the host? For an example, am I able to have a .cvs file with the host names and their desired sourcetypes? There are over 100 different hosts so manually importing them would be a bit of a hassle as it is done daily.

Any help would be appreciated!

0 Karma
1 Solution

Legend

There is certainly a way to do what you want - in fact, there are several ways.

While you could set the sourcetype to the device type, I would not do that. Within Splunk, sourcetype is used to group data based on the format/fields within the data. By using sourcetype for a different purpose, you will lose a lot of the built-in reporting capabilities of the various Splunk apps. I strongly suggest that you reserve sourcetype for its intended use, and leverage the Splunk pre-trained sourcetypes as much as you can.

There is another way to obtain the device types, which I think is superior for your case. Create a csv file that contains the host names, and the needed information about each. The CSV file must have a header line, like the example below.

host,devicetype,mfg,location
ajax,firewall,cisco,san francisco
achilles,firewall,cisco,austin

Note that the CSV file can contain a variety of relatively static attributes. Upload the CSV to Splunk as a lookup file, then define the lookup and make it automatic. Once you have done this, you will be able to use the field devicetype in searches. At the same time, you will be able to reload the CSV file as needed to add/remove/update hosts.

I think this is the easiest way to accomplish what you want; it is also the most flexible as your environment changes and grows. Here is a tutorial on how to set up the lookup .

View solution in original post

Legend

There is certainly a way to do what you want - in fact, there are several ways.

While you could set the sourcetype to the device type, I would not do that. Within Splunk, sourcetype is used to group data based on the format/fields within the data. By using sourcetype for a different purpose, you will lose a lot of the built-in reporting capabilities of the various Splunk apps. I strongly suggest that you reserve sourcetype for its intended use, and leverage the Splunk pre-trained sourcetypes as much as you can.

There is another way to obtain the device types, which I think is superior for your case. Create a csv file that contains the host names, and the needed information about each. The CSV file must have a header line, like the example below.

host,devicetype,mfg,location
ajax,firewall,cisco,san francisco
achilles,firewall,cisco,austin

Note that the CSV file can contain a variety of relatively static attributes. Upload the CSV to Splunk as a lookup file, then define the lookup and make it automatic. Once you have done this, you will be able to use the field devicetype in searches. At the same time, you will be able to reload the CSV file as needed to add/remove/update hosts.

I think this is the easiest way to accomplish what you want; it is also the most flexible as your environment changes and grows. Here is a tutorial on how to set up the lookup .

View solution in original post