assign sourcetype based upon file name

[a212830]

Hi,

Is there a way to dynamically assign sourcetype based upon input filename?

[mbonsack_splunk]

alacercogitatus's answer does work, though the separators for the MetaData keys are single colons, not double, i.e.

DEST_KEY = MetaData:Sourcetype
SOURCE_KEY = MetaData:Source

After that it worked great!

[jrodman]

alacercogitatus's answer is certainly correct. You can set the sourcetype at parsing time (typically on your indexer) with props and transforms. For some problems this is the way to go.

However, if you want the sourcetype set up from the beginning, so you can control things like the "CHARSET" (encoding), NO_BINARY_CHECK and so on, then you can also control the sourcetype set at the input layer in tailing. For this to work, you need to be willing to put configuration on your forwarder, and it would look something like

props.conf:

[source::< pattern >]
sourcetype = what_you_want

< pattern > is whatever you need to match the file. The simplest thing that gets everything you want and doesn't get false positives is kind of a local decision, but often it looks something like:

    [source::/var/log/my_app/*.log]

    sourcetype = my_app

If you use both these techniques, then the value set at the input layer will be used until it is possibly modified by the props/transforms pass. Usually though, you want to use one approach or the other for a particular dataset, lest you make things too confusing.

[alacercogitatus]

There is. You will need to specify this in your props/transforms files any where indexing is being performed.

props.conf
[source::...regex_to_match_filename]
TRANSFORMS-fs = force-sourcetype-st

transforms.conf
[force-sourcetype-st]
DEST_KEY = MetaData::Sourcetype
SOURCE_KEY = MetaData::Source
REGEX = YOUR_REGEX_TO_PULL_THE_FILENAME
FORMAT = sourcetype::$1
WRITE_META = true

Referances
http://docs.splunk.com/Documentation/Splunk/latest/admin/Transformsconf
http://docs.splunk.com/Documentation/Splunk/latest/admin/Propsconf