Getting Data In

assign sourcetype based upon file name

a212830
Champion

Hi,

Is there a way to dynamically assign sourcetype based upon input filename?

Tags (1)

mbonsack_splunk
Splunk Employee
Splunk Employee

alacercogitatus's answer does work, though the separators for the MetaData keys are single colons, not double, i.e.

DEST_KEY = MetaData:Sourcetype
SOURCE_KEY = MetaData:Source

After that it worked great!

0 Karma

jrodman
Splunk Employee
Splunk Employee

alacercogitatus's answer is certainly correct. You can set the sourcetype at parsing time (typically on your indexer) with props and transforms. For some problems this is the way to go.

However, if you want the sourcetype set up from the beginning, so you can control things like the "CHARSET" (encoding), NO_BINARY_CHECK and so on, then you can also control the sourcetype set at the input layer in tailing. For this to work, you need to be willing to put configuration on your forwarder, and it would look something like

props.conf:

[source::< pattern >]
sourcetype = what_you_want

< pattern > is whatever you need to match the file. The simplest thing that gets everything you want and doesn't get false positives is kind of a local decision, but often it looks something like:





[source::/var/log/my_app/*.log]
sourcetype = my_app




If you use both these techniques, then the value set at the input layer will be used until it is possibly modified by the props/transforms pass. Usually though, you want to use one approach or the other for a particular dataset, lest you make things too confusing.

alacercogitatus
SplunkTrust
SplunkTrust

There is. You will need to specify this in your props/transforms files any where indexing is being performed.

props.conf
[source::...regex_to_match_filename]
TRANSFORMS-fs = force-sourcetype-st

transforms.conf
[force-sourcetype-st]
DEST_KEY = MetaData::Sourcetype
SOURCE_KEY = MetaData::Source
REGEX = YOUR_REGEX_TO_PULL_THE_FILENAME
FORMAT = sourcetype::$1
WRITE_META = true

Referances
http://docs.splunk.com/Documentation/Splunk/latest/admin/Transformsconf
http://docs.splunk.com/Documentation/Splunk/latest/admin/Propsconf

Get Updates on the Splunk Community!

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...

Major Splunk Upgrade – Prepare your Environment for Splunk 10 Now!

Attention App Developers: Test Your Apps with the Splunk 10.0 Beta and Ensure Compatibility Before the ...

Stay Connected: Your Guide to June Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...