I want any logfile (local, or remote via a UniversalForwarder) with the filename "xyz.log" to have a sourcetype of XYZ, and get indexed in my xyz index (not the main index).
What do I need to put in props.conf?
Do I also need to configure transforms.conf?
I'm using Splunk Enterprise v8 on Windows.
current props.conf:
[source::...\\xyz.log]
sourcetype = XYZ
For the record,
remote UF inputs.conf:
[monitor://C:\pathname\xyz.log]
sourcetype = XYZ
index = xyz
disabled = 0
and Splunk Server props.conf:
[source::...\\xyz.log]
sourcetype = XYZ
doesn't do it; the data from remote UF gets the right sourcetype, but still gets indexed into main not xyz.
(The local xyz.log data coming from the Splunk Server directory gets the right sourcetype and goes into index xyz)
Something on the Splunk Server is over-riding the explicit index specification on the remote UF?
Going to try the other recommendation for props.conf and transforms.conf
In your input.conf on your UF, set a fixed sourcetype for your data, example "my_logdata"
Then on your Index server or HF add the following props and transforms add the following.
props.conf
[my_logdata]
TRANSFORMS-changesourcetype = changesourcetype-my_logdata
transforms.conf
[changesourcetype-my_logdata]
SOURCE_KEY = MetaData:Source
REGEX = ([^\\]+).log
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$1
We do use some like this to set sourcetype based on foldername
Fixed error in regex that did not work correctly
An even better regex to get just the file name (especially for long path name)
(?:[^\\]+\\)+([^.]+).log$
You could do a source entry in props.conf (source has higher priority than host and sourcetype) so that stanza should (with all additional remarks on config file priorities) get applied to all files for this source. But.
1) You would have to have this config on all your parsing components (which means first "heavy" - based on a full Splunk Enterprise package - instance in your event path. So probably all indexers and all HFs.
2) After initial ingestion the source field doesn't have to have anything to do with actual source file name. You can easily overwrite it with inputs.conf and vice versa - your syslog-received events can have a /tmp/whatever.log source because why not.
@PickleRick , it's a small system (mostly for dev work): everything on 1 Wintel Splunk Server; not distributed/clustered.
A couple of remote machines that use a UF.
Hi @NK,
did you tried to set up your inputs.conf:
[monitor:\\c:\your_path\xyz.log]
sourcetype = XYZ
index = xyz
disabled = 0
?
You can set up index and sourcetype in all the stanzas you want.
Ciao.
Giuseppe
@gcusello , that looks like a setup for local files.
(I have that set up via Splunk's "Data inputs" --> "Files & directories")
It does not seem to work for remote files coming in via the Splunk Universal Forwarder.
They may have different drive letters and pathnames, but the same filename (xyz.log).
Hi @NK,
for each input you can define index and sourcetype.
and anyway, you can override index and sourcetype on Indexers or (if present) on Heavy Forwarders.
Ciao.
Giuseppe