Getting Data In

How to override the sourcetype of events within the same source based on the event format?

norskedm
Explorer

I'm trying to override the sourcetype of events within the same source (for now, a file uploaded once and indexed - once I get it figured out, the source will be a scripted input from universal forwarders). I need to override the sourcetype of events in a source, based on the format of the event. If the event contains the word "share" (in a certain place) I'd like the source type to be "share"; likewise "dir" and "ext". The events' sourcetype defaults to "ext" (since this is the bulk of the events).

I am using Splunk documentation as a reference.

Here is a sample of the source data:

2014-08-11 22:14:54Z,foo900.example.com,share,seed,g:\seed,Disk,"General testing."
2014-08-11 17:14:54Z,foo900.example.com,dir,\\foo900.example.com\seed,182445977979,2014-07-17 17:00:28Z,2011-02-15 23:20:45Z
2014-08-11 17:14:54Z,foo900.example.com,ext,\\foo900.example.com\seed,.sgy,163108239992

Here is the props.conf:

[ext]
TRANSFORMS-change_sourcetype = transform_ext_sourcetype
SHOULD-LINEMERGE = false
REPORT-ext = transform_ext, transform_dir, transform_share

Here is the transforms.conf:

[transform_ext_sourcetype]
DEST_KEY = MetaData:Sourcetype
REGEX = [^,]+,[^,]+,(share|dir|ext),
FORMAT = sourcetype::$1

[transform_ext]
REGEX = (?<datetime>[^,]+),(?<hostname>[^,]+),ext,(?<share>[^,]+),(?<file_ext>[^,]*),(?<bytes_used>[^$]+)$

[transform_share]
REGEX = (?<datetime>[^,]+),(?<hostname>[^,]+),share,(?<share_name>[^,]+),(?<path>[^,]*),(?<share_type>[^,]+),(?<share_remarks>[^$]+)$

[transform_dir]
REGEX = (?<datetime>[^,]+),(?<hostname>[^,]+),dir,(?<share>[^,]+),(?<bytes_used>[^,]+),(?<last_access_datetime>[^,]+),(?<creation_datetime>[^$]+)$
0 Karma
1 Solution

norskedm
Explorer

The sourcetype had to be set for incoming data. This can be done in inputs.conf, as documented.

Since I was bringing everything in as sourcetype=ext, my inputs.conf looks like this

[foo.txt]
sourcetype=ext

View solution in original post

norskedm
Explorer

The sourcetype had to be set for incoming data. This can be done in inputs.conf, as documented.

Since I was bringing everything in as sourcetype=ext, my inputs.conf looks like this

[foo.txt]
sourcetype=ext

norskedm
Explorer

@lguinn The problem is it didn't work. A very-important part of the puzzle was left out of the documentation

0 Karma

lguinn2
Legend

What is the problem?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...