Getting Data In

How to override the sourcetype of events within the same source based on the event format?

norskedm
Explorer

I'm trying to override the sourcetype of events within the same source (for now, a file uploaded once and indexed - once I get it figured out, the source will be a scripted input from universal forwarders). I need to override the sourcetype of events in a source, based on the format of the event. If the event contains the word "share" (in a certain place) I'd like the source type to be "share"; likewise "dir" and "ext". The events' sourcetype defaults to "ext" (since this is the bulk of the events).

I am using Splunk documentation as a reference.

Here is a sample of the source data:

2014-08-11 22:14:54Z,foo900.example.com,share,seed,g:\seed,Disk,"General testing."
2014-08-11 17:14:54Z,foo900.example.com,dir,\\foo900.example.com\seed,182445977979,2014-07-17 17:00:28Z,2011-02-15 23:20:45Z
2014-08-11 17:14:54Z,foo900.example.com,ext,\\foo900.example.com\seed,.sgy,163108239992

Here is the props.conf:

[ext]
TRANSFORMS-change_sourcetype = transform_ext_sourcetype
SHOULD-LINEMERGE = false
REPORT-ext = transform_ext, transform_dir, transform_share

Here is the transforms.conf:

[transform_ext_sourcetype]
DEST_KEY = MetaData:Sourcetype
REGEX = [^,]+,[^,]+,(share|dir|ext),
FORMAT = sourcetype::$1

[transform_ext]
REGEX = (?<datetime>[^,]+),(?<hostname>[^,]+),ext,(?<share>[^,]+),(?<file_ext>[^,]*),(?<bytes_used>[^$]+)$

[transform_share]
REGEX = (?<datetime>[^,]+),(?<hostname>[^,]+),share,(?<share_name>[^,]+),(?<path>[^,]*),(?<share_type>[^,]+),(?<share_remarks>[^$]+)$

[transform_dir]
REGEX = (?<datetime>[^,]+),(?<hostname>[^,]+),dir,(?<share>[^,]+),(?<bytes_used>[^,]+),(?<last_access_datetime>[^,]+),(?<creation_datetime>[^$]+)$
0 Karma
1 Solution

norskedm
Explorer

The sourcetype had to be set for incoming data. This can be done in inputs.conf, as documented.

Since I was bringing everything in as sourcetype=ext, my inputs.conf looks like this

[foo.txt]
sourcetype=ext

View solution in original post

norskedm
Explorer

The sourcetype had to be set for incoming data. This can be done in inputs.conf, as documented.

Since I was bringing everything in as sourcetype=ext, my inputs.conf looks like this

[foo.txt]
sourcetype=ext

norskedm
Explorer

@lguinn The problem is it didn't work. A very-important part of the puzzle was left out of the documentation

0 Karma

lguinn2
Legend

What is the problem?

0 Karma
Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  &#x1f680; Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...