Getting Data In

Transforms not splitting sourcetypes

EdgarAllenProse
Path Finder

So I am trying to take a single monitored log, and split sourcetypes based off of the terms SCAN, RECV, SEND. I created my props and transforms first. I made sure that splunk recognized the sourcetype in props, this was successfull. I created inputs stanza for the monitored file and the only sourcetype I see is test_barracuda, which was from the props stanza, I am not getting the split transforms should be doing.

inputs.conf

[monitor://C:\\Users\\eap\\Desktop\\security\\Splunk\\DEVELOPMENT\\Test_Ingest\\test_raw_spam.txt]
disabled = false
sourcetype = test_barracuda
index = test

props.conf

[test_barracuda]
CHARSET=AUTO
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
category=Custom
disabled=false
pulldown_type=true
TRANSFORMS-overridest = set_sourcetype

transforms.conf

[set_sourcetype]
REGEX = \d+\s+(SEND|SCAN|RECV)\s
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::test_$1

This is an all in one test instance, so I placed these in SPLUNK_HOME/etc/apps/search/local/

Order of implementation: props.conf --> transforms.conf --> stop splunk --> clear event data from index test --> start splunk --> inputs.conf --> restart splunk.

Splunk gets the data. In the correct index, but only in sourcetype=test_barracuda.
I check to see if the regex in transforms is correct:

Query

index=test sourcetype="test_barracuda" | rex field=_raw "\d+\s+(?P<st>SEND|SCAN|RECV)\s"

Query works and I get exactly as many events in the 3 correct st fields.

So I do btools on props and transforms (note I'm not seeing any errors during /debug/refresh or splunk restart)

btools

C:\Program Files\Splunk\bin>splunk cmd btool transforms list set_sourcetype --debug

C:\Program Files\Splunk\etc\apps\search\local\transforms.conf [set_sourcetype]
C:\Program Files\Splunk\etc\system\default\transforms.conf    CAN_OPTIMIZE = True
C:\Program Files\Splunk\etc\system\default\transforms.conf    CLEAN_KEYS = True
C:\Program Files\Splunk\etc\system\default\transforms.conf    DEFAULT_VALUE =
C:\Program Files\Splunk\etc\apps\search\local\transforms.conf DEST_KEY = MetaData:Sourcetype
C:\Program Files\Splunk\etc\apps\search\local\transforms.conf FORMAT = sourcetype::test_$1
C:\Program Files\Splunk\etc\system\default\transforms.conf    KEEP_EMPTY_VALS = False
C:\Program Files\Splunk\etc\system\default\transforms.conf    LOOKAHEAD = 4096
C:\Program Files\Splunk\etc\system\default\transforms.conf    MV_ADD = False
C:\Program Files\Splunk\etc\apps\search\local\transforms.conf REGEX = \d+\s+(SEND|SCAN|RECV)\s
C:\Program Files\Splunk\etc\system\default\transforms.conf    SOURCE_KEY = _raw
C:\Program Files\Splunk\etc\system\default\transforms.conf    WRITE_META = False
C:\Program Files\Splunk\etc\system\default\transforms.conf    [set_sourcetype_to_stash]
C:\Program Files\Splunk\etc\system\default\transforms.conf    CAN_OPTIMIZE = True
C:\Program Files\Splunk\etc\system\default\transforms.conf    CLEAN_KEYS = True
C:\Program Files\Splunk\etc\system\default\transforms.conf    DEFAULT_VALUE =
C:\Program Files\Splunk\etc\system\default\transforms.conf    DEST_KEY = MetaData:Sourcetype
C:\Program Files\Splunk\etc\system\default\transforms.conf    FORMAT = sourcetype::stash
C:\Program Files\Splunk\etc\system\default\transforms.conf    KEEP_EMPTY_VALS = False
C:\Program Files\Splunk\etc\system\default\transforms.conf    LOOKAHEAD = 4096
C:\Program Files\Splunk\etc\system\default\transforms.conf    MV_ADD = False
C:\Program Files\Splunk\etc\system\default\transforms.conf    REGEX = .
C:\Program Files\Splunk\etc\system\default\transforms.conf    SOURCE_KEY = _raw
C:\Program Files\Splunk\etc\system\default\transforms.conf    WRITE_META = False

C:\Program Files\Splunk\bin>splunk cmd btool props list test_barracuda --debug

C:\Program Files\Splunk\etc\apps\search\local\props.conf [test_barracuda]
C:\Program Files\Splunk\etc\system\default\props.conf    ANNOTATE_PUNCT = True
C:\Program Files\Splunk\etc\system\default\props.conf    AUTO_KV_JSON = true
C:\Program Files\Splunk\etc\system\default\props.conf    BREAK_ONLY_BEFORE =
C:\Program Files\Splunk\etc\system\default\props.conf    BREAK_ONLY_BEFORE_DATE = True
C:\Program Files\Splunk\etc\apps\search\local\props.conf CHARSET = AUTO
C:\Program Files\Splunk\etc\system\default\props.conf    DATETIME_CONFIG = \etc\datetime.xml
C:\Program Files\Splunk\etc\system\default\props.conf    HEADER_MODE =
C:\Program Files\Splunk\etc\system\default\props.conf    LEARN_MODEL = true
C:\Program Files\Splunk\etc\system\default\props.conf    LEARN_SOURCETYPE = true
C:\Program Files\Splunk\etc\system\default\props.conf    LINE_BREAKER_LOOKBEHIND = 100
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_DAYS_AGO = 2000
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_DAYS_HENCE = 2
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_DIFF_SECS_AGO = 3600
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_DIFF_SECS_HENCE = 604800
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_EVENTS = 256
C:\Program Files\Splunk\etc\system\default\props.conf    MAX_TIMESTAMP_LOOKAHEAD = 128
C:\Program Files\Splunk\etc\system\default\props.conf    MUST_BREAK_AFTER =
C:\Program Files\Splunk\etc\system\default\props.conf    MUST_NOT_BREAK_AFTER =
C:\Program Files\Splunk\etc\system\default\props.conf    MUST_NOT_BREAK_BEFORE =
C:\Program Files\Splunk\etc\apps\search\local\props.conf NO_BINARY_CHECK = true
C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION = indexing
C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-all = full
C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-inner = inner
C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-outer = outer
C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-raw = none
C:\Program Files\Splunk\etc\system\default\props.conf    SEGMENTATION-standard = standard
C:\Program Files\Splunk\etc\apps\search\local\props.conf SHOULD_LINEMERGE = true
C:\Program Files\Splunk\etc\system\default\props.conf    TRANSFORMS =
C:\Program Files\Splunk\etc\apps\search\local\props.conf TRANSFORMS-overridest = set_sourcetype
C:\Program Files\Splunk\etc\system\default\props.conf    TRUNCATE = 10000
C:\Program Files\Splunk\etc\apps\search\local\props.conf category = Custom
C:\Program Files\Splunk\etc\system\default\props.conf    detect_trailing_nulls = auto
C:\Program Files\Splunk\etc\apps\search\local\props.conf disabled = false
C:\Program Files\Splunk\etc\system\default\props.conf    maxDist = 100
C:\Program Files\Splunk\etc\system\default\props.conf    priority =
C:\Program Files\Splunk\etc\apps\search\local\props.conf pulldown_type = true
C:\Program Files\Splunk\etc\system\default\props.conf    sourcetype =

No conflicts, but I'm not seeing anything for the expected sourcetypes test_SEND, test_RECV, or test_SCAN

Any idea where I messed up?

events for testing

In case you want to test, here are 3 events that match the criteria for each expected sourcetype

 Feb 13 12:14:57 192.168.x.x outbound/smtp: 127.0.0.1 1487013294-09b08c0e0d22d7b0001-vy5CMk 0 0 SEND - 1 0112918C8063 250 2.6.0 <2050829162.21743.1487013293752.JavaMail@dc1prjasszap434.whc> [InternalId=91736206477550, Hostname=host.prod.outlook.com] 11518 bytes in 0.192, 58.365 KB/sec Queued mail for delivery #to#name-com.mail.protection.outlook.com[8.8.8.8]:25

 Feb 13 12:14:56 192.168.x.x  scan: mail2-3.place.com[8.8.8.8] 1487013294-09b08c0e0d22d7b0001-vy5CMk 1487013294 1487013296 SCAN - prvs=7217d0fa1f=services_noreply@place.com name@otherplace.com - 7 88 corporate SZ:3263 SUBJ:Message: Attempt to retrieve your User ID

 Feb 13 12:14:15 192.168.x.x  inbound/pass1: name.place1.com[8.8.8.8] 1487013254-09b08c0e0e22d7a0001-VY5SBA 1487013254 1487013255 RECV information=place2.com@thing.com name@thingy1.com 2 3 blacklist.org[8.8.8.8]

woodcock
Esteemed Legend

I am sure that you read this:
http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Advancedsourcetypeoverrides

Your configurations look fine so:
Have you restarted splunkd on every indexer that is receiving these events?
Have you sent NEW events to the indexers (Indexed data is IMMUTABLE; only NEW events, post restart, will have the new configurations applied; previously indexed events will not be changed)?

0 Karma

EdgarAllenProse
Path Finder

Sorry I missed this response, I did take your steps, when initially setting up, restarting splunkd, ingesting new data and all, but still no luck. I'm kind of at a loss haha, I may at this point submit a support ticket to splunk.

0 Karma

EdgarAllenProse
Path Finder

I tried the responses on similar questions, but they don't seem to be working.

0 Karma

EdgarAllenProse
Path Finder

I'm setting up a universal forwarder to see if I need to do this there first.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...