I found this, but I am unable to replicate it.
I am not understanding where I am messing up here.
Problem:
I feed btool outputs into splunk, and chop them up by stanza.
/opt/splunk/etc/apps/Splunk_TA_windows/default/transforms.conf [xmlsecurity_eventcode_errorcode_action_lookup]
/opt/splunk/etc/system/default/transforms.conf CAN_OPTIMIZE = True
/opt/splunk/etc/system/default/transforms.conf CLEAN_KEYS = True
/opt/splunk/etc/system/default/transforms.conf DEFAULT_VALUE =
/opt/splunk/etc/system/default/transforms.conf DEPTH_LIMIT = 1000
/opt/splunk/etc/system/default/transforms.conf DEST_KEY =
/opt/splunk/etc/system/default/transforms.conf FORMAT =
/opt/splunk/etc/system/default/transforms.conf KEEP_EMPTY_VALS = False
/opt/splunk/etc/system/default/transforms.conf LOOKAHEAD = 4096
/opt/splunk/etc/system/default/transforms.conf MATCH_LIMIT = 100000
/opt/splunk/etc/system/default/transforms.conf MV_ADD = False
/opt/splunk/etc/system/default/transforms.conf REGEX =
/opt/splunk/etc/system/default/transforms.conf SOURCE_KEY = _raw
/opt/splunk/etc/system/default/transforms.conf WRITE_META = False
/opt/splunk/etc/apps/Splunk_TA_windows/default/transforms.conf case_sensitive_match = false
/opt/splunk/etc/apps/Splunk_TA_windows/default/transforms.conf filename = xmlsecurity_eventcode_errorcode_action.csv
I then wanted to extract the fields, for example "SOURCE_KEY = _raw" should be my key/value pair!
I hoped to accomplish this with
(transforms)
[dotheparsething]
REGEX = \s([\S-]+)\s=\s([^\/\n]+)
LOOKAHEAD = 100000
FORMAT = $1::$2
REPEAT_MATCH = true
(props)
[(?::){0}splunk:config:btool:*]
TRUNCATE=10000
MAX_EVENTS=10000
KV_MODE = none
BREAK_ONLY_BEFORE = conf[\s]+\[
#SEDCMD-removespaces = s/\ +/\ /g
REPORT-dotheparsething = dotheparsething
But I am getting nothing!
Regex101 seems happy with my search.
Did you know btool-equivalent data is available via REST? Check out
| rest /services/configs/conf-<<conf>>
where <<conf>> is a config file name (props, transforms, etc.).
When you say you are getting nothing, what exactly does that mean? How are you looking for the something? Recall the REPORT settings extract fields at search time as opposed to TRANSFORMS settings which apply at index time.
What sourcetype is specified in your inputs.conf?
Wow. I had no idea that was a thing! It never crossed my mind.
It wont work in this situation, as I use it to keep track of forwarders and stuff rest wouldn't have access to, but that is fascinating. Might be worth writing my own script to push out instead of the btool app...
Here is one of my input stanzas:
[script:///opt/splunk/etc/apps/btool/bin/btool.sh transforms /opt/splunk] _rcvbuf = 1572864
disabled = 0
host = cn02-k
index = splunk_insights
interval = 43200
sourcetype = splunk:config:btool:transforms
And I know my existing props works, because the rest of my stanza has been working:
[(?::){0}splunk:config:btool:*]
TRUNCATE=10000
MAX_EVENTS=10000
KV_MODE = none
BREAK_ONLY_BEFORE = conf[\s]+\[
REPORT-dotheparsething = dotheparsething <-----except this critter.
By "getting nothing", I mean that I was expecting my regex to go through and convert the output in the first post to key/value pairs when I search. I have this running on a single instance of splunk, so there's no search head/indexer confusions. So, given:
/opt/splunk/etc/apps/Splunk_TA_windows/default/transforms.conf [xmlsecurity_eventcode_errorcode_action_lookup]
/opt/splunk/etc/system/default/transforms.conf CAN_OPTIMIZE = True
/opt/splunk/etc/system/default/transforms.conf CLEAN_KEYS = True
/opt/splunk/etc/system/default/transforms.conf DEFAULT_VALUE =
/opt/splunk/etc/system/default/transforms.conf DEPTH_LIMIT = 1000
/opt/splunk/etc/system/default/transforms.conf DEST_KEY =
/opt/splunk/etc/system/default/transforms.conf FORMAT =
/opt/splunk/etc/system/default/transforms.conf KEEP_EMPTY_VALS = False
I am hoping to get
Match - $1 CAN_OPTIMIZE
Match - $2 TRUE
Match - $1 CLEAN_KEYS
Match - $2 TRUE
Match - $1 DEPTH_LIMIT
Match - $2 1000
Match - $1 KEEP_EMPTY_VALS
Match - $2 FALSE
Key=Value --> $1::$2
CAN_OPTIMIZE = TRUE
CLEAN_KEYS = TRUE
DEPTH_LIMIT = 1000
KEEP_EMPTY_VALS = FALSE
I was able to get it to work by removing the regex from the stanza name in props.conf.
[splunk:config:btool:*]
TRUNCATE=10000
MAX_EVENTS=10000
KV_MODE = none
BREAK_ONLY_BEFORE = conf[\s]+\[
REPORT-dotheparsething = dotheparsething
DATETIME_CONFIG = current
No, changing the regex just kills the whole stanza, because there's no wildcards in sourcetypes.
When my "break only before" is removed, each line is a new event, and splunk auto extracts the single fields.
I want the exact same extractions, but with the whole stanza as a single event
Perhaps it's a version thing, but it works in 9.0.0.1. Sourcetype splunk:config:btool:transforms was parsed from stanza [splunk:config:btool:*].