Hello
I set up custom field extractions for a facter app I created but it seems that it is not extracting the fields.
facter.sh outputs data almost in a field:value but uses "=>" instead of "=" so I had to create a REGEX to extract the data. Heres a sample of the output:
architecture => x86_64
augeasversion => 1.2.0
bios_release_date => 06/22/2012
bios_vendor => Phoenix Technologies LTD
bios_version => 6.00
blockdevice_fd0_size => 4096
blockdevice_hdc_size => 4294965248
blockdevice_sda_model => Virtual disk
blockdevice_sda_size => 85899345920
blockdevice_sda_vendor => VMware
boardmanufacturer => Intel Corporation
boardproductname => 440BX Desktop Reference Platform
boardserialnumber => None
facterversion => 1.7.4
My props.conf in SPLUNKHOME/etc/apps/myapp/local/:
[facter]
REPORT-facter=extract-puppet-fields
My transforms.conf in SPLUNKHOME/etc/apps/myapp/local/:
[extract-puppet-fields]
REGEX = [\t (),./0-9A-Z_a-z]{1,22}
FORMAT = $1::$2
My inputs.conf in SPLUNKHOME/etc/apps/myapp/default:
[script://./bin/facter.sh]
disabled = false
index = os
interval= 3600
sourcetype=facter
source=facter
Any ideas what I am missing? I can get the output of the script in inputs.conf when searching in splunkweb but none of the fields are being extracted.
As a side note: I had the extraction set to REGEX = (S+)s=>s(S+) as I saw it in an answer on another post but it didn't work either. I tested the one above in a regex tester before trying.
Thanks for the help!
Keep everything you originally posted but try this:
REGEX = ([^\s\n\r]+)\s+=>\s+([^\s\n\r]+)
MV_ADD = true
Try this alternate syntax:
[extract-puppet-fields]
REGEX = (?<_KEY_1>[^\s\n\r]+)\s+=>\s+(?<_VAL_1>[^\s\n\r]+)
MV_ADD = true
Keep everything you originally posted but try this:
REGEX = ([^\s\n\r]+)\s+=>\s+([^\s\n\r]+)
MV_ADD = true
Ok so with this in my transforms.conf in SPLUNKHOME/etc/apps/myapp/local:
[extract-puppet-fields]
REGEX = ([^\s\n\r]+)\s+=>\s+([^\s\n\r]+)
MV_ADD = true
FORMAT = $1::$2
Still extracts nothing but defaults
Did you deploy this to all of your Search Heads? That is the only thing I can think that could be wrong because the configurations are all correct.
Sorry for the delay, been working on other priorities, yes I did put this on my search heads. I just tested again searching on search head and it is extracting properly.
Thanks so much for the assistance, you are awesome!
I will test and post back an update once deployed. Thanks so much for your help sir!
In order for this to work, you need to have 2 RegEx capture groups
; the first will be used for $1
and the second for $2
. You have 0 capture groups set in you RegEx which is done by putting something inside parentheses: "(a captured group is here)".
So something like this for transforms.conf?
[extract-puppet-fields]
REGEX = (?<$1>[t (),./0-9A-Z_a-z]{1,22})
REGEX = (?<$2>[t (),./0-9A-Z_a-z]{1,22})
FORMAT = $1::$2
or just one line
[extract-puppet-fields]
REGEX = (?<$1>[t (),./0-9A-Z_a-z]{1,22}) (?<$2>[t (),./0-9A-Z_a-z]{1,22})
FORMAT = $1::$2
No, let's take the example you copied, which is REGEX = (\S+)\s=>\s(\S+)
. This says take a group of non-whitespace characters (\S+)
and capture it (but don't name the capture). Skip over 1 whitespace character followed by an equals sign followed by another whitespace character \s=>\s
and then take the next group of non-whitespace characters (\S+)
and capture it, which becomes your second captured group
(even though you have not explicitly named or numbered them). These 2 captured groups
are used on the FORMAT
line as $1
and $2
. You need to do it like that (if that is even what you are trying to do, which I am not sure).
I have one xml file
I want to extract (at search time) the fields/values IN BETWEEN and and throw away any of the lines before the very first and after the very last .
(In XML, the fields/values are located on each line in the form value)
4. Use the date in the ActionDate field and the time in the ActionTime field as the timestamp.
' <Interceptor>
<AttackCoords>-423423445345345.10742916222947</AttackCoords>
<Outcome>Inteccccn</Outcome>
<Infiltrators>20</Infiltrators>
<Enforcer>Iwildwood</Enforcer>
<ActionDate>2013-04-24</ActionDate>
<ActionTime>00:07:00</ActionTime>
<RecordNotes></RecordNotes>
<NumEscaped>0</NumEscaped>
<LaunchCoords>-80.23429525620114,24.08680387475695</LaunchCoords>
<AttackVessel>local</AttackVessel>
</Interceptor>'
below is my props.conf and transforms.conf
props.conf
[dreamcrusher]
BREAK_ONLY_BEFORE =
DATETIME_CONFIG =
NO_BINARY_CHECK = true
TIME_FORMAT =
TIME_PREFIX =
category = Custom
disabled = false
pulldown_type = true
PREAMBLE_REGEX = ^<\S+.*
REPORT-dream = dream
transforms.conf
[dream]
REGEX = ^<(.*?)>(\S+)<
when i check the events there are no search time extraction