Sorry for the cross post but after posting i saw a recommendation to use this forum instead of splunk.com
I am having issues getting REGEX to work my sample input is a simple two column csv for testing purposes (date,fakeip) the data looks like this:
8/31/2010 12:12:12,1.1.1.1
9/1/2010 12:12:12,1.1.1.1
9/2/2010 12:12:12,1.1.1.1
9/3/2010 12:12:12,1.1.1.1
my config files are as follows
props.conf
[c_netflowdata]
TRANSFORMS-foo=c_regex
Over the course of the past 8 hours I have tried a large number of different transforms files none which seem to do anything useful so there is obviously something that i am missing from the splunk documentation and peoples examples on this forum below are a few transforms.conf examples that i have tried. The have all produce varying results none of which have resulted in selectable fields in the Search app's field pick'em box
transforms.conf
[c_regex]
REGEX=(\d+/\d+/\d+\s\d+:\d+:\d+)
FORMAT=thedate::$1
[c_regex]
REGEX=(\d+/\d+/\d+\s\d+:\d+:\d+).*
FORMAT=thedate::$1
[c_regex]
REGEX=[^,],(\d+\.\d+\.\d+\.\d+)
FORMAT=src_ip::$1
[c_regex]
REGEX=(\d+/\d+/\d+\s\d+:\d+:\d+),(\d+\.\d+\.\d+\.\d+)
FORMAT=thedate::$1 src_ip::$2
I have successfully parsed the data using DELIMS and FIELDS - however there is a specific reason that I am trying to use REGEX - i.e. this is the first step in solving a larger issue that i am trying to address
Any help is appreciated, thanks in advance
then you definitely do not want to use index time extractions
yes thedate is just an example sorry for the confusion
Do you really want thedate
as a field, or is that just an example? If your data has been read in correctly, you should be able to use _time and format it out in any way you want, without using that. Otherwise, see answers below.
Yeap, this should work:
[c_regex]
REGEX=(\d+/\d+/\d+\s\d+:\d+:\d+)
FORMAT=thedate::$1
WRITE_META=true
Make sure that the data is actually coming in with sourcetype c_netflow.
When you specify your regex using TRANSFORMS-
in props.conf, this is run at index time and has different properties than search time regexes (specifically, the output needs to be written to _meta
with WRITE_META=true
). You can either specify it as REPORT-
in props.conf, and refer to transforms.conf. It's even easier to specify inline in props.conf:
[c_netflowdata]
EXTRACT-foo = (?<thedate>\d+/\d+/\d+\s\d+:\d+:\d+),(?<src_ip>\d+\.\d+\.\d+\.\d+)
I haven't attempted REPORT - but EXTRACT gets the job done. Thanks!