I'm using props and transforms to extract fields, all the fields are extracted properly, except the first one.
This is mainly because my transforms is currently:
[extract_fields] DELIMS = "," FIELDS = "field1", "field2", "field3", etc
The first line of the event itself is as follows:
Dec 2 15:47:55 foo.bar.ie IRL:"field1","field2","field3"
So it extracts Dec 2 15:47:55 foo.bar.ie IRL:"field1" as the first field.
How would I go about ignoring that first part, so the field extraction starts after IRL:?
I was looking for some kind of field extraction parameter similar to TIME_PREFIX but for field extractions, but there doesn't seem to be one or I'm just not understanding the docs correctly.
Alternative using just the props.conf.
[extract_fields] EXTRACT-fieldval = IRL:(?<field1>.*),(?<field2>.*),(?<field3>.*)
more fields can be added by appending ",(?
so length seems to be an issue (hur hur), I'm trying to extract 35 fields, so the line in props is pretty long.
When I test it using only the first five fields, it works and extracts these fields as it should.
Is there a limit to the number of fields to be extracted in props....? am I reaching a character limit meaning the whole line is ignored?
I was restarting after every change. Using rex the fields seem to extract fine, not in props though, so:
rex "ASM:.(?<field1>[^,]*).,.(?<field2>[^,]*). props EXTRACT-fieldval = ASM:.(?<field1>[^,]*).,.(?<field2>[^,]*).
so maybe the problem lies with the fact my fields do have literal quotes around them, and I'm trying to match everything between the quotes.
Above I match the quotation marks with . instead of matching the literal quotation marks. I also tried the two suggested regexes, work with rex, not props.
permissions seem to be fine on the files as well.
Maybe the regex needs to be revised to this
EXTRACT-fieldval = IRL:(?<field1>.*?),(?<field2>.*?),(?<field3>.*?)
as the first regex is "greedy" and that sometimes causes problems. Also, this regex assumes that there are no actual quotation marks in the data.
A final thought: there are rules for field names. A field name can contain only letters, numbers and the underscore character. It must begin with a letter. If your names were violating these rules, that could be a problem. Yes, field names can have special characters in some places if you use quotation marks - but I wouldn't do it.
You do not need to restart splunk if your
props.conf changes are only field extractions. But if you want to explicitly reload the field extractions, you can run this command
| extract reload=true
Yes, this search string starts with a
I hope you have restarted your splunk instance after changing the props.conf file. For troubleshooting, you can try running the provided regex in search using "|rex" command. "index=yourindex sourcetyo=pe=source_log | rex \"paste the regex"
I tired this, both with the field surrounded by quotes and without as that's the way they are in the log, although using this method it doesn't seem to be extracting any fields.
edit: should also mention, i also tried to name the stanza as [source_log] in props, which is my sourcetype I want this applied to.