Out of three ways to extract the fields,
1. BY using
eval command in search
2. By using
field extractor option
3. By adding entries to the
Currently, I am using 1st method, by using rex commands such as
index="PriorityLogs" sourcetype="WebPay" | rex field=_raw "ProcessId: (?<ProcessId>.*)" | rex field=_raw "EventId: (?<EventId>.*)"
which extracts the desired ProcessID with 99.9% of the events.
Coming to the field extractor, my logs don't have a proper structure which leads to poor field extractions when i use the Splunk field extractor.
Which one is the best method?
Technically, option 2 and option 3 are same (both are 'saved search time field extraction in configuration files'). The option 2 IFX provides an interactive method to do field extraction and can be used if you're not proficient with regular expression.
The option 1 and 3 is for advanced users who are good at regular expression. I'm guessing that efficiency of regex (99.9% ) is your issue here, so if you can provide sample data for which your inline extraction is working and for which it's not, splunkers here may be able to help you with that.
I've found ".*" to be a very generous grab, usually netting me everything up to the next line return (\r) or newline (\n) character.
You might have better luck limiting that in some fashion. Something like ".+" (slightly less generous grab) or "[\w\d]*" (doesn't grab spaces and various punctuation) might get you decent results.
After going through couple of splunk answers and documents. Search time field extractions are good compared to the index-time field extractions (editing conf files).