Which is the best way to extract fields: field extractor, rex and eval commands, or props and transforms?

[vkakani60]

Out of three ways to extract the fields,
1. BY using rex or eval command in search
2. By using field extractor option
3. By adding entries to the prop.conf and transforms.conf

Currently, I am using 1st method, by using rex commands such as

index="PriorityLogs" sourcetype="WebPay" | rex field=_raw "ProcessId: (?<ProcessId>.*)" |  rex field=_raw "EventId: (?<EventId>.*)" 

which extracts the desired ProcessID with 99.9% of the events.

Coming to the field extractor, my logs don't have a proper structure which leads to poor field extractions when i use the Splunk field extractor.

Which one is the best method?

[vkakani60 · Accepted solution]

After going through couple of splunk answers and documents. Search time field extractions are good compared to the index-time field extractions (editing conf files).

https://answers.splunk.com/answers/5817/search-time-versus-index-time-field-extractions.html
https://answers.splunk.com/answers/57247/index-time-field-extraction.html

[jeremiahc4]

I've found ".*" to be a very generous grab, usually netting me everything up to the next line return (\r) or newline (\n) character.
You might have better luck limiting that in some fashion. Something like ".+" (slightly less generous grab) or "[\w\d]*" (doesn't grab spaces and various punctuation) might get you decent results.

[somesoni2]

Technically, option 2 and option 3 are same (both are 'saved search time field extraction in configuration files'). The option 2 IFX provides an interactive method to do field extraction and can be used if you're not proficient with regular expression.

The option 1 and 3 is for advanced users who are good at regular expression. I'm guessing that efficiency of regex (99.9% ) is your issue here, so if you can provide sample data for which your inline extraction is working and for which it's not, splunkers here may be able to help you with that.