You can't automate a conditional delimiter-based field extraction, but you can define one unique field extraction per type of xferlog event and select which one to apply using the extract command.
First, define an automatic field extraction for the "operation" field, the value of which should determine which delimiter-based extraction we will apply.
props.conf:
[xferlog]
EXTRACT-operation = ^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}[^|]*\|\s+(? \w),
Now in transforms.conf, define one delimiter-based field extraction for each type of event. As an example, based on the xferlog reference page here's one definition for the S and R log entry types and one for the T (directory listing) log entry type.
transforms.conf:
[fields_store_retrieve]
DELIMS = ","
FIELDS = "Head","Pathname","Size","Duration","Rate","User","Email","Host","Suffix","Completion","Transfer_Type","Transfer_notes","Start_of_transfer","Session_ID","Starting_size","Starting_offset"
[fields_dirlist]
DELIMS = ","
FIELDS = "Head","Pathname","Completion","Pattern","Recursion","User","Email","Host","Session ID"
Now when you search, apply the appropriate field extraction depending on the value of the "operation" field of the events you are querying :
sourcetype=xferlog operation=R OR operation=S | extract fields_store_retrieve
or:
sourcetype=xferlog operation=T | extract fields_dirlist
It's too bad that one cannot define automatic field extractions based on event types because this would have been an ideal use-case for that.
... View more
