Hi nancylawrence007,
I understand the issue but I'm still not sure what you exactly want from the examples provided. As far as I know regex cannot count delimiters, so that is not an option. Basically, you need to normalise the logs so they always have the same number of fields, for which the sed command is perfect. So either,
add an extra ";" between "field6;field_7" in log1 (32 fields), or
substitute the ";" before DATABASE01 field in log2 (31 fields), or
remove the ";DATABASE01" section (31 fields)
I'd recommend #1, as #2 means you would need to perform another regex on the substituted field to extract its two values, and #3 removes a field (though it could be extracted earlier). Anyway, the "rex mode=sed ..." run anywhere example below would do the trick (assuming log1's 6 & 7 fields are consistently a "number;ip_addr") for #1 option.
|stats count as logs | eval logs="Jun 18 14:10:57 lec05674568 ABC[455135]: 2015-06-18 14:10:57;indexserver;lec056741;XBE;06;30893;11.91.6.128;lec056287312;9580;9599;LEC Audit - SYSTEM Logins;CRITICAL;CONNECT;SYSTEM;;;;;;DAPNAPS;UNSUCCESSFUL;;;;;;invalid username or password;;5562789;;|Jun 18 14:10:57 lec05674568 ABC[455135]: 2015-06-18 14:10:57;indexserver;lec056741;XBE;06;30893;DATABASE01;11.91.6.128;lec056287312;9580;9599;LEC Audit - SYSTEM Logins;CRITICAL;CONNECT;SYSTEM;;;;;;DAPNAPS;UNSUCCESSFUL;;;;;;invalid username or password;;5562789;;"
| eval logs=split(logs, "|") |mvexpand logs
| rex mode=sed field=logs "s/((?:[^;]+;){5}\d+;)(\d+\.\d+\.\d+\.\d+;.*)/\1;\2/"
| eval log_fields=split(logs, ";") |eval field_count = mvcount(log_fields)
Now, both logs have the same number of fields (as the field_count shows) and whatever regex extraction you are using will work.
... View more