I wrote the above questions, then realized it may not matter much though there are several ways to do this. They're probably separate events, so I think we first combine them. We can use transaction to do that. I'm picking a 1 minute max interval between the first line and the last line of the log file to keep it more efficient - adjust as necessary.
... | transaction startswith="Control File" endswith="CPU time was" maxspan=1m
That should group the events together. Now, let's extract the data you need with rex. To the end of the above...
... | rex "Data File: (?<data_file>[^\s]+)" | rex "Table (?<target_table>[^:]+)"
I took your string Data File: /dir/dir/dir/file_name.dat and made a field called "data_file" out of everything that isn't a space that followed the "Data File: " string. Right after that, I used rex to create a field called "target_table" out of everything that comes after the word "Table " up to the colon. Several of the other strings/captures will be much like that, I'm leaving it as an exercise for you to build them, but if you have any problems add a comment to this and I or someone will try to help with that particular problem!
One that's different will be 1 Row successfully loaded.
... | rex "(?<rows_success_string>\d+ Row successfully loaded.)" | rex field=rows_success_string "(?<rows_success_count>\d+)"
The creates a field called "rows_success_string" that is the full 1 Row successfully loaded. then immediately does a rex on that new field and pulls out the digits in the front and creates a field out of that called "rows_success_count", which you don't mention you need, but I thought I'd show the technique. You can easily do this in one rex, but this seemed like it should be easier to understand.
Those are the pieces I think you may need to get your data into fields. Next, you need to format it. I think the easiest way to format it might be to create a table out of your fields that you want, then "transpose" them to make it vertically oriented instead of left-right oriented. That would be something like
... | table data_file, target_table, rows_success, Field3, Field4, ... FieldN | transpose
Obviously, fill in the rest of the fields you need to show.
Last, create an alert from the "Save As" menu. Maybe have it run once per hour (or every 5 minutes, or once per day - whatever you want, keeping in mind system load) and alert when the result is greater than 1 and send an email with the contents sent in-line, and perhaps attached too.
... View more