Hi,
I am providing sample data below:
[2021-12-07 03:50:14,666] {{taskinstance.py:1532}} INFO - Marking task as FAILED. dag_id=any_bash_command_dag, task_id=bash_command, execution_date=20211207T035010, start_date=20211207T035013, end_date=20211207T035014
[2021-12-08 01:02:14,491] {{taskinstance.py:1192}} INFO - Marking task as SUCCESS. dag_id=Parent_dag, task_id=trigger_archive_files_dag, execution_date=20211207T000000, start_date=20211208T010213, end_date=20211208T010214
SPL:
index=cloud sourcetype=lambda:Airflow2Splunk "\"logGroup\"" "\"airflow-OnePIAirflowEnvironment-DEV-Task\"" "Marking task as*" dag_id=*
| rex field=_raw "task_id=(?P<task_id>\w+)"
| table _time dag_id task_id
| sort _time
Current Results in tabular form:
_time dag_id task_id Task_Status
--------------------------------------------------------------------------------------------------------------------------------------
2021-12-06 22:50:14.756 any_bash_command_dag bash_command
2021-12-07 20:02:14.626 Parent_dag trigger_archive_files_dag
Expected results in tabular form:
_time dag_id task_id Task_Status
--------------------------------------------------------------------------------------------------------------------------------------
2021-12-06 22:50:14.756 any_bash_command_dag bash_command Failed
2021-12-07 20:02:14.626 Parent_dag trigger_archive_files_dag Success
Can you please help me in modifying the SPL above which should result an additional column "Task_Status" and the values "Failed" for dag_id= any_bash_command_dag and "Success" for dag_id=Parent_dag?
Thanks,
Sumit
Add in
| rex field=_raw "Marking task as (?<Task_Status>\w+)"
| eval Task_Status=substr('Task_Status', 1, 1).lower(substr('Task_Status', 2))
The eval line is only necessary if you want to get a capitalised word, rather than all caps
Thank you very much! @bowesmana
Appreciate your help!