Hi everyone, I need help creating a table based on my logs. My logs are formatted as follows:
[2020-11-10 20:27:10,260]INFO - Logging info for Splunk:
[2020-11-10 20:27:10,260]INFO - spark_rc=0
[2020-11-10 20:27:10,260]INFO - status=success
[2020-11-10 20:27:10,260]INFO - clientName=foo
[2020-11-10 20:27:10,260]INFO - ID=123456
[2020-11-10 20:27:10,260] INFO - dag_ID=dag.py
I want to be able to express all this information from all of our logs in a table. For ex:
| Dag_ID | Client Name | Status |spark_rc|
|dag.py | foo | success. | 0 |
And more rows reading these fields from our other logs as well. For example, I want to see all these fields for our runs in the last 24 hrs. Can someone please help me with how to do this?
Am I correct in presuming each log line is a different event in Splunk? If so, what links related events together? I see nothing common except time and that's probably not reliable, especially if more than one run happens at the same time..
Hello and thank you for responding! I am new to Splunk and not sure. Should I be formatting my logs so that I have all this information printed within one line?
For example, I can format the logs to produce something like this :
splunk_log_info= [spark_rc=1, client_name=foo, dag_id=dag.py]