Creating table based on Logs

[christinaef07]

Hi everyone, I need help creating a table based on my logs. My logs are formatted as follows: 

[2020-11-10 20:27:10,260]INFO - Logging info for Splunk:

[2020-11-10 20:27:10,260]INFO - spark_rc=0

[2020-11-10 20:27:10,260]INFO - status=success

[2020-11-10 20:27:10,260]INFO - clientName=foo

[2020-11-10 20:27:10,260]INFO - ID=123456

[2020-11-10 20:27:10,260] INFO - dag_ID=dag.py

I want to be able to express all this information from all of our logs in a table. For ex:

| Dag_ID |  Client Name |  Status   |spark_rc|

|dag.py    |          foo             |  success.     | 0      |

And more rows reading these fields from our other logs as well. For example, I want to see all these fields for our runs in the last 24 hrs. Can someone please help me with how to do this?

[richgalloway]

Am I correct in presuming each log line is a different event in Splunk?  If so, what links related events together?  I see nothing common except time and that's probably not reliable, especially if more than one run happens at the same time..

[christinaef07]

Hello and thank you for responding! I am new to Splunk and not sure. Should I be formatting my logs so that I have all this information printed within one line? 

[christinaef07]

For example, I can format the logs to produce something like this : 

splunk_log_info= [spark_rc=1, client_name=foo, dag_id=dag.py]