topic Re: Log file is importing: How to parse the event? in Getting Data In

Log file is importing: How to parse the event?

richtate — Thu, 22 Jun 2023 17:58:39 GMT

I am getting the log file imported to Splunk, but each line is an event with no field name. Can I break up the line into columns? If not, how do I parse the line to extract a number?

Index is:

index=test_7d sourcetype=kafka:producer:bigfix

Events are:

2023-06-22 09:15:44,270 root - INFO - 114510 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:37,204 root - INFO - Executing getDatafromDB
2023-06-22 09:15:35,704 root - INFO - 35205 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:33,286 root - INFO - Executing getDatafromDB
2023-06-22 09:15:32,703 root - INFO - 167996 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:22,479 root - INFO - Executing getDatafromDB
2023-06-22 09:15:19,031 root - INFO - 181 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka

Each line/event starts with the date, the wordwrap is making it look incorrect. I need to parse the bold number of each line after '- INFO -' and add a zero if no number. I can do this with a eval, but how do I parse if there is no field name to add to the 'regex' command?

Thank you for looking at this problem!

Re: Log file is importing, need to parse the event.

richtate — Thu, 22 Jun 2023 17:58:19 GMT

For example, here I'm using 'regex' to remove Operating Systems from dataset on a fieldname 'operating_system' which is one column of an sourcetype:

Re: Log file is importing: How to parse the event?

richtate — Fri, 23 Jun 2023 18:26:45 GMT

I found the erex command that works,

| erex ImportCount examples="0,18729,49377"

But you have to enter a sample of the text you are looking for. So it only works for one day and it has to be changed. Can regex be used in place of the examples?

Re: Log file is importing: How to parse the event?

richtate — Mon, 26 Jun 2023 16:11:20 GMT

Any help is appreciated, even if it means this is in the wrong category..

Re: Log file is importing, need to parse the event.

richtate — Mon, 26 Jun 2023 17:48:38 GMT

Found the answer:

| rex "INFO - (?<eventCount>\d+)"
| fillnull value=0 eventCount