Getting Data In

Log file is importing: How to parse the event?

richtate
Path Finder

I am getting the log file imported to Splunk, but each line is an event with no field name.  Can I break up the line into columns?  If not, how do I parse the line to extract a number?

Index is:

index=test_7d sourcetype=kafka:producer:bigfix

Events are:

2023-06-22 09:15:44,270 root - INFO - 114510 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:37,204 root - INFO - Executing getDatafromDB
2023-06-22 09:15:35,704 root - INFO - 35205 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:33,286 root - INFO - Executing getDatafromDB
2023-06-22 09:15:32,703 root - INFO - 167996 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:22,479 root - INFO - Executing getDatafromDB
2023-06-22 09:15:19,031 root - INFO - 181 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka

Each line/event starts with the date, the wordwrap is making it look incorrect.  I need to parse the bold number of each line after '- INFO -' and add a zero if no number.  I can do this with a eval, but how do I parse if there is no field name to add to the 'regex' command?

Thank you for looking at this problem!

Labels (5)
Tags (2)
0 Karma
1 Solution

richtate
Path Finder

Found the answer:

| rex "INFO - (?<eventCount>\d+)"
| fillnull value=0 eventCount

View solution in original post

Tags (1)
0 Karma

richtate
Path Finder

Any help is appreciated, even if it means this is in the wrong category..

0 Karma

richtate
Path Finder

I found the erex command that works,

| erex ImportCount examples="0,18729,49377"

But you have to enter a sample of the text you are looking for.  So it only works for one day and it has to be changed.  Can regex be used in place of the examples?

0 Karma

richtate
Path Finder

For example, here I'm using 'regex' to remove Operating Systems from dataset on a fieldname 'operating_system' which is one column of an sourcetype:

| regex operating_system!="(Linux|AIX|CENTOS|WINDOWS|Digital UNIX|FreeBSD|HP-UX|Hyper-V|Juniper|Mac|Windows|NetBSD|OpenBSD|OpenVMS|Server 2012|Server Core 2012|Server 2016|Server 2019|Ubuntu|Solaris|Unix|ESX|vCenter Server|rbash|[\*\*\*\*\*\*]|\A[\-\-\-\-\-\-\-\-\-\-]|[\=\=\=\=\=\=\=\=\=\=])"

0 Karma

richtate
Path Finder

Found the answer:

| rex "INFO - (?<eventCount>\d+)"
| fillnull value=0 eventCount
Tags (1)
0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...