Getting Data In

Log file is importing: How to parse the event?

richtate
Path Finder

I am getting the log file imported to Splunk, but each line is an event with no field name.  Can I break up the line into columns?  If not, how do I parse the line to extract a number?

Index is:

index=test_7d sourcetype=kafka:producer:bigfix

Events are:

2023-06-22 09:15:44,270 root - INFO - 114510 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:37,204 root - INFO - Executing getDatafromDB
2023-06-22 09:15:35,704 root - INFO - 35205 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:33,286 root - INFO - Executing getDatafromDB
2023-06-22 09:15:32,703 root - INFO - 167996 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:22,479 root - INFO - Executing getDatafromDB
2023-06-22 09:15:19,031 root - INFO - 181 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka

Each line/event starts with the date, the wordwrap is making it look incorrect.  I need to parse the bold number of each line after '- INFO -' and add a zero if no number.  I can do this with a eval, but how do I parse if there is no field name to add to the 'regex' command?

Thank you for looking at this problem!

Labels (5)
Tags (2)
0 Karma
1 Solution

richtate
Path Finder

Found the answer:

| rex "INFO - (?<eventCount>\d+)"
| fillnull value=0 eventCount

View solution in original post

Tags (1)
0 Karma

richtate
Path Finder

Any help is appreciated, even if it means this is in the wrong category..

0 Karma

richtate
Path Finder

I found the erex command that works,

| erex ImportCount examples="0,18729,49377"

But you have to enter a sample of the text you are looking for.  So it only works for one day and it has to be changed.  Can regex be used in place of the examples?

0 Karma

richtate
Path Finder

For example, here I'm using 'regex' to remove Operating Systems from dataset on a fieldname 'operating_system' which is one column of an sourcetype:

| regex operating_system!="(Linux|AIX|CENTOS|WINDOWS|Digital UNIX|FreeBSD|HP-UX|Hyper-V|Juniper|Mac|Windows|NetBSD|OpenBSD|OpenVMS|Server 2012|Server Core 2012|Server 2016|Server 2019|Ubuntu|Solaris|Unix|ESX|vCenter Server|rbash|[\*\*\*\*\*\*]|\A[\-\-\-\-\-\-\-\-\-\-]|[\=\=\=\=\=\=\=\=\=\=])"

0 Karma

richtate
Path Finder

Found the answer:

| rex "INFO - (?<eventCount>\d+)"
| fillnull value=0 eventCount
Tags (1)
0 Karma
Get Updates on the Splunk Community!

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...

New Articles from Academic Learning Partners, Help Expand Lantern’s Use Case Library, ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...