Getting Data In

Log file is importing: How to parse the event?

richtate
Path Finder

I am getting the log file imported to Splunk, but each line is an event with no field name.  Can I break up the line into columns?  If not, how do I parse the line to extract a number?

Index is:

index=test_7d sourcetype=kafka:producer:bigfix

Events are:

2023-06-22 09:15:44,270 root - INFO - 114510 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:37,204 root - INFO - Executing getDatafromDB
2023-06-22 09:15:35,704 root - INFO - 35205 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:33,286 root - INFO - Executing getDatafromDB
2023-06-22 09:15:32,703 root - INFO - 167996 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka
2023-06-22 09:15:22,479 root - INFO - Executing getDatafromDB
2023-06-22 09:15:19,031 root - INFO - 181 events have been uploaded to topic DC2_Endpoint_Configuration_IBM_BigFix_Patch_Join on Kafka

Each line/event starts with the date, the wordwrap is making it look incorrect.  I need to parse the bold number of each line after '- INFO -' and add a zero if no number.  I can do this with a eval, but how do I parse if there is no field name to add to the 'regex' command?

Thank you for looking at this problem!

Labels (5)
Tags (2)
0 Karma
1 Solution

richtate
Path Finder

Found the answer:

| rex "INFO - (?<eventCount>\d+)"
| fillnull value=0 eventCount

View solution in original post

Tags (1)
0 Karma

richtate
Path Finder

Any help is appreciated, even if it means this is in the wrong category..

0 Karma

richtate
Path Finder

I found the erex command that works,

| erex ImportCount examples="0,18729,49377"

But you have to enter a sample of the text you are looking for.  So it only works for one day and it has to be changed.  Can regex be used in place of the examples?

0 Karma

richtate
Path Finder

For example, here I'm using 'regex' to remove Operating Systems from dataset on a fieldname 'operating_system' which is one column of an sourcetype:

| regex operating_system!="(Linux|AIX|CENTOS|WINDOWS|Digital UNIX|FreeBSD|HP-UX|Hyper-V|Juniper|Mac|Windows|NetBSD|OpenBSD|OpenVMS|Server 2012|Server Core 2012|Server 2016|Server 2019|Ubuntu|Solaris|Unix|ESX|vCenter Server|rbash|[\*\*\*\*\*\*]|\A[\-\-\-\-\-\-\-\-\-\-]|[\=\=\=\=\=\=\=\=\=\=])"

0 Karma

richtate
Path Finder

Found the answer:

| rex "INFO - (?<eventCount>\d+)"
| fillnull value=0 eventCount
Tags (1)
0 Karma
Get Updates on the Splunk Community!

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

[Coming Soon] Splunk Observability Cloud - Enhanced navigation with a modern look and ...

We are excited to introduce our enhanced UI that brings together AppDynamics and Splunk Observability. This is ...

Splunk Smartness with Patrick Tatro | Episode 4

Welcome to another episode of "Splunk Smartness," where we explore how Splunk Education can revolutionize your ...