Getting Data In

Need help with event breaking in XML log file

halr9000
Motivator

I have a log file that looks like the below. Events are denoted by a messages tag, with each having a timestamp tag. I've tried various things with props.conf, but so far, I only get a single event extracted.

My current config (which doesn't work, but at least explains what I'm trying to do):

BREAK_ONLY_BEFORE=<messages>
BREAK_ONLY_BEFORE_DATE=false
MUST_BREAK_AFTER=\/messages>
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true
TIME_PREFIX=<timestamp>
TRUNCATE=0

In case the angle brackets are eaten, here's the original: http://pastebin.com/1vSDC9Xr

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><logs schemaVersion="0"><messages><timestamp>2013-04-22T11:55:13.766-07:00</timestamp><level>PROGRESS</level><thread>backup4 ee5fa1cb0c31a3e56f4fed2c99ff7745</thread><location>com.netapp.common.flow.tasks.Log</location><msgKeyClass>com.netapp.smvi.SMMsgKey</msgKeyClass><msgKeyValue>PROGRESS_TASK_BACKUP_STARTING</msgKeyValue><parameters xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/><message>Starting backup request</message></messages><messages><timestamp>2013-04-22T11:55:14.156-07:00</timestamp><level>INFO</level><thread>backup4 ee5fa1cb0c31a3e56f4fed2c99ff7745</thread><location>com.netapp.smvi.task.validation.BackupValidation</location><msgKeyClass>com.netapp.smvi.SMMsgKey</msgKeyClass><msgKeyValue>BACKUP_VALIDATION_INTERNAL_BACKUP_NAME_FOR_SCHEDULE_JOB</msgKeyValue><parameters><parameter>66fc1387-594c-48cb-b35d-94ca319a4a3c</parameter><parameter>backup_PM cDOT Datastore_20130422115514</parameter></parameters><message>Generating backupName for the scheduleJob 66fc1387-594c-48cb-b35d-94ca319a4a3c is backup_PM cDOT Datastore_20130422115514</message></messages><messages><timestamp>2013-04-22T11:55:18.400-07:00</timestamp><level>PROGRESS</level><thread>backup4 ee5fa1cb0c31a3e56f4fed2c99ff7745</thread><location>com.netapp.smvi.task.vmware.VmGetVirtualMachinesToBackup</location><msgKeyClass>com.netapp.smvi.SMMsgKey</msgKeyClass><msgKeyValue>BACKUP_DATASTORE</msgKeyValue><parameters><parameter>[NetApp_cDOT_DS1 (netfs://172.17.47.235///NetApp_cDOT_DS1)]</parameter></parameters><message>Backing up datastore(s) ([NetApp_cDOT_DS1 (netfs://172.17.47.235///NetApp_cDOT_DS1)])</message></messages><messages><timestamp>2013-04-22T11:55:18.509-07:00</timestamp><level>PROGRESS</level><thread>backup4 ee5fa1cb0c31a3e56f4fed2c99ff7745</thread><location>com.netapp.smvi.task.vmware.VmGetVirtualMachinesToBackup</location><msgKeyClass>com.netapp.smvi.SMMsgKey</msgKeyClass><msgKeyValue>BACKUP_VIRTUAL_ENTITIES</msgKeyValue><parameters><parameter>[VMware vCenter Server Appliance, SN_RC1_Node1, 7M-VSA, UB_server, win08, esxi_1, SN_RC1_Node2, vc_5.5_va]</parameter></parameters><message>Backing up the following virtual machine(s) ([VMware vCenter Server Appliance, SN_RC1_Node1, 7M-VSA, UB_server, win08, esxi_1, SN_RC1_Node2, vc_5.5_va])</message></messages>
1 Solution

dwaddle
SplunkTrust
SplunkTrust

Let's try

LINE_BREAKER=([\r\n]*)<messages>

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

Let's try

LINE_BREAKER=([\r\n]*)<messages>

halr9000
Motivator

You da duck.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Dynamic formatting from XML events

This challenge was first posted on Slack #puzzles channelFor a previous puzzle, I needed a set of fixed-length ...

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  &#x1f680; Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...