Would like to parse out each PacketSignature as a separate event.
I have tried editing the props.conf and adding a custom data type
[PacketSignatureXML]
KV_MODE=xml
TRUNCATE=0
TIME_PREFIX=\<FirstPacketTime>
SHOULD_LINEMERGE=true
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%I
LINE_BREAKER=(<PacketSignature>)
MUST_BREAK_AFTER=(\</PacketSignature\>
<?xml version="1.0"?>
<PacketDataSummaryReport
xmlns="http://www.cablelabs.com/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.cablelabs.com/CBI-1.0.xsd">
<CaseIdentity>12345</CaseIdentity>
<IAPSystemIdentity>cable.com</IAPSystemIdentity>
<TimeStamp>2015-02-10T17:50:22.298912Z</TimeStamp>
<AccessSessionId>0</AccessSessionId>
<PacketSignature>
<sourceAddress>8.8.8.8</sourceAddress>
<destAddress>9.9.9.9</destAddress>
<sourcePort>53</sourcePort>
<destPort>23</destPort>
<protocol>18</protocol>
<NumPktsSinceLastReport>1</NumPktsSinceLastReport>
<NumBytesSinceLastReport>70</NumBytesSinceLastReport>
<FirstPacketTime>2015-02-10T17:43:32.747682Z</FirstPacketTime>
<LastPacketTime>2015-02-10T17:43:32.747682Z</LastPacketTime>
</PacketSignature>
<PacketSignature>
<sourceAddress>1.1.1.1</sourceAddress>
<destAddress>2.2.2.2</destAddress>
<sourcePort>50</sourcePort>
<destPort>24</destPort>
<protocol>20</protocol>
<NumPktsSinceLastReport>1</NumPktsSinceLastReport>
<NumBytesSinceLastReport>50</NumBytesSinceLastReport>
<FirstPacketTime>2015-02-10T17:43:35.747682Z</FirstPacketTime>
<LastPacketTime>2015-02-10T17:43:35.747682Z</LastPacketTime>
</PacketSignature>
....
</PacketDataSummaryReport>
Give this a try. THere will two extra events with header (everything before first PacketSignature) and footer (closing tag for PacketDataSummaryReport), which needs to be dropped, hence the transforms.conf is needed.
props.conf
[PacketSignatureXML]
KV_MODE=xml
TRUNCATE=0
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?=\s*\<PacketSignature\>)
TIME_PREFIX=\<FirstPacketTime\>
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%NZ
MAX_TIMESTAMP_LOOKAHEAD = 27
TRANSFORMS-removejunk = removeheaders
transforms.conf
[removeheaders]
REGEX = PacketDataSummaryReport
FORMAT = nullQueue
DEST_KEY=queue
Thanks for the XML dump. What exactly do you want to do with it?
You'll probably get more help if you ask specific questions rather a vague "help me" posting. Tell us what you've tried already.
Would like to parse out each PacketSignature as a separate event.
I have tried editing the props.conf and adding a custom data type
[PacketSignatureXML]
KV_MODE=xml
TRUNCATE=0
TIME_PREFIX=\<FirstPacketTime>
SHOULD_LINEMERGE=true
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%I
LINE_BREAKER=(<PacketSignature>)
MUST_BREAK_AFTER=(\</PacketSignature\>
There are lot of fields at the root level (outside PacketSignature) and a timestmap. Do you want to just ignore that? What should be the timestamp for each PacketSignature section?
The fields outside the PacketSignature, including the timestamp can be ignored. The FirstPacketTime inside of each PacketSignature can be used for the timestamp for each event.