Splunk Search

Help sending header and footer of CSV to nullqueue

msarro
Builder

Hey everyone.
The source files I am currently working with each contain a large amount of records. The problem is they follow a weird format. They begin with some numbers and symbols on a line. There is then a blank line. Then the actual body data starts.

After the body data, there is a blank line.
Finally, there is a footer line made of up some numbers and symbols.

Here is an example.

001;06.0.0;2011-08-01 09:31:02;CA114

DATA
...

10000;2011-08-01 09:34:18

I'm not sure how to ignore the header and footer lines. Any help would be very much appreciated.

Tags (2)
0 Karma
1 Solution

msarro
Builder

In props.conf (items to the left and right of = can be changed to suit your needs):

TRANSFORMS-PBTS-set1=setnull_pbts_head_cdr
TRANSFORMS-PBTS-set2=setnull_pbts_foot_cdr

In transforms.conf (added two stanzas, the regex just removes lines that start with 001 and 10000):

[setnull_pbts_head_cdr]
REGEX=^001;.*$
DEST_KEY=queue
FORMAT=nullQueue

[setnull_pbts_foot_cdr]
REGEX=^10000;.*$
DEST_KEY=queue
FORMAT=nullQueue

View solution in original post

0 Karma

msarro
Builder

In props.conf (items to the left and right of = can be changed to suit your needs):

TRANSFORMS-PBTS-set1=setnull_pbts_head_cdr
TRANSFORMS-PBTS-set2=setnull_pbts_foot_cdr

In transforms.conf (added two stanzas, the regex just removes lines that start with 001 and 10000):

[setnull_pbts_head_cdr]
REGEX=^001;.*$
DEST_KEY=queue
FORMAT=nullQueue

[setnull_pbts_foot_cdr]
REGEX=^10000;.*$
DEST_KEY=queue
FORMAT=nullQueue

View solution in original post

0 Karma

ftk
Motivator

Maybe you can post an answer to this question with what you did in order to make it work, so that other users can benefit from it? Thanks!

0 Karma

msarro
Builder

Realized that the header always starts with 001;, and the footer line always starts with 10000. Looks like it is working now. Thanks guys!

0 Karma

ftk
Motivator

Are any of the pieces in the header/footer static? If so which?

0 Karma