Help sending header and footer of CSV to nullqueue

[msarro]

Hey everyone.
The source files I am currently working with each contain a large amount of records. The problem is they follow a weird format. They begin with some numbers and symbols on a line. There is then a blank line. Then the actual body data starts.

After the body data, there is a blank line.
Finally, there is a footer line made of up some numbers and symbols.

Here is an example.

001;06.0.0;2011-08-01 09:31:02;CA114

DATA
...

10000;2011-08-01 09:34:18

I'm not sure how to ignore the header and footer lines. Any help would be very much appreciated.

[msarro · Accepted solution]

In props.conf (items to the left and right of = can be changed to suit your needs):

TRANSFORMS-PBTS-set1=setnull_pbts_head_cdr
TRANSFORMS-PBTS-set2=setnull_pbts_foot_cdr

In transforms.conf (added two stanzas, the regex just removes lines that start with 001 and 10000):

[setnull_pbts_head_cdr]
REGEX=^001;.*$
DEST_KEY=queue
FORMAT=nullQueue

[setnull_pbts_foot_cdr]
REGEX=^10000;.*$
DEST_KEY=queue
FORMAT=nullQueue

[ftk]

Maybe you can post an answer to this question with what you did in order to make it work, so that other users can benefit from it? Thanks!

[msarro]

Realized that the header always starts with 001;, and the footer line always starts with 10000. Looks like it is working now. Thanks guys!

[ftk]

Are any of the pieces in the header/footer static? If so which?