Are you a member of the Splunk Community?
- Find Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Help sending header and footer of CSV to nullqueue
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey everyone.
The source files I am currently working with each contain a large amount of records. The problem is they follow a weird format. They begin with some numbers and symbols on a line. There is then a blank line. Then the actual body data starts.
After the body data, there is a blank line.
Finally, there is a footer line made of up some numbers and symbols.
Here is an example.
001;06.0.0;2011-08-01 09:31:02;CA114
DATA
...
10000;2011-08-01 09:34:18
I'm not sure how to ignore the header and footer lines. Any help would be very much appreciated.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In props.conf (items to the left and right of = can be changed to suit your needs):
TRANSFORMS-PBTS-set1=setnull_pbts_head_cdr
TRANSFORMS-PBTS-set2=setnull_pbts_foot_cdr
In transforms.conf (added two stanzas, the regex just removes lines that start with 001 and 10000):
[setnull_pbts_head_cdr]
REGEX=^001;.*$
DEST_KEY=queue
FORMAT=nullQueue
[setnull_pbts_foot_cdr]
REGEX=^10000;.*$
DEST_KEY=queue
FORMAT=nullQueue
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In props.conf (items to the left and right of = can be changed to suit your needs):
TRANSFORMS-PBTS-set1=setnull_pbts_head_cdr
TRANSFORMS-PBTS-set2=setnull_pbts_foot_cdr
In transforms.conf (added two stanzas, the regex just removes lines that start with 001 and 10000):
[setnull_pbts_head_cdr]
REGEX=^001;.*$
DEST_KEY=queue
FORMAT=nullQueue
[setnull_pbts_foot_cdr]
REGEX=^10000;.*$
DEST_KEY=queue
FORMAT=nullQueue
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Maybe you can post an answer to this question with what you did in order to make it work, so that other users can benefit from it? Thanks!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Realized that the header always starts with 001;, and the footer line always starts with 10000. Looks like it is working now. Thanks guys!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Are any of the pieces in the header/footer static? If so which?
Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform
Calling All Security Pros: Ready to Race Through Boston?
Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...
