topic Re: Props Configuration with for Text File with First 2 lines contain header info in Splunk Search

Props Configuration with for Text File with First 2 lines contain header info

SplunkDash — Mon, 18 Oct 2021 23:23:44 GMT

Hello,

I have an issue writing props configuration for text source file which contains first 2 line (including "----" line) as header info. Please see 3 sample events along with 2 header lines below. I also included the props that I wrote for this source file, but not working as expected....getting some error message "failed to parse timestamp". Any help will he highly appreciated. Thank you so much.

Sample data

Event_id user_id group_id create_date create_login company_event_id event_name
----------------- ----------- ----------- ----------------------- ------------ ------------------------- --------------
105 346923 NULL 2021-10-07 14:13:21.160 783923 45655234 User Login
250 165223 NULL 2021-10-07 15:33:54.857 566923 92557239 User Login
25 1168923 NULL 2021-10-07 16:44:05.257 346923 34558242 User Login

props config file I wrote

SHOULD_LINEMERGE=false

INDEXED_EXTRACTIONS=csv

TIMESTAMP_FIELDS=create_date

TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N

HEADERFIELD_LINE_NUMBER=1

Re: Props Configuration with for Text File with First 2 lines contain header info

venkatasri — Tue, 19 Oct 2021 00:46:53 GMT

Hi @SplunkDash

You text file is not a qualified CSV as they don't have comma , separated values/header. To use space your event_name having value User Login which is having space that would not extract whole value of event_name.

one solution would be to drop the header and second line with ------ and use search time field extractions.

props shall be configured on HF/indexer to drop the header and ---- lines. ( use nullQueue ), Timestamp extraction you can use regex - TIME_PREFIX = ^\d+\s+\d+\s+\S+\s+ , TIME_FORMAT = <set_here>
props having search-time extractions shall go to SH.

In total you need to have two set's of props here.

An upvote would be appreciated if this reply helps!

Re: Props Configuration with for Text File with First 2 lines contain header info

SplunkDash — Tue, 19 Oct 2021 02:10:37 GMT

Thank you so much for your quick response, appreciated. But, TIME_PREFIX/TIME FORMAT is not working as expected, getting some error message couldn't "parse timestamp". Any help will be appreciated!

I used

TIME_PREFIX=^\d+\s+\d+\s+\w+\s+

TIME_FORMAT=%Y-%m-%d %H:%M%S.%3N

Re: Props Configuration with for Text File with First 2 lines contain header info

venkatasri — Tue, 19 Oct 2021 02:47:24 GMT

@SplunkDash

you have a typo in both TIME_PREFIX (capital \S in regex) and TIME_FORMAT (:%S missed). Try this, should work for other lines excluding first two lines.

[ __auto__learned__ ] SHOULD_LINEMERGE=false LINE_BREAKER=([\r\n]+) TIME_PREFIX=\d+\s+\d+\s+\S+\s+ TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3Q MAX_TIMESTAMP_LOOKAHEAD=40

---

An upvote would be appreciated if this reply helps!