Props Configuration with for Text File with First 2 lines contain header info

[SplunkDash]

Hello,

I have an issue writing props configuration for text source file which contains first 2 line (including "----" line) as header info. Please see 3 sample events along with 2 header lines below. I also included the props that I wrote for this source file, but not working as expected....getting some error message "failed to parse timestamp". Any help will he highly appreciated. Thank you so much.

Sample data

Event_id  user_id   group_id  create_date  create_login  company_event_id  event_name  
----------------- ----------- ----------- ----------------------- ------------ ------------------------- --------------
105  346923 NULL  2021-10-07 14:13:21.160 783923 45655234 User Login 
250 165223 NULL 2021-10-07 15:33:54.857   566923  92557239 User Login 
25 1168923 NULL 2021-10-07 16:44:05.257   346923  34558242 User Login 

props config file I wrote

SHOULD_LINEMERGE=false

INDEXED_EXTRACTIONS=csv

TIMESTAMP_FIELDS=create_date

TIME_FORMAT=%Y-%m-%d  %H:%M:%S.%3N

HEADERFIELD_LINE_NUMBER=1

[SplunkDash]

Thank you so much for your quick response, appreciated. But, TIME_PREFIX/TIME FORMAT is not working as expected, getting some error message couldn't "parse timestamp". Any help will be appreciated!

I used 

TIME_PREFIX=^\d+\s+\d+\s+\w+\s+

TIME_FORMAT=%Y-%m-%d %H:%M%S.%3N 

[venkatasri · Accepted solution]

@SplunkDash 

you have a typo in both TIME_PREFIX (capital \S in regex) and TIME_FORMAT (:%S missed). Try this, should work for other lines excluding first two lines.

[ __auto__learned__ ]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
TIME_PREFIX=\d+\s+\d+\s+\S+\s+
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3Q
MAX_TIMESTAMP_LOOKAHEAD=40

  ---

An upvote would be appreciated if this reply helps!

[venkatasri]

Hi @SplunkDash 

You text file is not a qualified CSV as they don't have comma , separated values/header. To use space your event_name having value User Login which is having space that would not extract whole value of event_name.

one solution would be to drop the header and second line with ------ and use search time field extractions.

• props shall be configured on HF/indexer to drop the header and ---- lines. ( use nullQueue ), Timestamp extraction you can use regex - TIME_PREFIX = ^\d+\s+\d+\s+\S+\s+ , TIME_FORMAT = <set_here>
• props having search-time extractions shall go to SH.

In total you need to have two set's of props here.

--

An upvote would be appreciated if this reply helps!