Splunk Search

Props Configuration with for Text File with First 2 lines contain header info

SplunkDash
Motivator

 

Hello,

I have an issue writing props configuration for text source file which contains first 2 line (including "----" line) as header info. Please see 3 sample events along with 2 header lines below. I also included the props that I wrote for this source file, but not working as expected....getting some error message "failed to parse timestamp". Any help will he highly appreciated. Thank you so much.

Sample data

Event_id  user_id   group_id  create_date  create_login  company_event_id  event_name  
----------------- ----------- ----------- ----------------------- ------------ ------------------------- --------------
105  346923 NULL  2021-10-07 14:13:21.160 783923 45655234 User Login 
250 165223 NULL 2021-10-07 15:33:54.857   566923  92557239 User Login 
25 1168923 NULL 2021-10-07 16:44:05.257   346923  34558242 User Login 

 

props config file I wrote

SHOULD_LINEMERGE=false

INDEXED_EXTRACTIONS=csv

TIMESTAMP_FIELDS=create_date

TIME_FORMAT=%Y-%m-%d  %H:%M:%S.%3N

HEADERFIELD_LINE_NUMBER=1

Labels (1)
Tags (1)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

@SplunkDash 

you have a typo in both TIME_PREFIX (capital \S in regex) and TIME_FORMAT (:%S missed). Try this, should work for other lines excluding first two lines.

[ __auto__learned__ ]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
TIME_PREFIX=\d+\s+\d+\s+\S+\s+
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3Q
MAX_TIMESTAMP_LOOKAHEAD=40

  ---

An upvote would be appreciated if this reply helps!

View solution in original post

SplunkDash
Motivator

Thank you so much for your quick response, appreciated. But, TIME_PREFIX/TIME FORMAT is not working as expected, getting some error message couldn't "parse timestamp". Any help will be appreciated!

I used 

TIME_PREFIX=^\d+\s+\d+\s+\w+\s+

TIME_FORMAT=%Y-%m-%d %H:%M%S.%3N 

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@SplunkDash 

you have a typo in both TIME_PREFIX (capital \S in regex) and TIME_FORMAT (:%S missed). Try this, should work for other lines excluding first two lines.

[ __auto__learned__ ]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
TIME_PREFIX=\d+\s+\d+\s+\S+\s+
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3Q
MAX_TIMESTAMP_LOOKAHEAD=40

  ---

An upvote would be appreciated if this reply helps!

venkatasri
SplunkTrust
SplunkTrust

Hi @SplunkDash 

You text file is not a qualified CSV as they don't have comma , separated values/header. To use space your event_name having value User Login which is having space that would not extract whole value of event_name.

one solution would be to drop the header and second line with ------ and use search time field extractions.

  • props shall be configured on HF/indexer to drop the header and ---- lines. ( use nullQueue ), Timestamp extraction you can use regex - TIME_PREFIX = ^\d+\s+\d+\s+\S+\s+ , TIME_FORMAT = <set_here>
  • props having search-time extractions shall go to SH.

In total you need to have two set's of props here.

--

An upvote would be appreciated if this reply helps!

Tags (3)
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...