Knowledge Management

Why are Tab Delimiter field extractions not working?

lavster
Path Finder

i've followed the documentation and also some examples on here but for some reason I cant seem to get these to extract

here is an example of the log

xxx localhost 9997 8003 test test endRequest 2266 2022-11-17T08:08:06.617 2022-11-17T08:08:06.640 23 0 - OK - - DESC EXTENDED VIEW test_data_imp DESC - Denodo-Scheduler JDBC 127.0.0.1 - -

the props are as follows

[denodo-vdp-queries]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
REPORT-denodo-vdp-queries-fields = REPORT-denodo-vdp-queries-fields

the transforms are as follows

[REPORT-denodo-vdp-queries-fields]
DELIMS = "\t"
FIELDS = "server_name","host","port","id","database","username","notification_type","sessionID","start_time","end_time","duration","waiting_time","num_rows","state","completed","cache","query","request_type","elements","user_agent","access_interface","client_ip","transaction_id","web_service_name"

 

i've pushed the app to the forwarders that sending in the data and its in the right sourcetype, i've also pushed the app across the SH cluster, however none of the fields are extracted, am i missing a step?

 

Labels (1)
0 Karma
1 Solution

lavster
Path Finder

Found out what it was, its because the app that i created wasnt shared globally so transforms wasnt visible.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

First, are you sure the delimiter is the tab character and not white space in general?

Second, you say you pushed the app to the forwarders, but are they universal or heavy forwarders?  If the former then the props will have no effect and must be pushed to the indexers.

---
If this reply helps you, Karma would be appreciated.
0 Karma

lavster
Path Finder

Thanks for the response, when i extract them using the gui and "Extract Fields" tab delimeter option does pick the fields out correctly. I thought that if we put the app across the search head it would extract them at search time, not at index time?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

My mistake.  Yes, REPORT transforms are performed at search time and as such should be pushed to search heads.

---
If this reply helps you, Karma would be appreciated.
0 Karma

lavster
Path Finder

Thanks, can you see if i've missed anything that may be causing it in that case?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Remove the quotes from the DELIMS setting.  The value of DELIMS is a regular expression rather than a quoted string.  Since your data does not contain quotes then the regex should not, either.

---
If this reply helps you, Karma would be appreciated.
0 Karma

lavster
Path Finder

Thanks for your reply, I have tried this and for some reason its not working, I cant work out at all why this isnt working. I've also tried to write regex for the transforms which works fine in regex101 but again doesnt extract.

0 Karma

lavster
Path Finder

I have found the following errors, so it looks like the transforms isnt being detected in splunk, do i need to make them global or something like that?

11-18-2022 09:30:37.315 +0000 WARN SearchOperator:kv [9210 TcpChannelThread] - Could not find a transform named denodo-vdp-fields
0 Karma

lavster
Path Finder

Found out what it was, its because the app that i created wasnt shared globally so transforms wasnt visible.

Get Updates on the Splunk Community!

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

[Coming Soon] Splunk Observability Cloud - Enhanced navigation with a modern look and ...

We are excited to introduce our enhanced UI that brings together AppDynamics and Splunk Observability. This is ...

Splunk Smartness with Patrick Tatro | Episode 4

Welcome to another episode of "Splunk Smartness," where we explore how Splunk Education can revolutionize your ...