Hi
I have the problem that I have different parsing results when I monitor a file (csv) on a universal forwarder or local on the SH+Indexer.
Local on the SH+Indexer the parsing with the field name assignment/extraction works.
When the same file was read by the UF and sent to the SAME Sourcetype the result is different - no fieldnames are applied. I think there is no parsing happening?
I think the config problem is on the UF. I only put there a inputs.conf with (monitor --> sourcetype + host and index)
Maybe someone can help me out?
IDX-props.conf
C:\Program Files\Splunk\bin>splunk btool props list --debug tv:star2
C:\Program Files\Splunk\etc\apps\search\local\props.conf [tv:star2]
C:\Program Files\Splunk\etc\system\default\props.conf ADD_EXTRA_TIME_FIELDS = True
C:\Program Files\Splunk\etc\system\default\props.conf ANNOTATE_PUNCT = True
C:\Program Files\Splunk\etc\system\default\props.conf AUTO_KV_JSON = true
C:\Program Files\Splunk\etc\system\default\props.conf BREAK_ONLY_BEFORE =
C:\Program Files\Splunk\etc\system\default\props.conf BREAK_ONLY_BEFORE_DATE = True
C:\Program Files\Splunk\etc\system\default\props.conf CHARSET = AUTO
C:\Program Files\Splunk\etc\apps\search\local\props.conf DATETIME_CONFIG =
C:\Program Files\Splunk\etc\system\default\props.conf DEPTH_LIMIT = 1000
C:\Program Files\Splunk\etc\apps\search\local\props.conf FIELD_DELIMITER = ,
C:\Program Files\Splunk\etc\apps\search\local\props.conf FIELD_NAMES = id,detection_time,log,parameter_list,text_english,error,sequence,roc_name,file_name,error_position,error_host,ack_required,acknowledged,ack_time,based_on_id,ack_by_user,job_id,recipient,destination,sender,subject,computed_size,spool
C:\Program Files\Splunk\etc\apps\search\local\props.conf FIELD_QUOTE = '
C:\Program Files\Splunk\etc\system\default\props.conf HEADER_MODE =
C:\Program Files\Splunk\etc\apps\search\local\props.conf INDEXED_EXTRACTIONS = csv
C:\Program Files\Splunk\etc\apps\search\local\props.conf KV_MODE = none
C:\Program Files\Splunk\etc\system\default\props.conf LEARN_MODEL = true
C:\Program Files\Splunk\etc\system\default\props.conf LEARN_SOURCETYPE = true
C:\Program Files\Splunk\etc\system\default\props.conf LINE_BREAKER_LOOKBEHIND = 100
C:\Program Files\Splunk\etc\system\default\props.conf MATCH_LIMIT = 100000
C:\Program Files\Splunk\etc\system\default\props.conf MAX_DAYS_AGO = 2000
C:\Program Files\Splunk\etc\system\default\props.conf MAX_DAYS_HENCE = 2
C:\Program Files\Splunk\etc\system\default\props.conf MAX_DIFF_SECS_AGO = 3600
C:\Program Files\Splunk\etc\system\default\props.conf MAX_DIFF_SECS_HENCE = 604800
C:\Program Files\Splunk\etc\system\default\props.conf MAX_EVENTS = 256
C:\Program Files\Splunk\etc\system\default\props.conf MAX_TIMESTAMP_LOOKAHEAD = 128
C:\Program Files\Splunk\etc\system\default\props.conf MUST_BREAK_AFTER =
C:\Program Files\Splunk\etc\system\default\props.conf MUST_NOT_BREAK_AFTER =
C:\Program Files\Splunk\etc\system\default\props.conf MUST_NOT_BREAK_BEFORE =
C:\Program Files\Splunk\etc\apps\search\local\props.conf NO_BINARY_CHECK = true
C:\Program Files\Splunk\etc\apps\search\local\props.conf REPORT-star2_2 = REPORT-star2_2
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION = indexing
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-all = full
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-inner = inner
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-outer = outer
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-raw = none
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-standard = standard
C:\Program Files\Splunk\etc\apps\search\local\props.conf SHOULD_LINEMERGE = False
C:\Program Files\Splunk\etc\apps\search\local\props.conf TIMESTAMP_FIELDS = detection_time
C:\Program Files\Splunk\etc\apps\search\local\props.conf TIME_FORMAT = %Y-%m-%d%H:%M:%S.%3Q
C:\Program Files\Splunk\etc\system\default\props.conf TRANSFORMS =
C:\Program Files\Splunk\etc\system\default\props.conf TRUNCATE = 10000
C:\Program Files\Splunk\etc\apps\search\local\props.conf category = Custom
C:\Program Files\Splunk\etc\apps\search\local\props.conf description = Comma-separated value format. star2 Eventlog
C:\Program Files\Splunk\etc\system\default\props.conf detect_trailing_nulls = auto
C:\Program Files\Splunk\etc\apps\search\local\props.conf disabled = false
C:\Program Files\Splunk\etc\system\default\props.conf maxDist = 100
C:\Program Files\Splunk\etc\system\default\props.conf priority =
C:\Program Files\Splunk\etc\apps\search\local\props.conf pulldown_type = true
C:\Program Files\Splunk\etc\system\default\props.conf sourcetype =
SH - inputs.conf
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf [monitor://C:\TV\star2_LOG\*_TV.csv]
C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf _rcvbuf = 1572864
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf disabled = false
C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf evt_dc_name =
C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf evt_dns_name =
C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf evt_resolve_ad_obj = 0
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf host = TV_TEST
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf index = tv
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf sourcetype = tv:star2
IDX-props.conf
C:\Program Files\Splunk\bin>splunk btool props list --debug tv:star2
C:\Program Files\Splunk\etc\apps\search\local\props.conf [tv:star2]
C:\Program Files\Splunk\etc\system\default\props.conf ADD_EXTRA_TIME_FIELDS = True
C:\Program Files\Splunk\etc\system\default\props.conf ANNOTATE_PUNCT = True
C:\Program Files\Splunk\etc\system\default\props.conf AUTO_KV_JSON = true
C:\Program Files\Splunk\etc\system\default\props.conf BREAK_ONLY_BEFORE =
C:\Program Files\Splunk\etc\system\default\props.conf BREAK_ONLY_BEFORE_DATE = True
C:\Program Files\Splunk\etc\system\default\props.conf CHARSET = AUTO
C:\Program Files\Splunk\etc\apps\search\local\props.conf DATETIME_CONFIG =
C:\Program Files\Splunk\etc\system\default\props.conf DEPTH_LIMIT = 1000
C:\Program Files\Splunk\etc\apps\search\local\props.conf FIELD_DELIMITER = ,
C:\Program Files\Splunk\etc\apps\search\local\props.conf FIELD_NAMES = id,detection_time,log,parameter_list,text_english,error,sequence,roc_name,file_name,error_position,error_host,ack_required,acknowledged,ack_time,based_on_id,ack_by_user,job_id,recipient,destination,sender,subject,computed_size,spool
C:\Program Files\Splunk\etc\apps\search\local\props.conf FIELD_QUOTE = '
C:\Program Files\Splunk\etc\system\default\props.conf HEADER_MODE =
C:\Program Files\Splunk\etc\apps\search\local\props.conf INDEXED_EXTRACTIONS = csv
C:\Program Files\Splunk\etc\apps\search\local\props.conf KV_MODE = none
C:\Program Files\Splunk\etc\system\default\props.conf LEARN_MODEL = true
C:\Program Files\Splunk\etc\system\default\props.conf LEARN_SOURCETYPE = true
C:\Program Files\Splunk\etc\system\default\props.conf LINE_BREAKER_LOOKBEHIND = 100
C:\Program Files\Splunk\etc\system\default\props.conf MATCH_LIMIT = 100000
C:\Program Files\Splunk\etc\system\default\props.conf MAX_DAYS_AGO = 2000
C:\Program Files\Splunk\etc\system\default\props.conf MAX_DAYS_HENCE = 2
C:\Program Files\Splunk\etc\system\default\props.conf MAX_DIFF_SECS_AGO = 3600
C:\Program Files\Splunk\etc\system\default\props.conf MAX_DIFF_SECS_HENCE = 604800
C:\Program Files\Splunk\etc\system\default\props.conf MAX_EVENTS = 256
C:\Program Files\Splunk\etc\system\default\props.conf MAX_TIMESTAMP_LOOKAHEAD = 128
C:\Program Files\Splunk\etc\system\default\props.conf MUST_BREAK_AFTER =
C:\Program Files\Splunk\etc\system\default\props.conf MUST_NOT_BREAK_AFTER =
C:\Program Files\Splunk\etc\system\default\props.conf MUST_NOT_BREAK_BEFORE =
C:\Program Files\Splunk\etc\apps\search\local\props.conf NO_BINARY_CHECK = true
C:\Program Files\Splunk\etc\apps\search\local\props.conf REPORT-star2_2 = REPORT-star2_2
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION = indexing
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-all = full
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-inner = inner
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-outer = outer
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-raw = none
C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-standard = standard
C:\Program Files\Splunk\etc\apps\search\local\props.conf SHOULD_LINEMERGE = False
C:\Program Files\Splunk\etc\apps\search\local\props.conf TIMESTAMP_FIELDS = detection_time
C:\Program Files\Splunk\etc\apps\search\local\props.conf TIME_FORMAT = %Y-%m-%d%H:%M:%S.%3Q
C:\Program Files\Splunk\etc\system\default\props.conf TRANSFORMS =
C:\Program Files\Splunk\etc\system\default\props.conf TRUNCATE = 10000
C:\Program Files\Splunk\etc\apps\search\local\props.conf category = Custom
C:\Program Files\Splunk\etc\apps\search\local\props.conf description = Comma-separated value format. star2 Eventlog
C:\Program Files\Splunk\etc\system\default\props.conf detect_trailing_nulls = auto
C:\Program Files\Splunk\etc\apps\search\local\props.conf disabled = false
C:\Program Files\Splunk\etc\system\default\props.conf maxDist = 100
C:\Program Files\Splunk\etc\system\default\props.conf priority =
C:\Program Files\Splunk\etc\apps\search\local\props.conf pulldown_type = true
C:\Program Files\Splunk\etc\system\default\props.conf sourcetype =
SH - inputs.conf
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf [monitor://C:\TV\star2_LOG\*_TV.csv]
C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf _rcvbuf = 1572864
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf disabled = false
C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf evt_dc_name =
C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf evt_dns_name =
C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf evt_resolve_ad_obj = 0
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf host = TV_TEST
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf index = tv
C:\Program Files\SplunkUniversalForwarder\etc\apps\uf_star2_srv\local\inputs.conf sourcetype = tv:star2
Hi Valiquet!
It works - BUT i do not understand this. I always thought the UF only makes the input and has nothing to do with parsing, this is the job of the indexer or heavy forwarder.
Do you know which of the props parameters are the ones which make this work? - Is the full parsing made on the UF? -no?
Maybe you can explain it to me so that I will know it better - I start splunking one month ago. I like Splunk, but this problem takes me now 4 day - and was very frustrating.
Thanks a lot for helping!!!
Max
Parsing don't always occurs at the same place depending of your architecture. If you have UF,HF,IDX and SHC it is more spread. You were missing the CSV settings it's only preparsing. Most of the other settings (date, line breaking, etc) are on the IDX. It's a bit of error and trial.
You can follow this wiki, it is mostly accurate. https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
It doesn't hurt to have the same props settings everywhere. Just don't do that with indexes.conf, server, etc. Also, _internal is a great source to TB shoot that.
Cheers
Not quite the wording on the documentation is;
Caveats to extracting fields from
structured data files Splunk software
does not parse structured data that
has been forwarded to an indexer
Refer to this page or for the more technical details refer to the wiki page you are effectively doing header extraction/structured parsing and will go down the structured parsing queue on the universal forwarder
Very cool -> you can see that the indexer is actually the last place to place the props.conf.
I would copy everything from local\props.conf to local\props.conf on UF! If it doesn't work, search for
index=_internal sourcetype=splunkd "tv:star2"
btool output props.conf on IDX
[tv:star2]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG =
DEPTH_LIMIT = 1000
FIELD_DELIMITER = ,
FIELD_NAMES = id,detection_time,log,parameter_list,text_english,error,sequence,roc_name,file_name,error_position,error_host,ack_required,acknowledged,ack_time,based_on_id,ack_by_user,job_id,recipient,destination,sender,subject,computed_size,spool
FIELD_QUOTE = '
HEADER_MODE =
INDEXED_EXTRACTIONS = csv
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
NO_BINARY_CHECK = true
REPORT-star2_2 = REPORT-star2_2
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = False
TIMESTAMP_FIELDS = detection_time
TIME_FORMAT = %Y-%m-%d%H:%M:%S.%3Q
TRANSFORMS =
TRUNCATE = 10000
category = Custom
description = Comma-separated value format. star2 log
detect_trailing_nulls = auto
disabled = false
maxDist = 100
priority =
pulldown_type = true
sourcetype =
btool output inputs.conf on UF
[monitor://C:\TV\star2_LOG\*_TV.csv]
_rcvbuf = 1572864
disabled = false
evt_dc_name =
evt_dns_name =
evt_resolve_ad_obj = 0
host = TV_TEST
index = tv
sourcetype = tv:star2
Can you --debug btool ?
It would be easier to compare. It looks like your indexer is configure to ingest CSV but not your forwarder. Use debug and Ill tell you what to move
Can you show us inputs props on FW and IDX?