topic Re: Splunk search doesn't extract fields on the forwarded data in Getting Data In

Splunk search doesn't extract fields on the forwarded data

kpragasam — Fri, 10 Nov 2017 17:05:29 GMT

Our forwarder sends the data to the Splunk Server & our config in the Splunk Server & forwarder looks like below. For some reason, when I do the search in the SPLUNK, it's not extracting the fields. Can anyone help us out??

**In Splunk Server:**
*transforms.conf*

[portal_eventlog_host]
REGEX = <computer>?(.*)</computer>
FORMAT = host::$1
DEST_KEY = MetaData:Host

*props.conf*

[iis_custom]
category = Web
description = W3C Extended log format produced by the Microsoft Internet Information Services (IIS) web server
SHOULD_LINEMERGE = False
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
INDEXED_EXTRACTIONS = w3c
detect_trailing_nulls = auto
FIELD_DELIMITER = whitespace
FIELD_HEADER_REGEX = ^#Fields:\s*(.*)
MISSING_VALUE_REGEX = -
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TZ = GMT
TIMESTAMP_FIELDS = date,time

[portal_eventlog]
category = Application
description = Portal event logs
disabled = false
pulldown_type = true
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
LINE_BREAKER = (<log>|</log>)
TIME_PREFIX = datetime
SEDCMD-format-text = s/<(\/{1}\w+)>/<\1>\n/g
TRANSFORMS-hostname = portal_eventlog_host
KV_MODE = xml

**In Forwarder:**
*inputs.conf*

[monitor://C:\TestLogs\IIS]
disabled = false
index = dotnet
sourcetype = iis_custom

[monitor://C:\TestLogs\PortalEventLog]
disabled = false
index = dotnet
sourcetype = portal_eventlog
crcSalt = <SOURCE>

Re: Splunk search doesn't extract fields on the forwarded data

sshelly_splunk — Fri, 10 Nov 2017 17:50:06 GMT

I apoligize if this seems like a stupid question :), but do you have a single splunk instance (indexer and search head are the same)? If distributed, you will need the props/etc on all instance (search heads and indexers).

Re: Splunk search doesn't extract fields on the forwarded data

kpragasam — Fri, 10 Nov 2017 19:38:22 GMT

It's distributed. We do have the props & transform configs in both indexer & SearchHead. But, that doesn't seems to help

Re: Splunk search doesn't extract fields on the forwarded data

skoelpin — Fri, 10 Nov 2017 21:20:12 GMT

What fields are not extracting? By default, it will extract all fields with some type of delimiter between values at search-time. How many fields do you currently have extracted? Do the fields you want have some type of delimiter between the key value pair in the event? If not, you will have to write some regex to get the field.

Remember that fields are relative to sourcetype

Re: Splunk search doesn't extract fields on the forwarded data

kpragasam — Tue, 29 Sep 2020 16:42:13 GMT

For "Portal_eventlog" sourcetype, I am seeing only the standards fields such as host, index, source, sourcetype, linecount, splunk_server. The log file is an XML. So, I am expecting each tag as a fields.

Note: Some people mentioned that we have to have props & transform.config in the forwarder too. I have tried that too & didn't work.