I am trying to send my cloudflare HTTP logs to my externally exposed splunk heavy forwarder (on prem). I have installed the Cloudflare App on the heavy forwarder and the searchead: https://splunkbase.splunk.com/app/4501/#/details I know the data is making it to my heavy forwarder that has the application installed. However, the data isn't being correctly ingested. I am finding this type of log on my _internal index on my forwarder, and it appears to be for each event that cloudflare has sent to my forwarder.  I have rebooted the forwarder since adding the application: 09-15-2022 10:16:22.804 -0400 WARN TcpOutputProc [5288 indexerPipe] - Pipeline data does not have indexKey. [_hecTeleVersionKey] = default\n[_hecTeleAppKey] = default\n[_raw] = \n[_meta] = punct::\n[MetaData:Source] = source::http:Cloudflare5xx\n[MetaData:Host] = host::readactedhost.com\n[MetaData:Sourcetype] = sourcetype::cloudflare:json\n[_done] = _done\n[_linebreaker] = _linebreaker\n[_time] = 1663251382\n[_conf] = source::http:Cloudflare5xx|host::readactedhost.com|cloudflare:json|\n My HEC token is configured as: [http://Cloudflare5xx] description = Used to get cloudflare logs into splunk for server 5xx errors disabled = 0 indexes = cloudflare token = 7xxxxxxxx I am stumped what "Pipeline data does not have indexKey" means and cannot find a next step.  If the logs are being sent, and making it to the forwarder, are there more steps than having the application there to interpret the information sent to the services/collector/raw endpoint?  I have never ingested on the /raw endpoint before so I wonder if something is missing. ... View more

I am sending information to splunk via an HTTP Event collector and specifying the index in the body of the HTTP POST. I have an HTTP Event collector on a heavy forwarder per this documentation: https://docs.splunk.com/Documentation/Splunk/8.2.0/Data/ScaleHTTPEventCollector However, I'd like to be able to define a token to a limited set of indexes per this documentation.  So the consumer can supply an index but cannot supply ANY index in their payload.  This appears to be the "indexes" property: https://docs.splunk.com/Documentation/SplunkCloud/8.2.2104/Data/UseHECusingconffiles When I use the GUI to create a token I only see indexes on the heavy forwarder to limit to, it does not show me the indexes actually on my clustered indexers.  When I manually edit the configuration for a token in the .conf file to have the index I want to limit to I receive an "Incorrect Index" error. Is it possible to have my HEC on a heavy forwarder but limit a token to indexes defined on my indexer cluster? ... View more

I have syslog information being sent to my heavy forwarder and I'd like to define a specific translation for one piece of information. The number in the <> brackets translates to Error, Warning, Info, Debug, etc. Jun 28 13:18:14 xxx.xxx.xxx.xxx Jun 28 13:16:44 vThunder a10logd: [SYSTEM]<6> Running co Jun 28 13:19:00 xxx.xxx.xxx.xxx Jun 28 13:17:31 vThunder a10logd: [SYSTEM]<4> Local auth Jun 28 13:19:00 xxx.xxx.xxx.xxx Jun 28 13:17:31 vThunder a10logd: [SYSTEM]<5> A web sess Jun 28 13:19:20 xxx.xxx.xxx.xxx Jun 28 13:17:50 vThunder a10logd: [CFGMGR]<7> Doesn't fi Jun 28 13:19:20 xxx.xxx.xxx.xxx Jun 28 13:17:50 vThunder a10logd: [VCS]<6> dcs config se Jun 28 13:19:20 xxx.xxx.xxx.xxx Jun 28 13:17:50 vThunder a10logd: [VCS]<6> dcs config se Jun 28 13:22:15 xxx.xxx.xxx.xxx Jun 28 13:20:46 vThunder a10logd: [SYSTEM]<5> Session ID Jun 28 13:22:15 xxx.xxx.xxx.xxx Jun 28 13:20:46 vThunder a10logd: [SYSTEM]<6> Session ti Jun 28 13:24:09 xxx.xxx.xxx.xxx Jun 28 13:22:39 vThunder a10logd: [SYSTEM]<4> Local auth Jun 28 13:24:09 xxx.xxx.xxx.xxx Jun 28 13:22:39 vThunder a10logd: [SYSTEM]<5> A web sess So: 7=Debug 6=Info 5=Warning 4=Error However, in my searching, I'm not sure the right way to accomplish this. What I would like in the search is to be able to filter to just the warnings (5), but do it with the word "warning" instead of remembering that number 5 is the warning level. Is that a new index-time field? Can I just add the field as a lookup to my sourcetype? ... View more

We are using the Splunk Add-on for Microsoft Windows to get Windows Event sourcetypes that we're forwarding from Universal Forwarders. We're managing our UF's with a deployment server. I would like to have different settings for inputs.conf with the Splunk_TA_windows add-on for different UF's, but would still like to maintain all types of configuration from the deployment server. Am I correct in thinking I can duplicate the app on my deployment server to apply to other types of UF's as "Splunk_TA_windows2"? Is there a better way to leverage the same app from a deployment server with different configurations for server classes? ... View more

We have a very simple inputs.conf stanza setup to monitor a file system: [monitor:|path|] disabled = false index = Index1 What I've noted is that this has resulted in many different sourcetypes for our data. Based on my research I believe I need to define the sourcetype in inputs.conf However, what's confusing me is that the "learned" sourcetypes are defined in props.conf and sourcetypes.conf on the Universal Forwarder rather than on the Indexers. On the UF there is a sourcetypes.conf stanza: [||filepath|] L-//::.*t. = 0.322842 L-//::._t. = 0.101574 _source = |filepath| _sourcetype = |typename-15| And an props.conf stanza: [typename-15] CHARSET = UTF-8 MAX_TIMESTAMP_LOOKAHEAD = 42 is_valid = True This leads me to two questions: When I define my actual sourcetype I should only define a stanza in props.conf on the INDEXER, correct? Define the inputs.conf with my location to monitor and sourcetype and nothing else? Shouldn't MAX_TIMESTAMP_LOOKAHEAD be part of the parsing logic needed on the Indexer? What is this doing on the UF which, I thought, only sends the data to the Indexer? ... View more