I have two indexes that have done to really high day age, I got a feeling a few of the PC's have had issues and gone back to default date.  I looked around and can't find a easy way to work out what events it was or the host, I have came close but want something to say here is the smoking gun.  Cheers for any help as per normal. ... View more

Hi, I have data going to my indexers and also selective data going though a HF off to a 3rd party via Syslog. I know splunk works off tubes and if one stops it blocks and queue's back up the chain. Is there any one done something clever basically they have internet issues it stops any data on that HF going to my indexer. The only think I can think of doing is having another HF that proxy's the data and I set a large queue size and say drop data after so long. There for technically never causing a block on our side. ... View more

Hi All, We been having issues with a CSV that is open for a long period of time while a script writes to it no my idea, but I can get the System Admins to budge.  I was using a normal monitor and props to do Field Extractions at index time.  I moved to MonitorNoHandle and it doesn't seem to do field extractions at index time, is this a known thing?  I know I can fix it by doing a Search time field extraction which I am prob going to do, but I can't see it documented anywhere that this is a known thing.  Cheers   ... View more

One of the system admins renewed the certificate and thought they needed to delete the splunk.key file in the mongo folder..  Looking at another server it looks like a password not a true key, how do I regenerate it... We don't have a backup either as they took to long to own up.  Cheers ... View more

We want to stop monitoring of /etc but it seems the /local/inputs.conf is dynamically created. What do I need to do to stop that being monitored. ... View more

Hello, We are sending data to a 3rd party raw, they want the IP of the sending host not the hostname, does splunk have a meta option for this I can update in transform's or can I get the uni forwarder to pass that information on? Any help would be great,I have a 3rd party we send data too, who want to have the IP address as well as the DNS name on the raw traffic. Is there a way of getting the uni forwards to send their IP as as there hostname when they send the events to splunk I can then use transforms to overwrite the hostname to a IP. Cheers ... View more

Hello, I am sending some source types to a 3rd party via SYSLOG as the output as TCP not UDP. All works fine until we lose contact with their syslog server. Then it breaks all indexing even to my cluster. The queues fill up and basically shuts down inputs. Works fine using UDP! Wondering if anyone has came across this issue. He is output of btool. [syslog] maxEventSize = 1024 priority = <13> type = udp [syslog:syslog_indexers] server = syslogserver:9997 timestampformat = %Y-%m-%dT%T.%S type = tcp [tcpout] ackTimeoutOnShutdown = 30 autoLBFrequency = 30 autoLBVolume = 0 blockOnCloning = true blockWarnThreshold = 100 cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-ECDSA-AES256-SHA384:ECDH-ECDSA-AES128-SHA256 compressed = false connectionTimeout = 20 defaultGroup = cluster_indexers disabled = false dropClonedEventsOnQueueFull = 5 dropEventsOnQueueFull = 5 ecdhCurves = prime256v1, secp384r1, secp521r1 forceTimebasedAutoLB = false forwardedindex.0.whitelist = .* forwardedindex.1.blacklist = _.* forwardedindex.2.whitelist = (_audit|_internal|_introspection|_telemetry) forwardedindex.filter.disable = false heartbeatFrequency = 30 indexAndForward = false maxConnectionsPerIndexer = 2 maxFailuresPerInterval = 2 maxQueueSize = auto readTimeout = 300 secsInFailureInterval = 1 sendCookedData = true sslQuietShutdown = false sslVersions = tls1.2 tcpSendBufSz = 0 useACK = true writeTimeout = 300 [tcpout:cluster_indexers] server = indexer1:9997,indexer2:9997 ... View more

Hi, I am getting a weird issue. If the syslog server fails, it stops all data being indexed by the default TCP out, and then Splunk fills its buckets and falls over. Am I missing something to set it to continue if it can't connect to a output. cat outputs.conf [syslog] defaultGroup = xxxxx_indexers [syslog:xxxxx_indexers] server = xxx.xxx.xxx.xxx:9997 type = tcp timestampformat = %Y-%m-%dT%T.%S cat transforms.conf [mehRouting] REGEX = . DEST_KEY = _TCP_ROUTING FORMAT = xxx_cluster_indexers [Routing_firewalls] SOURCE_KEY = MetaData:Sourcetype REGEX = (fgt_traffic|fgt_utm) DEST_KEY = _SYSLOG_ROUTING FORMAT = xxxx_indexers cat props.conf [host::xxxxxxx1c] TRANSFORMS-routing = mehRouting, Routing_firewalls [host::xxxxxc] TRANSFORMS-routing = mehRouting, Routing_firewalls ... View more

Hello, I need to send source types to my indexes as per normal. But I also got to send those same source types to a 3rd party in Syslog format. I can't seem to get the transforms to send to TCP and to Syslog. Has anyone got any experience with doing this? I can do both as TCP which works fine, but if I mix and match it has issues. ... View more

I am sending uncoooked data but the hostname the other end are seeing is the Heavy Forwarder. Is there anyway of sending the correct hostname? ... View more