Does anyone know the amount of time a universal forwarder takes to go and recheck the DNS entries of servers listed in the outputs.conf file. If the servers are listed by servername and not IP in the outputs file than Splunk would go out and check for the IP of those servers.  I know it does this at forwarder startup; but doe it also recheck periodically?   I am looking at a situation where the DNS entries for the backend servers get changed to new IPs (in a disaster recovery scenario) and want to know how long it would take the forwarders to start picking up on the new IPs (without having to go out and cycle the all of the forwarders that would talk to the indexers whose IPs got changed). Thanks. ... View more

I am working on seeing if there is a better way that I can consume the data I have from an Azure Blob storage. I am using the Splunk Addon for Microsoft Cloud Services; which is allowing me to grab the file from the Blob storage. Now the file itself is actually a CSV; but the app does not recognize CSV to split up the data so I had to create a props.conf and transforms.conf which does allow me to get the data in and into fields.  But it turns out that the CSV fields has 1 field in it that is the rawMessage and it is actually a JSON of all the field data (and sometimes will contain a field which is not broken into a column of the CSV.  For now I am bringing all of the data in and separated into fields and then at search time doing a spath on the rawMessage field to get the other fields that only sometimes appear. What I would like to do is instead of all of the data being brought in; I want to only bring in the rawMessage field (which is JSON) and have that indexed as it has all of the data/fields. What is the best way to write the inputs/props/transforms to only read that field in and then parse it as a JSON so it will do the autobreakdown for fields? Any help with this would be greatly appreciated. ... View more

I am utilizing the Website Monitoring app and was curious if I had something wrong in my head on the credentials. I always thought that if you did not put any credentials into the Website input then it would just utilize the credentials that the splunk service runs under for the system (on a Windows server, that would be the service account running the splunkd service).  Was my thinking incorrect in this? I am asking as I have a website I am checking that my service account that runs Splunk also has access to the webpage.  When I made the input and did not put any credentials into the input then the check results were always coming back as 401.  After I went in and put the service account credentials into the website check credentials section and let it run it is now coming back with 200. If this is what is needed then we can do that but one of the main reasons we were working with teams to get them to grant the permissions to our Splunk service account was to not have to always be inputting the credentials into all of these if it would just use what Splunk runs its process as. We are currently using version 2.74 of the app. ... View more

I am trying to figure out if there is a query that will tell me which forwarder some of the data I have in my indexers came from.  This question is coming because we have a syslog server that gathers files from multiple servers, and the forwarder on the syslog server reads those syslog and sends to the indexers.  But in doing this it updates the host to be the server that is in each of the files. What I want to know is which of our syslog servers did the actual forwarding of that data over to the indexer. Is that data in there metadata somewhere which can be queried? Thanks. ... View more

I am trying to be able to get data in from an azure function one our of team's has done. We are not able to get the data in through our HEC endpoints as all of those are internal to our network and the azure function sits outside of our network.  So we have gone the route of trying to have the function send into an EventHub  in Azure and then we pull the data from there. using the Microsoft Azure Add-on for Splunk: https://splunkbase.splunk.com/app/3757/    But it only seems to like the standard Azure diagnostic type logged data (admin/metrics).  But not any custom error logging we are sending into that EventHub. I am not seeing any errors for the app saying it can't connect or anything but no luck getting the custom erro logs.   What is the best way to get an applications custom error logs out of an Azure EventHub? ... View more

I am trying to figure out the best way to utilize a regkey we set on Windows server which indicates the Environment the server is for.  And then being able to use that to help limit the returned search events to that type of Environment. Meaning, we have the regkey set on our test servers as test.  So i would like to be able to set something like and eventtype or tag or some other simple way where I can read that regkey on all of our Windows servers and when users go to search they could do something like: host=web* eventtype=test And all of the events for the test servers would be show in the timeframe.  And if a user instead did the search: host=web* eventtype=preprod It would show all of the event for the preprod servers. This does not HAVE to be an eventtype but just using that as a field which is indicating the registry key value without having to create some extra search just for that every time as a user.   The reading of the regkey should also be such that if a server has the regkey changed from test to preprod then that is picked up and going forward the server would show as preprod. We use to use an eventtype that was basically equal to host="*PP*" but our servers now are getting more random names so that no longer will work. Thanks. ... View more

What permissions are required to be able to save a Slideshow?  I am able to save a new slideshow as an Admin in Splunk but when a user goes through the same process it does not save it (it gives them the options to create the slideshow and do the Save function but doesn't do anything and gives no error). Thanks. ... View more

I am experiencing an issue when I have made updates to the apps on the deployer for a search head cluster and trying to push it out to the nodes. The issue seems to be hitting on DBConnect but might be that is just the app that it is hitting it on. The error I am getting is about the file being used by another process. This is the error: Error while deploying apps to first member: Error while updating app=splunk_app_db_connect on target=https://dmmsplunksh 03:8089: Non-200/201 status_code=500; {"messages":[{"type":"ERROR","text":"\n In handler 'localapps': Error installing a pplication: Failed to copy: C:\Splunk\var\run\splunk\bundle_tmp\a1f2000df5b3093c\splunk_app_db_connect to C:\Spl unk\etc\apps\splunk_app_db_connect. 5 errors occurred. Description for first 5: [{operation:\"renaming .tmp file to d estination file\", error:\"The process cannot access the file because it is being used by another process.\", src:\"C:\ Splunk\var\run\splunk\bundle_tmp\a1f2000df5b3093c\splunk_app_db_connect\bin\lib\jtds-1.3.1.jar\", dest:\"C:\Sp lunk\etc\apps\splunk_app_db_connect\bin\lib\jtds-1.3.1.jar\"}, {operation:\"renaming .tmp file to destination file \", error:\"The process cannot access the file because it is being used by another process.\", src:\"C:\Splunk\var\ru n\splunk\bundle_tmp\a1f2000df5b3093c\splunk_app_db_connect\bin\lib\rpcserver-all.jar\", dest:\"C:\Splunk\etc\a pps\splunk_app_db_connect\bin\lib\rpcserver-all.jar\"}, {operation:\"copying contents from source to destination\", error:\"There are no more files.\", src:\"C:\Splunk\var\run\splunk\bundle_tmp\a1f2000df5b3093c\splunk_app_db_conn ect\bin\lib\", dest:\"C:\Splunk\etc\apps\splunk_app_db_connect\bin\lib\"}, {operation:\"copying contents from so urce to destination\", error:\"There are no more files.\", src:\"C:\Splunk\var\run\splunk\bundle_tmp\a1f2000df5b30 93c\splunk_app_db_connect\bin\", dest:\"C:\Splunk\etc\apps\splunk_app_db_connect\bin\"}, {operation:\"copying con tents from source to destination\", error:\"There are no more files.\", src:\"C:\Splunk\var\run\splunk\bundle_tmp\ a1f2000df5b3093c\splunk_app_db_connect\", dest:\"C:\Splunk\etc\apps\splunk_app_db_connect\"}]"}]} has anyone seen this before or know how to get around it so I can push out my updates? ... View more

is there a way to utilize Azure Tags for serverclasses so Splunk knows what deployment-apps need to be pushed out to the server? Thanks for any advice. ... View more

I am having a problem trying to monitor some files on some Windows servers. The directories that I am trying to pick up the files from are: D:\webroot\www.foo1.foo.cd\App_Data\logs D:\webroot\www.foo1.foo.cm\App_Data\logs And the file name will be similar to this: Aggregation.log.20191103.224856.txt I am needing to use wildcards in the path because the portion which is www.foo1.foo. changes based on which environment we are looking at and want to ensure as they bring up different environments the files are just automatically consumed. I tried using an * in the path and it did not work. this is the inputs section right now (which is still not working): Blockquote [monitor://D:\webroot] disabled = false sourcetype = SiteCore:cm whitelist = .+cm\App_Data\logs\Aggregation.+.txt$ [monitor://D:\webroot] disabled = false sourcetype = SiteCore:cd whitelist = .+cd\App_Data\logs\Aggregation.+.txt$ Blockquote any idea what I have wrong ? ... View more

I have a panel in a dashboard which is a search with 2 sub searches joined together. All of them are based off of the same starting criteria. Is there a way that I can have that starting search criteria used in each part of that search? The data would be something like this: Panel data: | other manipulation steps | join sessionId [ search | different manipulation steps] | join sessionId [ search | different manipulation steps] | steps done on the combined data Is this possible to do? ... View more

I am trying to figure out how I can audit who disabled alerts on one of our search heads. I was thinking it is in the _audit index but have not been able to find the information I would expect. I have enabled the alerts again as they need to be running but I do not see an entry in audit that shows I did that even. This is likely just a search that I am not writing correctly. Does anyone have a search that will show me alerts that were enabled/disabled during a time period and what user performed the action? Thanks. ... View more

I have the need to create a dashboard that will be showing a table of data based on date/time and showing the date/time as a column. The data will show the last 7 days. I want to be able to colorize the rows in the dashboard that are within the last 24 hours. Is there an easy way to do this in a dashboard. we are running Splunk Enterprise 7.2.4.2. Thanks. ... View more