What you actually mean with this "connection from host (nginx servers) to splunk gets cut (sometimes)."? Is the connection always down, will it start to working after some time or after something has done? Or something else? ... View more

If you are only people who have sc_admin role do as Rich propose. Other option is ask help from your Splunk/Partner account manager. Of course if there are other users which have sc_admin role, you could ask from them that they reset your password. ... View more

Here is Splunk's instructions for use DS currently named Client management or something similar https://help.splunk.com/en/splunk-enterprise/administer/update-your-deployment/9.4/deployment-server-and-forwarder-management/about-deployment-server-and-forwarder-management It needs that you have some apps defined for deploy to your clients. The easiest way to do it is just create e.g. Splunk App for collecting some WindowEvents from UF. Then install it in your server under /opt/splunk/etc/deployment-apps and restart your splunk instance. After that there should be Forwarder management in your Settings menu.   ... View more

Additional to other instructions, here is Splunk's best practice documentation for upgrade https://lantern.splunk.com/Manage_Performance_and_Health/Upgrading_the_Splunk_platform ... View more

As said this is usually done by PS service. Of course there are lot of things which you need to do by yourself. Here is documentation which told general process/project to do it https://lantern.splunk.com/Splunk_Cloud_Platform_Migration If you need to migrate your current data into SCP without reindexing it then you must use PS to help you. If you are just taking SCP into use without data migration then you can do it by yourself after you have gotten SCP stack into use. BUT if you haven't do it before, I strongly recommend to you to take onboard  someone who have already done it. And usually those are PS and/or your local Splunk Partner Company. ... View more

What you have already done?  Probably your listener is working as you can telnet to it. But are you using plain TCP (probably yes)? What you have defined on VM side? Have you define that outputs is sent to Splunk VM1? Have you anything in splunk.log on Server or UF side? Can you see UF's log in server's internal logs? ... View more

Also share your volume definitions if you are using volumes. ... View more

Please create a new question instead of using old one! In that way you will get easier answers for it. ... View more

Have you check that this user (which are running UF) have access to those logs in DC? ... View more

Also how long those events can be? There are defaults for auto extraction and those are not so big than you could expect. ... View more

1st are you sure that his is valid json? I have seen e.g. some weird UTF escaped message from MS databricks etc. Can you check what you have when you are opening event from this > mark and then select "Show Source" from "Event Actions" button. ... View more

Quite probably this is your issue. I expecting that you have some default in your SHC's Deployer for those apps, but then you have lot of changes done via SHC's GUI. Those are not present in Deployer. Then if you have also some configuration which permissions is private then those are not even inside .../etc/apps/<app name> and you cannot get that information before you have change permission for those KOs to app instead of private. Then probably biggest issue is that your users have done those changes inside default "search and reporting" app. Then basically what you should/must do is create a separate app then move those into that app and there export it and install it into your separate SH. But it's hard to say what you must exactly do, without seeing your environment. So maybe it's best to get some local Splunk Partner/Consultant which can come to your place and look it together with you. ... View more

Why you need to rerun it later? There is some things what you need to ensure in your alert and it's search span etc. to ensure that you will get (almost) same results when you rerun it. There is also possibility that e.g. some delayed events has indexed after original run and then you will get different result when you are rerunning it. IMHO: If the default time is not enough you should extend that in Alert configuration. Then you don't need to adjust all those stuff to ensuring that rerun gives you a same results than 1st run has given. ... View more

As already asked we need more information to  your needs and especially your business case! Yes, it's possible to use kafka as an output target in Splunk HF. But why you want to use it and is there any other  way to achieve Splunk -> ElasticSearch which is cheaper and has less moving parts and follow KISS principle? Then if not, which kafka installation you are using? Some open source, Confluent, Aiven or something else. And are you sending a new events which you also ingesting into splunk or are you ingesting those with UF/HF and only target is ES via Kafka. Or are you getting some events for Splunk which have already indexed? And your environment: OS, Splunk, etc. ... View more

One old answers where is instructions how to read Windows evtx exports in linux. https://community.splunk.com/t5/Getting-Data-In/Ingesting-offline-Windows-Event-logs-from-different-systems/m-p/649515   ... View more

If you really need to use those and need logs into splunk, I suppose that you need to do some scripting or programming for that. Just read those logs with your script/program and then send those via Splunk HEC into indexers.  https://dev.splunk.com/enterprise/docs/devtools/httpeventcollector/ told more how to use HEC. ... View more

Is the splunkd still up and running or is it also crashed? ... View more

Probably they have found some to bad bugs and removed it from splunkbase? At least in slack there are some discussions for issues after they have updated to 8.2.2. ... View more

And you have restarted this node after this change? Btool shows only what you have configured on disk but not what your currently running splunk process is using. If you want to see what are parameters in running splunkd you should use "splunk show config web" and look if max_upload_size is there. If not then it's using default value. ... View more

On additional comment. Don't use $SPLUNK_HOME/etc/system/local directory for (almost) any configuration. You cannot change those any other way than updating this manually (or your external configuration tool). It's always better to create a separate/own Splunk app for those and then install those into HF/UF. In that way you can keep better control for those. ... View more

Basically yes. I do it like: There some differences between SCP and Splunk Enterprise! outputs.conf SCP: Copy Splunk Universal Forwarder package from stack e.g. https://<STACK NAME>.splunkcloud.com/en-GB/app/splunkclouduf/setupuf (Download Universal Forwarder Credentials) Splunk Enterprise: Create / update / check Customer’s zzz_base_lnx_uf or zzz_base_win_uf configuration from GIT. There could be several if there are several environments and/or several target where clients are sending events. DS definitions Create / update / check Customer’s zzz_base_ds_uf configuration from GIT. There could be several if there are several environments and/or several DS defined. Copy msi for UF and base uf module to target node or whatever you are using for install it Next   msiexec.exe /i <path to temp>/splunkforwarder-<XXXXX>-x64-release.msi AGREETOLICENSE=yes LAUNCHSPLUNK=no SERVICESTARTTYPE=auto /quiet /l*v install-log.txt Add correct UF package under etc\apps\ (both DS and UF packages). Start SplunkForwarder service After that you should have connections between UF and DS and UF and indexers. Then you can see UF's internal logs from your normal SH and also check and modify UF's inputs on DS side. ... View more

It's exactly this what you need to do.  Is there any real reason (of course needed hw/virtual/storage capacity) which are needed that you haven't have indexer cluster in place? In long run indexer cluster is something which helps you to avoid this situation. Or have you already found what is the root cause to lost those events? ... View more

1st you must know what are those sources or access points what you want to calculate. Then you need to define which sources are coming through those. Then you need to calculate totals per input types like HEC, DBX and UF. You can use those examples from @gcusello and @PrewinThomas to get information from individual nodes, indexes etc. but you cannot get that by access type as easily. ... View more

Can you tell more about this? Where you are getting this? What you have actually try/done? What is your environment? What DB you have? What are your versions (splunk, dbx, jdbc drivers/packages, OS, target db etc.) Is this db_input or something else? Or are you initially configure your DB Connection to source DB? ... View more

Here is documentation what happening when you have license violations and when those are real violations and when those are more or less informative messages. https://help.splunk.com/en/splunk-enterprise/administer/admin-manual/9.3/manage-splunk-licenses/about-license-violations#ariaid-title4 e.g. with Splunk Free there is this limitation: If you generate three or more warnings in a rolling 30-day period, you are in violation of your license. Splunk Enterprise continues to index your data, but you cannot search it. The warnings persist for 14 days. No reset license is available. So after three warnings within 14 calendar days your searches have blocked until there is max two warning in 14 days. But e.g. with Enterprise with more than 100GB/d license you can still search even you have more than 45 breaches within rolling 60 day period. Of course you must contact to Splunk and agree for additional license quota. ... View more