I posted an edit to clarify what i have found so far. Sorry for not doing this earlier. Depending on how old your forwarder was before upgrade, remember that the direct upgrade to forwarder 9+ is only supported from 8.1.x and higher.  That said it I don't think we have seen the end of this yet 😞 ... View more

Not sure if you have seen it, but i posted an update. Unsure on how make it more visible. Any way, if you are using puppet, just ensure that the splunk user is created prior to installation. Then it should work fine. But yes it would be nice if the module was updated as well. ... View more

Hi Splunkers, Just an update on the original post, if you are finding this thread. So after back and forth with support, a few things was fixed:  First in 9.1.1 they fixed that the owner was forcefully changed to "splunkfwd" from "splunk" during an upgrade. But that version gave 1000+ warnings about the user splunk being absent. Then in 9.1.2 the warnings  was  fixed on a fresh install, but they came back when upgrading. Everything should now be fixed in 9.2.0 and later  On top of that, they have implemented that if the user "splunk" exist upon installation, "splunk" will be the owner and not "splunkfwd". So that said, your automation scripts needs to ensure that the "splunk" user exist prior to installation and then everything should be as it use to be. So it is still a change to all the automation out there, but a small one i believe. Now tell me again why this stunt was necessary, since the "splunk" user will be present if Splunk Enterprise is already installed.... 🙂 EDIT: A few packages have been released since this post was created. I want to correct some of my misunderstandings. I still believe this is a huge mistake, but now the warnings are gone and what will happen during installation and upgrade is bit more clear: RPM Installation: The forwarder will use splunkfwd as the owner, no matter what. You can chown the installation folder and change splunk-launch.conf to revert to splunk as the owner. But you have to do it in your script after the rpm installation. RPM Update:  The forwarder will retain splunk as the owner if the previous forwarder installation was owned by splunk. ... View more

I tested a fresh install of 9.1.1 using splunk documented installation procedure. It gave me 1400+ warnings with: warning: user splunk does not exist - using root warning: group splunk does not exist - using root If i created splunk user and group first there was no warning. But now i both have splunk and splunkfwd user!! They should never have change that. It feels like there are no testing of the rpm before shipping them? The installation might be working, but how can this be the quality of the rpm? I logged another support case for this. They have confirmed that this is an unresolved issue. ... View more

One small step for man...  I really hope that a fresh install will be reverted as well. ... View more

I am sorry, but this sounds like a bad excuse for not thinking this through. I have never seen that it was  popular, recommended or even supported to install the forwarder with the server. If you have any good links on this then please supply. If the docker people wants this, then create a solution for them, and leave the rest of us alone.  Imagine all the automation (puppet, ansible, self coded and so on) that now have to be changed. Monitoring of the user and service needs to be changed.  There must be a ton of code/checks/monitoring that needs to be changed.   In regards to when this change was implemented, i did a quick install test (wiped each time):  rpm -i splunkforwarder-7.3.0-657388c7a488-linux-2.6-x86_64.rpm - owner & group = splunk rpm -i splunkforwarder-8.0.4-767223ac207f-linux-2.6-x86_64.rpm - owner & group = splunk rpm -i splunkforwarder-8.2.6-a6fe1ee8894b-linux-2.6-x86_64.rpm - owner & group = splunk rpm -i splunkforwarder-9.0.0-6818ac46f2ec-linux-2.6-x86_64.rpm - owner & group = splunk rpm -i splunkforwarder-9.0.5-e9494146ae5c.x86_64.rpm - owner & group = splunk rpm -i splunkforwarder-9.1.0.1-77f73c9edb85.x86_64.rpm - owner & group = splunkfwd and just to verify i upgraded from 9.0.5 to 9.1.0.1 and yes the owner changed from splunk to splunkfwd. So be careful out there. To be fair support said that this should be fixed in coming 9.1.1 - retaining the previous user. Even the documentation uses "splunk" as the owner all the way from version 9.0 to 9.0.5 https://docs.splunk.com/Documentation/Forwarder/9.0.5/Forwarder/Installanixuniversalforwarder So i simply don't buy the excuse. Now if we are installing the 9.1.0.1 and wants to keep using "splunk" as the owner, we will have to manually , make the install, create "splunk" user, update the unit file, chown SPLUNK_HOME to splunk, update SPLUNK_OS_USER=splunk  in splunk-launch.conf and then delete "splunkfwd", According to support. Just why.  That said, good or bad reason, it does not change the fact that this is done out of the blue with no prior warning. Same happened with the change from initd/systemd and when you changed the service name.  Sorry for the rant, it just makes me annoyed that this should have been handled completely different imo. ... View more

I get it they want to distinguish between server and forwarder with different owners, but in my opinion that is way too late now. First the scenario of having both server and forwarder together must be a tiny tiny fraction of what is setup out there - finding it hard to see why you would at all. Second more than a decade with splunk as the owner makes this change something you should not take lightly if you have just a little respect for your customers. The impact on automation, security and plain familiarity far outweigh the need to separate the server and forwarder. At least this should have been an option not the default.  Then the unprofessional way of doing this in a minor patch without any warnings just makes me furious. I get the same ticks from when they also in a minor patch changed from initd to systemd.  I created a support case asking them to revert or come up with some options.  ... View more

I also had issues with the new Lookup editor and still have. I could not get the list of lookups, it was just blank. Changing /en-GB/ to /en-US/ worked for me. Not sure if it will help you.  ... View more

I actual started out by creating a case with splunk about the issues, but they said it was an intentional design choice to address issues with long text. Make no sense to me but what can I do. So had to move to ideas, where I have already created this idea https://ideas.splunk.com/ideas/APPSID-I-746  ... View more

They forgot to update the documentation link in the app descriptions. https://www.cisco.com/c/en/us/td/docs/security/firepower/670/api/eStreamer_enCore/eStreamereNcoreSplunkOperationsGuide_409.html?dtid=osscdc000283 I had the same issue and found it in https://community.splunk.com/t5/All-Apps-and-Add-ons/Cisco-eStreamer-eNcore-for-Splunk-Add-On-v4-6-0-Missing-Setup/m-p/552294 ... View more

Have the exact same issue - thanks for the workaround ... View more

Thank you so much - i battled with that all day 🙂 It worked like a charm and even made my other query more simpel. I see now that i simply did not understand the documentation of foreach. Now i do. ... View more

I have a support case running on the issue. ... View more

I have the exact same issue right after upgrade to 7.0 - 1 Searchhead - 2 Clustered Indexers ... View more

Thanks for follow up. I did the same. ... View more

Did you ever solve this, i face the same problem. ... View more

I have solved this by removing the Add-On and SA-nix app and reinstall them. ... View more