Any update on this. I am having same problem. I was breaking my head for days.  I confirmed same log behavior and same index/search field extraction problem. This is causing issues with apps like webayalytics. ... View more

Perfect, this gave me what I was looking for. And you gave me more ideas. I can also count field1 and changed IP entries over period of time which would indicate problem as well. thank you so much. ... View more

I had similar problem with inputs.conf that did not specify an index or sourcetype with monitor stanza. Example someone had entered monitor for SMTP logging of SMTP windows server but did not specify index to send data to. Example command: ./splunk add monitor C:\windows\system32\logfiles\SMTPSVC1 It created inputs.conf in default "Search" app on Windows universal forwarder [monitor://C:\Windows\System32\LogFiles\SMTPSVC1] Also assigned hostname as sourcetype All SMTP data was sent to "main" index. Search your inputs.conf for items without sourcetypes or specific index naming [monitor://C:\Windows\System32\LogFiles\SMTPSVC1] sourcetype = mssmtp index = mssmtp ,I had similar problem with inputs.conf that did not specify an index with monitor stanza. Example someone had entered monitor for SMTP logging of SMTP windows server but did not specify index to send data to. Example command: ./splunk add monitor C:\windows\system32\logfiles\SMTPSVC1 Created inputs.conf without specifying index. We modified inputs.conf to include index as follows. [monitor://C:\Windows\System32\LogFiles\SMTPSVC1] Also assigned hostname as sourcetype Search your inputs.conf for items without sourcetypes or specific index naming [monitor://C:\Windows\System32\LogFiles\SMTPSVC1] sourcetype = mssmtp index = mssmtp ... View more

** Modified powershell to skip if 7.2.4 already exists on system** ** Modified .bat file to only run once at service start** ** This allows you to install and not loop Uninstall-Install of same UF update** I used this process to remove older version of windows UF and upgrade using app deployment. Main piece was batch file in /bin/scripts directory that allowed me to run what ever I needed. You need to create app with files and directories listed below on deployment server, create a server class to add your old UF clients and push out using normal app deployment process. Make sure to remove clients from server class once they are upgraded. This process removed older version 7.2.x, and installs 7.2.4. It also removes old and adds new Deployment Server configuration to UF client that your upgrading. My Deployment server then pushed out any default apps my windows systems needed. Its not pretty but it works and you can improve it. I had to modify directory names so careful that scripts match what you create. App location: /opt/splunk/etc/deployment-apps/windows_ufupdate7.24/ App contents and folders you have to create /local/app.conf (blank file needed for Deployment server) /bin/splunkforwarder-7.2.4-8a94541dcfac-x64-release.msi (downloaded from splunk.com) /bin/SplunkFullInstall.ps1 (example below) /bin/CopyConfigs.ps1 (adds app to UF client with the Deployment server IP. Example below) /bin/scripts/installer.bat (example below) /bin/configfiles/deploymentclient/local/app.conf (blank file needed for Deployment server) /bin/configfiles/deploymentclient/local/deploymentclient.conf (example below) /default/inputs.conf (needed to execute windows bat file. Example below) /bin/SplunkFullInstall.ps1 $currentUser = New-Object Security.Principal.WindowsPrincipal $([Security.Principal.WindowsIdentity]::GetCurrent()) $testadmin = $currentUser.IsInRole([Security.Principal.WindowsBuiltinRole]::Administrator) if ($testadmin -eq $false) { Start-Process powershell.exe -Verb RunAs -ArgumentList ('-noprofile -noexit -file "{0}" -elevated' -f ($myinvocation.MyCommand.Definition)) exit $LASTEXITCODE } #### SPLUNK 7.2.4 New-PSDrive -Name HKCR -PSProvider Registry -Root HKEY_CLASSES_ROOT $regkeypath= "HKCR:\Installer\Products\F63C3FFC168A520418A4FF5C143E9D11" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Write-Host "The value does exist" Start-Sleep -s 25 exit} Else {Write-Host "The value does not exist"} $regkeypath= "HKCR:\Installer\Products\562176F993A508143953E0C4217E1283" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\562176F993A508143953E0C4217E1283 -Recurse} Else {Write-Host "The value does not exist"} $regkeypath= "HKCR:\Installer\Products\D8F76B23C2EAA254395F55894752D642" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\D8F76B23C2EAA254395F55894752D642 -Recurse} Else {Write-Host "The value does not exist"} $regkeypath= "HKCR:\Installer\Products\E698A50C4F506B941A19CF921B63B218" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\E698A50C4F506B941A19CF921B63B218 -Recurse} Else {Write-Host "The value does not exist"} #### SPLUNK 6.3.0 $regkeypath= "HKCR:\Installer\Products\C042F9A1CE44AA641A538C56CC9204B1" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\C042F9A1CE44AA641A538C56CC9204B1 -Recurse} Else {Write-Host "The value does not exist"} #### SPLUNK 6.6.1 $regkeypath= "HKCR:\Installer\Products\38133C98C7FC0BA42863BB91DCE49DC1" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\38133C98C7FC0BA42863BB91DCE49DC1 -Recurse} Else {Write-Host "The value does not exist"} #### SPLUNK 6.5.2 $regkeypath= "HKCR:\Installer\Products\D9D0997EE40A2E545AF41AA604E579C7" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\D9D0997EE40A2E545AF41AA604E579C7 -Recurse} Else {Write-Host "The value does not exist"} #### SPLUNK 6.5.3 $regkeypath= "HKCR:\Installer\Products\727981DA44845B34E97839CA1F7880DA" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\727981DA44845B34E97839CA1F7880DA -Recurse} Else {Write-Host "The value does not exist"} #### SPLUNK 6.6.3 $regkeypath= "HKCR:\Installer\Products\E6FA769F0A983BE4CABE3C4CB5E03FEE" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\E6FA769F0A983BE4CABE3C4CB5E03FEE -Recurse} Else {Write-Host "The value does not exist"} #### SPLUNK 7.0.1 $regkeypath= "HKCR:\Installer\Products\4E3F41596A1605B488E5410B7345105E" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\4E3F41596A1605B488E5410B7345105E -Recurse} Else {Write-Host "The value does not exist"} # SPLUNK 7.2.1 $regkeypath= "HKCR:\Installer\Products\1B38171A5C10C7C47A5E38811D50D4C8" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\1B38171A5C10C7C47A5E38811D50D4C8 -Recurse} Else {Write-Host "The value does not exist"} # SPLUNK 7.2.3 $regkeypath= "HKCR:\Installer\Products\ADD0EC2EB45B100469D56B727798A671" $value1 = (Get-Item $regkeypath) -eq $null If ($value1 -eq $False) {Remove-Item -path HKCR:\Installer\Products\ADD0EC2EB45B100469D56B727798A671 -Recurse} Else {Write-Host "The value does not exist"} stop-service SplunkForwarder stop-service Splunkd Start-Sleep -s 15 Remove-Item 'C:\Program Files\SplunkUniversalForwarder\etc\system\local\deploymentclient.conf' -Force -ErrorAction SilentlyContinue Invoke-Command -scriptblock { msiexec.exe /q ALLUSERS=2 /m MSIASQSH /i "$PSScriptRoot\splunkforwarder-7.2.4-8a94541dcfac-x64-release.msi" AGREETOLICENSE=Yes SPLUNKPASSWORD=buddy-holly-weezer } Start-Sleep -s 15 Copy-Item "$PSScriptRoot\ConfigFiles\*" $env:programfiles\SplunkUniversalForwarder\etc\apps -Force -Recurse Start-Sleep -s 30 restart-service SplunkForwarder Start-Sleep -s 30 start-service SplunkForwarder Batch file */bin/scripts/installer.bat * @ECHO OFF PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& "$env:programfiles\SplunkUniversalForwarder\etc\apps\utsa_windows_uf_7.2.4\bin\SplunkFullInstall.ps1"" inputs.conf tells app to launch batch file /default/inputs.conf [script://.\bin\scripts\installer.bat] interval = -1 ** point client to Deployment Server** ** /bin/configfiles/deploymentclient/local/deploymentclient.conf ** [deployment-client] [target-broker:deploymentServer] # Change the targetUri targetUri = 10.5.5.155:8089 ** update Deployment server on client** /bin/CopyConfigs.ps1 $currentUser = New-Object Security.Principal.WindowsPrincipal $([Security.Principal.WindowsIdentity]::GetCurrent()) $testadmin = $currentUser.IsInRole([Security.Principal.WindowsBuiltinRole]::Administrator) if ($testadmin -eq $false) { Start-Process powershell.exe -Verb RunAs -ArgumentList ('-noprofile -noexit -file "{0}" -elevated' -f ($myinvocation.MyCommand.Definition)) exit $LASTEXITCODE } Copy-Item "$PSScriptRoot\ConfigFiles\*" $env:programfiles\SplunkUniversalForwarder\etc\apps -Force -Recurse Start-Sleep -s 10 restart-service SplunkForwarder ... View more