All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

@marnall For your information I already tried with https before posting this to Splunk answers and for your information on windows server is using telnet instead to SSH.  Can you please help me to... See more...
@marnall For your information I already tried with https before posting this to Splunk answers and for your information on windows server is using telnet instead to SSH.  Can you please help me to understand the significance why you suggested https ? Because on other server posted command is working fine with “http” Please provide your more suggestion on this.
I am trying to setup Azure Event hub to Splunk using the tutorial here.   I followed the tutorial as is   I gave the right permissions (Azure Event hub Data owner as well) to the application but i... See more...
I am trying to setup Azure Event hub to Splunk using the tutorial here.   I followed the tutorial as is   I gave the right permissions (Azure Event hub Data owner as well) to the application but it always gives authentication failed. What am I doing wrong 
Hi everyone. I'm currently trying to install the Universal Forwarder on a Windows client. I haven't installed any previous versions of the Universal Forwarder on this client before. After reaching th... See more...
Hi everyone. I'm currently trying to install the Universal Forwarder on a Windows client. I haven't installed any previous versions of the Universal Forwarder on this client before. After reaching the final stages of the installation, unfortunately, it rolls back and displays a message indicating that the installation wizard did not complete. I'm also attaching the AppCrash report for your reference. Could you please provide some guidance on this? Edit 1: I would like to add that the client is part of a domain, and it is not beneficial whether I perform the installation with the domain admin user or the local admin user, as I still encounter errors. Version=1 EventType=APPCRASH EventTime=133562052818303743 ReportType=2 Consent=1 UploadTime=133562052827678946 ReportStatus=268435456 ReportIdentifier=6a213693-13e6-41a8-8c33-245355f1efbf IntegratorReportIdentifier=5ed072f0-3e6e-4ece-a001-6e76acdb8b27 Wow64Host=34404 NsAppName=splunkd.exe OriginalFilename=splunkd.exe AppSessionGuid=000031bc-0000-000c-9fd1-8bb8fa81da01 TargetAppId=W:00061d36d7ec41eb4da589a3b7ff905efd8600000904!00009bb194c1f79d67ef2b5434b1914ec98a520e1989!splunkd.exe TargetAppVer=2024//03//21:00:03:19!399d613!splunkd.exe BootId=4294967295 TargetAsId=32379 IsFatal=1 EtwNonCollectReason=1 Response.BucketId=bb8a2b9d5336153e35c1c445cd31e043 Response.BucketTable=4 Response.LegacyBucketId=1567749949376028739 Response.type=4 Sig[0].Name=Application Name Sig[0].Value=splunkd.exe Sig[1].Name=Application Version Sig[1].Value=2306.256.26107.30017 Sig[2].Name=Application Timestamp Sig[2].Value=65fb7947 Sig[3].Name=Fault Module Name Sig[3].Value=mimalloc-override.dll Sig[4].Name=Fault Module Version Sig[4].Value=0.0.0.0 Sig[5].Name=Fault Module Timestamp Sig[5].Value=65dfbfa9 Sig[6].Name=Exception Code Sig[6].Value=c0000005 Sig[7].Name=Exception Offset Sig[7].Value=0000000000002ad5 DynamicSig[1].Name=OS Version DynamicSig[1].Value=10.0.20348.2.0.0.400.8 DynamicSig[2].Name=Locale ID DynamicSig[2].Value=1033 DynamicSig[22].Name=Additional Information 1 DynamicSig[22].Value=c13a DynamicSig[23].Name=Additional Information 2 DynamicSig[23].Value=c13a0933a69b5a9aa04a609346aaa13d DynamicSig[24].Name=Additional Information 3 DynamicSig[24].Value=e9e6 DynamicSig[25].Name=Additional Information 4 DynamicSig[25].Value=e9e669e3acebdf636ea1556b4596e7dd UI[2]=C:\Program Files\SplunkUniversalForwarder\bin\splunkd.exe UI[5]=Close UI[8]=splunkd service stopped working and was closed UI[9]=A problem caused the application to stop working correctly. Windows will notify you if a solution is available. UI[10]=&Close LoadedModule[0]=C:\Program Files\SplunkUniversalForwarder\bin\splunkd.exe LoadedModule[1]=C:\Windows\SYSTEM32\ntdll.dll LoadedModule[2]=C:\Windows\System32\KERNEL32.DLL LoadedModule[3]=C:\Windows\System32\KERNELBASE.dll LoadedModule[4]=C:\Windows\System32\CRYPT32.dll LoadedModule[5]=C:\Windows\System32\ucrtbase.dll LoadedModule[6]=C:\Windows\System32\bcrypt.dll LoadedModule[7]=C:\Windows\System32\ADVAPI32.dll LoadedModule[8]=C:\Windows\System32\msvcrt.dll LoadedModule[9]=C:\Windows\System32\sechost.dll LoadedModule[10]=C:\Windows\System32\RPCRT4.dll LoadedModule[11]=C:\Program Files\SplunkUniversalForwarder\bin\mimalloc-override.dll LoadedModule[12]=C:\Windows\System32\WS2_32.dll LoadedModule[13]=C:\Windows\System32\USER32.dll LoadedModule[14]=C:\Windows\System32\win32u.dll LoadedModule[15]=C:\Windows\System32\GDI32.dll LoadedModule[16]=C:\Windows\System32\gdi32full.dll LoadedModule[17]=C:\Windows\System32\msvcp_win.dll LoadedModule[18]=C:\Windows\System32\SHELL32.dll LoadedModule[19]=C:\Windows\System32\ole32.dll LoadedModule[20]=C:\Windows\System32\combase.dll LoadedModule[21]=C:\Windows\SYSTEM32\ACTIVEDS.dll LoadedModule[22]=C:\Windows\SYSTEM32\pdh.dll LoadedModule[23]=C:\Windows\System32\OLEAUT32.dll LoadedModule[24]=C:\Program Files\SplunkUniversalForwarder\bin\libxmlsec.dll LoadedModule[25]=C:\Program Files\SplunkUniversalForwarder\bin\libxmlsec-openssl.dll LoadedModule[26]=C:\Windows\SYSTEM32\WINHTTP.dll LoadedModule[27]=C:\Program Files\SplunkUniversalForwarder\bin\SSLEAY32.dll LoadedModule[28]=C:\Program Files\SplunkUniversalForwarder\bin\libxml2.dll LoadedModule[29]=C:\Program Files\SplunkUniversalForwarder\bin\archive.dll LoadedModule[30]=C:\Program Files\SplunkUniversalForwarder\bin\mimalloc-redirect.dll LoadedModule[31]=C:\Program Files\SplunkUniversalForwarder\bin\VCRUNTIME140.dll LoadedModule[32]=C:\Program Files\SplunkUniversalForwarder\bin\libxslt.dll LoadedModule[33]=C:\Program Files\SplunkUniversalForwarder\bin\LIBEAY32.dll LoadedModule[34]=C:\Program Files\SplunkUniversalForwarder\bin\MSVCP140.dll LoadedModule[35]=C:\Windows\SYSTEM32\adsldpc.dll LoadedModule[36]=C:\Windows\System32\WLDAP32.dll LoadedModule[37]=C:\Windows\System32\bcryptprimitives.dll LoadedModule[38]=C:\Program Files\McAfee\Solidcore\SCINJECT_x64.DLL LoadedModule[39]=C:\Windows\System32\WINTRUST.dll LoadedModule[40]=C:\Windows\SYSTEM32\NETAPI32.dll LoadedModule[41]=C:\Windows\SYSTEM32\MPR.dll LoadedModule[42]=C:\Windows\SYSTEM32\SAMCLI.DLL LoadedModule[43]=C:\Windows\SYSTEM32\NETUTILS.DLL LoadedModule[44]=C:\Windows\SYSTEM32\MSASN1.dll LoadedModule[45]=C:\Windows\SYSTEM32\wkscli.dll State[0].Key=Transport.DoneStage1 State[0].Value=1 OsInfo[0].Key=vermaj OsInfo[0].Value=10 OsInfo[1].Key=vermin OsInfo[1].Value=0 OsInfo[2].Key=verbld OsInfo[2].Value=20348 OsInfo[3].Key=ubr OsInfo[3].Value=2322 OsInfo[4].Key=versp OsInfo[4].Value=0 OsInfo[5].Key=arch OsInfo[5].Value=9 OsInfo[6].Key=lcid OsInfo[6].Value=1033 OsInfo[7].Key=geoid OsInfo[7].Value=244 OsInfo[8].Key=sku OsInfo[8].Value=8 OsInfo[9].Key=domain OsInfo[9].Value=1 OsInfo[10].Key=prodsuite OsInfo[10].Value=400 OsInfo[11].Key=ntprodtype OsInfo[11].Value=3 OsInfo[12].Key=platid OsInfo[12].Value=10 OsInfo[13].Key=sr OsInfo[13].Value=0 OsInfo[14].Key=tmsi OsInfo[14].Value=222600573 OsInfo[15].Key=osinsty OsInfo[15].Value=2 OsInfo[16].Key=iever OsInfo[16].Value=11.1.20348.0-11.0.1000 OsInfo[17].Key=portos OsInfo[17].Value=0 OsInfo[18].Key=ram OsInfo[18].Value=32768 OsInfo[19].Key=svolsz OsInfo[19].Value=99 OsInfo[20].Key=wimbt OsInfo[20].Value=0 OsInfo[21].Key=blddt OsInfo[21].Value=210507 OsInfo[22].Key=bldtm OsInfo[22].Value=1500 OsInfo[23].Key=bldbrch OsInfo[23].Value=fe_release OsInfo[24].Key=bldchk OsInfo[24].Value=0 OsInfo[25].Key=wpvermaj OsInfo[25].Value=0 OsInfo[26].Key=wpvermin OsInfo[26].Value=0 OsInfo[27].Key=wpbuildmaj OsInfo[27].Value=0 OsInfo[28].Key=wpbuildmin OsInfo[28].Value=0 OsInfo[29].Key=osver OsInfo[29].Value=10.0.20348.2322.amd64fre.fe_release.210507-1500 OsInfo[30].Key=buildflightid OsInfo[31].Key=edition OsInfo[31].Value=ServerDatacenter OsInfo[32].Key=ring OsInfo[32].Value=Retail OsInfo[33].Key=expid OsInfo[34].Key=fconid OsInfo[35].Key=containerid OsInfo[36].Key=containertype OsInfo[37].Key=edu OsInfo[37].Value=0 OsInfo[38].Key=servicinginprogress OsInfo[38].Value=0 FriendlyEventName=Stopped working ConsentKey=APPCRASH AppName=splunkd service AppPath=C:\Program Files\SplunkUniversalForwarder\bin\splunkd.exe NsPartner=windows NsGroup=windows8 ApplicationIdentity=C750D84D7F48DB77161DC8FC07E09CE5 MetadataHash=1491437884
Hi @marnall, Thank you for your comment, I will definitely try that next time. At this time, I was able to workaround the issue and upload the app by extracting the add-on tar and performing the foll... See more...
Hi @marnall, Thank you for your comment, I will definitely try that next time. At this time, I was able to workaround the issue and upload the app by extracting the add-on tar and performing the following:  1-Navigate to the add-on default directory. 2- vi addon_builder.conf,  3- change this line accordingly: builder_version = 4.1.3 4- Run: find . -name .DS_Store -delete (macOS)  5- tar the app folder again and re-upload.  It would be beneficial for the vendor to update the add-on to ensure it meets the criteria for official support.   
Hello, does editing ES roles on Permissions page is same as editing ES roles in Splunk's native edit role page? I guess they both point to ES authorize.conf but native's one can work with custom ro... See more...
Hello, does editing ES roles on Permissions page is same as editing ES roles in Splunk's native edit role page? I guess they both point to ES authorize.conf but native's one can work with custom roles? Thanks.  
The query produces multiple pages of results. How do I move the total to the top (first) row for convenience?   search query | eval dayOfWeek=strftime(_time, "%A"), date=strftime(_time, "%Y-%m-%d... See more...
The query produces multiple pages of results. How do I move the total to the top (first) row for convenience?   search query | eval dayOfWeek=strftime(_time, "%A"), date=strftime(_time, "%Y-%m-%d") | eval dayNum=case(dayOfWeek=="Sunday", 1, dayOfWeek=="Monday", 2, dayOfWeek=="Tuesday", 3, dayOfWeek=="Wednesday", 4, dayOfWeek=="Thursday", 5, dayOfWeek=="Friday", 6, dayOfWeek=="Saturday", 7) | stats count as "Session count" by dayOfWeek, date | addtotals col=t row=f label="Month total" |sort date desc  
I am not a developer of the app so I can only speak from my experience: 1. No, the Splunk AI Assistant App translates your input prompt to SPL and/or explains a SPL prompt. It needs only to be insta... See more...
I am not a developer of the app so I can only speak from my experience: 1. No, the Splunk AI Assistant App translates your input prompt to SPL and/or explains a SPL prompt. It needs only to be installed on one server, and it can even be a separate experimental server that is disconnected from your production environment. 2. The Splunk AI Assistant App implements a command "splgen" which is a search command. Thus it is possible to run this command through a Splunk search dispatched via API. See: https://docs.splunk.com/Documentation/Splunk/9.2.1/RESTTUT/RESTsearches 3. The Splunk AI Assistant App provides the SPL query and an explanation if desired. No results of running it. 4. The Splunk AI Assistant App provides a single "best guess" SQL query based on your prompt. Not a selection of alternatives.  
I would not recommend posting valid authorization tokens on the internet, as unscrupulous people or bots could abuse them. Could you try curl-ing the collector health endpoint using HTTPS instead of... See more...
I would not recommend posting valid authorization tokens on the internet, as unscrupulous people or bots could abuse them. Could you try curl-ing the collector health endpoint using HTTPS instead of http? If it still does not give a response, it might be a firewall issue. Try connecting to the machine itself using ssh and then doing a curl on localhost, like this: curl -k https://127.0.0.1:8088/services/collector/health  
Does the app work if you delete that addon_builder.conf file from the tgz file? It seems to contain a lookup, event types, props, tags, and transforms. These should still work even if Splunk complain... See more...
Does the app work if you delete that addon_builder.conf file from the tgz file? It seems to contain a lookup, event types, props, tags, and transforms. These should still work even if Splunk complains about an older add-on builder version.
stoomart, your script appears to be exactly what I need, when I run the script in a RHEL 9 box it immediately transitions to the fsck command with a single bucket option, then displays an error messa... See more...
stoomart, your script appears to be exactly what I need, when I run the script in a RHEL 9 box it immediately transitions to the fsck command with a single bucket option, then displays an error message "path is not extant".  Your thoughts?  Also please clarify where you show "{index-name}", are you using the brackets to indicate placeholders or are they to be used in the thawed bucket path and at the end?   Thank you in advance for your help!  
In this case I suspect starting at the end and working backwards might be helpful. WMI - While it's not terrible for some small testing, I'd suggest not using it because it's *far* more difficult to... See more...
In this case I suspect starting at the end and working backwards might be helpful. WMI - While it's not terrible for some small testing, I'd suggest not using it because it's *far* more difficult to set up, manage, and deal with than using a Universal Forwarder on the actual endpoint.  The UF installs easily, is tiny and efficient, and *also uninstalls easily and completely too*.  And don't take my word for it, Splunk also has docs for this.  I know, it'll sound like they're "pushing the UF for some nefarious reason" but there's nothing nefarious about it, it's just better in nearly every way than using WMI.   https://docs.splunk.com/Documentation/Splunk/9.2.1/Data/ConsiderationsfordecidinghowtomonitorWindowsdata Even neater is to spend the few minutes - it's not terribly hard! - to set up the forwarders to use your splunk as a deployment server. Then on your Splunk you *can* create remote inputs, but instead of being some unreliable "pull" over wmi, it'll be configs sent to the UF to tell it how to collect them locally and send in those logs. And with those changes, all your complaints about WMI will disappear.  I mean, you may have new  complaints, but they won't be about WMI.  "Could not find userBaseDN on the LDAP server" is just generally just 'incorrect configuration'.  Some time in ADSI Edit and the various AD tools may help here. And network devices - it truly depends on your familiarity with syslog etc, but even having had been a Windows admin I found getting network device data into Splunk was at least as easy as getting Windows data in.  You literally started with what I think is the hard part.    There's one or two extra moving parts, but they're all simple, isolated parts in the device->syslog->UF->Splunk path that are easily understood and worked with, vs. the "magic" and weird stuff that the Windows event logs can sometimes conjure up. And a note - we're all 100% volunteers here.  I'm sure the comment about "no time wasters" was just frustration speaking, and that's understandable.  But it did come off as somewhat unkind and I'm sure you would have gotten something of an answer much quicker without that.  No one here that I've ever seen wants to waste your time.  We're all spending our free time trying to help people. 
Did you get this figured out? We are currently fighting the same issue.
Maybe just try to upgrade it to the oldest 8.x version you can get on the downloads page, then uninstall it after that?  
If I understand you correctly, you are configuring a linux host with a Splunk Enterprise installation (not Universal Forwarder installation?) and configuring it to retrieve deployment configurations ... See more...
If I understand you correctly, you are configuring a linux host with a Splunk Enterprise installation (not Universal Forwarder installation?) and configuring it to retrieve deployment configurations from a second server, and you are saying that the first machine properly appears on the "Deployment Clients" interface of the second server when its on version 9.1.3 of Splunk but not on version 9.2.0.1?
It's terrible, they're not easily accessible except through the UI.  It's a big ... sore spot for some of us who need to use these in a more programmatic way. But, there is a way using the REST inte... See more...
It's terrible, they're not easily accessible except through the UI.  It's a big ... sore spot for some of us who need to use these in a more programmatic way. But, there is a way using the REST interface from cURL. curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input Obviously fix the username and password to an admin one, and your hostname if it's not on localhost.  You might want to pipe that through jq to 'pretty print' it if you have jq installed because otherwise it's all smashed together and hard to read: curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input | jq .  You can also see only an individual one if you append the _key's value for the one you want to the end.  (The _key comes from the output of one of the earlier commands.) curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input/6452ce6e55102d0ad735ec31 | jq . You can also delete them or edit them, though ... obviously be careful and do this in a test environment at first! curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input/6452ce6e55102d0ad735ec31 -X DELETE And I've not found a good way to "edit" them, but it's pretty trivial to just edit the JSON you get from an individual entry, and load that back in wholesale. curl -k -u <username>:<password> https://localhost:8089/servicesNS/nobody/splunk_app_db_connect/storage/collections/data/dbx_db_input -d '{ "inputName" : "newEntryforMyDB", "value" : "200", "appVersion" : "3.16.0", "columnType" : 4, "timestamp" : "2024-03-21T13:11:41.633-05:00", "_user" : "nobody", "_key" : "65fc6ce1764e95450b0d98e1" }' -H "Content-Type: application/json" Which would overwrite entry 65fc6... with that new information. Happy Splunking, Rich  
Your *exact* example doesn't make much sense - why would y-d be y1 instead of y2? But at least some of this may be as simple as "makemv" and/or "mvexpand". In your example, it appears as if abcde a... See more...
Your *exact* example doesn't make much sense - why would y-d be y1 instead of y2? But at least some of this may be as simple as "makemv" and/or "mvexpand". In your example, it appears as if abcde are all multi-value fields (the "mv" in the two above commands).  If that's so, ... | mvexpand parameter should make the original into 13 rows.  Once they're separated, perhaps there's some other eval/conditionals you can use to get each output row to include the correct value? If that doesn't work, you may need something like ... ... | makemv delim=" " parameter | mvexpand parameter In any case I think you'll be two steps closer and we can iterate from there.   happy Splunking, Rich
@yuanliu apologies my bad - moving inputlookup at the end is returning all results (NOT just search results)   index="demo1" source="demo2" | rex field=_raw "id_num \{ data: (?P<id_num>\d+) \}" |... See more...
@yuanliu apologies my bad - moving inputlookup at the end is returning all results (NOT just search results)   index="demo1" source="demo2" | rex field=_raw "id_num \{ data: (?P<id_num>\d+) \}" | rex field=_raw "test_field_name=(?P<test_field_name>.+)]:" | search test_field_name="test_field_name_1" | table _raw id_num | reverse | filldown id_num [inputlookup sample.csv | fields FailureMsg | rename FailureMsg AS search | format]    Could you please help ?
@yuanliu Thank you for your response again. Apologies for my wording if it created any confusion. I will be more careful going forward. You're right, I meant my search did not return any results in m... See more...
@yuanliu Thank you for your response again. Apologies for my wording if it created any confusion. I will be more careful going forward. You're right, I meant my search did not return any results in my context.  This query returned my matching search results events . I noticed that id_num field in the search results was blank as I was using filldown to populate id_num fields   index="demo1" source="demo2" [inputlookup sample.csv | fields FailureMsg | rename FailureMsg AS search | format] | rex field=_raw "id_num \{ data: (?P<id_num>\d+) \}" | rex field=_raw "test_field_name=(?P<test_field_name>.+)]:" | search test_field_name="test_field_name_1" | table _raw id_num | reverse | filldown id_num     I moved lookup at the end after filldown and I see id_num field as well in search results table     index="demo1" source="demo2" | rex field=_raw "id_num \{ data: (?P<id_num>\d+) \}" | rex field=_raw "test_field_name=(?P<test_field_name>.+)]:" | search test_field_name="test_field_name_1" | table _raw id_num | reverse | filldown id_num [inputlookup sample.csv | fields FailureMsg | rename FailureMsg AS search | format]      
I'm also having a similar problem. The "user menu" for my Splunk UI is simply not there. With this being the case, I'm not able to change my preferences or simply logout. Any help would be greatly ap... See more...
I'm also having a similar problem. The "user menu" for my Splunk UI is simply not there. With this being the case, I'm not able to change my preferences or simply logout. Any help would be greatly appreciated. 
First, please do not use phrases like "does not work" because it conveys little information in the best scenario.  There are many ways a search "does not work".  There could be an error message.  The... See more...
First, please do not use phrases like "does not work" because it conveys little information in the best scenario.  There are many ways a search "does not work".  There could be an error message.  There could be no error, and no output.  There could be output, but not what you expected. And so on and so on. I assume that what you meant was that the search gave no output.  The problem, then, is that your raw events do NOT have a field named FailureMsg as your OP implied. (I tried to clarify in my previous response.) The fact that index="demo1" source="demo2" ("fail_msg1" OR "fail_msg2") returns results only means that the terms "fail_msg1", "fail_msg2" exist in some events; you need to be explicit about what fields are available at search time. If you do not have a suitable field name in raw events to limit the search, subsearch can still be used to match straight terms by using a pseudo keyword search. index="demo1" source="demo2" [inputlookup sample.csv | fields FailureMsg | rename FailureMsg AS search | format] | rex field=_raw "id_num \{ data: (?P<id_num>\d+) \}" | rex field=_raw "test_field_name=(?P<test_field_name>.+)]:" | search test_field_name=test_field_name_1 | table _raw id_num | reverse | filldown id_num