All Topics

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Topics

I set up a sample VM for myself to test out Splunk configuration. I wanted a stand-alone service just to make sure I can get my basic configuration running and forward logs from a Kubernetes instance... See more...
I set up a sample VM for myself to test out Splunk configuration. I wanted a stand-alone service just to make sure I can get my basic configuration running and forward logs from a Kubernetes instance. However, I am stuck in verification of the event receive resource. Here's the steps I followed: Setup a Linux VM Get Splunk installed Confirm web is working as expected Create an index called splunk_test_events that is of (Type: events, App: search) Go to Settings > Forwarding and Receiving and set up a port for 9997 In Settings > Data Inputs set up an HTTP Event Collector (details below) Ensure tokens are enabled (I forget where this was) Restart Splunk SSH into the machine and check the running ports (see below) Attempt to curl and event So the HTTP Event Collector I set up as: Name splunk_testing_events Source Type Entered Source Type Selected Allowed Indexes splunk_test_events Default Index splunk_test_events Output Group None Enable Indexer Acknowledgement On    I verified that the HTTP Event Collector is enabled. I log into the machine and check the ports that are active: $ sudo lsof -i -P -n | grep LISTEN systemd-r 649 systemd-resolve 13u IPv4 23727 0t0 TCP 127.0.0.53:53 (LISTEN) sshd 751 root 3u IPv4 26648 0t0 TCP *:22 (LISTEN) sshd 751 root 4u IPv6 26650 0t0 TCP *:22 (LISTEN) splunkd 6405 root 4u IPv4 63003 0t0 TCP *:8089 (LISTEN) splunkd 6405 root 60u IPv4 63818 0t0 TCP *:9997 (LISTEN) splunkd 6405 root 128u IPv4 123397 0t0 TCP *:8088 (LISTEN) splunkd 6405 root 156u IPv4 64895 0t0 TCP *:8000 (LISTEN) mongod 6482 root 10u IPv4 61364 0t0 TCP *:8191 (LISTEN) python3.7 6623 root 7u IPv4 63884 0t0 TCP 127.0.0.1:8065 (LISTEN)   Now I try and send a curl event over: curl -v -k -H "Authorization: Splunk GENERATED_HEC_TOKEN" http://VM_PUBLIC_IP:9997/services/collector/event -d '{ "event": "testing manually" }'    I get back an error: * Trying VM_PUBLIC_IP:9997... * Connected to VM_PUBLIC_IP (VM_PUBLIC_IP) port 9997 (#0) > POST /services/collector/event HTTP/1.1 > Host: VM_PUBLIC_IP:9997 > User-Agent: curl/7.74.0 > Accept: */* > Authorization: Splunk GENERATED_HEC_TOKEN > Content-Length: 31 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 31 out of 31 bytes * Empty reply from server * Connection #0 to host VM_PUBLIC_IP left intact curl: (52) Empty reply from server   I tried some of the other ports: 8088: Connection reset by peer 8089: Connection reset by peer 8000: HTTP/1.1 303 (which I expected in this case) What am I doing wrong here? 
I see the following errors when running a search against data in a vix. We have recently upgraded to 8.1.3 when I assume the thirdparty jar files changed from 1.10 to 1.19. I think there is som... See more...
I see the following errors when running a search against data in a vix. We have recently upgraded to 8.1.3 when I assume the thirdparty jar files changed from 1.10 to 1.19. I think there is some config that is pointing to the old 1.10.jar file I have looked in indexes.conf for the vix configuration which references the path  changed it to the new version of commons-compress-1.19.jar (see below) and deployed it to the SHC, however it does not seem to make any difference.  Can anyone help?
I need your help to backup the entire set of the .conf files in Splunk Ent. & ES separately please. Can this backup be scheduled? Is scheduled back up here recommended? Thanks a million in advance.
Hi there! Please allow me to admit, I'm newbie to splunk + sigma  rules for detection. In my test environment, I have imported windows sysmon event logs. I understand that using sigmac, I can creat... See more...
Hi there! Please allow me to admit, I'm newbie to splunk + sigma  rules for detection. In my test environment, I have imported windows sysmon event logs. I understand that using sigmac, I can create rules for splunk. My Q is how would I use those sigma rules for use with splunk for detection ?  My understanding is that when I ingest new logs, splunk would auto run those rules against newly ingested logs ? Thank you
I am ingesting a text file and I have created a field called Flag. I am looking to create a filter which only shows me events where the first two characters of that field are in capitals.   I.e. I ... See more...
I am ingesting a text file and I have created a field called Flag. I am looking to create a filter which only shows me events where the first two characters of that field are in capitals.   I.e. I want to see event where Flag is VMs, SVictor, ARev but not Amy, Fox or Dana.   Can you help?
Hello everyone. I'm getting Forced bundle replication failed. Reverting to old behavior - using most recent bundles on all on a search head, and I'm not sure how to fix this. I excluded heavy files f... See more...
Hello everyone. I'm getting Forced bundle replication failed. Reverting to old behavior - using most recent bundles on all on a search head, and I'm not sure how to fix this. I excluded heavy files from the bundle, also restarted the search head, but nothing changes. Where should I dig? I wasn't able to find this error message in Splunk documentation and on the internet. The closest topic on Splunk answers was related to search head clustering, but since I wasn't setting up SH clustering, I guess it's not applicable. Additional info. Before the issue occurred, I've noticed that disk usage on indexers went to 100%. I solved it by deleting data from /opt/splunk/var/run/searchpeers (except the latest files). My environment: - 4 indexer VMs. - 2 search head VMs (not clustered, just testing Splunk 7 and Splunk 8 in parallel). 4 indexers are connected as distributed search peers to each of those search heads. - No deployment server in use. Sometimes network connection is not good between indexers and search head, so maybe it contributes somehow. Any suggestions and ideas appreciated.  
Hi, I'am trying to map alerts for mitre_technique_id from one of my APIs, and I see a strange behaviour from splunk CIM pie chart where in it says "Your search returned no results". Although, I can ... See more...
Hi, I'am trying to map alerts for mitre_technique_id from one of my APIs, and I see a strange behaviour from splunk CIM pie chart where in it says "Your search returned no results". Although, I can see the mapped values dumped inside the splunk base while performing a search query. The data is dumped as expected but not being populated on the pie chart, giving the error message as in the picture below.   Please reply or comment if any known resolutions. Thank you!
we have indexer , search head and heavy forwarder in a vessel , the heavy forwarder will send the data to a head office , but due to the vessel is moving in international wate... See more...
we have indexer , search head and heavy forwarder in a vessel , the heavy forwarder will send the data to a head office , but due to the vessel is moving in international water or far from the head office , the head office indexers disconnected from the vessel , we know the heavy forwarder buffer the data until the indexers became available again , but the buffer is in memory(RAM) , and the buffered data will be very large -as the vessel disconnected long time -so the memory may be full and heavy forwarder will crash , now my question, do we can make the heavy forwarder buffer the data on the hard disk not on the memory ,or any other solution to this case ?  
Dear, Kindly please help with creating an official support account with case opening privileges. Best Regards
Hi, Im relatively new to Splunk, and am looking for some experiences and advice. A company I work for currently have a large (2TB index volume/day) on-prem deployment. It is going to be moved to c... See more...
Hi, Im relatively new to Splunk, and am looking for some experiences and advice. A company I work for currently have a large (2TB index volume/day) on-prem deployment. It is going to be moved to cloud at some point, and I am trying to get an overview over our three options which are to either deploy in Azure, AWS or purchase Splunk Cloud (business reasons). A hybrid solution may be an alternative as well. We are at a very early stage, and we will involve Splunk at some point. Right now Im just trying to get a sense of which aspects we need to consider and where to start looking for information. Has anybody done any assessments of one or more of these alternatives, or perhaps moved their on-prem to one of these clouds? Any main pros/cons, things to think about etc.? Any good source of information is highly appreciated. Thanks in advance!
Hello people, I'm very new to Splunk and I'm trying to create a dashboard with the "Statistics Table" Visualisation, that is more compact and takes up less space than what was there before. In orde... See more...
Hello people, I'm very new to Splunk and I'm trying to create a dashboard with the "Statistics Table" Visualisation, that is more compact and takes up less space than what was there before. In order to achieve this, I'm concatenating several strings from different fields, kind of like this: | eval compactfield="1. ".field1." 2. ".field2." 3. ".field3 and while that looks fine and works, it'd be great if I could add some kind of color tag (or maybe even bold, italic and so on) so that there'd be a different color for each line, making it easier to differentiate for people looking at the dashboard. I'm imagining something like this: | eval compactfield="<col="blue">1. ".field1." </col><col="red">2. ".field2." </col><col="purple">3. ".field3."</col>" Is there a way to achieve this? I'm really sorry if this question has been asked before, but I couldn't find anything. Thanks for your time, Cyd 
We are currently wanting to ingest logs from azure china into Splunk, while it seems this app only support public azure and government azure. Is there any changes need to be done to support Azure Ch... See more...
We are currently wanting to ingest logs from azure china into Splunk, while it seems this app only support public azure and government azure. Is there any changes need to be done to support Azure China? Thanks
Hello Everyone,   I need Help. We have a trouble  in Splunk Cluster and I want to find out and investigate this is bad for our system and what will be the effects. please Help me out
Hi, Based on my understanding, from the Splunk Guide, https://docs.splunk.com/Documentation/ES/6.6.0/Admin/Configurecorrelationsearches, I just need to add the notable under adaptive response. I did... See more...
Hi, Based on my understanding, from the Splunk Guide, https://docs.splunk.com/Documentation/ES/6.6.0/Admin/Configurecorrelationsearches, I just need to add the notable under adaptive response. I did   However, I try to fail log in, the notable events shown in the Incident Review dashboard, is 0. But when I run the spl, I embedded in my correlation search, the SPL  can fetch all the fail logged ins that I tried
Hello, I have some issues writing PROPS configuration for XML source file. Sample XML events (2 Events) are given below. Any help will be highly appreciated. Thank you so much. TIME_PREFIX= TIME_F... See more...
Hello, I have some issues writing PROPS configuration for XML source file. Sample XML events (2 Events) are given below. Any help will be highly appreciated. Thank you so much. TIME_PREFIX= TIME_FORMAT= LINE_BREAKER= -------------------------------- <a2ETraceEvent xmlns=http://schemas.test.com/2014/06/a2ETraceEvent>     <System xmlns=http://schemas.test.com/2014/08/windows/events/systems>         <EventID>0</EventID>         <Type>3</Type>         <SubType Name="Error">0</SubType>         <Level>2</Level>         <TimeCreated SystemTime="2021-07-20T04:00:53.4370283Z" />         <Source Name="ATech.Notifications" />         <Correlation ActivityID="{975c26b1-7acd-4ea0-8ad6-d7be1358e5fc}" />         <Execution ProcessName="ATech.JobFramework.Job" ProcessID="292132" ThreadID="1" />         <AssemblyVersion>6.4.10100.1051</AssemblyVersion>         <Channel />         <Computer>XVL0SMEMAPPAGR14</Computer>     </System>     <ApplicationData>         <TraceData>             <DataItem>                 <TraceRecord Severity="Error" xmlns=http://schemas.test.com/2014/10/a2ETraceEvent/TraceRecord>                     <TraceIdentifier>ATech.Notifications</TraceIdentifier>                     <Description>Error sending the email message generated for notification template 'Employee Training - ' with id = '12552'.</Description>                     <AppDomain>ATech.JobFramework.Job.exe</AppDomain>                     <Exception>                         <ExceptionType>ATech.Common.Exceptions.SendEmailNotificationException, ATech.Common, Version=6.4.10100.1051, Culture=neutral, PublicKeyToken=null</ExceptionType>                         <Message>Error sending the email message generated for notification template 'Employee Training - with id = '12552'.</Message>                         <Source />                         <ContextData>                             <Resolution>Please verify that the server configured in the ECPSA is reachable. For further support, please contact your system administrator.</Resolution>                             <ServerAddress>Changeit-mail-relay</ServerAddress>                         </ContextData>                         <StackTrace />                         <InnerException>                             <ExceptionType>System.Net.Mail.SmtpException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934a19</ExceptionType>                             <Message>Failure sending mail.</Message>                             <Source>System</Source>                             <StackTrace>   at System.Net.Mail.SmtpClient.Send(MailMessage message)    at ATech.Notifications.Providers.Mail.DefaultSmtpProvider.Send(MailMessage mailMessage, Notification notification)</StackTrace>                             <InnerException>                                 <ExceptionType>System.Net.WebException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=c77a52161934e08</ExceptionType>                                 <Message>The remote name could not be resolved</Message>                                 <Source>System</Source>                                 <StackTrace>       at System.Net.Mail.SmtpConnection.GetConnection(ServicePoint servicePoint)    at System.Net.Mail.SmtpClient.GetConnection()    at System.Net.Mail.SmtpClient.Send(MailMessage message)</StackTrace>                             </InnerException>                         </InnerException>                     </Exception>                 </TraceRecord>             </DataItem>         </TraceData>     </ApplicationData> </a2ETraceEvent> <a2ETraceEvent xmlns=http://schemas.test.com/2014/06/a2ETraceEvent>     <System xmlns=http://schemas.test.com/2014/08/windows/events/systems>         <EventID>1</EventID>         <Type>3</Type>         <SubType Name="Error">2</SubType>         <Level>1</Level>         <TimeCreated SystemTime="2021-07-20T04:00:54.4370283Z" />         <Source Name="ATech.Notifications" />         <Correlation ActivityID="{875c26b1-7acd-2ea0-8ad6-d7be1358e5f1}" />         <Execution ProcessName="ATech.JobFramework.Job" ProcessID="122132" ThreadID="1" />         <AssemblyVersion>6.4.10101.1061</AssemblyVersion>         <Channel />         <Computer>XVL0SMEMAPPAGR14</Computer>     </System>     <ApplicationData>         <TraceData>             <DataItem>                 <TraceRecord Severity="Error" xmlns=http://schemas.test.com/2014/10/a2ETraceEvent/TraceRecord>                     <TraceIdentifier>ATech.Notifications</TraceIdentifier>                     <Description>Error sending the email message generated for notification template 'Employee Training - ' with id = '237521.</Description>                     <AppDomain>ATech.JobFramework.Job.exe</AppDomain>                     <Exception>                         <ExceptionType>ATech.Common.Exceptions.SendEmailNotificationException, ATech.Common, Version=6.4.10100.1051, Culture=neutral, PublicKeyToken=null</ExceptionType>                         <Message>Error sending the email message generated for notification template 'Employee Training - with id = '237521'.</Message>                         <Source />                         <ContextData>                             <Resolution>Please verify that the server configured in the ECPSA is reachable. For further support, please contact your system administrator.</Resolution>                             <ServerAddress>Changeit-mail-relay</ServerAddress>                         </ContextData>                         <StackTrace />                         <InnerException>                             <ExceptionType>System.Net.Mail.SmtpException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=g77a5c561944t16</ExceptionType>                             <Message>Failure sending mail.</Message>                             <Source>System</Source>                             <StackTrace>      at ATech.Notifications.Providers.Mail.DefaultSmtpProvider.Send(MailMessage mailMessage, Notification notification)</StackTrace>                             <InnerException>                                 <ExceptionType>System.Net.WebException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=c77c52161934r19</ExceptionType>                                 <Message>The remote name could not be resolved</Message>                                 <Source>System</Source>                                 <StackTrace>   at System.Net.ServicePoint.GetConnection(PooledStream PooledStream, Object owner, Boolean async, IPAddress&amp; address, Socket&amp; abortSocket, Socket&amp;)    at System.Net.Mail.SmtpConnection.GetConnection(ServicePoint servicePoint)    at System.Net.Mail.SmtpClient.GetConnection()    at System.Net.Mail.SmtpClient.Send(MailMessage message)</StackTrace>                             </InnerException>                         </InnerException>                     </Exception>                 </TraceRecord>             </DataItem>         </TraceData>     </ApplicationData> </a2ETraceEvent>
Hallo, i am trying to make a Dashboard that takes the time from reports of jobs. That time is not the same as the time in Splunk. So the problem is that the time picker like  "last 7 days" shows m... See more...
Hallo, i am trying to make a Dashboard that takes the time from reports of jobs. That time is not the same as the time in Splunk. So the problem is that the time picker like  "last 7 days" shows more than 7 days and the First and Last are not complete. That is a Problem that i cant fix in the search code itself, i think. Code: | eval NewTime=strptime(StartDateTZ,"%Y-%m-%d %H:%M:%S") | eval _time=NewTime | timechart span=1d@d count by TestName limit=0
Hello support, I'm planning to use edit_tcp to send data for indexing to an REST endpoint in Splunk (no need to use a forwarder). My question: - Is it possible the send data to a specific Index or... See more...
Hello support, I'm planning to use edit_tcp to send data for indexing to an REST endpoint in Splunk (no need to use a forwarder). My question: - Is it possible the send data to a specific Index or by default, it will send to all indexes ? - Is it possible to restrict sending data to only one index ?   Thank you.
The name of each dashboard in the browser tab is merely "Dashboard".  How do I configure a custom name?  I would expect the default name for the tab to be like the dashboard name, like in the xml d... See more...
The name of each dashboard in the browser tab is merely "Dashboard".  How do I configure a custom name?  I would expect the default name for the tab to be like the dashboard name, like in the xml dashboards.
I have this result response[sample]:   "{\"meta\":{\"code\":400}},[Content-Type:\"application/json\", Transfer-Encoding:\"chunked\", Date:\"Mon, 13 Sep 2021 17:25:12 GMT\", Keep-Alive:\"timeout=60\... See more...
I have this result response[sample]:   "{\"meta\":{\"code\":400}},[Content-Type:\"application/json\", Transfer-Encoding:\"chunked\", Date:\"Mon, 13 Sep 2021 17:25:12 GMT\", Keep-Alive:\"timeout=60\", Connection:\"keep-alive\"]" I want value of  field code to be extracted I tried first to extract json out of this string "{\"meta\":{\"code\":400}},' but it looks i dont need to do these because i just want value of  field code I tried below but got stuck to remove "/" .  It would be nice to extract json and get code value but just getting the field code from above will also suffice | eval responseJson0 = replace(responseJson,"\/", "") | eval responseJson1 = replace(responseJson,"<", "") | eval responseJson2 = replace(responseJson1,">", "") | eval responseJson3 = replace(responseJson2,"200,", "")
Hello,   I am trying to build a chart based on 3 fields: 2 calculated fields and a simple one: |  query="select OPEN_FY, OPEN_QUARTER, CLOSURE_FY, VULNERABILITY_LIFECYCLE, SOURCE, LAYER from tabl... See more...
Hello,   I am trying to build a chart based on 3 fields: 2 calculated fields and a simple one: |  query="select OPEN_FY, OPEN_QUARTER, CLOSURE_FY, VULNERABILITY_LIFECYCLE, SOURCE, LAYER from table | [evaluate] DETECTION_TIME=if((OPEN_FY="21/22" AND OPEN_QUARTER ="Q2"),"new" , "old") | [evaluate]  SOURCE=if((SOURCE!="QUALYS-P"), "Confirmed", "Potential") | chart count(DETECTION_TIME) by SOURCE over(LAYER) the last line won't work. I would need to see the total number of vulnerabilities by source by  detection time and by layer. Is that possible? Thanks