All Topics

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Topics

Hi, Everyone!   Planning your Smart Agent installation mise-en-place? Get the facts, encouragement, and inspiration here: Smart Agent FAQ | Getting Started: Installation What do you think? D... See more...
Hi, Everyone!   Planning your Smart Agent installation mise-en-place? Get the facts, encouragement, and inspiration here: Smart Agent FAQ | Getting Started: Installation What do you think? Don't forget to ask your own questions and share your insights! As a community, we are all here to help! Our team would love to hear your thoughts, including how we can add to and improve the FAQ Please do share your impressions, considerations, and questions below. Our Smart Agent FAQ has a lot of information about Smart Agent and related features. We thought you might appreciate this quick way to get to the topics that most interest you, paired with a place to ask questions and enlarge on your take... 
Hi, Everyone!   So, what can you expect from the Agent Management UI?  There are many new improvements to the user interface to support agent management, and we have been addressing many questions ... See more...
Hi, Everyone!   So, what can you expect from the Agent Management UI?  There are many new improvements to the user interface to support agent management, and we have been addressing many questions that are sure to help you on your journey. Controller Agent Management console? RBAC? Check out these and other UI-related frequently asked questions here: Smart Agent FAQ | Agent Management User Interface What do you think? Have a question of your own? Post it here and let us help. Our team would love to hear your thoughts, including how we can add to and improve the FAQ. Post your questions here and let us help. Please do share your impressions, considerations, and questions below. Our Smart Agent FAQ has a lot of information about Smart Agent and related features. We thought you might appreciate this quick way to get to the topics that most interest you, paired with a place to ask questions and enlarge on your take... 
Hi, Everyone!   Thinking about how Smart Agent integrates with your CI/CD pipelines? Is agent management encouraged for existing CI/CD pipelines? Um..., yes! See the details here, and please sha... See more...
Hi, Everyone!   Thinking about how Smart Agent integrates with your CI/CD pipelines? Is agent management encouraged for existing CI/CD pipelines? Um..., yes! See the details here, and please share your questions and impressions below: Smart Agent FAQ | Tooling Pipeline Guidelines What do you think? Our team would love to hear your thoughts, including how we can add to and improve the FAQ Please do share your impressions, considerations, and questions below. Our Smart Agent FAQ has a lot of information about Smart Agent and related features. We thought you might appreciate a quick way to get to the topics that most interest you, paired with a place to ask questions and enlarge on your take... 
Hi, Everyone!   Starting to think about your agent management strategy?  Check out topical questions to spur your planning and inspire more questions: Smart Agent FAQ | Strategy How does Smart ... See more...
Hi, Everyone!   Starting to think about your agent management strategy?  Check out topical questions to spur your planning and inspire more questions: Smart Agent FAQ | Strategy How does Smart Agent manage existing agents—or new planned ones? What if there are hundreds, or more? How will it really work for your day-to-day? What do you think? Our team would love to hear your thoughts, including how we can add to and improve the FAQ Please do share your impressions, considerations, and questions below. Our Smart Agent FAQ has a lot of information about Smart Agent and related features. We thought you might appreciate this quick way to get to the topics that most interest you, paired with a place to ask questions and enlarge on your take... 
Hi, Everyone!   What are Smart Agent's requirements?  Here's a key question before getting started: What are the requirements?  Smart Agent FAQ | Requirements What do you think? We've starte... See more...
Hi, Everyone!   What are Smart Agent's requirements?  Here's a key question before getting started: What are the requirements?  Smart Agent FAQ | Requirements What do you think? We've started with the requirements questions we knew you'd want, plus questions others have already asked. Our team would love to hear your thoughts, including how we can add to and improve the FAQ Please do share your impressions, considerations, and questions below. Our Smart Agent FAQ has a lot of information about Smart Agent and related features. We thought you might appreciate this quick way to get to the topics that most interest you, paired with a place to ask questions and enlarge on your take... 
Hi, Everyone!   What about environments supported with Smart Agent?  Find out what environments and features are supported as what may be coming down the line. Post your questions and we will be su... See more...
Hi, Everyone!   What about environments supported with Smart Agent?  Find out what environments and features are supported as what may be coming down the line. Post your questions and we will be sure to address them. The future of Smart Agent depends on your needs! Smart Agent FAQ | Supported Environments What do you think? Our team would love to hear your thoughts, including how we can add to and improve the FAQ Please do share your impressions, considerations, and questions below. Our Smart Agent FAQ has a lot of information about Smart Agent and related features. We thought you might appreciate this quick way to get to the topics that most interest you, paired with a place to ask questions and enlarge on your take... 
Hi, Everyone!   Concerned about costs or support with Smart Agent?  Spoiler alert: You don't need to buy additional licenses to use Smart Agent. Check out the other most frequently asked about this... See more...
Hi, Everyone!   Concerned about costs or support with Smart Agent?  Spoiler alert: You don't need to buy additional licenses to use Smart Agent. Check out the other most frequently asked about this here:  Smart Agent FAQ | Licenses and Packages  What do you think? Our team would love to hear your thoughts, including how we can add to and improve the FAQ Please do share your impressions, considerations, and questions below. Our Smart Agent FAQ has a lot of information about Smart Agent and related features. We thought you might appreciate this quick way to get to the topics that most interest you, paired with a place to ask questions and enlarge on your take... 
Hi, Everyone!   Get to know some of the basics around using Smart Agent to simplify agent management tasks, such as it's value and what specific features are available.  Smart Agent FAQ | Simplifi... See more...
Hi, Everyone!   Get to know some of the basics around using Smart Agent to simplify agent management tasks, such as it's value and what specific features are available.  Smart Agent FAQ | Simplified Agent Management Basics There, find out how we define smart agent management, installation requirements and details, a high-level outline of value in this v23.11 release,  and more...  What do you think? Our team would love to hear your thoughts, and how we can improve the FAQ Please do share your impressions, considerations, and questions below. Our Smart Agent FAQ has a lot of information about Smart Agent and related features. We thought you might appreciate this quick way to get to the topics that most interest you, paired with a place to ask questions and enlarge on your take... 
Hi, I have noticed over the last 4 days I had an increased number of Search Bundle replication errors: 12-21-2023 09:50:12.604 +0000 WARN ConfReplicationThread [9209 ConfReplicationThread] - Error ... See more...
Hi, I have noticed over the last 4 days I had an increased number of Search Bundle replication errors: 12-21-2023 09:50:12.604 +0000 WARN ConfReplicationThread [9209 ConfReplicationThread] - Error pushing configurations to captain=https://searchHeadCaptain:8089, consecutiveErrors=1 msg="Error in acceptPush: Non-200 status_code=400: ConfReplicationException: Cannot accept push with outdated_baseline_op_id=16ed9160640170315673324237791a4cfe256d59; current_baseline_op_id=cd93950208af34df00957e721b87128d3629d2d1" These occur in groups every 4 hours. I have also seen CPU spikes on the Search Heads that started occuring at the same time and also every 4 hours. Further investigation has shown that the following events from conf.log have also been occuring at the same time every 4 hours { [-]    component: ConfOp    data: { [-]      applied_at: 1703264397      asset_id: 220d8bbce6d790850cda3980c5784c62b1a9f9ff      asset_uri: [ [+]      ]      from_repo: https://searchHeadCaptain:8089      op_id: 102aa206f930da5eef0d47163b354c61254566c5      optype: 2      optype_desc: WRITE_STANZA      payload: { [-]        alias: Risk        metadata: { [-]          permissions: { [-]          }        }        value: ***TRANSIENT***://6613      }      payload_extra: ***ALLOW_SKIP_ON_WRITE***      status: applied      task: pullFrom      to_repo: https://searchHeadPeer.com:8089      to_repo_change_count: 20214    }    datetime: 12-22-2023 16:59:57.097 +0000    log_level: INFO } Does anyone know what these events mean and how I can find out what is causing them? Bundle replication errors:   conf.log events:   CPU spikes:  
I am running the current search using the network toolkit but will not show the hostname field from the csv, do I need to do another inputlookup at the end of the search. | inputlookup iphost.csv |... See more...
I am running the current search using the network toolkit but will not show the hostname field from the csv, do I need to do another inputlookup at the end of the search. | inputlookup iphost.csv | search src_ipV4=* hostname=* | rename src_ipV4 as host | stats values(host) as host | mvexpand host | map maxsearches=50 search="| ping host=$host$ count=1 | eval dest=if(isnull(dest),host,dest) | fields host dest received" | table host dest received hostname
We have data coming in that we need to alert on, however because of the formatting of the data, this is very hard to do.   The data is coming in as key value pairs but the values are not encapsulated... See more...
We have data coming in that we need to alert on, however because of the formatting of the data, this is very hard to do.   The data is coming in as key value pairs but the values are not encapsulated in quotes and is being truncated.  For example _Raw - filepath=c:\program files\abc123\ What we end up getting is Parsed - filepath=c:\program Everything after the space is ignored. If I wanted to find all occurrences where the path was c:\program files\abc123, I can't. We are sending the data via syslog to the splunk servers Thanks in advance!      
Is it standard for the Splunk server itself to be over 50% of the daily indexing total? In our production environment, we are starting run over the daily and simply because of the splunk server itsel... See more...
Is it standard for the Splunk server itself to be over 50% of the daily indexing total? In our production environment, we are starting run over the daily and simply because of the splunk server itself. I understand its what does the heavy lifting, but its hard to base how much licensing you may need when you dont know how to gauge what the server will utilize    
Hi, How to add MSAL4J.jar to DB_Connect. I am getting error: Failed to load MSAL4J Java library for performing ActiveDirectoryServicePrincipal authentication.
Hi all, We need to add a couple dozen new search head peers to search head deployer, as well as adding a couple dozen indexers to a cluster master and would like to script this implementation.  I n... See more...
Hi all, We need to add a couple dozen new search head peers to search head deployer, as well as adding a couple dozen indexers to a cluster master and would like to script this implementation.  I need to know what configuration files need to be modified to join these new search head peers and indexer to the existing Splunk environment.  We are plan on running an Ansible script for this implementation project.  /Paul
Hello Experts, I'm facing challenge where I need to automatically load data from Python script results into a metric index in Splunk. Is it possible? I'd appreciate any guidance or examples how to... See more...
Hello Experts, I'm facing challenge where I need to automatically load data from Python script results into a metric index in Splunk. Is it possible? I'd appreciate any guidance or examples how to achieve this? Thanks
Hi everyone, I am in trouble. I need help. We are performing an UPGRADE of splunk ITSI. Following the upgrade path of ITSI, we are now handling the following. 4.9.x → 4.11.x → 4.13.x → 4.15.x Tr... See more...
Hi everyone, I am in trouble. I need help. We are performing an UPGRADE of splunk ITSI. Following the upgrade path of ITSI, we are now handling the following. 4.9.x → 4.11.x → 4.13.x → 4.15.x Trouble is occurring in the following cases 4.9.6 → 4.11.6 The server configuration is a cluster. The Splunk version is as follows. Search head:Splunk 9.1.2 indexer:Splunk 9.1.2 api(HF):Splunk 9.1.2   Migration logs at the time the trouble occurred are as follows. -------------------------------------------------------- 2023/12/22 14:45:20.640 2023-12-22 14:45:20,640+0900 process:2531 thread:MainThread ERROR [itsi.migration] [itsi_migration:4543] [run_migration] Migration from 4.9.2 to 4.10.0 did not finish successfully. host = logmng-st-splunk_srch01source = /opt/splunk/var/log/splunk/itsi_migration_queue.logsourcetype = itsi_internal_log 2023/12/22 14:45:20.636 2023-12-22 14:45:20,636+0900 process:2531 thread:MainThread ERROR [itsi.migration] [__init__:1433] [exception] 4.9.2 to 4.10.0: [HTTP 404] https://127.0.0.1:8089/servicesNS/nobody/SA-ITOA/storage/collections/data/itsi_entity_management_rules?fields=object_type; [{'type': 'ERROR', 'code': None, 'text': 'An object with name=itsi_entity_management_rules does not exist'}] Traceback (most recent call last): File "/opt/splunk/etc/apps/SA-ITOA/lib/migration/migration.py", line 310, in run if not command.execute(): File "/opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/itsi_migration.py", line 249, in execute backup.execute() File "/opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py", line 1244, in execute self.backup() File "/opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py", line 973, in backup raise e File "/opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py", line 942, in backup object_types = self._get_object_type_from_collection(collection) File "/opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py", line 601, in _get_object_type_from_collection rsp, content = simpleRequest(location, sessionKey=self.session_key, raiseAllErrors=False, getargs=getargs) File "/opt/splunk/lib/python3.7/site-packages/splunk/rest/__init__.py", line 669, in simpleRequest raise splunk.ResourceNotFound(uri, extendedMessages=extractMessages(body)) splunk.ResourceNotFound: [HTTP 404] https://127.0.0.1:8089/servicesNS/nobody/SA-ITOA/storage/collections/data/itsi_entity_management_rules?fields=object_type; [{'type': 'ERROR', 'code': None, 'text': 'An object with name=itsi_entity_management_rules does not exist'}] host = logmng-st-splunk_srch01source = /opt/splunk/var/log/splunk/itsi_migration_queue.logsourcetype = itsi_internal_log   2023/12/22 14:45:20.635 2023-12-22 14:45:20,635+0900 process:2531 thread:MainThread ERROR [itsi.migration] [__init__:1433] [exception] 4.9.2 to 4.10.0: BackupRestore: [HTTP 404] https://127.0.0.1:8089/servicesNS/nobody/SA-ITOA/storage/collections/data/itsi_entity_management_rules?fields=object_type; [{'type': 'ERROR', 'code': None, 'text': 'An object with name=itsi_entity_management_rules does not exist'}] Traceback (most recent call last): File "/opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py", line 1244, in execute self.backup() File "/opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py", line 973, in backup raise e File "/opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py", line 942, in backup object_types = self._get_object_type_from_collection(collection) File "/opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py", line 601, in _get_object_type_from_collection rsp, content = simpleRequest(location, sessionKey=self.session_key, raiseAllErrors=False, getargs=getargs) File "/opt/splunk/lib/python3.7/site-packages/splunk/rest/__init__.py", line 669, in simpleRequest raise splunk.ResourceNotFound(uri, extendedMessages=extractMessages(body)) splunk.ResourceNotFound: [HTTP 404] https://127.0.0.1:8089/servicesNS/nobody/SA-ITOA/storage/collections/data/itsi_entity_management_rules?fields=object_type; [{'type': 'ERROR', 'code': None, 'text': 'An object with name=itsi_entity_management_rules does not exist'}] host = logmng-st-splunk_srch01source = /opt/splunk/var/log/splunk/itsi_migration_queue.logsourcetype = itsi_internal_log -------------------------------------------------------- How do you deal with errors? Any help would be good! thanks, shinsuke
Hi, We initially deployed a heavy forwarder on-prem to collect data from our passive devices (syslogs, security devices, etc) however per talking with a splunk represent he recommended to have a s... See more...
Hi, We initially deployed a heavy forwarder on-prem to collect data from our passive devices (syslogs, security devices, etc) however per talking with a splunk represent he recommended to have a splunk connect for syslog to collect the data. Per him Syslog connect is the recommended method of collection for passive devices and also helps with parsing/normalization of the data when it goes to our Enterprise Security. Can both HF and SC4S be in the server ? If yes how will that work? Can SC4S direct data to the cloud indexer? And for future, do we just go for SC4S instead on the HF on-prem for the passive devices?  Thank you
Hi there. I would like to know about Splunk Health engine, Enterprise 8.2.12, 3 SHC,     HOW it considers a savedsearch a Lagged search? Based on same previous 24h search runs and doing an a... See more...
Hi there. I would like to know about Splunk Health engine, Enterprise 8.2.12, 3 SHC,     HOW it considers a savedsearch a Lagged search? Based on same previous 24h search runs and doing an average running times? Since we have many many heavy searches that end up also in 10/15m WHY, sometimes, i found in Skipped search monitor a 100% of skipped search (1 from 1, when we have hundreds of scheduled searches)? WHILE, searching the scheduler log, i found something like 70.000 success / 68 skipped (scheduled every minute or every two, concurrency is a factor i calculate and there's no problem) in last 24h ? WHY 100%? Is it a bug? I also search for a single scheduled search per day savedsearches, but all (few) are in "success" status When those strange things occur, sometimes, restarting the cluster, make health monitor to reset without warnings!!! Other times, in reverse, restarting the cluster make a clean health monitor to start giving warnings from point 1 & 2 ... strange behaviour!!! Thanks.
Recently configured a new input that has successfully ingesting logs but appears to be working intermittently. There is large gaps in logs that we have confirmed are present and being created regular... See more...
Recently configured a new input that has successfully ingesting logs but appears to be working intermittently. There is large gaps in logs that we have confirmed are present and being created regularly from the source server. Example : Logs are captured 8th December and 16 December only. So, here 9th December to 15nth December logs are not captured We have created custom app on our deployment server and push that app across all the deployment slaves. The data flow is coming from source-->Universal forwarder-->Heavy forwarder--> Splunk cloud we have created Inputs.conf  [monitor://F:\Polarion\data\logs\main\*.log.*] sourcetype = catalina index = ito_app disabled = false ignoreOlderThan = 7d initCrcLength = 10000 Please help on the issue Thank you
Agent configuration and maintenance don't have to be complex. With our new Smart Agent, managing upgrades and installations is as easy as a few clicks.  See for yourself! Check out this click-throu... See more...
Agent configuration and maintenance don't have to be complex. With our new Smart Agent, managing upgrades and installations is as easy as a few clicks.  See for yourself! Check out this click-through demo to see it in action: