All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Hi @manhdt , as I said, ITSI is a premium App, this means that you need an additional license for it, the normal Splunk Enterprise license isn't sufficient. The solutions to see ITSI are the ones I... See more...
Hi @manhdt , as I said, ITSI is a premium App, this means that you need an additional license for it, the normal Splunk Enterprise license isn't sufficient. The solutions to see ITSI are the ones I described in my previous post. If instead you acquired an ITSI license for yourself, you must receive the link to download it, if you acquired for a final customer, the eMail arrived to it. Ciao. Giuseppe
Hi all, I'm in the process of migrating our single hosted Splunk installation to a new server. After setting up a new Splunk instance and feeding it data from a few devices, I notice an oddity I nev... See more...
Hi all, I'm in the process of migrating our single hosted Splunk installation to a new server. After setting up a new Splunk instance and feeding it data from a few devices, I notice an oddity I never noticed before. Logging in and getting to search & reporting all works at the expected speed. But every time I start a new search, 18 to 19 seconds are spend with a POST call to the URL (host and user obfuscated) https://hostname/en-US/splunkd/__raw/servicesNS/myusername/search/search/ast The result is always a 200, but it always takes those 18 to 19 seconds to finish. When I have the results, everything is fast: selections in the timeline, paging through results and changing the "results per page" value. It seems like the system is trying something, runs into a timeout and then proceeds with normal work, but I cannot figure out what that would be. I have not done much customizations yet, but we are in a heavily firewalled environment. Am I overlooking something here?  
Hi @skramp , I am currently using a license Splunk Enterprise, but at the moment I do not have the email information used to purchase the license.
UPDATE: Here are the setPushEventSettings public IPs that need to be whitelisted: setPushEventSettings Sup friends, So I just came across this Bitdefender issue and here's what worked for me: ... See more...
UPDATE: Here are the setPushEventSettings public IPs that need to be whitelisted: setPushEventSettings Sup friends, So I just came across this Bitdefender issue and here's what worked for me: 1. Ensure your HEC endpoint supports TLS 1.2 (it most certainly does):   openssl s_client -connect http-inputs-namehere.splunkcloud.com:443 -tls1_2   2. Ensure your Splunk Cloud HEC access for ingestion IP Allow List has the IP ranges for IPs Bitdefender Cloud API responses - Splunk Cloud > Settings > Server settings > IP allow list - I'm still not sure what they are, but you probably get these from Bitdefender Support. 3. Ensure the integration command is properly formatted - if your stack is on GCP the HEC URL will be different. I believe it would be http-inputs.namehere.splunkcloud.com. My example below is for stacks hosted in AWS (more info on that here https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/UsetheHTTPEventCollector   curl -k -X POST https://cloud.gravityzone.bitdefender.com/api/v1.0/jsonrpc/push -H 'authorization: Basic <Auth header base64>' -H 'cache-control: no-cache' -H 'content-type: application/json' -d '{"params": {"status": 1, "serviceType": "splunk", "serviceSettings": {"url": "https://http-inputs-namehere.splunkcloud.com:443/services/collector", "requireValidSslCertificate": false, "splunkAuthorization": "Splunk <Splunk Cloud HEC Token>"}, "subscribeToEventTypes": {"hwid-change": true,"modules": true,"sva": true,"registration": true,"supa-update-status": true,"av": true,"aph": true,"fw": true,"avc": true,"uc": true,"dp": true,"device-control": true,"sva-load": true,"task-status": true,"exchange-malware": true,"network-sandboxing": true,"malware-outbreak": true,"adcloud": true,"exchange-user-credentials": true,"exchange-organization-info": true,"hd": true,"antiexploit": true}}, "jsonrpc": "2.0", "method":"setPushEventSettings", "id": "1"}'   So without the Bitdefender IPs, I had to test by opening up the HEC allow list with 0.0.0.0/0 (takes a couple of  minutes for the change to take effect), getting a successful response, and then immediately removing it, but this will let you know if this is the issue. Or you could wait to get the IPs from Bitdefender. If you do get a successful response, you can send a test event with this:   curl -k -X POST https://cloud.gravityzone.bitdefender.com/api/v1.0/jsonrpc/push -H 'authorization: Basic <Auth header base64>' -H 'cache-control: no-cache' -H 'content-type: application/json' -d '{"params": {"eventType": "av"}, "jsonrpc": "2.0", "method": "sendTestPushEvent", "id": "3"}'   Hope this helps!
What is the fastest way to migrate Splunk objects dashboard , alerts, reports from one of these old version ( 6.5, 7)  to latest cloud.  . thanks 
Thanks for your help with this. In the meantime I've run into another problem. Could you please help me? This is the topic: https://community.splunk.com/t5/Getting-Data-In/conditional-whitespace-in... See more...
Thanks for your help with this. In the meantime I've run into another problem. Could you please help me? This is the topic: https://community.splunk.com/t5/Getting-Data-In/conditional-whitespace-in-transform/m-p/708831
Hello everyone! I am experimenting with the SC4S transforms that are posted here: https://splunk.github.io/splunk-connect-for-syslog/main/sources/vendor/Splunk/heavyforwarder/ My goal is to send... See more...
Hello everyone! I am experimenting with the SC4S transforms that are posted here: https://splunk.github.io/splunk-connect-for-syslog/main/sources/vendor/Splunk/heavyforwarder/ My goal is to send the logs to a syslog-ng instance running with a custom config. My current problem is that the SC4S config contains a part where it checks for subseconds, and appends the value to the timestamp, if found.   [metadata_source] SOURCE_KEY = MetaData:Source REGEX = ^source::(.*)$ FORMAT = _s=$1 $0 DEST_KEY = _raw [metadata_sourcetype] SOURCE_KEY = MetaData:Sourcetype REGEX = ^sourcetype::(.*)$ FORMAT = _st=$1 $0 DEST_KEY = _raw [metadata_index] SOURCE_KEY = _MetaData:Index REGEX = (.*) FORMAT = _idx=$1 $0 DEST_KEY = _raw [metadata_host] SOURCE_KEY = MetaData:Host REGEX = ^host::(.*)$ FORMAT = _h=$1 $0 DEST_KEY = _raw [metadata_time] SOURCE_KEY = _time REGEX = (.*) FORMAT = _ts=$1$0 DEST_KEY = _raw [metadata_subsecond] SOURCE_KEY = _meta REGEX = \_subsecond\:\:(\.\d+) FORMAT = $1 $0 DEST_KEY = _raw   In my case however, when it's not found, the timestamp field will not get a whitespace appended, and thus it will be practically concatenated with the following field, which is not what I want. How could set up the config so that there will always be a whitespace before the next field (the host/_h field)? I tried adding an extra whitespace in front of the _h in the FORMAT part of the metadata_host stanza, but that seems to be ignored. This is what I see:   05:58:07.270973 lo In ifindex 1 00:00:00:00:00:00 ethertype IPv4 (0x0800), length 16712: (tos 0x0, ttl 64, id 49071, offset 0, flags [DF], proto TCP (6), length 16692) 127.0.0.1.49916 > 127.0.0.1.cslistener: Flags [.], cksum 0x3f29 (incorrect -> 0x5743), seq 1:16641, ack 1, win 260, options [nop,nop,TS val 804630966 ecr 804630966], length 16640 0x0000: 0800 0000 0000 0001 0304 0006 0000 0000 ................ 0x0010: 0000 0000 4500 4134 bfaf 4000 4006 3c12 ....E.A4..@.@.<. 0x0020: 7f00 0001 7f00 0001 c2fc 2328 021a 7392 ..........#(..s. 0x0030: 486d 209f 8010 0104 3f29 0000 0101 080a Hm......?)...... 0x0040: 2ff5 b1b6 2ff5 b1b6 5f74 733d 3137 3336 /.../..._ts=1736 0x0050: 3931 3730 3739 5f68 3d73 706c 756e 6b2d 917079_h=splunk- 0x0060: 6866 205f 6964 783d 5f6d 6574 7269 6373 hf._idx=_metrics 0x0070: 205f 7374 3d73 706c 756e 6b5f 696e 7472 ._st=splunk_intr    This is the interesting part:   0x0040: 2ff5 b1b6 2ff5 b1b6 5f74 733d 3137 3336 /.../..._ts=1736 0x0050: 3931 3730 3739 5f68 3d73 706c 756e 6b2d 917079_h=splunk- 0x0060: 6866 205f 6964 783d 5f6d 6574 7269 6373 hf._idx=_metrics   The _h will come right after the end of the _ts field, without any clear separation.
I set up DoDBanner when logging in by putting the following in Web.conf. It was displayed in Splunk 9.2.1, but it was no longer displayed in Splunk 9.2.2. $SPLUNK_HOME$\etc\system\local\web.conf [... See more...
I set up DoDBanner when logging in by putting the following in Web.conf. It was displayed in Splunk 9.2.1, but it was no longer displayed in Splunk 9.2.2. $SPLUNK_HOME$\etc\system\local\web.conf [settings] login_content = <script>function DoDBanner() {alert("Hello World");}DoDBanner();</script> Is DoDBanner no longer supported from a certain version?
This is actually a MSI quirk. I've seen it happen with various softwares over last 20+ years. And yes, it's frustrating.
Hi @gcusello , I am currently using a license Splunk Enterprise, but at the moment I do not have the email information used to purchase the license.
Ultimately, it was time for a new machine.   Upshot: Not installing Splunk on the new machine to avoid this from happening again.
I'm trying to take a single node Splunk Enterprise system and expand it to a cluster with an additional search head and indexes. I copied the existing install to a new system and that worked perfect... See more...
I'm trying to take a single node Splunk Enterprise system and expand it to a cluster with an additional search head and indexes. I copied the existing install to a new system and that worked perfectly. Then I added the cluster manager and indexes and all of the settings that were in the old system that were copied to the search head were gone. I'm assuming that I put the copy of the single node into the wrong role, but I'm not sure which role I should have picked.
Yes, the replication_host setting only applies to search head clusters.  That is what the second sentence in the description is trying to say. Have you tried KVStore backup/restore?  It will transfe... See more...
Yes, the replication_host setting only applies to search head clusters.  That is what the second sentence in the description is trying to say. Have you tried KVStore backup/restore?  It will transfer data from one SH to another, but is not suitable for keeping the two in sync - that is what replication is for.
i want to replicate search1's kvstore to search2. that two cluster is non-clustering and standalone.   And i found option in server.conf [kvstore] stanza replication_host. Does this option only o... See more...
i want to replicate search1's kvstore to search2. that two cluster is non-clustering and standalone.   And i found option in server.conf [kvstore] stanza replication_host. Does this option only operated in clustering environment?   
This is the error I get when trying to start SOAR on the Warmstandby: Splunk SOAR is in standby. Starting all services except automation daemons Starting Database server (PostgreSQL): [ OK ] Sta... See more...
This is the error I get when trying to start SOAR on the Warmstandby: Splunk SOAR is in standby. Starting all services except automation daemons Starting Database server (PostgreSQL): [ OK ] Starting Connection pooler (PgBouncer): [ OK ] Checking database connectivity: [ OK ] Checking component versions: [ OK ] Starting Supervisord: [ OK ] Starting Splunk SOAR daemons: [ OK ] Checking Supervisord processes: [FAILED] add_to_es_index failed to start. Check /opt/soar/var/log/phantom/add-es-index-stderr.log for more info. Splunk SOAR startup failed.   I saw there is a known issue for this: https://docs.splunk.com/Documentation/SOARonprem/6.3.1/ReleaseNotes/KnownIssues 2024-12-03 PSAAS-20901 supervisord failing to start on warm standby instance   Does anyone have a workaround or fix for this issue?
1. Why would you fiddle with license manager? (unless it's on the CM which is not a very good idea) 2. Why copy anything from var/run? 3. Switching indexers between CMs is asking for trouble. I'd r... See more...
1. Why would you fiddle with license manager? (unless it's on the CM which is not a very good idea) 2. Why copy anything from var/run? 3. Switching indexers between CMs is asking for trouble. I'd replace a CM in place.
You can Download IT Essentials Work, it's an ITSI without a license. But you can upgrade it to a full ITSI if you insert a license once. https://splunkbase.splunk.com/app/5403
There are some apps on splunkbase which might help you: https://splunkbase.splunk.com/app/7384 https://splunkbase.splunk.com/app/6724  
Hi all, I would like to migrate our current cluster master to the a new server. Here's what I gather the process to do so. If someone can take a look and let me know if there's anything missing that... See more...
Hi all, I would like to migrate our current cluster master to the a new server. Here's what I gather the process to do so. If someone can take a look and let me know if there's anything missing that'll be much appreciated. Thank you! Additionally, should I enable cluster maintenance mode on the old cluster master prior to the migration?  ======================================================================== ======================== Migrate the Cluster Master ==================== ======================================================================== - Stop the splunk service on both the old and new cluster master /opt/splunk/bin/splunk stop - On the old Cluster Master change encrypted passwords to clear text and save theses find /opt/splunk/etc -name '*.conf' -exec grep -inH '\$[0-9]\$' {} \; /opt/splunk/bin/splunk show-decrypted --value '$encryptedpassword' - - Copy files to the new Cluster Master scp -r /opt/splunk/var/run/splunk/cluster/remote-bundle/ new_splunkmaster:/opt/splunk/var/run/splunk/cluster/remote-bundle/ scp -r /opt/splunk/etc/master-apps/ new_splunkmaster:/opt/splunk/etc/ scp -r /opt/splunk/etc/system/local/server.conf new_splunkmaster:/opt/splunk/etc/system/local/ - Make sure the above decrypted the main 2 passwords below and replace them in the copied server.conf, in clear text, on the new Cluster Master until it is restarted when it will then encrypt. [general] sslPassword= [clustering] pass4SymmKey= - Start splunk on the new Cluster Master /opt/splunk/bin/splunk start - Point indexers to the new Cluster Master /opt/splunk/bin/splunk edit cluster-config -mode peer -manager_uri https://new_splunkmaster:8089 -replication_port 9887 -secret new_splunkmaster - Point the search heads to the new Cluster Master /opt/splunk/bin/splunk edit cluster-config -mode searchhead -manager_uri https://new_splunkmaster:8089 -secret new_splunkmaster ======================================================================== ======================== Migrate the License Manager ==================== ======================================================================== - Promote a license peer to be the manager: On the peer, navigate to Settings > Licensing. Click Switch to local manager. On the Change manager association page, choose Designate this Splunk instance as the manager license server. Click Save. Restart the Splunk Enterprise services. On the new license manager, install your licenses. See Install a license. Configure the license peers to use the new license manager: - On the peer (indexer / search heads / deployer), navigate to Settings > Licensing. Click Switch to local manager. Update the Manager license server URI to point at the new license manager. Click Save. Restart the Splunk Enterprise services. Demote the old license manager to be a peer: - On the old license manager, navigate to Settings > Licensing. Click Change to peer. Click Designate a different Splunk instance as the manager license server. Update the Manager license server URI to point at the new license manager. Click Save. Stop the Splunk Enterprise services. Using the CLI, delete any license files under $SPLUNK_HOME/etc/licenses/enterprise/. Start the Splunk Enterprise services.  
Settings in the [default] stanza apply if they are not mentioned in a specific stanza.  IOW, if you have [default] frozenTimePeriodInSecs = 36000000 [ubuntu] frozenTimePeriodInSecs = 72000000 [rhe... See more...
Settings in the [default] stanza apply if they are not mentioned in a specific stanza.  IOW, if you have [default] frozenTimePeriodInSecs = 36000000 [ubuntu] frozenTimePeriodInSecs = 72000000 [rhel] someOtherSetting = foo  The ubuntu index will have a retention period of 72,000,000 seconds and the rhel index will have a retention period of 36,000,000 seconds.