You may want to look into this, as it looks somewhat similar to your requirements. Mind you, there are going to be a lot of false positives. How to audit changes in savedsearches.conf  Hope this helps, S   ... View more

You're restarting splunkd using root, instead of Splunk. This usually causes such problems. Are all of your Indexers running as root? I'd try to check the permissions of the files, get rid of the PID and restart splunk using the user which was used to install the software, in most cases, it's Splunk. ... View more

I'd consider ingesting System monitoring (sysmon) logs, and then make use cases out of them. There's a great Splunk app ES Content Updates, which can give you various use cases along with SPL, based on different stages of kill chain. You can look into MITRE framework app, which will give you use cases, along with SPL regarding modern attacks. Sysmon logs will go a long way for you to get deep insights into your Indexer. If your organization has an EDR, like Crowdstrike or Carbon black, I'd recommend installating that on your Indexer server. Also having an Antivirus solution on the server always helps. Hope this helps. S ... View more

Hello @thambisetty, Thanks for the guidance. Since there are chances of errors of inconsistency, if I do not remove any of the previous search bundles,  blacklist the lookups, and restart the service, will it cause Splunk to create a new knowledge bundle, or do I have to get rid of the previous knowledge bundle? Also, do you think the way my blacklist is setup, will work? Since it'll be done on a production environment, I don't want to cause any troubles 🙂 Thanks, S ... View more

Hello, In order to find the root cause, in your Indexers, go to Monitoring console -> Resource usage -> Resource usage: Instance. Under the snapshots section, you'll find two very important graphs. Physical memory usage by process class and CPU Usage by process class. You'd want to look at the CPU usage graphs and see what's causing the hogging of CPU utilisation. If it says Search, that means you have to look at people running resource intensive searches in your environment. For that, you'd want to go to Search -> Search Activity: Instance and check everything out from there. You can always create alerts off the audit data to find violators, who are running long running searches, or too many searches etc. If it's not a search issue, please contact Splunk support, as it maybe a case of memory leakage. Here's the article that you can also go through for this: Resouce Usage CPU - Splunk Docs  Hope this helps, S. Note: If this helped, please mark it as an accepted answer.     ... View more

If you can give some sample data, which has the data of pre and post group changes, we can help you to build the search. I believe an EventCode is being generated, everytime there's some changes in the OU. Like 4727 for a group creation, 4728 when a member is added, 4729 if a member is removed and so on. If that's the case, I'd look out for those eventcodes and use the table command to pipe the required values in tabular format. Again, some sample data and more light on what you want to achieve will be helpful. Thanks, S ... View more

Yes you can. You can bring your AD logs to Splunk using the following app Splunk App For Windows  Using ldapsearch command, and with a bit of SPL magic, you can customise the data that you want to pull from AD. Since Splunk will store your old, as well as new data, you can easily compare them and schedule alerts, reports, create tickets in a ITSM ticketing tool like Servicenow, Remedy etc and much more. Hope this helps, S Note: If it helped, please mark this as an accepted answer.   ... View more

Unfortunately you cannot limit the usage of Splunk web like this. However, in a firewall, you can have multiple rules for the same port. For your requirement, I'd suggest to keep the incoming connection limited only for Microsoft team's server IP (as it'll send call record data on it. Just create an inbound rule for it. If you want Splunk to connect to it as well, create an outbound rule and it'll do the trick) and your localhost. This will fulfill your requirement of no one being able to access your Splunk web portal apart from being accessed locally, and your webhook will still be working. In the future, if you want someone else to access the port on your Splunk server, you just have to create a firewall rule accordingly. This is usually how servers in a DMZ are set up as well. Hope this helps, S ... View more

I got your requirement now, here's what you can try: 1. In the Index field in your datamodel, append the results of your lookup (inputlookup append=t your_lookup.csv) 2. In the calculated fields, use the option of extract more fields, and use Auto extracted fields and check if you can find your desired field there, if yes, just add it to your datamodel. 3. If you cannot find it via Auto extract, you can always go for the trusted Regular Expressions. Try this and let me know if it works. S If it helps, please accept it as an answer. ... View more

You're merging two different aspects into one here.  The setting ```allowRemoteLogin```, is only applicable for Splunkd service, not the web UI. The document says the very same. When set to "never", only local logins to splunkd are allowed. Note that this still allows remote management through Splunk Web if Splunk Web is on the same server. I believe you're trying to restrict web UI access towards your Splunk server. For that, in the firewall of your server, add the rule, which will allow only localhost to access the Splunk web port. This will do the trick, as this requirement falls more towards server administration, rather than Splunk. Also, please don't hash anything out or make any sort of changes in default Splunk configuration files, else you're going to find a lot of error messages in Splunk, complaining about Files integrity monitoring. Splunk highly recommends against modifying the default files and configurations in the local folder are always preferred over default anyways. Hope that helps, S If it helps, please accept it as an answer.   ... View more

Oh yes. I missed to add the line about the public URL. Since it's public, you'll have to modify its DNS entry on Route 53, or get a new one created, if it doesn't exist already, then have your ALB route the connections to your instances. In this case, none of the changes are required on your end. Once the changes are made and the ALB directs the requests successfully to your instance, you should be all set. I'm sure you're already aware of it, but: 1. Never expose your EC2 instances to the web, use an Application Load balancer or Cloudfront. 2. Encryption is transit is always a better option than no encryption at all. (connection encryption between your instance and ALB) If it helps, please mark it as accepted. Thanks, S ... View more

Yes you can. But it's a bit of a manual effort, but worth the effort. The supression for 1 hour for user and dest will remain as it is in your alert's throttling. Now let's assume that user=Adam and dest=abc. A notable event is fired for this and is now in your incident review dashboard. You triage and decide to send it for remediation, as the system is infected by a malware, which will take 7 days. To enable suppression for these particular values of user and dest, follow these steps: 1. In Incident review dashboard, find the Incident. 2. Click on Actions tab's dropdown, belonging to this notable event, using which you decided to invoke supression. 3. Select Supress Notable Events. 4. Name your suppression whatever you want and set the duration for the suppression. In your case, for the next 7 days. Select the date range accordingly. 5. Selected fields option will be greyed out, but look at it's description and click change. 6. Verify if the values of user and dest are the same that you intended to, which should be the case as we properly chose the notable. If not, I'd recommend searching the notable properly, instead of modifying. That should be the last resort. 7. If you're happy with everything, click save. This will make sure that for a particular user and dest, you won't see a notable event for the next 7 days for this particular alert only. If the same user and dest are found in some other alert, let's say DLP Violations, you'll get an notable event. For the rest of the notable events, throttling remains at 1 hour. Let me know if this helps and how this works out. If it does, please mark this as an accepted answer. Thanks, S   ... View more

Rich is correct. This should stop the flow of logs. If they're still coming in, you can always remove all the 3 stanzas from inputs.conf, copy them in a notepad or elsewhere to revisit them if you need to in the future,and restart splunkd. That surely will do it. FYI: When we mean restart splunkd, we mean restart it on the system on which the forwarder is installed. If you're managing this from the deployment master, then make sure that deployment app, under which this system falls, has the capability to Restart splunkd. Only then will you be able to make the changes effective from the deployment master. You can always edit the app and add the capability of restarting splunkd and save it. Make sure that the app is still enabled after the change. I've had times where after making the changes, the app gets disabled.   Hope this helps. Whichever solution works for you, please accept it as answer. S ... View more

The name in the url is usually picked from your system's hostname. You'll have to contact your loval administrator for changing that entry for the particular Splunk instance that you're trying to change the URL for. Then if you have ssl certificates enabled as well, then you'll have to get them adjusted as well (The value CNAME is also based on your system's hostname).  Once that's sorted, supply the required values in web.conf and restart splunkd. This will take care of the URL change. Hope this helps. Please accept it as a solution if it helped or let us know if it causes problems.   ... View more

A common trouble with JSON data. Can you share your some sample data? Please mask any sensitive data like IP addresses or anything related to your company, as its risky and usually leads to the question being deleted. If that's not possible, try modifying snd implementing something like this in your props.conf. [Your_sourcetype] KVMODE=json LINEBREAKER= your_regex_to_identify_the_whitespace_between_closing_and_opening_braces_of_first_and_second_events MAXTIMESTAMPLOOKAHEAD=30 NOBINARYCHECK=true SHOULDLINEMERGE=false     ... View more

For anyone looking how to do this for your report/alert already created: 1. Edit savedsearches.conf on your search head. Find the name of the alert, for which you want to use the new email address to send emails to the recipients and add the following: action.email.from = your_email_address action.email.mailserver = your_smtp_server_name 2. Restart Splunkd.  Works like a charm. The best part is that all of your other alerts will still use the default email address, supplied in the server settings of your search head. Hope  this helps. ... View more

@to4kawa, I agree. Hence I wrote I'm not sure if it'll work. It shouldn't, as parsing doesn't happen on a UF. Just FYI: Transforms.conf doesn't necessarily work on SH. If you are doing search time extractions, then you put your transforms and props on search head. In this case, we're doing Index time extractions, as we don't want to ingest a certain set of logs.  Hence they'll go on either on a Heavy Forwarder, or on an Indexer. If there are multiple indexers, then via Cluster master (My second suggestion on the previous answer). @alexspunkshell, Please put the props and transforms on your Indexers, and set the regex accurately, as everything that'll match it, will not be indexed.  Hope this helps, ... View more

Hello, Props and transforms don't work on a Universal forwarder, since these features are only supported by a Splunk enterprise version (Heavy forwarder, Indexers, Search Heads). Hence you're not able to drop the data. You can do either of the following: 1. Use blacklist. Not exactly 100% sure if it'll work here, as I've only used it with files and event codes. In your inputs.conf, edit and paste the following.   blacklist = your_field_name_which_contains_domain = (.+Domain.+)   If you can give some sample data, maybe I can help with writing more accurate regex. Or you can play around here and build your own regex: https://regex101.com/r/8rBc6U/1 . Once you know that your regex will filter out all the fields containing the keyword "Domain", use: blacklist = <your_custom_regex>   2. Move your props and transforms to the Indexers, and have them drop the data. Let me know if it helps. Thank you, S ... View more

Hello, HEC Event Collector on Splunk cloud is entirely managed by Splunk support (Almost everything is, apart from your forwarders). Filing a ticket with them is the best way to resolve the issue, as you won't be able to making any configuration changes on your own. Thank you, ... View more

Hey Rich, Thanks for answering. I've opened a support ticket already, but as you're aware it takes a bit of time for them to reply and catch up with the issues (No fault of theirs. They have multiple other things to do). I was wondering if anyone in the community had any ideas as to how can we handle this issue 🙂 Thank you, ... View more