Since the data is not structured as expected by the "default add-on", you will have to craft your own add-on to parse the fields. ... View more

That's worth trying. ... View more

Splunk has a host field that identifies the sending server, but there is nothing built-in that identifies the source location (site, IP, etc).  You would need to add fields to include that information.  Use INGEST_EVAL in transforms.conf to do that. ... View more

What are looking at to determine there is an indexing delay?  Another possibility is event timestamps are not honoring the correct time zone and so events just *appear* to be delayed. ... View more

A field that will only accept a standard email address certainly will not accept a PowerShell script. ... View more

Splunk probably expects a dot and TLD in the domain name of the email address.  Go to https://ideas.splunk.com to ask for support for local email addresses.  Until that is added, use a standard email address. ... View more

That you are ingesting Management events, but not Network Activity events tells me there is an input defined for the former, but not the latter.  Perhaps this is not the intent, but it is the effect. Can you share the CloudTrail input(s)? ... View more

You need an input to tell Splunk to ingest the Network Activity data. ... View more

Have you configured and enabled an input as described in https://splunk.github.io/splunk-add-on-for-amazon-web-services/CloudTrail/ ? P.S. It appears the version 7.10.0 is no longer available to consider upgrading to 7.11.0 or newer. ... View more

We need more information. Are you saying to want to send data collected by Splunk to GCP?  Is that all data or just some of it? Can you be more specific about the destination?  Saying "GCP" is like saying "AWS".  Both offer a large number of services with different requirements and capabilities. What is your SIEM? It may be necessary to write code that uses the API to fetch data and then ship it to GCP. ... View more

Does your Splunk Cloud stack have REST API enabled (it's off by default)?  If not, open a support case. Is the IP address of the server running the script on your stack's IP Allow List?  If not, get a Splunk admin to add it. ... View more

They are not "steps".  They're separate checks/measures to perform to try to alleviate delayed searches. See https://help.splunk.com/en/splunk-enterprise/get-started/deployment-capacity-manual/9.4/performance-reference/reference-hardware and https://help.splunk.com/en/splunk-enterprise-security-8/install/8.1/planning/minimum-specifications-for-a-production-deployment Enterprise Security is a very resource-intensive application.  Therefore, it is recommended to install ES on a separate Splunk instance.  It can, however, share indexers with other search heads. This should require no explanation. Real-time searches pin themselves to a CPU, preventing other searches from running there.  Don't use real-time searches.  See https://help.splunk.com/en/splunk-cloud-platform/search/search-manual/9.2.2406/search-and-report-in-real-time/about-real-time-searches-and-reports for more. Entire books could be written on making searches more efficient.  Splunk has one at https://help.splunk.com/en/splunk-enterprise/search/search-manual/9.4/optimizing-searches/about-search-optimization Allow Skew gives the search scheduler permission to adjust the run time of a scheduled search to one with fewer other searches scheduled.  Search Windows allow the scheduler to delay the start of a scheduled search in the event that resources are not yet available.  See https://help.splunk.com/en/splunk-enterprise/create-dashboards-and-reports/reporting-manual/10.0/report-management/offset-scheduled-search-start-times and https://www.splunk.com/en_us/blog/platform/schedule-windows-vs-skewing.html There is a strong tendency among Splunk users to run their scheduled searches at the top of an hour.  At most of the customers I've visited, this accounts for about half of all scheduled searches and is a source of most of their delayed and skipped searches.  It also doesn't account for the 30-90 seconds of delay between when an event is generated and when it is searchable by Splunk.  It's far better to use a cron schedule to have the search run at 2-3 minutes after the hour.  Other peak search periods to avoid are 15, 30, and 45 minutes into any hour of the day. Splunk's Workload Management feature gives Splunk admins some control over how resource contention (CPU and memory) is handled.  It also can be used to stop long-running searches, prevent real-time searches, and prevent users from running searches during peak times.  See https://help.splunk.com/en/splunk-enterprise/administer/manage-workloads/9.4/workload-management-overview/about-workload-management for details. I've seen plenty of instances where a reports run once a day or even every week at 8 or 9 in the morning.  This usually is unnecessary and takes away "slots" from other searches.  Instead, these types of "batch" reports should run in less busy times of day such as 3am or on weekends.   ... View more

If there were no events in the last 24 hours then there is no last timestamp to display. The only way to get the timestamp would be to join the current search with one that scans the logs for the most recent entry for each sourcetype in some larger time window (perhaps 30 days). ... View more

It's not clear if you want to correlate Splunk data in your ITSM product or ITSM data in Splunk. There are a few ways to onboard data into Splunk. Install a universal forwarder on the server to send log files to Splunk Have the server send syslog data to Splunk via a syslog server or Splunk Connect for Syslog Use the server's API to extract data for indexing Use Splunk DB Connect to pull data from the server's SQL database. Have the application send data directly to Splunk using HTTP Event Collector (HEC). There are REST API calls available for getting data out of Splunk. Let us know which you are trying to do and we'll point you toward to relevant docs.   ... View more

Use the table command to put the fields in the desired order | chart sparkline count by a,b | table a b count sparkline   ... View more

There is no way to mitigate the vulnerability except to install the next version of Splunk that includes a fix.  We don't know which version that will be, but Splunk typically announces when vulnerabilities are fixed.  Check out https://advisory.splunk.com/ ... View more

Thanks for clarifying. While Splunk Enterprise can be installed on Windows workstations, that's typically done for dev/test purposes.  Workstations don't have the horsepower to handle enterprise quantities of data. The typical model is to install SE on shared servers (on-prem or virtual) with all other enterprise devices running a Universal Forwarder to send data to SE. ... View more

These are two different problems. For the first, know that Splunk indexes are immutable.   Data cannot be removed selectively.  My advice is to copy  the desired data to a new index (use the collect command) and then delete the contents of main. For the second, I presume you're referring to the list of forwarders in the Monitoring Console.  The "missing" forwarders can be removed by clicking the "Rebuild asset list" button. ... View more

Why are you installing Splunk on every Windows system?  It should be installed on a central server and accessed via web browser. Perhaps you're referring to the Splunk Universal Forwarder.  That should indeed be installed on each machine, but do it as part of the system image.  If that's not possible then I know customers have installed it using GPO. To prevent Splunk from asking for credentials, supply them when launching the installation.  See the installation manual.   ... View more

You may want to contact the developer directly at support@pingidentity.com ... View more

Searches are delayed because too many searches are trying to run at the same time.  There are a few things you can do about it. Ensure all instances meet or exceed Splunk's Reference Hardware specification. Ensure Splunk Enterprise Security runs on a dedicated Search Head (or SHC).  Do not run searches unrelated to ES on that SH. Eliminate unneeded scheduled searches. Prohibit real-time searches. Verify all scheduled searches complete as quickly as possible.  Do this by minimizing their search windows and using efficient searches. Implement Allow Skew and Schedule Windows. Distribute search run times evenly across the clock.  Avoid running at peak times such as 0, 15, 30, or 45 minutes of each hour. Consider using Workload Management to control the search behavior of users. Move daily/weekly/monthly scheduled searches to off hours. ... View more

Please contact your Splunk account team or the Partner from which you purchased Splunk for answers to pricing questions.  Only they will have the correct answers. ... View more

No, you are not losing "Running" from each event.  Only the first capture group in LINE_BREAKER is discarded from the event. However, positive and negative lookaheads do not perform well in Splunk so they should be avoided. ... View more

Always.  It never hurts to have it and can help when the UF has to break an S2S packet.  Don't forget to also set EVENT_BREAKER_ENABLE=true. ... View more

Positive lookahead wastes resources.  Over millions of events, those extra cycles add up to excess SVC consumption.  This works better LINE_BREAKER = ([\r\n]+)Running ... View more