Let's try one last time! In props.conf, the source stanza actually follows a modified regex syntax. (See my other comment from 19-nov-2024 or just read props.conf.spec). So the following should work [source::/opt/splunk/var/log/syslog/*lx0001*/*] or perhaps [source::/opt/splunk/var/log/syslog/*lx0001*/*.log] But re-reading the original post, @jonatanjosefson was actually trying to set some property for all syslog events corresponding to a particular host name pattern - you can do that using a host stanza instead of a source stanza: host::*lx0001* To me, this seems easier to understand and maintain, as it will work even if the directory structure of the syslog files changes over time. It depends only on the hostname, and not the file name or location.       ... View more

For the source stanza, Splunk uses regular expressions that are PCRE (Perl Compatible Regular Expressions). From props.conf.spec **[source::<source>] and [host::<host>] stanza match language:** Match expressions must match the entire name, not just a substring. Match expressions are based on a full implementation of Perl-compatible regular expressions (PCRE) with the translation of "...", "*", and "." Thus, "." matches a period, "*" matches non-directory separators, and "..." matches any number of any characters. Also from props.conf.spec When setting a [<spec>] stanza, you can use the following regex-type syntax: ... recurses through directories until the match is met or equivalently, matches any number of characters. * matches anything but the path separator 0 or more times. The path separator is '/' on unix, or '\' on Windows. Intended to match a partial or complete directory or filename. | is equivalent to 'or' ( ) are used to limit scope of |. \\ = matches a literal backslash '\'. So for mylog_*  you could specify source::.../mylog_* It's been a few years on this one, so hope I am right this time! ... View more

You can't tell Splunk not to store the job, the search would break. The solution is to make sure that this directory/volume has sufficient space to store the temporary files for the searches that you run. ... View more

What is the version of Splunk, plus the size of your environment? ... View more

The SEDCMD needs to go in a props.conf on the indexers (or wherever the data is being parsed, but definitely not the search head). ... View more

The difference between The Simpsons and South Park in a single comparison... ... View more

Try adding these 4 lines ... | eval temp=hits * 'AVG RT' | eventstats sum(hits) as totalhits sum(temp) as sumtemp | eval wt_rt = sumtemp / totalhits | fields - sumtemp totalhits ... View more

If you want all the data in the file to be a single event, you can probably do that better with LINE_BREAKER Try this in props.conf [yoursourcetype] SHOULD_LINEMERGE=false LINE_BREAKER=((thismustneverappearinyourfile)) TRUNCATE=0 I don't think that you need to set MAX_EVENTS at all when using this method. But feel free to add in MAX_EVENTS as well... This technique works by using LINE_BREAKER to define the split between events - and then assigning an "impossible" character string as the line-breaking condition. ... View more

Where do you want it to break? What is one event in this file? Is the whole body a single event? ... View more

Indexed real-time reads the disk instead of watching the data flow between indexing steps in memory. When a search is running though, it consumes a memory core, regardless of whether it is a real-time search or an indexed real-time search or a historical search. You can set different limits on the number of concurrent searches by type (in limits.conf) - but using indexed real-time does not inherently increase the number of possible parallel searches. While running, an indexed real-time search performs like a historical (normal, not real-time) search at any point in time, in terms of memory use and cpu workload. The difference is that an indexed real-time search never finishes reading the data, as it is always waiting for more data to arrive on disk. However, because it uses the normal OS I/O mechanisms, an indexed realtime search can take advantage of disk caching, concurrent reads, etc., making it more efficient than a real-time search. A real-time search monitors memory and tends to slow down the indexing of incoming data, which can lead to other problems. However, these differences occur on the indexers - where the data is being read and indexed - and not really on the search heads (AFAIK). So you might think, "this isn't going to affect my search head at all" - but it does. All searches essentially compete for time on the indexers, so when a less efficient search runs (like any real-time search), it can affect the throughput of all searches running on the search head. AFAIK, there are no hard numbers, as the cost/impact will depend on the workload mix (indexing vs. searching) on the indexers as well as the workload (number, size, and types of searches) running on the search head. I think the effects will be more obvious with a higher system load; for example, systems with unused cores might show little difference. But even though there are no hard numbers, I think it means a lot when Splunk says "All real-time searches in Splunk Enterprise Security use the indexed real-time setting to improve indexing performance" in the Enterprise Security Installation and Upgrade Manual under Deployment Planning. But notice it says "indexing performance" not "search performance." I think your main performance improvement happens by not bogging down the indexers, which in turn will make the searches run faster on the search head. Hope this helps! You might want to benchmark this a bit in your own environment. I know this answer is not definitive and I would love to hear more from others... ... View more

I don't mean to be a jerk, but my bet is that your user name or password is invalid. Did you change the admin password on the forwarder? If not, the default user name is admin and the default password is changeme. You really don't need any other Splunk user on your forwarder except admin. ... View more

In order for Splunk to send an email, Splunk must be able to connect to a valid email server with an appropriate user name (and password.) This is configured by the Splunk system admin; in the GUI, the email settings can be found under Settings >> Server settings >> Email settings ... View more

How do you know which employee made which sale? Is it possible to have 2 employees at the same location at the same time? ... View more

When you need to group events using the startswith= and endswith= options, it is generally very difficult to replace that with stats. Is there another field that represents something like a "session id" that could be used to group events, instead of the startswith/endswith? ... View more

Now documented here: http://docs.splunk.com/Documentation/Splunk/latest/DistSearch/SHCsystemrequirements At 7.0.2, the max is 100 search heads in a cluster; Thanks @hexx ... View more

When I look at the bucket_flags using | rest /services/cluster/master/buckets , I see the following: 0x0 0x2 0x3 0x4 0x5 0x7 This is a multisite cluster with 2 sites; each site has 2 indexers. There have never been any more indexers or any other sites. The cluster is complete and valid; it is not in maintenance mode. What do these flags mean? I would expect to see 0x3 and 0x5 only... I do have some nonclustered buckets (with flags of 0x7) and some single-site buckets (with a variety of flags) ... View more

Lesson learned: Both the rest command in Splunk (from the master node) | rest /services/cluster/master/control/control/rebalance_primaries splunk_server=local and the browser interface use an HTTP GET. This is great for retrieving info, but doesn't work for triggering the rebalancing action. ... View more

This is not a bug with the transaction command, it is inherent in how the command works. The transaction command creates a new event from the original events, so there is nowhere to "drill down." If you want a drill-down in the dashboard, you can write a custom drill-down; here is the documentation: http://docs.splunk.com/Documentation/Splunk/latest/Viz/DrilldownIntro I generally recommend custom drill-downs for dashboards; they are pretty easy and provide a much cleaner interface for users. Note that most dashboard panels have an "open in search" magnifying glass; for more sophisticated users, this is an alternative to drill-down that allows them to see the underlying search and manipulate it as desired. ... View more

You might want to restart the master node. ... View more

(blush) You are welcome ... View more

So could you identify logins/logouts using Windows Event Codes? (I am not a Windows person.) What event codes do you use for valid logins and logouts? ... View more