Looks like you already figured out the first part, but just in case, here's a good reference:      http://www.splunk.com/base/Documentation/latest/Knowledge/Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles#First.2C_define_a_field_transform If you want the extraction to apply to all sourcetypes, then just create your entry in props.conf at the top of the file, outside of any stanza heading. Entries in the file that are not beneath a [stanza] header are considered to be global. There's no way to do a wildcard like [log4j-*] , at least as of Splunk 4.1. There are three main options: Define the extraction globally, putting your EXTRACT-xxx or REPORT-xxx line at the top of props.conf. Define the extraction multiple times, with a separate stanza for each sourcetype Define the extraction for sourcetype [log4j] , and rename each log4j-xxx sourcetype. Going forward, manually assign the sourcetype so that you don't get the variations. Option 3 is usually preferable. ... View more

The default Unix app already contains a scripted input that runs ps . That should get you most of the way, but there are several different ways you might construct the search. Here's one approach, which will generate a new field is_running : index=os sourcetype=ps | head 1 | eval is_running=if(match(_raw, "\shttpd\b"), 1, 0) head 1 will retrieve only the latest polling cycle, giving you the "current" status. If you want to do things like charting status over time, leave that bit out. Using 1 or 0 makes charting easier, but you can also replace them with text values like "Running" or "Not Running" , etc. ... View more

More detail on your environment and what you are trying to accomplish would be helpful. However, the following may be helpful: In general, you may wish to consult Set up user authentication in the Splunk docs. The options are: Manually create an account with the same name and password on each indexer. Use LDAP Authentication with an external account store. Use a scripted authentication for RADIUS or to allow complete customization. Use an external authentication source and Splunk's support for single-sign-on. Options (1) or (2) are usually the easiest to maintain for a small number of indexers. Option (2) is particularly recommended if Active Directory or another LDAP provider is available. ... View more

Your extract line is missing an '=' after EXTRACT-snort_signature -- just a typo? ... View more

Or if you need a fully search-time approach: ... | rex max_match=100 field=_raw "\((?<subrecord>.*?)\)" | mvexpand subrecord | eval _raw = record | extract pairdelim=";", kvdelim="=-", auto=f | table _time, src_ip, ver, n, err, s, conn, fg, bg, sd, chan, dur, bytes, a, subrecord Advantage: Your web server logs are kept intact. Disadvantage: Messy search strings ... View more

host is one of the very few fields assigned at index time, so it works a little differently than most field extractions. The link you already found is the right starting point, but maybe this will help clarify. transforms.conf: [override-hostname] DEST_KEY = MetaData:Host REGEX = ^(\S+) FORMAT = host::$1 props.conf: [ErrorMissingObject] TRANSFORMS-host = override-hostname In transforms.conf, we're telling it to match as many non-space characters as possible at the beginning of a line. Then, assign it to the hosts value. In props.conf, note that you need to use TRANSFORMS-xxx instead of REPORT-xxx , since this is happening at index time. Props.conf is what tells Splunk to actually apply the transform we defined to your data. I'm assuming from your current props.conf that ErrorMissingObject is the sourcetype for these entries. You can just add the TRANSFORMS-host line to what's already there. ... View more

Agreed - this is a better way. ... View more

There isn't a native mechanism for that, at least as of 4.1. Your best approaches are to either use a scripted input to read the data, or to have an external script pre-process the log files before moving them into a directory monitored by Splunk. You might also want to submit an enhancement request:      http://answers.splunk.com/questions/4844/how-can-i-submit-an-enhancement-request ... View more

At some level, it becomes increasingly important to define the rules you want to follow. The more precise the definition, and the nature of the constraints, will dictate how complex the regex gets. More complex expressions also may suffer in terms of performance. In particular, the regex engine may start having to scan the same text multiple times looking for different ways to match. For example, here's one way to look at defining the domain name that would fit most purposes: A domain name is a single (optional) segment of the hostname, plus a TLD If the last two segments are each two characters, the TLD is two segments ( .co.uk ) If the TLD is not in .co.uk form, then it is one segment If the hostname is an IP address, then capture the entire string Hostnames of only 1-2 characters are considered valid Hostnames beginning with a dot are not considered valid. A simplistic approach is to break that down into three possibilities, and check each one: An IP address A string with no dots A fullly qualified domain name Given those three possibilities, here's one (ugly and inefficient) regex that would work: (?<domainname>(?<ip>^[A-Fa-f\d\.:]+$)|(?<nodots>^[^\.]+$)|(?<fqdomain>(?:(?:[^\.]+\.)?(?<tld>(?:[^\.\s]{2})(?:(?:\.[^\.\s][^\.\s])|(?:[^\.\s]+)))))$) That should produce the following results: u -> u uk -> uk hostname -> hostname 0.track.ning.com -> ning.com 0.tqn.com -> tqn.com 0.r.msn.com -> msn.com 0.52.channel.facebook.com -> facebook.com fe80::48e:b5c4:5670 -> fe80::48e:b5c4:5670 fe80::f002:192.168.1.1 -> fe80::f002:192.168.1.1 127.0.0.1 -> 127.0.0.1 gentoo.co.uk -> gentoo.co.uk what.are.you.talking.about -> talking.about gentoo.what.bar -> what.bar foo.ba -> oo.ba boo.com -> boo.com xx.boo.com -> boo.com ... View more

This previous question may help get you started:      http://answers.splunk.com/questions/2040/what-is-the-most-effective-ways-to-poll-databases-such-as-ms-sql-sybase-oracle ... View more

Another option is to use a third-party product to poll the database for you and convert it to a flat file or syslog output. Adiscon makes one, and there are almost certainly others out there. Buying additional software is far from ideal, but may be preferable for people without a strong scripting background. ... View more

Everything depends on your router model. The normal way to collect log entries from a router is via syslog. There are two things you'll need to do: Configure the router to send syslog data Configure Splunk to receive the data. For Cisco routers, the command is: logging 172.16.1.20 Of course, replace the IP address with that of your Splunk server. There are plenty of other syslog references out there. For Juniper, this may help. Many home-class routers have an option to export syslog as well, typically buried under an "Advanced" menu somewhere. On the Splunk side, you can either configure a Splunk listener on port 514/udp, or you can configure your syslog server to write out to a file and index that. For the latter approach, here's how to configure using syslog-ng. It's also possible to enable remote logging with other syslog daemons, though you may lose some flexibility. On Windows, look for Kiwi Syslog. ... View more

What kind of metrics are you looking for? Search performance? Frequency of user access? Something else entirely? ... View more

Maybe, or maybe it's a bug. What version of Splunk are you running? I seem to remember seeing doing something like that once before, on an older version. Or try the revised breaker string above - I gave that one a (very quick) test using your sample data and it worked fine on 4.1.6. ... View more

TimeGenerated is when the event occurred; TimeWritten is when it was written to the event log. Usually they'll be the same or very close. See also:      http://msdn.microsoft.com/en-us/library/system.diagnostics.eventlogentry.timegenerated(v=vs.71).aspx ... View more

Generally, you want to either use rex or create a dedicated field extraction. For more complete information, look here. Using rex: In the search string, add the following to your search: | rex field=xx "^\d+:\d+: DT : (?<txid>.*?) : EF" For something more permanent, you can use: transforms.conf: [mytxid] REGEX = "^\d+:\d+: DT : (.*?) : EF" FORMAT = txid::$1 props.conf: [yoursourcetype] REPORT-txid = mytxid ... View more

The best approach is almost certainly going to be to use a lookup table. See the link for information on setting up a CSV-based lookup. Once you have such a lookup, you'll be able to do a search such as: | metadata hosts | lookup mylookup host OUTPUT operating_system If you have a scripted input running uname -a as Genti suggests, that can be used to populate your table, e.g.: sourcetype=uname | fields host, operating_system | outputlookup mylookup Another approach, if you're lucky enough to have all systems in some form of directory would be to use a scripted lookup that leverages LDAP to query (for example) Active Directory. In a real pinch, you may be able to partially fill your CSV file from data within Splunk. For example, if you see WMI events, you can safely assume that it's a Windows system, and if you see 'ASA' or 'PIX' in syslog data, it's clearly a Cisco firewall. Ultimately though, the chances are you'll need to manually populate the CSV file. ... View more

You can add endswith=("Success") or similar in the 'transaction' command to force the end of a transaction as soon as it sees a success. The command also adds a new field called linecount to each combined result, so at that point you can search for linecount>10. ... View more