This isn't something you want to do in fields.conf. You'll want to do this in transforms/props. In transforms.conf: [foo] REGEX = ^(?<date>[^ ]+)\s[^ ]+\,[0-9]+ \[(?<codetype>[^ ]+)\] New documents[ ]+: (?<newdocs>[^a-zA-Z._]*) \<[0-9 ]+ \((?<processid>[^ ]+)\)\> *DATASOURCE.(?<datasource>.*)\. And then in props.conf: [sourcetype] REPORT-sourcetype = foo Check the documentation for both props.conf and transforms.conf for more details. If it's a simple situation where you just have one extract for one sourcetype, you can eliminate the transforms.conf step, and do it all in props.conf: [sourcetype] EXTRACT-sourcetype = ^(?<date>[^ ]+)\s[^ ]+\,[0-9]+ \[(?<codetype>[^ ]+)\] New documents[ ]+: (?<newdocs>[^a-zA-Z._]*) \<[0-9 ]+ \((?<processid>[^ ]+)\)\> *DATASOURCE.(?<datasource>.*)\. ... View more

The way you would implement these in your transforms/props.conf is as follows: In transforms.conf you would actually define two separate transforms. [foo] REGEX = .*,Admin:\s(?<user>\w+),(?<message>.*),(?<policy>.*) [bar] REGEX = .*PES0:\s(?<machine>\w+),(?<srvr_action_taken>\w+),,(?<user_action_taken>\w+\s\w+),Begin:.*Rule:\s(?<rule_used>.*),\d+,(?<process_called>.*),\d+,No\sModule\sName,(?<filename>.*),User:\s(?<user>\w+),Domain:\s(?<domain>\w+) Then in your props.conf you reference the above transforms like so: [syslog] REPORT-syslog = foo, bar ... View more

I would recommend the use of xyseries here. Try the following to see it in action: index=_internal | stats count by host sourcetype | xyseries host sourcetype count This will use the values for the host and sourcetype fields for your row and column headers, respectively. (The format is 'xyseries row_identifier column_identifier data_value') Try swapping host and sourcetype in the above example to see how the output changes. This should accomplish what you're looking for nicely. ... View more

This sounds like an excellent case for event types. (Event type Documentation) Since your saved searches simply return events that match those criteria, you can use those searches as the basis for event type categorization. For example, if you use the following search as the basis for the event type 'tomcat_error'... sourcetype=tomcat logLevel="ERROR" Then any event that matches that criteria will automatically be flagged as eventtype=tomcat_error and you can then search for: eventtype="tomcat_error" and get all events that match the original search criteria. Then instead of a hundred saved searches, you just define all of those as event types, and they become very easy to aggregate. * | timechart count by eventtype You can continue to flag new event types as you discover them, and the reports won't need to be changed as they're just working with that field. ... View more

So the fundamental problem is that you want to limit the scope of the search for the start of the event to the timeframe of the summary search, while having a wider time range for subsequent entries so you can get complete transactions even if they cross over. I think this is something set union can help with. First of all, you need to know the maximum expected duration of a transaction, and make sure that you're running your summary search with enough room after the end of the timeframe for transactions to complete. So if a transaction can last 30 seconds at most, and you're capturing 15 minutes of transactions, run the search for -16@m through now. Then, use something like the following: | set union [search earliest=-16m@m latest=-1m@m "criteria to get only events that start transactions"] [search "criteria that gets all relevant events EXCEPT the ones that start the transaction"] | transaction startswith="starting qualifier" endswith="ending qualifier" This will take the results of the two sub searches, which have different time ranges, and unions them into a single result set. THEN you transaction on that result set. This makes sure you only get transactions that start within your specific 15 minute timeframe, but they can extend past it. With this, an event that crosses over will be counted once, in the span it starts in, but not in the subsequent span. Not sure if there is an easier way to do this, but this does work. EDIT: It's worth noting that the set command doesn't return events, it returns fields. (which include _raw by default) but it would be cleaner to have the two sub-searches return the fields you care about in your final transaction and aggregation. ... View more

There is a reference for High Availability that I think would be a good start here. ... View more

One thing to keep in mind is that a bucket won't transition over to frozen until ALL events in the bucket exceed the timeframe given. On low volume indexes, it's possible for a bucket to have data from a wider date range, and thus would hang around longer than expected. Another way to approach this is to estimate how much data you index in the time period desired, and then set the maxTotalDataSizeMB accordingly. ... View more

Saved searches can be stored in a couple of places. If the search is private to the user, then it would be under: $SPLUNK_HOME/etc/users/USERNAME/APPNAME/local/savedsearches.conf If it's a saved search that's shared, then it gets moved into the app itself: $SPLUNK_HOME/etc/apps/APPNAME/local/savedsearches.conf Look in these places for the missing content. These are what you'd want to restore from a backup to correct any corruption or missing configurations. ... View more