If you have to stay in 2.x, I would go with 2.4 If you can go to 3.x I would do that as it has some major performance/design enhancements. http://blogs.splunk.com/2017/02/20/splunk-db-connect-3-released/ Splunk DB Connect 3.0 is a major release to one of the most popular Splunk add-ons. Splunk DB Connect enables powerful linkages between Splunk and the structured data world of SQL and JDBC. The major improvements of this release are: Performance improvement. Under similar hardware conditions and environment, DB Connect V3 is 2 to 10 times faster than DB Connect V2, depending on the task. Usability improvement. A new SQL Explorer interface assists with SQL and SPL report creation. Improved support for scripted configuration, via reorganized configuration files and redesigned checkpointing system. Note that rising column checkpoints are no longer stored in configuration files. Stored procedures support in dbxquery. Retry policy on scheduled tasks is improved (no more need for auto_disable) Either way be sure to check out the release notes/known issues here: http://docs.splunk.com/Documentation/DBX/latest/ReleaseNotes/Releasenotes ... View more

Hey joesrepsol! I like building on top of the meta woot! app: https://splunkbase.splunk.com/app/2949/ https://discoveredintelligence.ca/meta-woot-update/ I pair the app's approach to summary indexing and it's trend and compliance views with the Machine Learning toolkit ( https://splunkbase.splunk.com/app/2890/ )to apply algorithms that alarm when a sourcetype or host deviates from normal trend. It populates a summary index at a chosen interval (you chose 5, 15 or 30 mins) using tstats similar to your example. This provides a highly customizable and advanced analytical view of your data while providing some really sweet tools for your Splunk arsenal! Eventually you can not only build out better views of what data is in Splunk for your users, but you can also provide data feed integrity at a granular level. For example, here is an example of looking at a sourctype trend over the last 4 hours to find deviations in the 5 min sum of events: Whether monitoring for spikes or dips in ingest, alerting the search that counts deviations by upper or lover bounds is useful: index=`meta_woot_summary` sourcetype=meta_woot orig_sourcetype!=stash orig_sourcetype=juniper:junos:firewall orig_host=* orig_index=n00blab | timechart limit=20 span=5m sum(count) as totalEvents by orig_sourcetype | streamstats window=12 current=true median("juniper:junos:firewall") as median | eval absDev=(abs('juniper:junos:firewall'-median)) | streamstats window=12 current=true median(absDev) as medianAbsDev | eval lowerBound=(median-medianAbsDev*3), upperBound=(median+medianAbsDev*3) | eval isOutlierLower=if('juniper:junos:firewall' < lowerBound, 1, 0), isOutlierUpper=if('juniper:junos:firewall' > upperBound, 1, 0) | timechart sum(isOutlierUpper), sum(isOutlierLower) As you can see the toolkit lets you spit out the SPL you can use and provides some cool vizualizations that you can leverage. Then once you get comfortable, you could even timewrap your event trends and then run the deviations against the timeseries which would provide a real nice view of "are we normal" Lots of options! Maybe start with simply using the trends to set static lowerbound thresholds and work your way up to a more dynamic trending analysis! ... View more

No upper limit is posted in the spec file maxEventSize = <integer> * If specified, sets the maximum size of an event that splunk will transmit. * All events excedding this size will be truncated. * Defaults to 1024 bytes. https://docs.splunk.com/Documentation/Splunk/6.5.2/Admin/Outputsconf Are you forwarding from Splunk to rsyslog? ... View more

Hey Rich! Your regex is fine, I think the issue is that rex didn't like the leading underscores on your field and value names...(remember, leading underscores are reserved for Splunk internal use) This worked for me: | rex max_match=0 field=_raw ".+?name="(?<KEY_1>.+?)"\>(?<VAL_1>.+?)\<\/col>" Once you have the rex the way you like it, you can implement this with props and transforms... http://docs.splunk.com/Documentation/Splunk/6.5.1/Knowledge/Configureadvancedextractionswithfieldtransforms I tried using KV_MODE= xml but it didn't extract what I think you will want...the whole "col name =" got in the way, and made it messy.. the main thing is we need to keep the KV pair relationship to do the reporting you want...will play in the lab and see what I can come up with then update this post. UPDATE: Here is the props.conf and transforms.conf I used to parse the fields and keep the KV pairs. Big Up MuS for the optimized regex to deal with null values in the XML. Previous regex was too greedy for null fields. props.conf [answers494268] BREAK_ONLY_BEFORE=<row> CHARSET=UTF-8 KV_MODE=none MAX_TIMESTAMP_LOOKAHEAD=125 NO_BINARY_CHECK=true SHOULD_LINEMERGE=true category=Custom disabled=false pulldown_type=true TIME_FORMAT=%m/%d/%Y %H:%M:%S %p TIME_PREFIX=<col name="time"> REPORT-xml = answers494268xml transforms.conf [answers494268xml] #REGEX = .+?name="(.+?)">(.+?)</col> #Above regex too greedy for null values REGEX = .+?name=\"([^\"\>]+)\"\>([^\<]+)\<\/col> FORMAT = $1::$2 One thing to note in this data...there are some fields with no values....not sure if this is because you were scrubbing the data to be shared...but just keep in mind the first regex wont match if there is no value for the key, thus messing up the transform...updated rex ensures no erroneous matches ie. <col name="User"></col> <col name="Message"></col> <col name="Datasource"></col> ... View more