Could you please share the docs that specify the maximum value that can be used for maxQueueSize in outputs. conf for Splunk version 6.6.0?  ... View more

Hello, is this still the case in V9? Thanks. ... View more

That is a good question. The goal is to have Splunk "auto" discover a sourcetype based on a known list. The Splunk UI doesn't have this capability yet, so I have been trying to find methods for automation. I may have missed this in my first post, but I hope this update helps to solidify my first question. I would like for Splunk to auto discover the sourcetype based on a known list. I can then setup a response to a customer or an admin, based on the results of a Splunk automated sourcetype recognition of a "monitor" input. ... View more

Personally, I don't think that you need this. If you want to do data loss prevention 1: Turn on the useAck option on every forwarder 2: Make sure that you are not overloading the middle forwarder tier 3: Size the overall forwarder queue appropriately (maxQueueSize) - this is especially necessary on the middle forwarder tier Finally, you should not be using heavy forwarders as a middle tier unless you absolutely must. Universal forwarders are preferred as they are more efficient. I know that is counter-intuitive for most people, but it is true. Here is a very recent blog post that addresses this: Universal or Heavy, that is the question ... View more

Hi Lisaac, For number 2, Fluentd offers a great alternative to HEC as it allows you to bifurcate/copy/process/transform/parse the data at the node level and then send to the specific data store you need. For example, you can take all warning, error, critical priority Syslog messages and send those to Splunk while sending the lower priority + warning, error, criticial messages to HDFS, Amazon S3, etc. This allows you to reduce the volume that hits the Splunk indexers. Additionally, Fluentd offers an enterprise version that offers SLA based support for each of the outputs you mentioned (Splunk Enterprise, HDFS, Kafka, Amazin S3) - if you are interested email me at a@treasuredata dot com. More information can be found at https://fluentd.treasuredata.com Thanks, Anurag ... View more

Here is what i arrived at based on input from my Remedy team. I listed out the original Nix example along with a suggested MS-Windows example: [monitor:///opt/remedy_data/user/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db\?] “I don’t know what files this might map too? Any thoughts on this one?” disabled = false sourcetype = remedy_user [monitor:///opt/remedy_data/api/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db\arapi.log] disabled = false sourcetype = remedy_api [monitor:///opt/remedy_data/ar-error/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db\arerror.log] disabled = false sourcetype = remedy_arerror [monitor:///opt/remedy_data/filter/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db\arfilter.log] Do you think this might map to the file CAIFilterPluging.log as well? disabled = false sourcetype = remedy_filter [monitor:///opt/remedy_data/escalation/] “I don’t know what files this might map too? Any thoughts on this one?” disabled = false sourcetype = remedy_escalation [monitor:///opt/remedy_data/approval/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db\Approval*.log] disabled = true sourcetype = remedy_approval [monitor:///opt/remedy_data/ar-monitor/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db\armonitor.log] disabled = true sourcetype = remedy_armonitor [monitor:///opt/remedy_data/arplugin/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db\ar*plugin.log] covers arjavaplugin.log and arftsplugin.log covers arftsplugin-std*.log & arjavaplugin-std*.log disabled = true sourcetype = remedy_arplugin [monitor:///opt/remedy_data/cmdb/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db*cmdb*.log] covers arcmdb.log and cmdbengdebug.log disabled = true sourcetype = remedy_cmdb [monitor:///opt/remedy_data/fulltextsearch/] “I don’t know what files this might map too? Any thoughts on this one?” disabled = true sourcetype = remedy_fulltextsrh [monitor:///opt/remedy_data/mid-tier/] [monitor:/// mware\vfabric-tc-server-standard-2.6.5.RELEASE\BMC_MT_64Bit_V7\logs\armmidtier*.log] disabled = true sourcetype = remedy_midtier [monitor:///opt/remedy_data/servergroup/] “I don’t know what files this might map too? Any thoughts on this one?” disabled = true sourcetype = remedy_servergroup [monitor:///opt/remedy_data/sql/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db\arsql.log] disabled = true sourcetype = remedy_sql [monitor:///opt/remedy_data/thread/] [monitor:/// E:\Program Files\BMC Software\ARSystem\Arserver\Db\arthread_BMC.log] disabled = true sourcetype = remedy_thread [monitor:///opt/remedy_data/tomcat/] [monitor:/// mware\vfabric-tc-server-standard-2.6.5.RELEASE\BMC_MT_64Bit_V7\logs\catalina*.log] disabled = true sourcetype = apache_tomcat ... View more

Give this a try [monitor://E:\Program Files\Microsoft SQL Server\MSRS11.MSSQLSERVER\Reporting Services\LogFiles\] disabled = false index = app_ops_prod whitelist=ReportServerService.*\.log$ sourcetype = mssql:ilink:rptsvrsvc ignoreOlderThan = 3d ... View more

Hi lisaac, I've had experience with hundreds of applications with thousands of clients, and have found the biggest issue was utilizing the web interface for deployment server. I think that the known issues indicate this. With a growing number clients/apps, you'll have serious issues using the deployment server console. Of course, you can still do whatever you want from the command/conf files. To answer the question, I don't think there would be any problem with the operation of the deployment server in principle. Depending on how many clients are checking in to each individual DS, you'll need to increase the phonehome interval so the clients aren't hammering the server. Beyond that, it's a simple process where client checks in, compare's checksums of the app to the app on the DS, and then downloads and unpacks as needed. Please let me know if this answers your question 😄 ... View more

Adding to what muebel said, please read migrating search app twice if not more.....That's is going to tricky and please check the app.conf thoroughly after you consolidate all of the artifacts on Deployer before you push. Might take a lot of time to consolidate initially but will save you from this hassle on a long run 🙂 Thanks, Raghav ... View more

Problem: License violation alert and license_usage.log reportedly did not match Problem Summary: A license violation alert had been sent to a customer on a particular day (2/27), but the license summary page reportedly did not indicate that any license was exceeded. The customer did receive an alert mail message saying that splunk indexed over 300GB on the previous day (2/26), but the summary page reportedly showed nothing. What the customer did to troubleshoot the issue: Checked indexed license_usage.log in relation to Feb 27 from SH, but most of indexed data was already deleted due to misconfiguration. Configurations were fixed by PS and the customer can search entire indexed data since March 7th. Questions from the customer: How did the license master calculate the amount of data indexed on Feb 27th Did this unexpected license violation is occur due to the general error messages we found in splunkd.log. How to Troubleshoot: navigate to license_usage_summary.log' grep -rin --color "RolloverSummary" (or the date) "02-27" license_usage_summary.log look for the number behind the 'b' which is the amount of data actually indexed on the previous day. Resolution: Provided customer with explanation as to why customer was receiving license alerts on Feb. 27, which are illustrated below: Two alerts with different amounts of usage were seen on Feb. 27 because both slaves reported their usage to the master and each slave naturally reported different degrees of usage. Both these reports can be seen in the license_usage_summary.log report at the end of the day. (THIS LOG IS GENERATED AT THE END OF THE DAY BASED ON DATA FROM THE PREVIOUS DAY) There is no relation to the splunkd.log errors customer provided in the creation of this ticket and the 400GB license usage alerts customer received from the DMC. The DMC alert is what correctly pointed out the +400GB license usage which can be found in license_usage_summary.log. ... View more

The app (I'm presuming you are talking about this one?) does not mention anything about another input method. I can't imagine Tivoli not being able to write its own data out reasonably well and it hopefully is fairly efficient about the way it reads its own data. Splunk should not have any issues ingesting even rather large amounts of this data rapidly since it is apparently a text based, easily parsed input. Between those two, my guess would be that the majority of the work isn't going to be done in the transfer of data to Splunk so it shouldn't add a huge added load on any of the systems involved. Of course, if by "busy" you mean "running at 99.8% CPU and with disks steadily at 97% busy" then all bets are off. Hopefully that's not the case, though. 🙂 ... View more

the sample gateway .props creates a '!' delimited key/value flatfile. each alerts.status action dumps all values for that row. the TA-tivoli_netcool_object_server then parses all key/value pairs via KV_MODE = auto_escaped. ... View more

i've basically found that no splunk supported app works with shclustering no matter how much effort you put into it. don't waste your time. ... View more

1) splunk provides a True() function, which you should use in place of the 1=1. 2) I believe that should be 1==1. ... View more

In Pre 6.3: The only way to read archives/files in parallel is by spawning multiple instances of splunk on the forwarder. Splunk 6.3 release has a new feature where you can spawn multiple ingestion pipelines, where each pipeline can read one archive/file independently. So essentially with multiple ingestion pipelines, splunk will read multiple archives/files in parallel. Documentation: http://docs.splunk.com/Documentation/Splunk/6.3.0/Indexer/Pipelinesets ... View more

Technically possible. But, Splunk app is not officially supporting vertical scalability more than 8 workers at this time. Please create a Support case if you really need to do this. By the way, horizontal scaling by adding more DCNs is better practice. Scalability for 32 workers; - Vertical Scale: One DCN with 32 CPU cores - Horizontal Scale: 4 x DCNs ( 8 CPU cores for each DCN) Using horizontal scaling, when one DCN is down, three DCNs are still available. ... View more

Hi lisaac, Based on the busyness of the hosts involved in the search, it seems reasonable that there could be momentary periods of high latency that could generate these messages. There are various timeout settings described in http://docs.splunk.com/Documentation/Splunk/6.3.0/Admin/Distsearchconf that could adjust the environment's expectations, for instance: # this stanza controls the timing settings for connecting to a remote peer and # the send timeout [replicationSettings] connectionTimeout = 10 sendRcvTimeout = 60 ... View more

Yes, that is how I would do it! ... View more

I like the idea of changing the TRANSFORMS name, I have definitely gotten bitten in the past buy using reserved words in stanza names before and "host" has a high probability of that. ... View more

Short answer: no. Each indexer can belong to only a single cluster, with one cluster master. A cluster master defines the mode of the cluster: single site or multi-site, but not a mixture. ... View more

Your query is returning values for the Label and values for the Value. At least two different Label values correspond to the exact same Value value. That's the issue causing this warning. Your code will still work (hence warning, rather than error.) There are many ways to solve it. In my case it's really the logged records that were wrong, not the Splunk query, so I left it alone but asked for a fix to the logging. ... View more

This issue (SPL-79421) has been resolved in Splunk 6.3 and is incorporated in Splunk 6.4. You can get the latest Splunk release from: https://www.splunk.com/en_us/download/splunk-enterprise.html ... View more

Thank you for the information. This is what I needed. ... View more