One license can only be used on a single instance without violating the terms of the license. However, if you kept a copy of the license on the second instance in the event of a disaster, you could add that license to make it the main instance while the other is down. After you've recovered, then you can merge the data using another method after taking the disaster instance down. ... View more

If the data isn't text, it's binary. If thats the case, you need this setting in props.conf: #****************************************************************************** # Binary file configuration #****************************************************************************** NO_BINARY_CHECK = [true|false] * When set to true, Splunk processes binary files. * Can only be used on the basis of [<sourcetype>], or [source::<source>], not [host::<host>]. * Defaults to false (binary files are ignored). http://docs.splunk.com/Documentation/Splunk/5.0/admin/Propsconf The data you provided in the sample clearly isn't text data, which means it will be considered as 'binary', irrespective of what is contained in the file. ... View more

last updated means the last time an event was received for that particular host. If you're seeing old hostnames, chances are that you've still got older indexed events containing the old hostname configurations, as you can't edit data in Splunk retroactively. Keep in mind what last updated doesn't mean. It does not mean the last indexed timestamp of an event which Splunk has processed, but rather the last time something was updated, irrespective of the timestamp. It is entirely possible for this scenario to occur if for some reason you're having issues with timestamp extraction. It could be that 1) your indexing historical data, or 2) Splunk is having problems extracting the timestamps from your events. ... View more

When you run into problems with timestamp recognition out of the box, the proper course of action is to provide Splunk with a set of instructions telling it where the time stamp in the event is, the format it follows, and how long the time stamp is within the event. These settings are TIME_PREFIX, TIME_FORMAT, and MAX_TIMESTAMP_LOOKAHEAD. These steps are outlined here with examples: http://docs.splunk.com/Documentation/Splunk/5.0/Data/Configuretimestamprecognition ... View more

All indexed data is not stored in 'defaultdb'. It could be that you're indexing data into other indexes. Defaultdb is the 'main' index, where data is sent if you haven't specified any other index. If you've got a custom index, data will be written to that index, and not 'defaultdb' in $SPLUNK_HOME/var/lib/splunk/. You've also got the internal indexes within Splunk, but that data shouldn't be counted by the indexing volume page. Keep in mind that warm buckets only contain data that is not going to be written to again, so if you're looking at warm buckets that contain data for a specific time period, there could also be hot buckets that are still being written and haven't yet been rolled to warm. Data is also compressed when it is written to disk, so although you're charged for 300mb, it isn't a 1:1 storage ratio. Usually the figure that is tossed around is ~50%. It might also be possible for a person to configure index retention in such a way as to clear out data older than a certain period of time/over a certain size. If that is done, you won't be able to tell by the size of the index about the indexing volume for that day. For these reasons, I wouldn't recommend that you use a method of looking at physical space used on disk to determine how much size is used. The page you're looking at hits the rest endpoint and that counts the totality of indexed data in non-internal indexes from midnight until the time you view the page. If you think you're having problems, start with this page: http://wiki.splunk.com/Community:TroubleshootingIndexedDataVolume ... View more

You can use the same REST endpoint that you use for regular Splunk searches - just change the search string to use mysqlquery instead Regular search: | search foobar | stats count dbquery search: | mysqlquery spec=mydb query="SELECT * FROM foobar LIMIT 10" Here's how you can use the REST endpoint to create searches [http://docs.splunk.com/Documentation/Splunk/5.0/RESTAPI/RESTsearches] ... View more

Yes, the universal forwarder can do remote wmi polling: http://docs.splunk.com/Documentation/Splunk/latest/Deploy/DeployaWindowsdfmanually http://docs.splunk.com/Documentation/Splunk/latest/Data/ConsiderationsfordecidinghowtomonitorWindowsdata#Polling_data_remotely_over_WMI.3F_Be_sure_to_read_this ... View more

We've seen this before, and what we noticed was that when we looked at the single line event data the instance was generating, it was all showing up in QRadar normally. The problem turned out to be that QRadar doesn't understand how to deal with multiline events, so each line is handled as an individual event. We confirmed the behavior with the vendor, presently it is a limitation of the product. ... View more

This was a problem that was fixed with 4.3.3, Splunk previously ignored files that we'd recorded as having a problem and wouldn't re read the files until after a restart. As of 4.3.3, this is no longer an issue, so you should probably upgrade. If this doesn't solve your issue, you should open a support incident, but it sounds like a good match for the problem you're describing in your post. ... View more

The log you want to examine is splunkd.log in $SPLUNK_HOME/var/log/splunk/. You can get more information if you turn up debugging in log.cfg on the Deployment client and server. In $SPLUNK_HOME/etc/, you'll find log.cfg. On the deployment server, change category.DeploymentServer=WARN to category.DeploymentServer=DEBUG On the Deployment Client, change category.DeploymentClient=WARN to category.DeploymentClient=DEBUG Then restart both the client and server instances, and you'll get quite a bit more information on what is occurring. Best Regards, Joshua Backing Splunk Support ... View more

Yes, see here: http://splunk-base.splunk.com/answers/38091/best-practices-to-deploy-the-splunk-on-splunk-app-in-a-distributed-search-environment ... View more

you can use 'splunk list deploy-clients' on the deployment server to see who checked in and when they last checked in with the DS. I don't think that will tell you who got what app, but at least you can see if the clients have checked in. ... View more

I would start by looking at splunkd.log on the forwarder in the $SPLUNK_HOME/var/log/splunk folder for messages that are from 'TcpOutputProc', they should give you an indication as to what is occurring when the forwarder tries to connect to the indexer. ... View more

On your heavy forwarder, you've made some mistakes. For props.conf, I would add TIME_PREFIX with a regex to match the data that comes before the timestamp you want to extract [host::192.168.1.1] MAX_TIMESTAMP_LOOKAHEAD = 30 TIME_PREFIX = REGEX TIME_FORMAT = %b %d %H:%M:%S.%6N This transform is incorrect: FORMAT = sourcetype::newsourcetype What you should be doing looks like this: [new_sourcetype] DEST_KEY = MetaData:Sourcetype REGEX = mydata FORMAT = newsourcetype ... View more