Does this search do it for you? index=_internal source="C:\\Program Files\\Splunk\\var\\log\\splunk\\metrics.log" earliest=-7d@d | table sourceHost | dedup sourceHost | stats count with the source path changed accordingly of course! ... View more

Is there anything stopping you from installing python? I am using universal forwarders but created a deployment app that contains a portable Python version and a script to add the app directory to the PATH environment variable. You could also just install python to %Splunk Install Path%\Splunk\bin. ... View more

Remember to accept hazedav's answer as correct! ... View more

Have you recently upgraded to v4.2.1 from a previous Splunk version? I had a similar problem when I upgraded with the wrong user account (a different one then I used to originally install). ... View more

I believe they are equal because in the end, they both are probably making the same WMI query. But, that said, I usually turn off the WMI one because the Sourcetype is longer. They both come from the Windows app, the first one comes from an entry in wmi.conf and the second from an inputs.conf entry. I believe they both get turned on if you choose to turn on every WMI input by default and also turn on every WinEventLog input as well during install, though that might not be the case. I have only noticed it happening on my 4.2 Indexer after I upgraded from 4.1.x with upgraded windows app as well. ... View more

This happens especially when bringing new Splunk forwarders online, they try and report every historic log on the machine. ... View more

If you are looking to change the port that the deployment client (forwarder) looks for on the deployment server to receive its configurations from, then it is set at *\SplunkUniversalForwarder\etc\system\local in deploymentclient.conf which looks like this: [target-broker:deploymentServer] targetUri = deployservername:8089 If you are looking for the port that a deployment server manages from, then use mgmtHostPort in web.conf as others have answered. ... View more

Are you using Heavyweight Forwarders? If you are, the props.conf should be set on the forwarder. ... View more

I am a little confused at your setup, could you expand on it? -Are you using a Indexer/Forwarder relationship or is this all on one box? -Is "iis-2" a sourcetype? a source? a host? If you can answer those then I may be able to help more, in the mean time I made a post about timezones in a different question, it may be able to help you HERE Here is the excerpt from that post: I had a pain syncing my timezones too, here is the info that helped me out: First, you should note that the Splunk Indexer sets everything relative to its own time zone. Thus if you want to have the Logs be indexed based on CST, the indexers timezone must be set to CST. The Indexer gets its timezone info from the clock set on the machine its installed on, so to reiterate, the Indexer machines Time and Date settings should be set to CST if that is the timezone you want to base inputs off of. Yes, it is kinda annoying, w/e. Secondly, all machines that are in a different time zone from your Indexer (anything not in CST) will need to have a TZ setting in props.conf. The TZ setting will be set to whatever timezone the forwarding host is in, thus if the Indexer is in CST and the Forwarder is in EST then the TZ set in props.conf on the Indexer for the Forwarder would be set to EST. Splunk will then figure out the difference between the two timezones and mark inputs accordingly. I also noticed that you were not using correct TZ codes for the TZ setting, "GMT" is not a correct TZ code. The list of TZ codes can be found here: http://en.wikipedia.org/wiki/List_of_zoneinfo_timezones (Look under the TZ column) ... View more

Hey MasterOogway, I had a pain syncing my timezones too, here is the info that helped me out: First, you should note that the Splunk Indexer sets everything relative to its own time zone. Thus if you want to have the Logs be indexed based on CST, the indexers timezone must be set to CST. The Indexer gets its timezone info from the clock set on the machine its installed on, so to reiterate, the Indexer machines Time and Date settings should be set to CST if that is the timezone you want to base inputs off of. Yes, it is kinda annoying, w/e. Secondly, all machines that are in a different time zone from your Indexer (anything not in CST) will need to have a TZ setting in props.conf. The TZ setting will be set to whatever timezone the forwarding host is in, thus if the Indexer is in CST and the Forwarder is in EST then the TZ set in props.conf on the Indexer for the Forwarder would be set to EST. Splunk will then figure out the difference between the two timezones and mark inputs accordingly. I also noticed that you were not using correct TZ codes for the TZ setting, "GMT" is not a correct TZ code. The list of TZ codes can be found here: http://en.wikipedia.org/wiki/List_of_zoneinfo_timezones (Look under the TZ column) So to wrap up, I think your entry looks good except that "GMT" is not a correct TZ. I would double check that you use the right TZ for daylight savings time, UTC has more zones then GMT with more specific daylight savings times. Use the wiki page above and pay attention to the Standard Time vs Summer Time, for example America/Dawson_Creek has a standard time zone of UTC-07 and no summer time while America/Cambridge_Bay also has a standard time zone of UTC-07 but it has a summer time of UTC-06. If the Indexers timezone and the Forwarders time zones are not synched for daylight savings time then you could hit some nasty bugs whenever daylights savings time changes. Good Luck and Happy Splunking! ... View more

Hey Wlhuang, The host value gets set by the server.conf configuration file, this is located at %SPLUNK_HOME%\etc\system\local or \etc\system\default if one has not been made in local. If this config file was messed up then it would give you problems with the host value. It seems weird that this would happen on only one machine if you are ghosting/cloning machines. I assume you are using some sort of forwarder on your machines being cloned? To locate which machine is being weird, I would suggest taking a look at the metrics.log file located on the Splunk Indexer at %SPLUNK_HOME%\var\log\splunk. Inside this log file there should be a "group=tcpin_connections" INFO line for each machine that is talking to the indexer. I would search for "l3win7clone" which should return a result for the hostname field on one of the INFO lines, i.e. "hostname=l3win7clone". At the beginging of this line should by an IPaddress field. You should be able to find the server once you have the ip address. Hope this helps! ... View more

Hey Msarro, Have you tried using the Interactive Field Extractor? Maybe look Here. This is a great tool, especially for us who are hesitant in out abilities with regex. Zero padding should not matter, you will probably be using "\d" for digits, and just throwing on a + will give you "one or more times", thus, \d+ means 1 or more digits. For example it would match 0, 02312300123, or 23. Either way, starting with the IFE to give you a good guess at the regex and then all that matters is making sure you understand what Splunk is saying with the regex it generates and editing it if you notice and errors. GL! ... View more

It seems like some Release Notes are posted, the Change Log at least (this seems to be a minor bug fix release). They can be found here: 4.1.8 Release Notes Here is the Excerpt from that site: 4.1.8 The following issues have been resolved in this release of Splunk: * Epoch timestamps not parsed correctly after March 12, 2011, (SPL-38016) * A crash issue has been identified with SuSE Linux Enterprise Server 11 / OpenSuSE 11.0,11.1,11.2 and 11.3, combined with Splunk on x86_64. It is caused by a SuSE patch to glibc which breaks programs using external mallocs. The problem can be identified by __res_iclose() in the backtrace in the crash log. (SPL-37331) * Improvements to IndexProcessor service performance. (SPL-37390) * Annoying but harmless "ERROR Timeliner - Failed to rm dir" (SPL-35722) * Scheduled searches stop firing with occasional "WARN SavedSplunker - Saved splunk failed to get current user context" error in scheduler.log. (SPL-38047) * Crash in AD with a large number of .dmp files whenever the AD message has %s or similar string. (SPL-37640) * Window installation erroneously creates splunk-regmon.py and splunk-wmi.py stanzas in inputs.conf. (SPL-37440) * When registry monitoring baseline data from multiple hives are selected to be monitored, the baseline events from the first collection are repeated multiple times. The baseline data from the other stanzas do not show up. (SPL-37381) * Incomplete events in parsingQueue are indexed as 'complete' when the 10 second idle timeout limit is hit. (SPL-37252) * Diag is failing due to token files in $SPLUNK_HOME/etc/alive_tokens/alive_xxxx.token. (SPL-37934) * Issues with getting results for certain session IDs with error "ERROR SearchResults - Error reading internal file header". (SPL-37712) * If splunk-regmon baseline=1 is set for a stanza, regmon collects events even if it the collection is disabled. (SPL-37620) * Regmon filters are case sensitive and shouldn't be. (SPL-37619) ... View more

Perfmon does usually have a Process Query, you would have to add the counter for it in perfmon.conf. The entry would look something like this: [PERFMON:Process Monitor] counters = ID Process;% User Time disabled = 0 instances = * interval = 10 object = Process index = some_Index The instances field could be * (all processes) like mine or a specific process, ex. notepad, splunkd, or explorer. The Counters field would be aspects you want to monitor of the process, PID, Memory Usage, CPU Usage, etc. EDIT: Switched the counters and object, got messed up in translation. It is correct now, or look at Dokian's, Thx Dokian! EDITx2: Clarity/Grammar ... View more

I am just wondering if you are using the Free License on the indexer or the forwarders? If you were talking about the forwarders, then do you mean the Free License or the Forwarder License. I am wondering if this might change how configurations are handled. ... View more

I only used this solution for a brief period to make sure that I did not blow the license I had in the interim before getting a new license. I should also note that once the data gets into the Indexer then its time stamp is based off of when it was logged by the forwarder or if it had time stamp given by the reporting mechanism. Example: if it was currently 5pm and I did a search from 4-5pm and some piece of data was in the cache to be indexed with a timestamp of 4:30pm, then my search would not return it. later if I did the same search after the data had been indexed then it would show up. ... View more

Yes, the server would cache the data and trickle it into the indexer at the fixed rate. This is why it is so ugly, if you have a spike in data input then there is a slower response before it shows up in the index and if the data spike is larger then your daily limit then the cache could get larger and larger. What I am not so sure is if the thruput is limited at input or output on the indexer. If it is limited in input then the forwarders would hold the cache because they couldn't push to the indexer, if output is blocked then the cache would be on the indexer before it puts it on disk. ... View more

Disclaimer: CAUTION: Do not alter the settings in limits.conf unless you know what you are doing. Improperly configured limits may result in splunkd crashes and/or memory overuse. -I can't say I really know what I am doing so take this answer with a grain of salt. A 100% effective, although unconventional, way to ensure that you never go over your indexing limit is to limit how fast the indexer can run. On the Splunk index server, add a stanza to the limits.conf file: $SPLUNK/etc/system/local/limits.conf: [thruput] maxKBps = # To figure out what the # should be, divide the daily license cap (1GB: 1073741824 bytes) by 86400 (seconds in a day), to get your max Kbps rate (12427 bytes/sec, or 12KB). This doesn't sound like much, and it isn't for a single second, but if splunk runs steadily all day long, you'll get close to your limit, but not go over it. Yes, this is ugly but it is the only sure-fire way I have found to limit Splunk's insatiable hunger. I hope someone has a better answers because I would sure like to know it. ... View more

Hey bshuford, Simple answer: Use just @mon. This will tell Splunk to look at the last month, if you put something in front (i.e. -1mon@mon) it looks for the last month from that point going back (the first start of month since 1mon ago). ... View more

Awesome, very informative. Thanks for the response! ... View more

Hello rgraetz, Here are some assumptions I am making about your question, if these are incorrect then please clarify in your question. 1: That you have the Windows App installed and are using it to gather WMI information. 2: That you are not using deployment clients or light weight forwarders. If that is the case then in the folder %SPLUNK-INSTALL%\etc\apps\windows\local (ex. C:\Program Files\Splunk\etc\apps\windows\local) on the server you want to monitor there should be a file called wmi.conf that holds configurations for WMI inputs. It's got stanzas like this inside: [WMI:FreeDiskSpace] interval = 60 disabled = 0 server = localhost You want to Add (or un-disable) the LocalNetwork one, which will look something like this: [WMI:LocalNetwork] interval = 300 disabled = 0 TL;DR: The correct config file is wmi.conf in the folder %SPLUNK-INSTALL%\etc\apps\windows\local and the syntax is like above 🙂 ... View more