This typically happens for two reasons.
The value of slotID, here "1234", is not searchable on its own. We optimize searches by replacing the equality with the value, and post-filter. If this is the case, you can mark "slotID" as "INDEXED_VALUE = false" in fields.conf.
If the slotID extraction is configured via an eventtype in props.conf, it will be extracted, but not searchable in the first search clause. There is no workaround for this.
... View more
Instead of stats , try chart, like:
... | chart count by WEBSPHERE_FUNCTION WEB_STATUS
Or you could do:
... | stats count by WEBSPHERE_FUNCTION WEB_STATUS
... View more
Replication is the process of moving the search-time configuration from the search head to the indexers. This is usually very fast, but can take some time if the indexers are across a WAN or if the apps on the search head are particularly large. We've made some improvements in 4.1.5 to make replication faster, but it can still be slow for large apps.
If you are on an intranet, you can make it faster by adding to server.conf :
[sslConfig]
useSplunkdClientSSLCompression = false
Or you could even disable SSL entirely:
[sslConfig]
enableSplunkdSSL = false
Finally, you can find large assets in your apps, and disable replication in distsearch.conf :
[replicationWhitelist]
largeFiles = .../largefile.*
... View more
When you specify your regex using TRANSFORMS- in props.conf, this is run at index time and has different properties than search time regexes (specifically, the output needs to be written to _meta with WRITE_META=true ). You can either specify it as REPORT- in props.conf, and refer to transforms.conf. It's even easier to specify inline in props.conf:
[c_netflowdata]
EXTRACT-foo = (?<thedate>\d+/\d+/\d+\s\d+:\d+:\d+),(?<src_ip>\d+\.\d+\.\d+\.\d+)
... View more
Your search syntax looks right, and the stats should calculate the sum of the duration field. You should note that your stats is calculating the total duration for the month, not for the day, so it's not unreasonable for it to be greater than 86,400s.
Another possibility is that vpn_user has become a multivalued field and certain sessions are counting against multiple users.
I'd pipe the output of transaction to stats sum(duration) count to see what the total duration and number of sessions is, as a sanity check.
... View more
The "(?=\d{4})" is a positive lookahead assertion. It's not strictly necessary, but just ensures that we're only looking for \s+Y immediately followed by four numbers. Your TIME_PREFIX would work equally well.
... View more
If you find yourself setting MAX_EVENTS to be very large, you should probably consider changing the LINE_BREAKER to break the input stream into larger chunks (that aren't single lines).
At index time, data is broken into lines (usually by \r\n or \n ). It then may be put back together using line merging. It's significantly more efficient to not have to reassemble the data with string concatenation. You then have to set the maximum "line" length using the TRUNCATE attribute. If you know that every event is demarcated by two newlines, you could set in props.conf:
[<sourcetype>]
LINE_BREAKER = ((\r?\n){2})
TRUNCATE = 1000000
SHOULD_LINEMERGE = false
... View more
Assuming that you can use a regex to determine which particular events are of interest to you, routing to the nullQueue is the best solution: http://answers.splunk.com/questions/96/how-do-i-exclude-some-events-from-being-indexed-by-splunk.
If the decision is on a file-by-file basis, whitelists and blacklists in inputs.conf is the best solution.
... View more
The following configuration should work in props.conf:
[<sourcetype>]
TIME_PREFIX = \s+Y(?=\d{4})
TIME_FORMAT = %YM%mD%dH%HN%MS%S
Using just (1), you will find the right location to look for the timestamp, but none of our default rules will apply. Using just (2), strptime parsing will try to match that pattern at the beginning of the line.
... View more
Subsearches are run with the same time range as the time picker selects (which are passed as an API parameter to the search). If you set a time range in the search string, say with earliest=... , it will not be used by the subsearch, only the API parameter.
... View more
_indextime is the time of arrival and is the primary sort order. we break ties by the place it was written. When we roll from one bucket to the next, the address will drop back to zero and increase from there. You may actually want to "| sort _indextime bucket address" to better handle the bucket roll.
... View more
Your sort here doesn't work because the _cd field looks like <bucketid>:<address> , so it's not sortable as a number or a string. You'll have to break it apart, say by ... | rex field=_cd "(?<bucket>\d+):(?<address>\d+)" . You can then | sort + _indextime address to get the data in arrival order. You can't use the bucket here since with multiple hot buckets possible, the bucket id doesn't help much.
Splunk has plenty of other internal fields. Most are used to make sure that enough information is preserved in the search to enable the UI to work as expected for events. Very few internal fields exist after a transforming command (one notable one is the _span for the result of the timeline command).
... View more
This is just a bug with the specification of the xmlkv command in commands.conf . It is implemented in python, in $SPLUNK_HOME/etc/apps/search/bin/xmlkv.py and its properties are described in $SPLUNK_HOME/etc/apps/search/default/commands.conf .
It is erroneously not declared as "streaming" which allows python commands to go past 50k events. You can workaround this by adding to $SPLUNK_HOME/etc/apps/search/local/commands.conf :
[xmlkv]
streaming = true
An alternate (and probably higher performing) solution would be to use a regex based extraction for fields like this, say in props.conf :
[<sourcetype>]
EXTRACT-mac = <MAC>(?<MAC>[^<]+)</MAC>
... View more
This means that the "host" field was not found in the table passed into the anomalies command. What does the rest of the search look like?
... View more
Yes, this is an undocumented feature of inputs.conf. Any DEST_KEY in transforms.conf can be set directly in inputs.conf. You are correct that the inputs.conf setting goes on the forwarder but fields.conf goes on the indexer.
... View more
The concurrency limit applies independently at each search head only. This means that any individual indexer could have as many concurrent jobs as the product of the per-search head limit and the number of search heads. However, this is unlikely to be a problem in most deployments since the load factor on each indexer is less than one per search.
... View more
Adding samples to ngram-models simply assists Splunk in guessing a CHARSET that we already support. It cannot be used to add support for a new charset. We have in product support for GB18030, GB231280 and GBK in addition to GB2312.
... View more
Events are persisted to disk in their original arrival order. Events are retrieved in inverse time order, with inverse arrival order used to break ties.
The "code" of the event, stored in the field _cd stores the pair (bucket id, arrival address) . You can sort by ascending arrival address to see events in arrival order.
... View more
If you have repeated "sub" values as in your example, you may want to consider a search like:
... | chart count by cat sub
This will make a table like:
cat 1 2 3
--- - - -
a . . .
b . . .
c . . .
This is useful if you want a stacked column chart to be rendered in the UI, so that you can see each combination as well as the total for each category.
... View more
If you want to hardcode the date here, you can just add | eval Date = "8/31/2010" but I suppose you want to be more clever and have it be based on the search itself.
In that case you should add:
... | addinfo | eval Date = strftime(info_min_time, "%m/%d/%Y") | fields - info_*
... View more
This means that the CLI thought that it should render raw results but was not given any. If you can share your search, I might be able to give some insight as to why (or file a bug). A quick workaround is to add "-output table" to your argument list.
... View more