That works fine for me. You can use makeresults and eval to fake results to try things out on. For example: | makeresults | eval timeStr="2018-08-30T07:47:16.127+01:00" | eval n=strptime(timeStr, "%Y-%m-%dT%H:%M:%S.%3N%:z") Are you sure your timeStr is being extracted correctly? ... View more

It's not quite defined.... import splunklib.six.moves.http_client makes splunklib.six.moves.http_client available but that's very different than six.moves.http_client (which is missing the leading splunklib. part) As it seems they've been fairly consistent here in their invocation, I would alter that import statement like so: import splunklib.six as six Thus making six available in as a name, and thus everything resolves properly ... View more

You should try it out yourself! The beautiful thing about makeresults and eval is that you can use them to make synthetic results to try things out on: | makeresults | eval fieldX="12345",fieldY="A" | eval fieldX-{fieldY} = fieldX ... View more

So I decided to play with taking those timestamps and uploading it to my local instance through the Add Data page Something that was interesting to me, with these dates on my laptop's 7.1.2 instance, it seems like Splunk was getting tripped up by the leading square bracket and or single digit date... which I was able to fix in two different ways... either by specifying a TIME_PREFIX = \[ or starting your TIME_FORMAT with [ . If you specify TIME_PREFIX , you could also reduce the search space Splunk uses to identify a timestamp by setting MAX_TIMESTAMP_LOOKAHEAD down to around 26 or 28. If the timestamp is the first thing in your event, then you could even anchor it with TIME_PREFIX = ^\[ As @cpetterborg mentions since your data has timezones, you really should be including %Z in your format... and I would also encourage use of the more familiar %d and %H over the %e and %k equivalences (but the bracket fix it seems to work either way). Additionally, if you can guarantee only single line events, I'd encourage making use of SHOULD_LINEMERGE = false instead of BREAK_ONLY_BEFORE = regex or the default BREAK_ONLY_BEFORE_DATE = true . As you have what seems to be an event-breaking regex, you may even consider altering LINE_BREAKER to have only single "line" events, but that's assuming that this timestamp is the first thing after a linebreak in your file. (if you go that route you might also need to be aware that TRUNCATE has a maximum default 10000 bytes per "line" ... View more

This doesn't read like you have a question, but I'll give you an answer... If you have a need for such a feature you should submit a P4 (Enhancement Request) ticket on your support entitlement, and/or talk to your account / partner teams about it. In order for anything to have a chance of being worked on by the development teams, it needs to get to Splunk's JIRA queues. While some employees may troll Splunk Answers, you have a better chance of getting it into their JIRA to be prioritized if it's logged through proper channels. See also: https://answers.splunk.com/answers/4844/how-can-i-submit-an-enhancement-request.html ... View more

Splunk may not be the correct tool for your use case. First of all fschange monitors have been deprecated since Splunk 5 and could be removed at any time. Second: Splunk is more about recording events, extracting information and correlating them. If you had something producing events into Splunk (like the fschange monitor) and you had a scheduled search on your search head, you could kick off custom alert action to execute your script from the search head, but that may not be what you're looking to do. I am not as familiar as I should be with all the ins and outs of Phantom yet, however based on signals, they too can invoke playbook actions to automate tasks, but I'm not exactly sure of the mechanics there. I suspect however, if you have access to the NFS server, you may be looking for an inotify based tool as have been suggested on this stack overflow question: https://stackoverflow.com/q/14692353/504685 But also if you don't have that sort of access to the NFS server you may run into issues, and are likely looking for a different solution: https://stackoverflow.com/a/4231277/504685 ... View more

Please read my response more thoroughly, make an attempt at building the query on your own, and ask specific questions in a manner that shows you are attempting to understand the information already given to you. I already gave you not only the methodology for working with data that had those three fields per event, but also links straight to Splunk's documentation on all the commands and functions you will likely have to use. The people who volunteer their time to on this site are here out of their own personal passion to help others learn about Splunk and learn to help use Splunk better. Right now your comment reads as if you are asking me to do your work for you, which is disrespectful of the time I donated to try to help you learn to solve your problem and future problems like it on your own. If you are not interested in learning but rather just want someone do your work for you, Splunk and many Splunk Partners offer for hire a wide variety of professional services, from Admin on Demand credits, to full blown engagements. ... View more

So assuming that all three of those are in the same event, your first step would be to use the eval command to convert the timestamps to epoch time using the strftime() function. You can then take a difference between them to get the duration between the request and the response in seconds. Then you could use stats command to calculate avg(duration) by app. If however the request time and response time per app appears in different events, then you are likely looking at finagling with the transaction command to stitch the pairs of events together by a request identifier. Assuming your data is being parsed properly and Request time and response time are just _time, transaction would calculate duration for you. Finally stats to get the average by app again. That's the overall gist, but of course a lot depends on the structure of your data and the field names you have extracted for the details. (arguably the time between the request and the response is the processing time, as opposed to the time the client started the request to the time the client finished receiving the response.. but again... details) ... View more

In moving this post, a comment providing more details from @psmp was lost: Thanks for the reply. index=powershell sourcetype=sw_list source="powershell://cw_list" host=prod* | stats list(host) This Query Yields 5 hosts. Host 1 Host2 Host3 Host4 Host5 index=powershell sourcetype=sw_list source="powershell://cw_list" host=prod* packagename="Mysoftware" | stats count(host) by packagename packageversion host packageName packageVersion Host1 Mysoftware 1.0 Host3 Mysoftware 1.1 Host5 Mysoftware 1.0 I am looking for a Query that Yields below report: PackageName PackageVersion Total_hosts_with_pkg Total_hosts_without ListOfHostMissingPack Mysoftware 1.0 2 2 Host2 Mysoftware 1.1 1 Host4 ... View more

On my phone right now so I have neither a Splunk nor RDBMS instance to play with off hand so the syntax may be slightly off, and depending on your time field in the database, you may need to do some conversion... but I would use a subsearch with addinfo to build the SQL statement including the time based where clauses, and then insert that query into the parent dbx query... e.g. |dbxquery connection="HealthMon" [makeresults | addinfo | eval query="SELECT * FROM x WHERE x.time >=".info_min_time." AND x.time <=".info_max_time | return query] ... View more

"Best" is a function of details of your environment... Are you on Windows or Linux. If on Linux are your existing cold volume on an LVM powered lv? By "new set of storage" do you mean entirely new indexers, or new disk presented to existing indexers? Or if just existing indexers do you have any spare hardware or is it all provisioned? Is new disk SAN or physically presented? Are your existing index definitions defined in terms of volumes in indexes.conf or static paths? How much data are you needing to move (approximately, per indexer). All of these things would feed into a number of different options that could be used to plan a successful migration, and set expectations with your users. On the plus side with an indexer cluster this is a very doable thing either by yourself or with the help of Professional Services, just a lot of details to iron out to make "best". ... View more

So it's an HTTP POST already... if you can't insert a token in from the source system, then setup a level 7 proxy/load balancer (F5 LTM, or Apache mod_proxy, or HAProxy, or others), to front your HTTP Event Collectors and insert the Authorization header with token as it's passed back to your HEC(s) If you need more manipulation... you can develop an App server that does the transformation and pass it back... ... View more

I have not dealt with the ThreatConnect app, but their User Guide including setup instructions is available through their site: https://kb.threatconnect.com/customer/portal/articles/2146321--threatconnect-app-for-splunk-enterprise-users-guide I suspect that the python script in question designed to be launched by Splunk / using the built in python that is shipped with Splunk, and is not supposed to be launched using your system python installation. I reach that conclusion through two older answers posts: First this answer discussing how launching a script directly from the command line is not the same as launching it through Splunk : https://answers.splunk.com/answering/417389/view.html And then this answer about invoking the python interpreter through Splunk to find out about splunk.Intersplunk from a Splunk installation: https://answers.splunk.com/answering/7910/view.html ... View more

It looks like you're using an HTTP POST to send JSON with the wrong Content-Type (it should be application/json if you were sending application/x-www-form-urlencoded the body would look like device=sx20&state=Registered ) to the TCP port, and the 10 lines that you want to get rid of are HTTP headers. Why not setup and leverage the HTTP Event Collector instead? That way it properly handles the HTTP request, and you get your message indexed without them. You may need to do some manipulation either with the source python code, or possibly with the Gateway, but it seems like a good option to me. Check out the Developer Site on the HTTP Event Collector: http://dev.splunk.com/view/event-collector/SP-CAAAE6M ... View more

Unfortunately there's not really a good answer here if you want host based reporting of license usage when you hit the squash_threshold. You can increase the squash_threshold in your indexers` server.conf files, but the tradeoff is of course that you're retaining more tuples of information during periodic reporting and therefore more overhead on your indexers before they dump their data to the license master. (There was a bug where this setting didn't work for a some versions: https://answers.splunk.com/answers/403171/higher-value-of-squash-threshold-no-longer-making.html but that should be resolved now ) Even if you hit the squash_threshold, your license usage by sourcetype, index, and indexer will be accurate. Only source and host are dropped from the reporting dimensions. (Depending on your index / indexer layout, this may be sufficient for your needs). Searching for per_X_thruput is of course rather interesting... as it'll report the top 10 series per reporting period, of course host=HOSTHERE means you're getting the thruput off of your forwarder (it'll include any data dropped on the indexer), but host=INDEXER* series=HOSTHERE will only be included if HOSTHERE is in the top 10 hosts for that reporting period... and both of these include _internal logs which do not count toward your license. A brute force method is of course: index=* host=HOSTHERE sourcetype!=stash | eval size=len(_raw) | stats sum(size) by sourcetype index This query kinda sucks when you have a lot of events, and it's making 2 assumptions 1) is that your event timestamp _time is being parsed properly and is close enough to your _indextime (as license usage is done based on _indextime) otherwise you need to search all time (which is awful for sizable indexes) and use _index_earliest and _index_latest to constrain your index time... 2) That all your events are using characters that are represented by single bytes in UTF-8. len gives you the number of characters in a string not bytes, so you wind up with a lower bound of license usage by this method. ... View more

From the indexes.conf spec file: Note that this it will act only on those indexes which reference this volume, not on the total size of the path set in the path attribute of this volume. Therefore you need to configure your indexes to reference the volume and not just put the indexes to happen to reside in the same path. (You're not showing this configuration, so maybe you are already, but you ask if you need per index configuration so I figured better to not assume) Secondly, you have two volumes referencing the same path, assuming your indexes are utilizing volume definitions, that means that each part is using its own quota. So 5400000 MB for summaries and 5400000 MB for hot, which would be twice the expected quota. Third 5400000 MB would 5,529,600,000 1K blocks... That's more than the 5,272,422,384 used that your df output states... If your target is 85% of your disk that would be 5105645 MB... (5100000 for a nice round number). Splunk tends to operate in scales of 1024 not 1000. ... View more

You're correct in that +? is the lazy 1 or more quantifier that I was talking about and is enough for the problem at hand... The (?:x|y) syntax denotes a non-capturing group with two options x and y. Since you don't care to capture either the comma nor the end of line character, by habit, I typically mark such as non-capturing since there's no need for your regex processor to spend time keeping the value of that group around. There are some other group syntaxes (atomic groups for example) that have other characteristics in terms of what they do vs performance ... View more

All previous releases of Splunk Enterprise that are available for download by platform can be found: https://www.splunk.com/page/previous_releases (You will likely have to log in with your splunk.com credentials to download) ... View more

Move your where clause to before your table clause. You are correct that table is acting as a filter to eliminate fields, much like where is acting as a filter to eliminate rows. The problem is that your where clause is using fields that were eliminated by your table clause... so either, you need to include those fields in your table, or you move your where clause to be earlier in your search. ... View more