I would have your match simply be anchored at the beginning and match anything that's not your delimiter character e.g.: ^(?<error>[^,]++) The problem with your existing regular expression, is that . matches any string and + matches greedily, so .+ consumes the entire string first, and then it checks for either a comma or the end of the string, because it's at the end of the string, must be a successful match (despite containing delimiters). a second option would be to use a lazy quantifier e.g. ^(?<error>.+?)(?:[,]|$) but of course that takes more steps of processing with these particular test cases. A regular expression debugger like: https://regex101.com can help you see the steps of processing your regular expression against test cases. ... View more

If you are on a new enough version of Splunk, there's now UI pieces that cover this use case exactly. Relevant Docs: http://docs.splunk.com/Documentation/Splunk/7.1.1/Knowledge/Resolveorphanedsearches#Bulk-reassign_multiple_knowledge_objects_to_another_owner If not, the REST API has a method of updating the ACLs of a knowledge object: https://docs.splunk.com/Documentation/Splunk/7.1.1/RESTUM/RESTusing#Access_Control_List Otherwise you're looking inside of metadata directories inside of apps and the system dir for default.meta and/or local.meta files and changing those then restarting: http://docs.splunk.com/Documentation/Splunk/7.1.1/Admin/defaultmetaconf ... View more

Through the REST API, there are earliest_time and latest_time parameters that the POST to the /search/jobs endpoint takes, these are the equivalent to your time picker selected values (and you could use these instead of embedding earliest and latest in your query string) http://docs.splunk.com/Documentation/Splunk/7.1.1/RESTREF/RESTsearch#search.2Fjobs For example using a quick dummy splunk and the makeresults command which also does not take earliest or latest, and the addinfo command which can add information like earliest and latest time to my search results.... $ curl -k -u admin:changeme https://localhost:8089/services/search/jobs -d search="makeresults | addinfo | convert ctime(*time*)" -d earliest_time=-h@h -d latest_time=@h -d exec_mode=oneshot <?xml version='1.0' encoding='UTF-8'?> <results preview='0'> <meta> <fieldOrder> <field>_time</field> <field>info_max_time</field> <field>info_min_time</field> <field>info_search_time</field> <field>info_sid</field> </fieldOrder> </meta> <result offset='0'> <field k='_time'> <value><text>2018-05-30T19:40:14.000-05:00</text></value> </field> <field k='info_max_time'> <value><text>05/30/2018 19:00:00.000</text></value> </field> <field k='info_min_time'> <value><text>05/30/2018 18:00:00.000</text></value> </field> <field k='info_search_time'> <value><text>05/30/2018 19:40:14.343</text></value> </field> <field k='info_sid'> <value><text>1527727214.99</text></value> </field> </result> </results> ... View more

While the HP/UX forwarder was deprecated with Splunk 7.x, there are various Splunk 6.x universal forwarder packages for HP-UX v11 depending on your platform. See: https://www.splunk.com/page/previous_releases/universalforwarder These will not be officially supported after the release of Splunk 8.x (whenever that happens), but they may still work. See Splunk's support policy: https://www.splunk.com/en_us/legal/splunk-software-support-policy.html You'll also want to keep an eye on the compatibility between Forwarders and Indexers as you upgrade your indexing tier, as certain features may not work with an older forwarder, but for raw events no problem: http://docs.splunk.com/Documentation/Forwarder/latest/Forwarder/Compatibilitybetweenforwardersandindexers As you mention an Oracle Database, DBConnect may also be an option, where you would have JDBC connectivity from a different host to the database, but it may not be the logs that you are looking for. ... View more

There are multiple ways of skinning this proverbial cat a lot depends on the scale of data you're dealing with and what you're doing with this data afterwards. A couple of options could use the stats command. <base search> | stats count by user would give you a single row per user, and the count of events where user has a particular value. if you don't care about count, you could get rid of it using fields: <base search> | stats count by user | fields - count Alternatively you could use the value stats function: <base search> | stats values(user) as users Which returns a single record, with multiple values, one per each unique user. Which could be expanded to multiple records with one value per record with the mvexpand command. <base search> | stats values(user) as user | mvexpand user Now stats gets rid of the actual events, a bit more costly execution wise, you could use dedup to keep a single event per user <base search> | dedup user Hopefully this gives you a number of options, one or more of which you may want to try... and this doesn't even get into chart and timechart for when you want that information not in isolation but against other variables... ... View more

Doing some tests myself, it seems that SA-ldapsearch doesn't properly handle attribute range retrieval with active directory. As @richgalloway suggests you should file a bug report for it. On the plus side, given that range=0-1499 is the first retrieval set, and that ldapsearch generates a JSON event, and it seems to actually retrieve the range attribute... you can use spath to extract the ranged field, and grab it: | ldapsearch search="(cn=ReallyBigGroup)" attrs="member" | spath | table "member;range=0-1499{}" Obviously it doesn't work for any other ranges: | ldapsearch search="(cn=ReallyBigGroup)" attrs="member;range=1500-*" | spath Yields: External search command 'ldapsearch' returned error code 1. Script output = "error_message=Invalid attribute types in attrs list: member;range=1500-* " (which now i'm looking through the python script to figure out why it can't parse the attribute range) EDIT: it doesn't help that it seems that the packaged version of the ldap3 library that's packaged is really old... version 0.9.5.3 which isn't a tagged version in github... 0.9.5.2 was released 4 years ago. The first error seems to come from SA-ldapsearch/bin/packages/app/init.py:437 but then if you edit the code to bypass that you run smack into an error from: SA-ldapsearch/bin/packages/ldap3/operation/search.py:315 This is of course looking at SA-ldapsearch v2.1.6 ... View more

Ok, so let's start that you are ingesting 300 GB/day. As Splunk compresses the raw data that it stores, what that page is saying is that an ingestion of 300 GB/day of logs could be stored on disk at a rate of 150 GB/day or so. (Note, that this compression ratio is a general estimate, and is highly dependent on the type of data you're ingesting as to how well it compresses, and how many terms Splunk pulls out into metadata). Now let's assume you have a single server, and you just want to store these logs (in Splunk) for 365 days, by default that would be 150GB/day * 365 days -> 53.5 TB of storage at the end of the first year, at which point the first logs will start being frozen based on time (by default - deleted), as new ones come in. If you're wanting to ramp up to that amount... the first month you'll need 150GB/day * 30 days worth of data -> 4.4 TB, during the second month you'll need a total of (150GB/day * 60 days worth of data) -> 8.8 TB total (as you're keeping the first month in addition to the second month) and so on. If you're not deleting data at the end of your Splunk retention, then you'll need to figure in the amount of time that you keep the data in frozen storage on disk. Splunk itself doesn't manage the lifecycle of data after it has been frozen. When you introduce features like Indexer Clustering, this gets more complex, as you now store multiple copies of the raw data (replication factor or RF - estimated at 15% of the raw size) and the search metadata (search factor or SF - estimated as 35% of the raw size) across multiple servers, providing you with nice safety guarantees... Let's say you have a cluster with SF=3 and RF=3 and keeping the same amount of data... your 53.5 TB has now turned into 160.4 TB in total disk space... If you have a cluster of 7 indexers this means around 22.9 TB per indexer for the data storage alone. We can add other features to your environment like Data Model Acceleration, Report Acceleration, and Summary Indexing where Splunk is automatically or on a schedule running searches and gathering statistics about your data so that your searches can leverage those statistics and perform faster/better/for longer time periods on demand at the cost of execution time in off hours, as well as some additional disk space to store the summary data. There's also a possibility of reducing space required on disk through TSIDX Reduction, where after a certain age Splunk throws away portions of the search metadata, saving disk, at the cost of possibly requiring rebuilding the same metadata during a search whose time period crosses a particular threshold. If you want to play with some of the basic storage options, there's a tool available at https://splunk-sizing.appspot.com/ But I don't know of any good way to estimate storage requirements for leveraging any of these other features that I've mentioned. ... View more

I'm guessing that this props.conf isn't present on your forwarders as well and that's causing you issues. The thing to note here is the line: INDEXED_EXTRACTIONS = csv This means that your linebreaking and extractions of fields actually happens on your universal forwarder (where the header row of your CSV file is located) and not your indexers... see the diagram on (especially the structured parsing pipeline): https://wiki.splunk.com/Community:HowIndexingWorks Being a custom sourcetype, your forwarders need to know about it otherwise they won't perform the index time extractions you're expecting that they should be doing. There are some other side effects to this as well such as what is mentioned in this Splunk Reaction. ... View more