Mus, If I understand correctly you are interested in conditional Role-Based access depending on physical location. This could certainly be satisfied by introducing multiple search heads and using a combination of NAC and Splunk's RBAC features. Search Head 1: High Security Environment Using NAC this search head would only be network accessible from "High Security Environments" and using Splunk's RBAC would allow a user to see all his/her classified Star fleet messages. Search Head 2: Bridge Using NAC this search head would only be network accessible from "the Bridge" and using Splunk's RBAC would allow a user to see all his/her non-classified messages. ... View more

You are correct in that real-time searches grab the data before it hits the index queue, however real-time searches do have access to search time field extractions which happen in the parsing queue. Can you successfully search for "error=ANR1234E" via non RT search. This would rule out the field extraction as the culprit? ... View more

msarro, When writing regular expressions for TRANSFORMS statements, you don't necessarily need to capture anything. Simply write a regular expression that identifies things you want to nullQueue. For instance if you wanted to nullQueue whenever there is a '3' in column 3 you would specify: ## props.conf [<spec>] TRANSFORMS-nullQueue_for_spec = nullQueue_for_spec ## transforms.conf [nullQueue_for_spec] ## Here we use * to specify 0 or more REGEX = [^,]*,[^,]*,3, DEST_KEY = queue FORMAT = nullQueue We can add field extraction to events that make it past the nullQueue by adding the following props/transforms: ## props.conf [<spec>] KV_MODE = none REPORT-kv_for_spec = kv_for_spec ## transforms.conf [kv_for_spec] DELIMS = "," FIELDS = column1,column2,column3... If you really don't want certain columns extracted, instead of specifying FIELDS/DELIMS in your REPORT-kv_for_spec property use a similar regular expression to the nullQueue with capture groups around the columns you want to pull out. The point is separate the nullQueue process from the field extraction process. Update, If you were interested in capturing all events, but only certain columns try: ## props.conf [<spec>] TRANSFORMS-make_raw_for_spec = make_raw_for_spec ## transforms.conf [make_raw_for_spec] DEST_KEY = _raw REGEX = ([^,]*),([^,]*),([^,]*), ## Drop column 2 FORMAT = $1,$3 ... View more

remy, Data will definitely not be indexed while the indexer is off. It is the responsibility of the forwarder to cache the data locally or maintain state (in the case of Splunk "monitor://" file monitors). I am confused by the mention of "using forwarders and udp to send data". Are you using forwarders to collect UDP and forward to the indexer? Are the indexers collecting UDP directly? In general if you are using Splunk forwarders to collect data via "monitor://" inputs, Splunk, in the event the forwarder cannot contact the indexer, will maintain state and pick up where it left off. Other data (i.e. TCP, UDP, or scripted inputs) is maintained by the forwarder based on several output settings. To name a few: maxQueueSize = [<integer>|<integer>[KB|MB|GB]] dropEventsOnQueueFull = <integer> useACK = [true|false] If you are indexing data directly (on the indexer) via a "udp://" inputs statement, data will be dropped if splunkd is stopped. Finally, most of these issues are mitigated with the introduction of a mult-indexer environment which allows forwarders to auto load balance. I would also not recommend indexing UDP data directly using Splunk. Instead data could be collected via syslogd and indexed via "monitor://" or collected via a Splunk forwarder and indexed via "splunktcp://" . ... View more

The UI will create a stanza which did not work in my testing. ## props.conf [*] LOOKUP-all_test = my_test_lookup sourcetype OUTPUTNEW foo If you want this lookup to be global I would recommend specifying this property in props.conf without a stanza: ## props.conf LOOKUP-all_test = my_test_lookup sourcetype OUTPUTNEW foo Based on a similar Q/A it is also possible use wildcards in sourcetype for props.conf if you don't want to make the lookup global: wildcards in sourcetypes.conf ... View more

rgcox1, We use the following configurations for WindowsUpdateLog to create the following fields: package (multi-value KB #) package_title (mult-value package title) status (available, installed, restart required, failed) This will allow you to search for the most recent status of an update based on "stats first(status)" . I would recommend using a lookup (i.e. update_tracker.csv) to track this information over time: sourcetype=WindowsUpdateLog | stats first(status) as status,min(_time) as firstTime,max(_time) as lastTime by host,package,package_title ## Copyright (C) 2005-2011 Splunk Inc. All Rights Reserved. ## props.conf [WindowsUpdateLog] FIELDALIAS-dest_for_windowsupdatelog = host as dest REPORT-0package_message_for_windowsupdatelog = package_message_for_windowsupdatelog REPORT-1package_title_for_windowsupdatelog = package_title_for_windowsupdatelog,package_title_for_windowsupdatelog_restartrequired,package_title_for_windowsupdatelog_package_message REPORT-package_for_windowsupdatelog = package_for_windowsupdatelog REPORT-pid-tid-component_for_windowsupdatelog = pid-tid-component_for_windowsupdatelog LOOKUP-status_for_windowsupdatelog = windows_update_status_lookup vendor_status OUTPUTNEW status ## transforms.conf [windows_update_status_lookup] filename = windows_update_statii.csv [package_message_for_windowsupdatelog] REGEX = (Content\s+Install\s+((Restart\s+Required)|(Installation\s+Ready)).*) FORMAT = package_message::"$1" vendor_status::"$2" [package_title_for_windowsupdatelog] REGEX = Content\s+Install\s+(Installation\s+(Successful|Failure)):\s+Windows.*the\s+following\s+update.*?:\s+(.*) FORMAT = vendor_status::"$1" package_title::"$3" [package_title_for_windowsupdatelog_restartrequired] REGEX = Content\s+Install\s+(Installation\s+successful\s+and\s+restart\s+required)\s+for\s+the\s+following\s+update:\s+(.*) FORMAT = vendor_status::"$1" package_title::"$2" [package_title_for_windowsupdatelog_package_message] SOURCE_KEY = package_message REGEX = \-\s+([^)]+\)(\,\s+\d+\-[bB]it\s+Edition)?) FORMAT = package_title::"$1" MV_ADD = True [package_for_windowsupdatelog] SOURCE_KEY = package_title REGEX = (KB\d+) FORMAT = package::$1 MV_ADD = True [pid-tid-component_for_windowsupdatelog] REGEX = ^\S+\s+\S+\s+(\S+)\s+(\S+)\s+(\S+) FORMAT = pid::$1 tid::$2 component::$3 ## windows_update_statii.csv vendor_status,status "Installation Ready",available "Installation Successful",installed "Restart Required","restart required" "Installation successful and restart required","restart required" "Installation Failure",failure ... View more

I would recommend adjusting earliestTime values to create a real-time time window. This helps Splunk collect real-time events with drift between indextime and _time . Try: <earliestTime>rt-5m</earliestTime> <latestTime>rt</latestTime> Update: Glad this worked. If you would like to always have the latest event consider adding the "head" command. <your search> | head 1 ... View more