look like 1 of the 3 lookup pieces is missing read permissions. ... View more

Not like that. license pools are used to segregate indexers volume. Eventually, with a heavy forwarder routing some data to specific indexers, and attach the indexers to a specific license pool, you could achieve that. But this is a bit of a burden to maintain, maybe not worth it. ... View more

rename the lookup field "Account" to the search field name "act", then use the lookup in a subsearch, (it will convert to condition like A=1 OR A=2 OS A=3 etc...) Index=myIndex [ | inputlookup mylookup.csv | eval act=Account | table act ] see https://docs.splunk.com/Documentation/Splunk/latest/SearchTutorial/Useasubsearch ... View more

We expect to use the collectd setup script, that will send data over HEC to the indexers (and skip the UF) the SAI app does not use UDP Setup: collectd -> localhost udp port 5000 -> indexer (via system/local/outputs.conf) If you send data over UDP, the format may not be recognized, as many transformations are done for the em_metrics sourcetypes. ... View more

what about an OR condition ? FileName=Test.json OR FileName=Test.json.pgp Or if you have 2 searches and try to JOIN them, normalize the fields name or content, then use a join search1withoutpgp | eval FileName=FileName.".pgp" | join FileName [ search search2withpgp | table myotherfield FileName] ... View more

Understood, if you have a support contract, you can still open an enhancement request case on the product, to request outputlookup options. ... View more

second remark , do not use a table and dedup in that order, it's expensive as the table can only happen on the sh, rely on the map reduce capabilities of splunk to get the work done on the indexers. |table host Domaine |dedup host try |dedup host | table host Domaine ... View more

if your lookup file is a csv "mycsvlookup.csv" or "mycsvlookup", with a single column "host" You should be able to call it with | inputlookup mycsvlookup now try index="dc_winaudit" [ | inputlookup mycsvlookup | table host ] or maybe index="dc_winaudit" [ search | inputlookup mycsvlookup | table host ] ... View more

To have a UF forward to syslog, check https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Forwarddatatothird-partysystemsd Keep in mind that if a forwarder send to a splunk indexer AND to a syslog server. It will stop sending to both if one of them is not responding, not great for HA. ... View more

Why not add a fake event at the end of each search, to never return an empty table ? search .... | append [ | stats count | eval fieldA="null", fieldB="null" | fields - count ] | table fieldA fieldB ... View more

You see a delay on the windows events, from forwarders. Can you also check the internal logs of those forwarders, it will tell you if the problem is on the forwarding or now. (if internal logs are also delayed, then you have a bottleneck after the forwarder) If only the Windows events are often delayed at the collection level, because the forwarder may be waiting for the AD to do the objects names resolution. ... View more

The problem may be the higher volume of windows events to read during the business hours. The modular inputs doing the collection may be hammering the windows API or waiting for it to respond. see https://docs.splunk.com/Documentation/Splunk/latest/Data/MonitorWindowseventlogData Try to reduce the collection by adding whitelist and blacklist on the forwarder inputs.conf. Maybe some verbose eventcodes are not useful to collect for you and may reduce the volume. see if your input is not waiting for the AD server to resolve the objects names. check if you need to disable evt_resolve_ad_obj, or ensure that you are querying the closest/fastest AD evt_dc_name by default, the forwarder may be querying a remote busy AD. ... View more

2 approaches define your event breaking rules at indextime for your sourcetype to break that xml into several small events. but then all the other fields will be disconnected from the events. https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureeventlinebreaking or index at it is, and at search time, try to use the | spath command to parse the xml and get the fields stored in a multivalue field structure. https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Xpath ... View more

No, the search restrictions are controlled by their search-head, so they can allow access to what they want. If you were the admin of their SH, you could enforce role permissions and restrictions. ... View more

it will find events index=os sourcetype=splunk_health_check calculate some additional fields, value, message, status and score, drop other fields, and add the search-head name |eval value=delay_status |eval message=if(status=="OK","",status) |eval status=if(status=="OK",status,"NG") |eval score=if(value==99,-1,value) |table _time sh status score message |rename sh to searchhead and add a condition to drop all the events that do not meet the condition on score ,status and message |search score<5 AND status!="OK" message!="*No error" AND message!="Proxy Error" ... View more

Ivan, please post your method as an answer instead of a comment. It will help mark the question as answered. ... View more

If you are splunkcloud customer, you should have a support entitlement contract, and be able to reach cloud support directly for those questions. Try the splunk.com support portal page to open a case. ... View more

food for thoughts Have you thought monitoring the rotated file too ? Splunk had a logic to avoid reindexing the file Sometimes it helps if the last events of the file were not indexed before the file rotated. if the problem is during the rotation process, maybe disable the monitoring before the rotation and re-enable after. ... View more

Look at sub searches to add search conditions to the main search. index=AAAA [ |inputlookup mylookup.csv | table host ] will turn into index=AAAA ( host=1 OR host=2 OR host=3 ) ... View more

Can you make a more generic search, that could handle them all at once, instead of using appends? It would be more efficient and faster. Or on recent spunk, if you need to append, take a look at the alternate command union. instead of "| append" try " | union " to run sub searches in parallel instead of serialized. search 1 | union [ search 2 ] https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Union ... View more

If you have automatic lookups, then you should have a way to edit the permissions on it to only apply to certain roles. That way the user that is not a member should not try and fail and trigger an error. An automatic lookup is composed of : lookup table, lookup definition, automatic lookup definition check the permissions of all those 2 objects. ... View more

on more recent splunk please prefer modular inputs instead of scripts : https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ModInputsIntro ... View more