They are not "secret". I raised my values. Anyone can raise their actual values 😉 Bacause they are strictly based on your personal infrastructure. ... View more

I raised the values for limits on all SHs nodes, [search] base_max_searches = xx max_searches_per_cpu = xx [scheduler] max_searches_perc = xx ... after restarting, when cluster goes online, dispatching seems to have more efficency. Monitoring the system. ... View more

Thanks. So, the problem is "phisiological"... no workaround possible. Tried to optimize the system, already, talking about making Dashboards, by default, to take less range times searchers and make auto refreshes at least at 5m. We have 12 vCpus per node (Indexers 4*12), already. While SHs run with 6 vCpus per node. We take care about this... thanks. ... View more

Ok. Taken from documentation, https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf ... i missed this directive... need to try. Event Log filtering Filtering at the input layer is desirable to reduce the total processing load in network transfer and computation on the Splunk platform nodes that acquire and processing Event Log data. whitelist = <list of eventIDs> | key=regex [key=regex] blacklist = <list of eventIDs> | key=regex [key=regex] whitelist1 = <list of eventIDs> | key=regex [key=regex] whitelist2 = <list of eventIDs> | key=regex [key=regex] whitelist3 = <list of eventIDs> | key=regex [key=regex] whitelist4 = <list of eventIDs> | key=regex [key=regex] whitelist5 = <list of eventIDs> | key=regex [key=regex] whitelist6 = <list of eventIDs> | key=regex [key=regex] whitelist7 = <list of eventIDs> | key=regex [key=regex] whitelist8 = <list of eventIDs> | key=regex [key=regex] whitelist9 = <list of eventIDs> | key=regex [key=regex] blacklist1 = <list of eventIDs> | key=regex [key=regex] blacklist2 = <list of eventIDs> | key=regex [key=regex] blacklist3 = <list of eventIDs> | key=regex [key=regex] blacklist4 = <list of eventIDs> | key=regex [key=regex] blacklist5 = <list of eventIDs> | key=regex [key=regex] blacklist6 = <list of eventIDs> | key=regex [key=regex] blacklist7 = <list of eventIDs> | key=regex [key=regex] blacklist8 = <list of eventIDs> | key=regex [key=regex] blacklist9 = <list of eventIDs> | key=regex [key=regex] ... clear! Need to try, later. Thanks. ps. this is documented under "Windows Event Log Monitor" however... not sure will work in a normal log file... i'll try. ... View more

Not sure having understood. Can you do an exact example with real inputs as above? You're saying these inputs will work? "whitelistX=" directive in stanzas works? addon: etc/apps/addon1/default/inputs.conf [monitor:///tmp/] whitelist1=.*\.log$|.*\.txt$ index=blabla sourcetype=blabla addon: etc/apps/addon2/default/inputs.conf [monitor:///tmp/] whitelist2=.*\.json$|.*\.dat$ index=blabla sourcetype=blabla ... View more

Hi. I took out this thread for an addition... for a problem i found in my infrastructure... The Infrastructure UFs -- [HF] -- IDX The problem ... do to firewall problems, all UFs have outputs to point to both HF and IDX, at the same time, in default stanza... some hosts join IDX directly (since fw blocks HF flow), some join HF only (same as before), some join both... i found that when some inputs go to IDX direcly i got my props (IDX only) parsed right, when passing through HF (no props), parsing is broken (HF-->IDX, HF parses source by default and passes wrong events to IDX which do not elaborate/parse props as well). The solution Deploying same props both to IDX, as well, and also to HF, by giving HF same DS as IDX has. Ok. The workaround (if possible, i don't think can do it) Bypass parsing (props) in HF and forwarding datas as a common UF to IDX. Is there a conf to match this behaviour? Thanks. ... View more

Thanks again. ... View more

Anyway, yes, this props did it, TIME_FORMAT=[%d/%m/%Y %T] SHOULD_LINEMERGE=false ... View more

Ok, thanks. Is right the TIME_FORMAT = %m/%d/%Y %H:%M:%S you wrote up? Shouldn't be TIME_FORMAT = %d/%m/%Y %H:%M:%S ? date +'%d/%m/%Y %H:%M:%S' 05/11/2019 13:10:24 Right? ... View more

Thanks. As i wrote in main post, i'll (eventually) write the correct props in Indexer(s) asap. The question now is: why i have many others logs, from other forwarders, logging like 2019/05/11 without their sourcetype stanza in props, and are still logging Europe since 01/11? Also, on same host, same forwarder, others logs are logged randomly 05/11 and 11/05, today. Example, 3 events are 05/11, other 3 are 11/05! Strange behaviour..... Thanks, anyway. ... View more

Found the problem. Inputlookup passes variables to the map with double-quotes,so a single host is get, a list in IN clausole not. Ex. host1,host2 become, literally, "host1,host2" so IN("host1,host2") is not parsed good, host1 IN(host1) in parsed good, without double quotes, but also making quoted works with single host. Null-string is passed as "". Trying to remove the quotes. Another workaround, bad, but working,is something like adding single host fields in csv, h1,h2,h3,h4 and (host=$h1$ OR host=$h2$ OR host=$h3$ OR host=$h4$) in the map search. Very bad, but it works. ... View more

Thanx a lot. But i have built this little "Engine" for the "map" command; in reality the csv has many many more fields (earliest, latest, thresholds, pattern to find etc.....). MAP command works perfecly, only the IN clausole does not when i have more than 1 host!!!!!!! 😐 .....also, the "table host,source" up is only an example, in the real searches i have more than 10 piped commands. It works, but the "host IN" with multiple hosts NOT 😐 😐 😐 With the subsearch i need to change many things. Also, the subsearch up, i think, should be ended with a "format" command and also hostnames renamed in "host" to work fine. I prefer to fix the "map", for now. Thanks a lot. ps. i fixed as workaround the "IN problem" with a tag/eventtype in front who make me permit to search what i need |map maxsearches=50 search="search tag=MYTAG source=$source$|table host,source" ....tag/eventtype contain index and hosts i need.................. i want to bypass also tag. ... View more

Resolved also the "props path problem", adjusting the "metadata/local.meta" of the TA, setting global. ... View more

Another strange behaviour. This is the perfect/right props.conf to index the json, also more complicated (1000 records with multiple arrays), [my_json] pulldown_type = true INDEXED_EXTRACTIONS = json KV_MODE = none category = Structured TRUNCATE=0 JSON_TRIM_BRACES_IN_ARRAY_NAMES=true NOW, on my test-Environment (single istance idx,sh all), 1) if i let props.conf stay in etc/apps/MY_TA/default/ , fields are duplicated, no truncate (seems the TA works as a simple Forwarder) 2) if i move the props.conf in etc/system/local, json is perfectly indexed (now seems the props is used as Indexer) ... ... View more

The only problem is now with much complex json format, like {} formatted, { "data": [ { "displayName": "First Name", "rank": 1, "value": "VALUE" }, { "displayName": "Last Name", "rank": 2, "value": "VALUE" }, { "displayName": "Position", "rank": 3, "value": "VALUE" }, { "displayName": "Company Name", "rank": 4, "value": "VALUE" }, { "displayName": "Country", "rank": 5, "value": "VALUE" } ] } This is not indexed with multiple events, but single events and multivalue fields... ... View more