How many directories needs to be scanned by the UF to reach those two files? Also can you try truss the process and see what it actually does? ... View more

Try accessing this REST endpoint on your UF https://localhost:8089/services/admin/inputstatus/TailingProcessor:FileStatus to see how may files are being monitored. High numbers of monitored files can cause such behaviour ... ... View more

Hi thsvinayb4u, you are looking for custom alert action; find the docs here http://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ModAlertsIntro All you need to know about passing tokens to your alert script can be found here http://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ModAlertsLog Hope this helps to get you started ... cheers, MuS ... View more

It fails for me when trying inline.... ... View more

This is what @somesoni2 is referring to MAX_TIMESTAMP_LOOKAHEAD=32 this is set too low and therefore Splunk never reaches the correct time stamp. ... View more

Hi HattrickNZ, you can hide the real-time presets using this css: /* hide real-time accordion */ div[id^='realtime_view'] { display: none; } /* hide real-time presets */ div[id^='presets_view'] > div.accordion-body > div > ul:nth-child(1) { display:none; } /* hide divider after real-time presets */ div[id^='presets_view'] > div.accordion-body > div > div:nth-child(2) { display:none; } Put it in your app directory in app server/static and use it in the dash board with stylesheet="yourcssfilenamehere.css" , depending on your Splunk version you can either restart Splunkweb or _bump it by accessing this URL http[s]?://<SPLUNK_IP_ADDRESS>:8000/en-GB/_bump . After that the real-time presets are hidden: But one can still used them with the advanced options in the time range picker, so you additionally need to remove the rtsearch capabilities from the user role to make sure user don't run a manual real-time search. Hope this helps ... cheers, MuS ... View more

Hi dadepu, there is an App already on Splunkbase https://splunkbase.splunk.com/app/1249/ 😉 cheers, MuS ... View more

Well, the above settings are the Splunk default settings so they really should work. ... View more

Hi there, here is a hint how it could be done https://answers.splunk.com/answers/176466/how-to-use-eval-if-there-is-no-result-from-the-bas-1.html or read this to replace or this one https://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-join-append-or-use-of-subsearches.html to replace the join with a stats 😉 cheers, MuS ... View more

If you are brave enough to use tmpfs for your hot storage, then yes 😉 Maybe this claim was related to the storage system has some internal cache which keeps open files in its internal memory before writing to disk? ¯\_(ツ)_/¯ ... View more

Did you mean inputs.conf http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf ? props.conf only has initCrcLength but anything else is configurable in inputs.conf cheers, MuS ... View more

Hi walkerhound, One thing I spot is that the docs mentions this value for the metric_name FORMAT to be FORMAT = metric_name::graphite.$1 maybe try to use FORMAT = metric_name::channel1.$1 in your case. For the _value, try this regex Value=\"([^\"]+)\" or if you really have literary Value="1234" in your logs try this: [test_value] REGEX = (.+) SOURCE_KEY = Value FORMAT = _value::$1 WRITE_META = true Hope this helps ... cheers, MuS ... View more

Hi kutsyy, if I got your question right, you can get the expect result by adding a chart to your mstats search like this: | mstats avg(_value) AS avg_v max(_value) AS max_v min(_value) AS min_v WHERE index="meh-tricks" AND source="http:collectd" AND metric_name=cpu.percent* span=1m by host | eval idle=if(metric_name=="cpu.percent.idle.value", max_v, null()), user=if(metric_name=="cpu.percent.user.value", max_v, null()) | chart values(idle) AS idle values(user) AS user over _time by host This will get you a chart view with column contains the values for different metric, like the screenshot below: Please adjust to your needs, because I'm pretty sure you will not have an index called meh-tricks 😉 Hope this helps ... cheers, MuS ... View more

So, looking at all the examples and after having a night rest over it, I gave it another try and came up with a bit a different approach which builds a dynamic list of field names and uses a threshold to match on the value. Here is the run everywhere example: index=_internal date_minute=* | stats count by date_minute | eval header{date_minute}=date_minute | where [ search index=_internal date_minute=* | stats count by date_minute | eval header{date_minute}=date_minute | eval threshold=15, connector=" OR " | foreach header* [ eval total=if(isnull(total), "<<FIELD>>" .">=". threshold, total ."". connector ." ". "<<FIELD>>" .">=". threshold) ] | table total | rename total AS search ] | stats count by date_minute The important part is the foreach() in the subsearch which basically builds a list like this: | where header0>=15 OR header1>=15 OR header10>=15 OR header11>=15 OR header12>=15 OR header13>=15 OR header14>=15 OR header15>=15 OR header16>=15 OR header17>=15 OR header18>=15 OR header19>=15 OR header2>=15 OR header20>=15 OR header21>=15 OR header22>=15 OR header23>=15 OR header24>=15 OR header25>=15 OR header26>=15 OR header27>=15 OR header28>=15 OR header29>=15 OR header3>=15 OR header30>=15 OR header31>=15 OR header32>=15 OR header33>=15 OR header34>=15 OR header35>=15 OR header36>=15 OR header37>=15 OR header38>=15 OR header39>=15 OR header4>=15 OR header40>=15 OR header41>=15 OR header42>=15 OR header43>=15 OR header44>=15 OR header45>=15 OR header46>=15 OR header47>=15 OR header48>=15 OR header49>=15 OR header5>=15 OR header50>=15 OR header51>=15 OR header52>=15 OR header53>=15 OR header54>=15 OR header55>=15 OR header56>=15 OR header57>=15 OR header58>=15 OR header59>=15 OR header6>=15 OR header7>=15 OR header8>=15 OR header9>=15 The threshold value would be 1 in your case and you have to tweak the example of course, but it will get you a dynamic search approach and something to start with. cheers, MuS ... View more

I still don't think this works, because your regex is using a specific field and the regex is on the value of that field not the field name itself. It could work if the field name P-CSCF-02 is actually in _raw , but we don't know for sure. Don't take me wrong, my answer is also not really a good solution and just a workaround 😉 cheers, MuS ... View more

and if you want to catch as well events outside of your defined sets of durations, just add a catch-all clause at the end of the case() like this: case(duration < 10, "<10d", duration = 10 AND duration <30, "10-30d", duration < 30, ">30d", 1=1, "older") cheers, MuS ... View more

I just checked the default settings for [splunk_pdfgen] and it actually has this option set: TIME_FORMAT = %m-%d-%Y %H:%M%S,%l So, please remove the TIME_FORMAT you added and try again - really wired... Can you run this command /opt/splunk/bin/splunk btool props list splunk_pdfgen --debug and compare to this list of options please: /opt/splunk/etc/system/default/props.conf [splunk_pdfgen] /opt/splunk/etc/system/default/props.conf ADD_EXTRA_TIME_FIELDS = True /opt/splunk/etc/system/default/props.conf ANNOTATE_PUNCT = True /opt/splunk/etc/system/default/props.conf AUTO_KV_JSON = true /opt/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE = /opt/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE_DATE = True /opt/splunk/etc/system/default/props.conf CHARSET = UTF-8 /opt/splunk/etc/system/default/props.conf DATETIME_CONFIG = /etc/datetime.xml /opt/splunk/etc/system/default/props.conf HEADER_MODE = /opt/splunk/etc/system/default/props.conf LEARN_MODEL = true /opt/splunk/etc/system/default/props.conf LEARN_SOURCETYPE = true /opt/splunk/etc/system/default/props.conf LINE_BREAKER_LOOKBEHIND = 100 /opt/splunk/etc/system/default/props.conf MATCH_LIMIT = 100000 /opt/splunk/etc/system/default/props.conf MAX_DAYS_AGO = 2000 /opt/splunk/etc/system/default/props.conf MAX_DAYS_HENCE = 2 /opt/splunk/etc/system/default/props.conf MAX_DIFF_SECS_AGO = 3600 /opt/splunk/etc/system/default/props.conf MAX_DIFF_SECS_HENCE = 604800 /opt/splunk/etc/system/default/props.conf MAX_EVENTS = 256 /opt/splunk/etc/system/default/props.conf MAX_TIMESTAMP_LOOKAHEAD = 40 /opt/splunk/etc/system/default/props.conf MUST_BREAK_AFTER = /opt/splunk/etc/system/default/props.conf MUST_NOT_BREAK_AFTER = /opt/splunk/etc/system/default/props.conf MUST_NOT_BREAK_BEFORE = /opt/splunk/etc/system/default/props.conf SEGMENTATION = indexing /opt/splunk/etc/system/default/props.conf SEGMENTATION-all = full /opt/splunk/etc/system/default/props.conf SEGMENTATION-inner = inner /opt/splunk/etc/system/default/props.conf SEGMENTATION-outer = outer /opt/splunk/etc/system/default/props.conf SEGMENTATION-raw = none /opt/splunk/etc/system/default/props.conf SEGMENTATION-standard = standard /opt/splunk/etc/system/default/props.conf SHOULD_LINEMERGE = False /opt/splunk/etc/system/default/props.conf TIME_FORMAT = %m-%d-%Y %H:%M%S,%l /opt/splunk/etc/system/default/props.conf TRANSFORMS = /opt/splunk/etc/system/default/props.conf TRUNCATE = 10000 /opt/splunk/etc/system/default/props.conf detect_trailing_nulls = false /opt/splunk/etc/system/default/props.conf maxDist = 100 /opt/splunk/etc/system/default/props.conf priority = /opt/splunk/etc/system/default/props.conf sourcetype = ... View more

hmm, looks like I don't understand how this works ?!? ¯\_(ツ)_/¯ ... View more

Hi HattrickNZ, after checking the other answer, I must admin I don't understand how it works and thought of a different approach to make it dynamic. Take this run everywhere search: index=_internal date_minute=* | stats count by date_minute | eval header{date_minute}=date_minute | foreach header* [ eval match=if(date_minute >= "30", "yes", "no")] | table date_minute match The first 3 lines just make up dummy data and the foreach eval actually would be threshold to filter on later. Hope this makes sense ... cheers, MuS ... View more

If you run a Splunk bash session $SPKUNK_HOME/bin/splunk cmd bash and within this session do a nslookup , dig , ping tests and other network DNS related tests, do all the 12 configured idx ever show up as DNS response? cheers, MuS ... View more

Please use the code function while selecting the regex to keep formatting; code can be applied by clicking the 101010 icon or pressing CTRL-k cheers, MuS ... View more

Check your permission settings on the SHC, the message says only power and admin user can write. cheers, MuS ... View more

Hi chandanaberi, as you said correctly, there is no need for any subsearch here. A single stats search with additional eval will do it: index=_internal clientip=* status=* | eval dc_200=case(status="200", 1, 1=1, 0), dc_404=case(status="404", 1, 1=1, 0) | stats dc(eval(dc_200 + dc_404)) AS sum_status count by clientip | eval frequent=case(sum_status="2", "common", sum_status="1", "uncommon", 1=1, "unknown") This is a run everywhere search and you need to adapt it to your needs 😉 Hope this helps to get you started ... cheers, MuS ... View more