windows or nix? ... View more

Can you share the entire config for this input? ./splunk btool inputs list --debug look for the stanza for this particular input and share here! ... View more

Hey mfrost8! You are on the right track with suspecting the mgmt port, but lets start by focusing on a particular host and we can walk through validating that. Filter down to a host and examine the ports splunk is running and we can go from there. Based on what you have advised I would suggest you don't have too much to worry about. It is likely the communication to the Deployment server it is complaining about...but lets prove it! ... View more

ah! I re-read your post and see what you mean now! Glad Somesoni2 got you sorted! ... View more

Try adding _time to the by clause in your sistats command: sourcetype=mail | sistats count as sbj_count by subject, _time To test it out first, just use the regular stats command to ensure you get what you expect: sourcetype=mail | stats count as sbj_count by subject, _time Depending on what your data looks like you may have multiple timestamps to deal with, so something like | stats max(_time) AS _time may also be helpful if you want the last timestamp. Would likely then need to convert the epoch to human readable. Really depends what the data looks like and what your desired outcome is. Feel free to post a couple example events to allow us to help further. ... View more

What Sourcetype are you using for the following hosts/sources? c:\inetpub\wwwroot\sf-api\logs\v3API.txt|host::SFUTIL1-TEST D:\sc\logs\StorageCenterLog_20170405.log|host::storage-usw-85 D:\sc\logs\StorageCenterLog_20170405.log|host::storage-usw-83 Can you share the props.conf for those sourcetypes? It looks like there could be an opportunity to improve timestamp recognition settings in props.conf ie. TIME_PREFIX , TIME_FORMAT , MAX_TIMESTAMP_LOOKAHEAD to improve timestamping. There are times when ignoring these messages may be required, if the data you are ingesting is void of timestamps or has strange formatting. But let's start by ensuring your sourcetype has been configured optimally. Add data wizard on a dev machine can make this much easier as well. ... View more

Hi GArienti! You can debug scripted inputs by checking index=_internal source=*splunkd.log ExecProcessor *win_All_TCP_ports.bat for any stderr that may have been emitted from your script at this time. Do you see any errors? ... View more

totally understood and is why i wish it was something avail in our indexing pipeline. fingers crossed. ... View more

sad panda. unfortunately it sounds like dedup is the easiest option here... ... View more

Ha! @woodcock knows I thought about answering this one. Tricky part here is the use of HEC which doesn't give us the chance to put the json to disk so we can pre-parse. Would need jq integrated into the indexing pipeline...here's hoping my enhancement request gets some eyes one day. @burras what is sending the json to you? I can tell you about another method that may work for you...but it would include some changes in your solution. The problem here is that, in order for Splunk to see these as individual events, yet keep the json format, we need to unwrap the array...something the indexing pipeline doesn't handle all that well today...check out jq and if you can catch these json events and put them to disk, you can pre-parse them into single events then ingest... ... View more

why is the sourcetype _undefined on your 9999 input? also INDEXED_EXTRACTIONS=JSON is not set in your props? Also not seeing your transforms being set in props.... ... View more

Hi deepak02! Splunk has both indexed extractions and searchtime extractions for json. They are found in props.conf. http://docs.splunk.com/Documentation/Splunk/6.5.2/Admin/Propsconf INDEXED_EXTRACTIONS = < CSV|W3C|TSV|PSV|JSON > * Tells Splunk the type of file and the extraction and/or parsing method Splunk should use on the file. CSV - Comma separated value format TSV - Tab-separated value format PSV - pipe "|" separated value format W3C - W3C Extended Extended Log File Format JSON - JavaScript Object Notation format * These settings default the values of the remaining settings to the appropriate values for these known formats. * Defaults to unset. *If you are using a forwarder, be sure to put the props.conf on the forwarder! not just the indexer! Also as an FYI, Splunk has a searchtime extractions available: KV_MODE = [none|auto|auto_escaped|multi|json|xml] * Used for search-time field extractions only. * Specifies the field/value extraction mode for the data. * Set KV_MODE to one of the following: * none: if you want no field/value extraction to take place. * auto: extracts field/value pairs separated by equal signs. * auto_escaped: extracts fields/value pairs separated by equal signs and honors \" and \\ as escaped sequences within quoted values, e.g field="value with \"nested\" quotes" * multi: invokes the multikv search command to expand a tabular event into multiple events. * xml : automatically extracts fields from XML data. * json: automatically extracts fields from JSON data. * Setting to 'none' can ensure that one or more user-created regexes are not overridden by automatic field/value extraction for a particular host, source, or source type, and also increases search performance. * Defaults to auto. * The 'xml' and 'json' modes will not extract any fields when used on data that isn't of the correct format (JSON or XML). OR AUTO_KV_JSON = [true|false] * Used for search-time field extractions only. * Specifies whether to try json extraction automatically. * Defaults to true. What ever way you decide, I encourage you to try a sample of your json using the Add Data wizard, to ensure you are getting the extractions you expect. http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Setsourcetype ... View more

please do. the more tickets the better ... View more

Meh, worth a shot. Many application are able to use syslog to both send to remote host and to write to disk..If you know syslog is not an option for remote logging here, then I guess the quest continues.... ... View more

ah nice! now I can sleep better at night! 😉 Good reference here regarding what config changes require restart, cause editing the conf files won't alert you to needing to restart... http://docs.splunk.com/Documentation/Splunk/latest/Admin/Configurationfilechangesthatrequirerestart ... View more

ah yes, that'll work! did u end up editing the view to just keep the table? or just kept the whole view? ... View more

np, the view is called showcase_classification.xml so i think showcase_classification.js is def where you wanna start ... View more

Hi Ram, I recommend turning on the developer tools in chrome of firefox to help you examine the code to ensure you get the pieces you are looking for. ... View more

can you provide the output of: ./splunk btool inputs list tcp --debug and ./splunk btool props list fluentd --debug One thing I could think of would be that the json indexed extractions aren't happening for some reason? Your sourcetype is relying on fields to be present to make the re-writes, so maybe its not available....hard to say for me without seeing the system. Do you have some sample events? ... View more

looks good, did you restart splunk? ./splunk restart ... View more

good call, dont forget restart! Abilan ... View more

./splunk btool props list test --debug need the sourcetype on the forwarder and indexer. EDIT : updated command to reflect different soucretype. as you have it called test now...other thread is sched ... View more

your user is set to UTC in the Splunk GUI, correct? click the username dropdown (top right of your screen) > Account Settings > Set timezone With my user set in UTC, I see these events at 3AM GMT in my Time field....is that what you are trying to achieve? Note the time in the event has not changed, just the Time field now has a local conversion in the gui. can you please collect the output of ./splunk btool props list sched --debug on your forwarder and your indexer? ( not sure of your setup, I think you said forwarder) you have the sched sourcetype updated in both a forwarder and indexer, correct? `[splunker@n00bserver bin]$ ./splunk btool props list sched --debug /home/splunker/splunk/etc/apps/search/local/props.conf [sched] ......... /home/splunker/splunk/etc/apps/search/local/props.conf TIME_FORMAT = %Y-%m-%d %H:%M:%S,%f /home/splunker/splunk/etc/apps/search/local/props.conf TIME_PREFIX = ^ /home/splunker/splunk/etc/system/default/props.conf TRANSFORMS = /home/splunker/splunk/etc/system/default/props.conf TRUNCATE = 10000 /home/splunker/splunk/etc/apps/search/local/props.conf TZ = Brazil/East ........ ` Brasil event time, UTC Splunk web user, splunker sitting in Canada 😉 ... View more