Do you happen to have the full example to hand? I'm have trouble getting the javascript to be rendered. When I view the html source the javascript is not appearing. ... View more

So I recently wrote a modular input that needed to record its status between runs. The way I tackled it was to work out the checkpoint directory then create a file in that directory to store its current status (in stream_events) checkpoint_directory = str(self._input_definition.metadata['checkpoint_dir']) timestamp_progress_file = os.path.join(checkpoint_directory, self.stanza + '_last_timestamp.txt') This file is then read/updated like any other file accessed through Python. ... View more

http://dev.splunk.com/view/event-collector/SP-CAAAE6P covers the format of batch events within the HTTP event collector. Following the below examples you will not need to worry about things like LINE_BREAKER in props.conf Just ensure things like time, host, source are broken out from the event itself (so your original JSON will need some tweaking). This applies when using the https://example.com:8088/services/collector/event endpoint not the http://example.com:8088/services/collector/raw endpoint. From the link...... { "time": 1437522387, "host": "dataserver992.example.com", "source": "testapp", "event": { "message": "Something happened", "severity": "INFO" } } To batch (from link)...... { "event":"event 1", "time": 1447828325 } { "event":"event 2", "time": 1447828326 } Combining these you would have something like this { "time": 1437522387, "host": "dataserver992.example.com", "source": "testapp", "event": { "message": "Something happened", "severity": "INFO" } } { "time": 1437522388, "host": "dataserver993.example.com", "source": "testapp", "event": { "message": "Something else happened", "severity": "DEBUG" } } ... View more

So the following should get you started. | stats count as host | eval host="jboss-vm-2f5r8.prod.d2-nonpfv-brown.cloud.pi.com" | rex field=host "(?<host_part1>[^\.]*)\.(?<host_part2>[^\.]*)\.(?<host_part3>[^\-]*).*" | eval sitename=case(host_part3=="d2","BOHO",host_part2=="d3","Cranebank") | eval environment=case(host_part2=="prod","Production",host_part2=="sys","System testing") The trick is to break up the host string using rex into fields with the text to check, then use a case command to test the text and then set it to the value you want. Depending on how consistent the naming convention for the host is the rex may need some tweaking. ... View more

To your point about scripting in python, to make this a bit more "splunkish" you could create a custom search command http://dev.splunk.com/view/python-sdk/SP-CAAAEU2. So your python code could be called as a command from within the query (SPL) ... View more

So this feels a bit like a follow up question. Would you mind accepting the answer for the D3 question (if it answers the D3 question). Then re-asking the follow up question as a new question. It should help other people who come to visit these questions at a later date. (Also a bit more detail on the follow question would be good). So short answer, what you are asking for feels doable. ... View more

So the following will work and/or provide pointers how to do this. index="462049" | rex field=_raw ".*`savedsearch_name`.*`savedsearch_name`=\"(?<savedsearch_name_second>)\".*" | eval savedsearch_name_second_is_blank=if(savedsearch_name_second="",1,0) | where savedsearch_name_second_is_blank = 0 So the trick is to use regex to find a second savedsearch_name by looking directly against the _raw event field. Then apply additional logic against this field. ... View more

So a few further questions for clarification - Can savedsearch_name appear more than twice? Can both savedsearch_names be empty? Are the only events you want exclude are match on first and nothing on second? ( So nothing on first and match on second is allowed ) ... View more

https://docs.splunk.com/Documentation/Splunk/6.5.0/AdvancedDev/CustomVizTutorial covers how to build a custom visualization in Splunk. The example in the link also happens to use D3 as the visualization library. Also worth looking at Splunkbase as a few custom visualizations ( https://splunkbase.splunk.com/apps/#/app_content/visualizations ) are already up there and some of which also use D3. It might be your D3 visualization you want to use has already be done and you can use that. Or you could install them and see how they work as a point of reference. ... View more

To build modular inputs in Python have a look at http://dev.splunk.com/view/python-sdk/SP-CAAAER3 However you may be able to achieve what you need without building a modular input, for example if the XML is created as a file in a directory, you could poll the file and index the XML without needing to create the modular input. Also looking at Splunkbase (https://splunkbase.splunk.com/) there are several technical methods to collect data, i.e. JMS, MQTT, AMQP, etc that you may also be able to use. ... View more

In addition to the fields extracted above you could further enrich the data with lookups or eval statements to create additional fields. For example converting the "pr" to "prod" and "l" to "linux" ... View more

I've successfully used the HTTP event collector ( http://dev.splunk.com/view/event-collector/SP-CAAAE6M ) mechanism to send data to the cloud. It works using a token rather than username/password (also different port). As the HTTP event collector (HEC) is based on REST you can create a java client (with your preferred HTTP java library) that passes data to Splunk Cloud. Example curl - curl -X POST -d @**FILENAME** -H "Content-Type: application/json" -k https://input-**SERVER**.cloud.splunk.com:8088/services/collector/event -H "Authorization: Splunk **HEC_TOKEN**" ... View more

When you say Solar do you mean Apache Solr? ... View more

Splunk accepts both structured and unstructured data. Regarding Splunk vs Solar vs Big data what are the use cases you are trying to solve? ... View more

Hi, Just to clarify have I only provided one answer to your two questions? The one that I think I have answered is changing the query based on the drop down. Thanks Mark ... View more

Another option for the visualization that uses the min/max value query index="456278" | timechart span=7d max(state) as max, min(state) as min by series format="$VAL$ $AGG$" Is the "Horizon Chart - Custom Visualization" on Splunkbase https://splunkbase.splunk.com/app/3117/ This has the advantage of being able to pull a line across the charts that shows the values. So you can see the min/max for a certain time period. So you end up with something a bit like this - ... View more

So heres a simpler example of filtering by lookup value which should hopefully get you started. Using data ( put into index 456446 ) - time,column_a,column_b 09-21-2016 01:00:00,11,10 09-22-2016 01:00:00,9,10 09-23-2016 01:00:00,7,10 09-24-2016 01:00:00,5,10 09-25-2016 01:00:00,3,10 09-26-2016 01:00:00,1,10 09-27-2016 01:00:00,0,10 With lookup ( put into 456446_lookup ) - component_id, value 1,1 2,2 3,3 4,5 5,8 6,13 7,21 8,34 9,55 10,89 You can use this - index="456446" | lookup 456446_lookup component_id as column_a outputnew value as comparison_field | table column_a, column_b, comparison_field | where column_b < comparison_field ... View more

Using the seriesColors option you don't need to know the names of your series ahead of time. So you can create pairs of colors instead. <option name="charting.seriesColors">[0x1e93c6, 0x1e93c6, 0xf2b827, 0xf2b827, 0xd6563c, 0xd6563c, 0x6a5c9e, 0x6a5c9e</option> Here is a full example - <dashboard> <label>456278</label> <row> <panel> <chart> <search> <query>index="456278" | timechart span=7d max(state) as max, min(state) as min by series format="$$VAL$$ $$AGG$$"</query> <sampleRatio>1</sampleRatio> </search> <option name="charting.axisLabelsX.majorLabelStyle.overflowMode">ellipsisNone</option> <option name="charting.axisLabelsX.majorLabelStyle.rotation">0</option> <option name="charting.axisTitleX.visibility">visible</option> <option name="charting.axisTitleY.visibility">visible</option> <option name="charting.axisTitleY2.visibility">visible</option> <option name="charting.axisX.scale">linear</option> <option name="charting.axisY.scale">linear</option> <option name="charting.axisY2.enabled">0</option> <option name="charting.axisY2.scale">inherit</option> <option name="charting.chart">line</option> <option name="charting.chart.bubbleMaximumSize">50</option> <option name="charting.chart.bubbleMinimumSize">10</option> <option name="charting.chart.bubbleSizeBy">area</option> <option name="charting.chart.nullValueMode">gaps</option> <option name="charting.chart.showDataLabels">none</option> <option name="charting.chart.sliceCollapsingThreshold">0.01</option> <option name="charting.chart.stackMode">default</option> <option name="charting.chart.style">shiny</option> <option name="charting.drilldown">all</option> <option name="charting.layout.splitSeries">0</option> <option name="charting.layout.splitSeries.allowIndependentYRanges">0</option> <option name="charting.legend.labelStyle.overflowMode">ellipsisMiddle</option> <option name="charting.legend.placement">right</option> <option name="charting.seriesColors">[0x1e93c6, 0x1e93c6, 0xf2b827, 0xf2b827, 0xd6563c, 0xd6563c, 0x6a5c9e, 0x6a5c9e</option> </chart> </panel> </row> </dashboard> Also note the use of double $ ($$) in the dashboard as the $ needs to be escaped. So the above works with the standard charting options within Splunk. Another alternative would be to build a custom vizualization but this would likely take a bit of time to create. ... View more

Using index="456278" | timechart span=7d max(state) as max, min(state) as min by series format="$VAL$ $AGG$" with format will place max/min legends next to each other, for easier comparison between values. ... View more