Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- About phoenixdigital
phoenixdigital
Builder
Member since:
02-08-2011
06-05-2020
Community Statistics
| Posts | 279 |
| Solutions | 25 |
| Karma Given | 33 |
| Karma Received | 75 |
| Member Since | 02-08-2011 |
Activity Feed
- Got Karma for Re: Terminate User Session. 08-05-2022 12:07 AM
- Got Karma for Re: Realtime clock of Splunk server time on dashboard?. 06-10-2022 05:49 AM
- Got Karma for Indexing a CSV data file with more than one set of data. 04-06-2021 04:39 PM
- Got Karma for Can you hard code a color for a value range in Calendar Heat Map?. 06-05-2020 12:49 AM
- Karma Re: unarchive_cmd for decoding binary file with python script for MuS. 06-05-2020 12:48 AM
- Karma Re: unarchive_cmd for decoding binary file with python script for micahkemp. 06-05-2020 12:48 AM
- Karma Re: Splunk App for Stream: How to configure a universal forwarder to monitor DNS and DHCP? for vshcherbakov_sp. 06-05-2020 12:48 AM
- Karma Re: Is there a logging or debug tool to identify all props and transforms that are applied to a particular sourcetype to isolate rogue field extractions? for rjthibod. 06-05-2020 12:48 AM
- Karma Re: Is there documentation on forwarder behavior for various types of inputs when an indexer goes offline? for ryanoconnor. 06-05-2020 12:48 AM
- Karma Re: Search Head Deployer in a SH Cluster: What happens to local? for teunlaan. 06-05-2020 12:48 AM
- Karma Re: How is Server Identified After clone-prep-clear-config Script is Run? for jconger. 06-05-2020 12:48 AM
- Got Karma for Re: Chart not obeying charting.axisY.includeZero=false. 06-05-2020 12:48 AM
- Got Karma for setup.xml to modify/create an entry in app.conf (or anywhere else). 06-05-2020 12:48 AM
- Got Karma for How to update a global lookup file via REST API for a particular app in a search head cluster?. 06-05-2020 12:48 AM
- Got Karma for How to update a global lookup file via REST API for a particular app in a search head cluster?. 06-05-2020 12:48 AM
- Got Karma for Is there a logging or debug tool to identify all props and transforms that are applied to a particular sourcetype to isolate rogue field extractions?. 06-05-2020 12:48 AM
- Got Karma for Search Head Deployer in a SH Cluster: What happens to local?. 06-05-2020 12:48 AM
- Karma SA-Eventgen doesn't work? for laiyongmao. 06-05-2020 12:47 AM
- Karma Re: Splunk App for Stream: Is it possible to add my own custom protocol? for vshcherbakov_sp. 06-05-2020 12:47 AM
- Karma Re: 6.3 Upgrade - Missing HTTP Event Collector Data Input Option for MaverickT. 06-05-2020 12:47 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 1 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
12-08-2015
12:20 PM
Turns out the issue is a Windows issue and not Splunk. The XML format can be viewed with Windows Event Viewer so if something is missing it is because Windows did not put it there. 😞
Sadly we have switched back to plaintext events for Applocker. Windows disappoints yet again.
More information can be found here.
https://answers.splunk.com/answers/290844/testing-renderxml1-for-windows-event-logs-in-splun.html
... View more
01-24-2016
01:09 PM
To answer this question, you can use Search -> Pivot to look at the data models in question. Network Traffic and maybe Network Sessions are the most likely to be interesting. Limit to the sourcetypes in question and a reasonable timeframe, then add the fields you're concerned about. If it shows up in a data model, then ES cares about it. If it doesn't, it doesn't.
... View more
01-05-2016
09:05 AM
Dude
please try yyyy instead of YYYY in your inputs
regards
... View more
04-20-2018
01:10 AM
@phoenixdigital, set the default value of Text input to $component$ i.e. <default>$component$</default> and in the <init> section set component token to asterisk ( * ). Please try out and confirm.
PS: <init> section is available in version 6.5 and above. You would need a dummy search to set component to asterisk using search event handler in case you are on Splunk version 6.4 or previous.
<form>
<label>Drilldown Example Table Complex</label>
<init>
<set token="component">*</set>
</init>
<fieldset submitButton="false">
<input type="text" token="component">
<default>$component$</default>
</input>
<input type="time" token="tokTime" searchWhenChanged="true">
<label></label>
<default>
<earliest>-24h@h</earliest>
<latest>now</latest>
</default>
</input>
</fieldset>
<row>
<panel>
<table>
<search>
<query>index=_internal sourcetype=splunkd| stats count(eval(log_level="INFO")) as INFO count(eval(log_level="ERROR")) as ERROR by component</query>
<earliest>$tokTime.earliest$</earliest>
<latest>$tokTime.latest$</latest>
</search>
<option name="count">10</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="refresh.display">progressbar</option>
<option name="rowNumbers">false</option>
<option name="wrap">true</option>
<drilldown>
<condition field="component">
<set token="component">$click.value$</set>
<set token="log_level">*</set>
</condition>
<condition field="INFO">
<set token="log_level">$click.name2$</set>
<set token="component">$click.value$</set>
</condition>
<condition field="ERROR">
<set token="log_level">$click.name2$</set>
<set token="component">$click.value$</set>
</condition>
</drilldown>
</table>
</panel>
</row>
<row>
<panel>
<table>
<title>Search = index=_internal component="$component$" log_level="$log_level$"</title>
<search>
<query>index=_internal component="$component$" log_level="$log_level$"</query>
<earliest>$tokTime.earliest$</earliest>
<latest>$tokTime.latest$</latest>
</search>
<option name="count">10</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">cell</option>
<option name="refresh.display">progressbar</option>
<option name="rowNumbers">false</option>
<option name="wrap">true</option>
</table>
</panel>
</row>
</form>
... View more
09-06-2017
04:28 PM
Good to know. I haven't looked much at Eventgen since I posted this ~2 years ago it has likely changed a bit since then.
https://github.com/splunk/eventgen
Back then if eventgen didn't find a timestamp on every line it borked out as shown above.
... View more
06-05-2016
05:40 PM
This turned out to be an issue of "nested lookups" which is not supported in any of splunk versions as of this writing.
You would see inconsistent results - The situation is it works under certain instance and not on another. Please avoid any chain lookups.
... View more
09-10-2015
11:46 AM
1 Karma
Enterprise Security Deployments always need tuning and tweaking. This tuning is usually a major portion of a professional services engagement when they are deploying Enterprise Security app. Most of the technology add-ons that are put out by splunk are pretty good, but they all usually require some customization at deployment time. I would validate that the version of the TA that you are using is the most recent. Sometimes the version that is bundled with Enterprise Security isn't the latest and greatest and you will find an updated version out on splunkbase. Other times, you may have to add a local/props.conf with an alias or an extract statement etc.
... View more
12-07-2015
11:49 PM
Thanks for the answer but I already have the DB admins breathing down my neck regarding database input queries. I fear that adding any more complexity to the query would push them over the edge and they might go all "soup nazi" on me "No DB input for you!!!!"
Ideally having it supported like it was for DB Connect 1 would be the best option.
... View more
08-09-2015
07:54 PM
OK I have modified eventgen to handle this plus another feature
lib/eventgenconfig.py - http://pastebin.com/pwKXe0mL
lib/eventgensamples.py - http://pastebin.com/dNnX3vNB
I have implemented a new replacementType of randomRated there was something close to this called rated in the code but it only worked off global values.
token.1.token = @@fastdaytime
token.1.replacementType = randomRated
token.1.replacement = integer[1000:1100]
token.1.hourOfDayMultiplier = { "0": 0.2, "1": 0.3, "2": 0.4, "3": 0.5, "4": 0.5, "5": 0.5, "6": 0.5, "7": 0.5, "8": 1, "9": 1.5, "10": 4, "11": 5, "12": 6, "13": 6, "14": 5, "15": 2, "16": 1.5, "17": 1, "18": 0.8, "19": 0.7, "20": 0.5, "21": 0.5, "22": 0.5, "23": 0.5 }
token.1.dayOfWeekMultiplier = { "0": 1, "1": 1, "2": 1, "3": 1, "4": 1, "5": 1, "6": 1 }
So what will happen is if you have an integer or float random generated value you can also apply a multiplier to the resulting random number based on the time of day and/or day of the week. You can configure hourOfDayMultiplier or dayOfWeekMultiplier or both.
The other feature I added was the ability to gradually increase the size of a field by a certain amount +- a random number. This generates a gradual increase in value with some slight randomness to it.
token.3.token = @@NT
token.3.replacementType = integerid
token.3.replacement = 101
token.3.replacementIncrementAmount = 20
token.3.replacementIncrementAmountRandomness = 0.2
So in the instance above the value will start at 101 then increase by 18 - 22 each subsequent event. 10% variation either side of 20.
Does anyone know who I can submit the changes to? I made sure I did not impact on any existing code so it should be backwards compatible
... View more
07-15-2015
04:16 PM
Well, it's intrinsic to what an eventtype actually is:
It's a set of search filters that determines what eventtypes an event gets annotated with. You use it two ways - either to filter based on the eventtype, or to look at the eventtype field in events returned by any search.
A | inputlookup doesn't make sense in this concept. Have you considered a macro?
... View more
06-23-2015
11:40 PM
Thanks for that.
Weird/annoying that the configs are so inconsistent.
... View more
08-19-2015
08:40 AM
Because of these issues splunk does not recommend using VMs. Having said that, the best way to convince any one is to monitor CPU usage after the SH is in production. You will know very quickly (based on load) if you are CPU bound or not. Make sure you have historical view the CPU levels an not just snapshots in time.
... View more
06-23-2015
11:00 PM
This is the route we ended up going for but if the customer had insisted they replicated the old data I would likely have followed acharlieh's advise posted here.
... View more
08-03-2016
04:27 PM
3 Karma
The answer by mdessus describes how to detect this issue.
Firstly, the ES Identity and Assets are merged every 5 minutes as a modular input, that explains why sometimes it will happen instantly and other times it can take a few minutes: http://docs.splunk.com/Documentation/ES/4.1.1/User/Identitymanagement#Merging_the_asset_and_identity_lists
What worked for me was the following:
Background: you have a lookup, ad_identity_list that is silently failing to load in to ES. The lookup is populated with good data, you've checked the logs for modular inputs and have seen that the merge is running properly, but no data Identity data is being populated in ES.
Make an interim lookup, called something like ad_identity_interim.
Copy whole ad_identity_list into ad_identity_interim.
Execute the following, to place only a few entries into the Identity lookup ES is trying to merge.
| inputlookup ad_identity_interim | head 5 | outputlookup ad_identity_list
Wait until the merge occurs and you should see the five entries in your Identity Center.
Continue adding incrementally until you have the whole list in there, making sure you wait for the merge to occur between each execution.
| inputlookup ad_identity_interim | head 50 | outputlookup ad_identity_list
| inputlookup ad_identity_interim | head 100 | outputlookup ad_identity_list
| inputlookup ad_identity_interim | head 500 | outputlookup ad_identity_list
| inputlookup ad_identity_interim | head 1000 | outputlookup ad_identity_list
You should now have all your identities in ES.
I'm unsure as to why this works, but the issue has occurred and this fix has worked for me in several completely different architectures. It seems as though once the initial list has populated that the updates to the lookup are loaded properly, so I haven't had to make a chain of saved searches to behave as described above; it works as expected once it's all initially loaded -- noting that I have only ever made minor changes on an ongoing basis.
... View more
03-17-2015
06:01 PM
Agreed it would make things a hell of a lot easier. I am guessing they don't want to support tweaking the Python libraries and in a way I can see why.
Sadly it really limits the power of what we can achieve with modular and scripted inputs though.
... View more
03-31-2015
05:06 PM
Agreed, however Australia/Hobart? That's pretty odd? No offence to anyone from Hobart of course but Hobart is undoubtedly the least significant of the Australian capital cities and the absence of Sydney seems particularly strange as Sydney is often, incorrectly, referred to as the national capital... I would hope Splunk review this at some point as there are can be some peculiar timezone offsets for daylight savings time along the Australian east coast.
... View more
07-01-2015
09:45 PM
OK it seems my previous post got lost in mod approval limbo. Likely due to the wall of code I had in the post. So here goes the second attempt with the code up on pastebin.
You likely have already resolved this issue or have given up. However this answer is for future people looking for an answer.
This is the solution I just discovered that works quite well using the library
http://docs.python-requests.org/en/latest/index.html
Code examples
http://pastebin.com/5LG8YAr1
... View more
02-07-2017
03:45 PM
| rest /servicesNS/-/-/search/jobs splunk_server=local | search eventSorting=realtime
Works for me in Splunk 6.5
| rest /servicesNS/-/-/search/jobs | search eventSorting=realtime
Works for me on Splunk 6.5
... View more
10-29-2014
09:29 PM
OK based on that other answer I linked to in the question it was definitely the right track.
I messed around with the "Tag Cloud Autodiscover" dashboard in the Splunk 6 Example searches app and got it sorted.
Ref : http://your.splunk.intance:8000/en-US/app/simple_xml_examples/custom_viz_tag_cloud_autodiscover?earliest=0⪭st=
The key bit was just making a postprocessmanager which referenced the original manager.
Full code here
<dashboard script="autodiscover.js">
<label>Custom Visualization with Auto-Discovery</label>
<row>
<table>
<title>Table</title>
<searchString>index=_internal source=*metrics.log group=pipeline | stats max(cpu_seconds) as cpu_seconds by processor | sort - cpu_seconds</searchString>
<earliestTime>-24h</earliestTime>
<latestTime>now</latestTime>
</table>
<html>
<h2>Custom Visualization: Tag Cloud</h2>
<div id="tagcloud-search"
class="splunk-manager splunk-searchmanager "
data-require="splunkjs/mvc/searchmanager"
data-options='{
"app": "myapp",
"preview": true,
"search": "index=_internal source=*metrics.log group=pipeline | stats max(cpu_seconds) as cpu_seconds by processor",
"earliest_time": "-24h"
}'>
</div>
<div id="tagcloud-searchpostprocess"
class="splunk-manager splunk-searchmanager "
data-require="splunkjs/mvc/postprocessmanager"
data-options='{
"managerid": "tagcloud-search",
"search": "| where like(processor,\"s%\")"
}'>
</div>
<div id="tagcloud"
class="splunk-view"
data-require="app/infratil/components/rawdump/rawdump"
data-options='{
"minFontSize": 14,
"maxFontSize": 55,
"managerid": "tagcloud-search",
"valueField": "cpu_seconds",
"labelField": "processor"
}'>
</div>
</html>
</row>
<row>
<html>
<h2>Custom Visualization: Tag Cloud 2</h2>
<div id="tagcloud2"
class="splunk-view"
data-require="app/infratil/components/rawdump/rawdump"
data-options='{
"minFontSize": 14,
"maxFontSize": 55,
"managerid": "tagcloud-searchpostprocess",
"valueField": "cpu_seconds",
"labelField": "processor"
}'>
</div>
</html>
</row>
</dashboard>
... View more
11-05-2014
05:20 PM
1 Karma
OK I upgraded Splunk and the DBX app and things are back to normal again.
The original search (database injection) is working as expected.
After updating the DBX app I had to run setup again for the app which I suspect removed whatever corrupt cache data file was there.
If a DBX support person can tell us where the DBX app might have been caching information that would be great. I am certain if I could have just deleted the relevant corrupted file this would have been resolvable within minutes not 10 days.
... View more
10-15-2014
06:07 PM
2 Karma
Ok it appears that threading was the wrong track to go down.
The SDK supports single or multiple instances meaning that
single instance - one script handles all of the data inputs
multi instance - each input fires off its own version of the script
http://docs.splunk.com/Documentation/Splunk/6.1.4/AdvancedDev/ModInputsScripts#Single_or_multiple_instances_of_a_script
I tested it out again with the random numbers example
single instance - http://pastebin.com/77MMLqPz
multi instance example - http://pastebin.com/idDqG7XY
Now to build the systemd modular input based on this new found knowledge.
... View more
05-22-2014
05:07 PM
As Nick pointed out to me the search in my original post was not actually the one dying it was an inline one further down that was causing the issue.
... View more
04-08-2014
05:36 PM
dropEventsOnQueueFull (in outputs.conf) seems to have resolved it even though the manual seems to indicate it does the exact opposite and drops NEW events.
http://docs.splunk.com/Documentation/Splunk/latest/admin/outputsconf
May I recommend to a Splunk staff member to reword the manual entry for this to be less amiguous.
If set to a positive number, wait seconds before throwing out all new events until the output queue has space.
change to
If set to a positive number, wait seconds before throwing out all new events (already in the queue) until the output queue has space. New events arriving at the indexer will still be placed onto the queue.
The way it is currently worded seems to indicate that once the queue is full any new events arriving at the indexer will be dropped. It makes no mention of removing/dropping data from the queue itself.
Is there a better solution here?
I have also tried setting queues in inputs.conf which has no effect.
... View more
04-04-2014
01:04 AM
Excellent thanks again Nick. See I knew it would be a simple answer and thanks for clarifying why I had trouble seeing it with firebug.
Infact it is actually so simple I am a little embarrassed I didnt just take a stab in the dark and test that.
... View more
04-01-2014
08:33 PM
Simple error. I didn't have the app installed on indexer2 which means that the index did not exist for the Unix TA. So the data had nowhere to go.
Installed the app and everything is working as expected.
... View more
