About spavin

spavin · ‎04-04-2024

Hi @theprophet01, If you have a itoa_admin role, you can export services, entities, glass tables, KPI searches, templates etc from the following menu: Configuration > Backup/Restore Click Create Job > Create Backup Job Select Partial Backup, give it a name and description, uncheck include conf files, then click next Select the services you'd like to backup Click Save and Backup You will be taken to the Backup/Restore jobs page, where your job will be queued. When it's finished, usually after a few minutes, you can download the backup as a zip. Go to the same page on a different Splunk instance to restore it - this time select Restore Job and upload the zip file. See the capabilities here: https://docs.splunk.com/Documentation/ITSI/4.18.1/Configure/Capabilities You'll need the ones listed under "Backup/Restore" which by default is only given to itoa_admin.

spavin · ‎07-14-2023

Hi @rikinet., I've had a crack at this and I think I've found a working solution. Splunk has a few JSON commands (see docs) that come in handy. | makeresults | eval json="{\"data\":{\"192.168.1.1\":{\"ip\":\"192.168.1.1\",\"number\":0,\"list\":[{\"msg\":\"msg1\",\"time\":\"2023-07-01T01:00:00\"},{\"msg\":\"msg2\",\"time\":\"2023-07-01T02:00:00\"},{\"msg\":\"msg3\",\"time\":\"2023-07-01T03:00:00\"}]},\"192.168.1.2\":{\"ip\":\"192.168.1.2\",\"number\":2,\"list\":[{\"msg\":\"msg1\",\"time\":\"2023-07-02T01:00:00\"},{\"msg\":\"msg2\",\"time\":\"2023-07-02T02:00:00\"}]}}}" | fromjson json ``` Above is just creating the test data ``` ``` Get the "data" field from the JSON``` | eval ips = json_extract(json, "data") | fields ips ``` Extract all the IPs - we'll use these later ``` | eval keys = json_keys(ips) | eval keys= json_array_to_mv(keys) | mvexpand keys ``` Now we have one IP address per row. Next we get each IPs specific data ``` | foreach keys [ | eval values = json_extract_exact(ips, <<FIELD>>)] | fields keys, values | fromjson values | mvexpand list | fromjson list ``` Now table it out``` | table ip, number, msg, time That results in a table like this: As per your notes: The key names are dynamically identified under "data". The fact that they have dots doesn't really matter for the json commands used. We mvxpand the "data.X.list" variable to make sure we get them all. "data.X.number" is just any number - it's just an extra field in the list. Hopefully this query gets you closer to what you're looking for. Cheers, Daniel

spavin · ‎03-30-2021

Hi all, I have recently upgraded the Splunk_TA_New_Relic to v 2.2.0 on Splunk 8.0.7. Version 2.1.0 worked fine, but after the upgrade to 2.2.0 I started getting a lot of python errors. Has anyone found this before or know a solution? I'm going to try to set the python version to use 2.7, but wondering if there is something else we need to do to get the new version up and running. Here's a sample of the python error: 2021-03-30 13:18:42,644 ERROR The script at path=/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/Splunk_TA_New_Relic_rh_new_relic_insights.py has thrown an exception=Traceback (most recent call last): File "/opt/splunk/bin/runScript.py", line 82, in <module> exec(open(REAL_SCRIPT_NAME).read()) File "<string>", line 4, in <module> File "/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/splunk_ta_new_relic/aob_py3/splunktaucclib/rest_handler/endpoint/validator.py", line 8, in <module> from past.builtins import basestring File "/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/splunk_ta_new_relic/aob_py3/past/__init__.py", line 88, in <module> from past.translation import install_hooks as autotranslate File "/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/splunk_ta_new_relic/aob_py3/past/translation/__init__.py", line 42, in <module> from lib2to3.refactor import RefactoringTool File "/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/splunk_ta_new_relic/aob_py3/lib2to3/refactor.py", line 25, in <module> from .fixer_util import find_root File "/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/splunk_ta_new_relic/aob_py3/lib2to3/fixer_util.py", line 7, in <module> from .pygram import python_symbols as syms File "/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/splunk_ta_new_relic/aob_py3/lib2to3/pygram.py", line 32, in <module> python_grammar = driver.load_packaged_grammar("lib2to3", _GRAMMAR_FILE) File "/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/splunk_ta_new_relic/aob_py3/lib2to3/pgen2/driver.py", line 156, in load_packaged_grammar return load_grammar(grammar_source) File "/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/splunk_ta_new_relic/aob_py3/lib2to3/pgen2/driver.py", line 131, in load_grammar g.load(gp) File "/opt/splunk/etc/apps/Splunk_TA_New_Relic/bin/splunk_ta_new_relic/aob_py3/lib2to3/pgen2/grammar.py", line 108, in load d = pickle.load(f) AttributeError: 'collections.OrderedDict' object has no attribute 'append' Thanks in advance.

spavin · ‎04-24-2020

Hi @ksoori, The heatmap viz can show all 100 servers - I suspect you're running into the default limit for | timechart Try adding this to your timechart command: limit=0 e.g: |index=main |timechart avg(cpu) by host limit=0 That will allow all 100 servers to be included in the output, rather than aggregating most into an "other" field. Once you've done that there are a couple of options you should probably set to accommodate so many rows: Set the height to 1000. That's the most you can set a visualisation in a dashboard, and should give you enough room to display all 100 hosts. <option name="height">1000</option> Set the font-height to 12px - that will make the visualization more readable: <option name="heat-map-viz.heat-map-viz.labelFontSize">12</option> Turn off animations. You may still like it twinkling, but it could get annoying with so many data points: <option name="heat-map-viz.heat-map-viz.enableAnimation">false</option> Cheers, Daniel

spavin · ‎02-01-2020

You can do either or both, depending on how you want the extracted fields to look. KV_MODE will give extractions in the form of: {}.fieldname EXTRACT- will give extractions in the form of: fieldname

spavin · ‎01-31-2020

Hi @ylucena, One way to filter out already defined data is to use streamstats combined with evals. First thing to do is sort the table by Host, interface, ip_address and OS. Then you can compare each row with the value of the previous row. If they are the same as the previous line, then change the value to "". |...previous search here... | sort Host, interface, ip_address, OS | streamstats current=false last(Host) as prev_host, last(interface) as prev_interface, last(ip_address) as prev_ipaddress | eval Host=if(Host==prev_host,"",Host) | eval interface=if(Host=="" AND interface == prev_interface,"",interface) | eval ip_address=if(Host=="" AND interface=="" AND ip_address==prev_ipaddress,"",ip_address) | table Host, interface, ip_address, OS Checks: This search checks the Host to see if it's the same as the last one, and if so sets it to "". Next it looks at the Interface - if the host is already set to "" and the interface is the same as the last one, then it sets the interface to "". Finally, a similar check is done for ip_address. As long as the Host and interface are blank, it will set the ip_address to "" if it's the same as the previous one. In all other cases the fields are left as-is. That results it the following: Cheers, Daniel

spavin · ‎01-31-2020

Hi @YuliyaVassilyeva, To get different values, you can use <condition>...</condition> If you have 3 known split-by names, then you can show/hide the panels by selectively setting and unsetting tokens that the panels depend upon. EG, if your split by fields were APLPHA, BETA, and GAMMA, then this will show the ALPHA panel when you click on ALPHA, and hide the other panels. <drilldown> <condition match="$trellis.value$="ALPHA""> <set token="show_panel1">true</set> <unset token="show_panel2"></unset> <unset token="show_panel3"></unset> <set token="event_type">Epics</set> <set token="label">Status</set> </condition> <condition match="$trellis.value$="BETA""> <set token="show_panel2">true</set> <unset token="show_panel1"></unset> <unset token="show_panel3"></unset> <set token="event_type">Epics</set> <set token="label">Status</set> </condition> <condition match="$trellis.value$="GAMMA""> <set token="show_panel3">true</set> <unset token="show_panel2"></unset> <unset token="show_panel1"></unset> <set token="event_type">Epics</set> <set token="label">Status</set> </condition> <condition> <unset token="show_panel2"></unset> <unset token="show_panel1"></unset> <unset token="show_panel3"></unset> <unset token="event_type"></unset> <unset token="label"></unset> </condition> </drilldown> Then have your panels set up: <row> <panel depends="$show_panel1$">...</panel> <panel depends="$show_panel2$">...</panel> <panel depends="$show_panel3$">...</panel> </row> Cheers, Daniel

spavin · ‎01-31-2020

Hi @beingkaran, If you on-board the JSON as-is, you can create search-time extractions to get all the fields, then you won't need to use spath to use them. E.G. the following props.conf: [ANSWERS_SEARCHTIME] CHARSET=UTF-8 DATETIME_CONFIG=CURRENT KV_MODE=JSON LINE_BREAKER=\}(,?) NO_BINARY_CHECK=true SHOULD_LINEMERGE=true SEDCMD-firstbracket=s/^\[// SEDCMD-lastbracket=s/\]$// EXTRACT-ticket_id = "ticket_id"\s*:\s*"([^"]+)" EXTRACT-circuit_id = "circuit_id"\s*:\s*"([^"]+)" EXTRACT-customer_id = "customer_id"\s*:\s*"([^"]+)" EXTRACT-prob_type = "prob_type"\s*:\s*"([^"]+)" EXTRACT-severity = "severity"\s*:\s*"([^"]+)" That will break the event up (assuming no nested brackets) and create search-time fields for all the keys. The props.conf will need to go on both the search head (for search-time extractions) and indexer (for the rest of the props config). An alternative is to use indexed extractions. You can take advantage of the INDEXED_EXTRACTIONS setting: [SPLUNK_ANSWERS] DATETIME_CONFIG = CURRENT INDEXED_EXTRACTIONS = json LINE_BREAKER = ([\r\n]+) This results in all the fields being available, with no spath required: Note that indexed extractions will take more space on indexers, and will need more CPU to initially on-board. You get the ability to use tstats though, which you can't do with search-time extractions.

spavin · ‎01-31-2020

Hi @neluvasilica, What you have so far is: If you click on a row that starts with "(master)" then $master$ is set and $non_master$ is unset If you click on a row that doesn't start with "(master)" then $master$ is unset and $non_master$ is set The order of the conditions is important. If you want to match another option - eg "bla" then you'll need to check for that before you do the click.value != "(master)" check - because that will match everything that's not "(master)" and the processing will stop there. Try this order: <drilldown> <condition match="$click.value$ = "bla""> <set token="bla">$click.value$</set> <unset token="non_master"></unset> <unset token="master"></unset> </condition> <condition match="$click.value$ = "(master)""> <set token="master">$click.value$</set> <unset token="non_master"></unset> <unset token="blah" </condition> <condition match="$click.value$ != "(master)""> <set token="non_master">$click.value$</set> <unset token="master"></unset> <unset token="bla"></unset> </condition> </drilldown> If you want a generic catch-all condition to set $non_master$, you can simply put <conditon>...</conditon> as the last block. It will only match if none of the other conditions matched. I assumed you also wanted a token $bla$ created - if not, just set master / non_master to suit your needs when it matches bla. Cheers, Daniel

spavin · ‎01-31-2020

Hi Manikanthkoti, Can you provide an example of coloring by appName? The heat map is designed to show variation in the data - in this case the count. Are you looking for static colors for each app?

spavin · ‎01-31-2020

Hi @charithaa15, I believe you can achieve the desired outcome using a slightly different search. The viz only accepts numbers, so we need to convert "Failed" and "Success" to numbers. We can do that with an eval statement. For example, have a look at the follwoing search. This segment will give you some test data similar to your own data: | gentimes start=-1 increment=1h | rename starttime as _time | eval "Jobname" = if(random()%2==0,"REQUEST_QUOTE_JOB","ADP_EMAIL_JOB"), "result" = if(random()%2==0,"Failed","Success") | table _time, Jobname, result This will convert Failed to 0 and Success to 1: | eval result=if(result="Failed",0,1) Now we can timechart: | timechart sum(result) by Jobname limit=100 | fillnull Putting it all together: : | gentimes start=-1 increment=1h | rename starttime as _time | eval "Jobname" = if(random()%2==0,"REQUEST_QUOTE_JOB","ADP_EMAIL_JOB"), "result" = if(random()%2==0,"Failed","Success") | table _time, Jobname, result | eval result=if(result="Failed",0,1) | timechart sum(result) by Jobname limit=100 | fillnull The dashboard then shows this: Cheers, Daniel

spavin · ‎11-19-2019

Hi @bryanttfeliciano, I have just uploaded a new version of the Event Timeline Viz that lets you do exactly that! You can download it here: https://splunkbase.splunk.com/app/4370/ Under General options you can choose: Show current time: Which shows a red line indicating the current date: Cheers, Daniel

spavin · ‎07-29-2019

Hi, The Splunk Add-on for Citrix NetScaler explicitly states that it supports Citrix NetScaler versions 10-12. https://docs.splunk.com/Documentation/AddOns/latest/CitrixNetScaler/About Will this work on Citrix NetScaler version 13? If not, is support for NetScaler 13 planned in a future release? Thanks, Daniel

spavin · ‎04-23-2019

Hi @ryhluc01, You can compare the previous event with the current event using a combination of streamstats current=f and last() | makeresults | eval raw="Difference=0.000, DifferenceDuration=00:00:00.000,FirstCreatedByDate=1554766782,FirstModifiedByDate=1554766782,Status=New ### Difference=66383.000, DifferenceDuration=18:26:23.000,FirstCreatedByDate=1554766782,FirstModifiedByDate=1554833165, Status= ### Difference=67674.000, DifferenceDuration=18:47:54.000,FirstCreatedByDate=1554766782,FirstModifiedByDate=1554834456, Status=" | makemv delim="###" raw | mvexpand raw | rename raw as _raw | extract | eval Comment = "----- This will create a delta field if the previous Status field was 'New' ----" | fillnull value="blank" Status | streamstats current=false last(Status) as PrevStatus | eval Delta=if(PrevStatus=="New" AND Status=="blank",DifferenceDuration,null) | eval Status=if(Status=="blank",null,Status) | fields - PrevStatus - Comment The start of this search just generates your dummy data. The second part works like this: Fill all the blank status fields with a value ("blank") This will get undone later Use streamstats to get the previous Status value, and put it in the current event Create a delta field, and set it to the duration only if the previous status was 'New' Clean up - remove the 'blank' values and delete the PrevStatus and Comment fields. Does that do what you were looking for?

spavin · ‎04-01-2019

Hi @vas123, You can do this via the rex command: | makeresults count=4 | streamstats count as id | eval url=case(id=1,"/", id=2,"/?AID=10654946&PID= 40", id=3,"/topic1.jsp?redirectPage=/main/word/undefined 50",id=4,"/topic1.jsp?redirectPage=/site.webmaster 200") | rex field=url "^(?<page>[^?]+)(?<querystring>\?[^\s]+)?\s(?<count>[0-9]+)$" | stats sum(count) as "Total Count" by page The first few lines are just to setup the test data - you'd only need the last two lines. If you are interested in the querystring, it's captured too.

spavin · ‎04-01-2019

spavin · ‎03-29-2019

Hi @raja8220, Without seeing your search, or any details about what you're searching for, I can only give a high-level view of how to go about this. You can structure your search like this: 1. Main search: index=blah | stats count by host (or use | metadata for speed) 2. Append your lookup of known devices, and create a field called "totalCount" that equals zero 3. Use stats to get the first value for each host. 4. Filter out all hosts with a totalCount greater than zero This method will get all hosts that are forwarding data, and then add a list of all known hosts. There will be duplicates - so we need to tidy it up a bit. By using stats we weed out all the duplicate events, and are left with a unique list of all the host names, and a count of how many messages were received. Now you can filter out all hosts that have a count >0. The remaining hosts haven't communicated in your selected time period. Here's an example: | metadata type=hosts index=my_index | fields host, totalCount | append [| inputlookup assets.csv | eval totalCount=0 | fields totalCount, host] | stats first(totalCount) as totalCount by host | where totalCount=0

spavin · ‎03-10-2019

Hi @Shuhei052492, What you are seeing means you have set up forwarding correctly. The universal forwarder has 2 indexers set up, and it will automatically load balance between them. By default, it will send to one indexer for 30 seconds, then switch to the other for 30 seconds, then back to the first. It will only send data to one indexer at a time. You can test this out by running splunk list forward-server then waiting 35 seconds, then running it again. You can also configure the forwarders to load balance based on data volume, to try to get an even spread of data in your indexers. Check out the docs here: https://docs.splunk.com/Documentation/Splunk/7.2.4/Forwarding/Setuploadbalancingd

spavin · ‎03-08-2019

The search head cluster will make sure that the edited lookup is synced across all search heads - no matter which one you edit on. To make sure you don't obliterate the changes when you push out apps from the deployer you should use "-preserve-lookups true": splunk apply shcluster-bundle -target <URI>:<management_port> -preserve-lookups true -auth <username>:<password> That makes the deployer ignore lookups on the deployer that already exist on the search heads.

spavin · ‎03-06-2019

Can you give us a sample field to work with? Does it look like: Machine 1 Machine 2?

spavin · ‎03-06-2019

Hi @tchaeunsang1, The index you created may have looked something like this: [testindexsize] homePath = $SPLUNK_DB\$_index_name\db coldPath = $SPLUNK_DB\$_index_name\colddb thawedPath = $SPLUNK_DB\testindexsize\thaweddb homePath.maxDataSizeMB = 0 coldPath.maxDataSizeMB = 0 maxTotalDataSizeMB = 20 Which makes an index with a maximum size of 20MB, with no other limits on how big the hot/warm or cold dbs can get. When you add data it will go into a hot bucket. The bucket size defaults to "auto" which is 750MB. That means your data will grow into a 750MB bucket before rolling to warm (assuming you have data constantly coming in). Once it rolls to warm, that's when the maxTotalDataSizeMB kicks in. It sees that the hot/warm db is taking up too much space, and so it starts to roll buckets to get back down to 20MB. That means the warm bucket is removed, and you're back to under 20MB. The rolling process happens when: The DB size gets higher than maxTotalDataSizeMB (it takes into account hot buckets, but will only roll from warm to cold and freeze cold, so your hot bucket is safe) The most recent timestamp in the bucket is older than the frozenTimePeriodInSecs splunkd is restarted - this will roll all hot buckets into warm buckets You manually run: splunk _internal call /data/indexes/testindexsize/roll-hot-buckets Try repeating your experiement with maxDataSize=1 This represents the maximum size in MB for a hot DB to reach before a roll to warm is triggered. You should then see your index keeping much closer to the 20MB limit.

spavin · ‎03-05-2019

I have the same thing happening with my thawed path - it is configured as: [my_index] homePath = volume_hot:/$_index_name/db coldPath = volume_cold:/$_index_name/colddb thawedPath = $SPLUNK_DB/$_index_name/thaweddb ... The hot and cold work as intended, however the thawed db ends up in: /opt/splunk/var/lib/splunk/$_index_name/thaweddb I checked the docs for indexes.conf, and while it doesn't explicitly tell you to not use $_index_name for thawedDB path, it doesn't use it in the example (but it does for homePath/ coldPath). It says that the thawedPath should contain an absolute path that contains the thawed (resurrected) databases for the index. Of note, it mentions: We strongly recommend that you avoid the use of environment variables in index paths, aside from the possible exception of SPLUNK_DB. See homePath for the complete rationale. However, that warning is also mentioned for homePath and cold Path. For now, I'll stick to writing out the index name in the path.

spavin · ‎03-04-2019

This comment is quite old now, but I thought I'd mention in case others had a similar issue. If you want multiple deployment servers behind a load balancer you can configure crossServerChecksum=true in the serverclass.conf on the deployment server. From docs: * Ensures that each app will have the same checksum across different deployment servers. * Useful if you have multiple deployment servers behind a load-balancer. * Defaults to false.

spavin · ‎03-04-2019

Sure - update the last line to change it from: | rename minid as id to: | table _time, did, callerid, extension_number

spavin · ‎03-02-2019

Hi @sathwikakamai, If you are confident that the HEC endpoint is set up correctly, there are no firewall restrictions between you and it, and the authorization token is correct, you should be able to run the following curl command: curl -k https://localhost:8088/services/collector -H 'Authorization: Splunk d51d25fc-6fa2-4841-b430-be55876332b7' -d '' It will return something like: {"text":"No data","code":5} It will not submit any data to the index, but it will check that: There are no firewall issues between where you ran the command and the HEC endpoint The authentication token is correct, and working You can add an additional check by supplying an index that you expect to be valid, while still not sending an event: curl -k https://localhost:8088/services/collector -H 'Authorization: Splunk d51d25fc-6fa2-4841-b430-be55876332b7' -d '{"index":"my_index"}' The response will either tell you that the index is incorrect (meaning the HEC endpoint doesn't allow adding events to that index), or that there was no event supplied - indicating that the index is allowed.

Posts	32
Solutions	10
Karma Given	6
Karma Received	25
Member Since	‎11-09-2018

Online Status	Offline
Date Last Visited	‎07-10-2025 11:53 PM

Python errors when upgrading Splunk_TA_New_Relic t...

Support for Netscaler 13

Setting up AppDynamics for Oracle 9i

Re: Is it possible to export ITSI Services?

Re: Flatten JSON associative array with variable k...

Python errors when upgrading Splunk_TA_New_Relic t...

Re: How to increase row count on the heatmap viz c...

Re: What will the JSON Path for breaking this JSON...

Re: Cascade Table View

Re: How to get different drill down for different ...

Re: What will the JSON Path for breaking this JSON...

Re: drilldown condition set token based on click.v...

Re: How To Create Heat Map time as X axis?

Re: Heat Map Viz Limitations

Re: Is there a way to display current time with ma...

Support for Netscaler 13

Re: Delta Conditional Statement?

Re: How do you group by substring of URI?

Re: How to get the non reporting device list??

Re: How to get the non reporting device list??

Re: CLI "list forwarder" does not show all active ...

Re: How do you handle a lookup in search head clus...

Re: How do you split two string values joined in o...

Re: Why my current index size is larger than the m...

Re: index_name variable to use in all index paths

Re: Large number of Forwarders and Deployment serv...

Re: Extract fields from multiple events based off ...

Re: How to check if an HEC is up or not before pos...

Are you a member of the Splunk Community?