About yshen

yshen · ‎04-07-2022

I also note that with Splunk SDK (Python), at the end of the embedded query, using 'fields' to select the returned fields, it does not work as I desired with all fields returned. But 'table' would result in only the listed fields returned.

yshen · ‎04-07-2022

With hint by https://splunk-usergroups.slack.com/team/UB5DA9L02, it turns out that as the sourcetype is only known in the context of my application ics_analytics, in the service definition with SDK, I must indicate the application context with app= argument. Here is the corrected service definition: service = client.connect( host= 'splunk.bart.gov', app='ics_analysis', port = '8089', username = 'userid', password = 'secrete', ) once the sourcetype is properly declared to be known, the same code as above would be able to retrieve the field value of ENTRY. Here is the link to the relevant documentation: https://docs.splunk.com/DocumentationStatic/PythonSDK/1.6.16/client.html#splunklib.client.Service This is post is a capture of Slack discussion: https://splunk-usergroups.slack.com/archives/C04DC8JJ6/p1649351828984919?thread_ts=1649265592.685629&cid=C04DC8JJ6

yshen · ‎04-07-2022

When I used the following code to perform a query: service = client.connect( host= 'splunk.bart.gov', port = '8089', username = 'userid', password = 'secrete', ) query = "search index=slog_ics sourcetype=occ_mgr | table _time, ENTRY | head 3" query_results = service.jobs.oneshot(query) reader = res.ResultsReader(query_results) results = [] for item in reader: print(item) results.append(item) print("results[1]:") print(results[1]) In the above result, I cannot see the value for the field ENTRY. ENTRY is a field defined by the sourcetype occ_mgr in my application ics_analytics. While in Splunk web UI, in the context of the application ics_analytics using the same query, I can see the field value of ENTRY: index=slog_ics sourcetype=occ_mgr | fields _time, ENTRY | head 3 with the result: _time ENTRY 4/6/22 2:11:00.000 AM EOR. 4/6/22 1:48:00.000 AM (ref 0120) T203 released ATO, (762) second delay. 4/6/22 1:36:00.000 AM CORE Blanket established. What could be the root cause of the problem?

yshen · ‎02-12-2022

@gcusello Clever idea! I tried your suggestion. Here is the outcome: I use https://splunk.bart.gov/en-US/static/app/ics_analysis/../../weatherAssets/lowTemp.png it is translated by the browser (Chrome) or Splunk server as: https://splunk.bart.gov/en-US/static/weatherAssets/lowTemp.png again failed. I tried: https://splunk.bart.gov/en-US/static/app/ics_analysis/../weatherAssets/lowTemp.png it is converted as https://splunk.bart.gov/en-US/static/app/weatherAssets/lowTemp.png It looks that the browser performed the translation of the /.. literally before reaching the web server. Yes, I may try to manually upload the asset individually as last resort. Indeed, the manual approach works! Here is the proof: At https://splunk.bart.gov/en-US/static/app/ics_analysis/lowTemp.png The indented image is shown. Furthermore, the following also works in the dashboard: | eval coldImg = "/static/app/ics_analysis/lowTemp.png" Disappointed with Splunk's implementation, I guess, this is the best I can do. (Why the disappointment, we're working with a big app, there are 3 to 5 engineers improving it, if I have to manually add about a dozen icons to apps/ics_analysis/appserver/static, suffering from the unnecessary repetition. That directory will be a big soup of mess, that no one can understand the files left there completely.) Thanks again!

yshen · ‎02-11-2022

@gcusello Thank for your further help! From the information that you shared me, it seems that the image MUST be at /opt/splunk/etc/apps/ics_analysis/appserver/static/lowTemp.png ? I wonder if it will work as follows: /opt/splunk/etc/apps/ics_analysis/appserver/static/weatherAssets/lowTemp.png as the application is large. I hope to have better separation within. But even more unfortunate, the administrator is not responsive to help me at the moment. Therefore, I'm very interested in your mentioned "absolute path". I wish that it would refer and access to my image at its current, non-standard location. But I could not find any information of how to express the absolute path in the Simple XML. Could you elaborate it? Thanks again!

yshen · ‎02-11-2022

@gcusello thanks for helping! As I don't have the write access to the Splunk server, I had to ask the admin to place the folder weatherAssets for me. Now, it has been placed at /opt/splunk/etc/apps/ics_analysis/weatherAssets/lowTemp.png I wonder if there is a way to use the above path somehow still make the image accessible to the dashboard? Or do I have to place the image at /opt/splunk/etc/apps/ics_analysis/appserver/static/lowTemp.png to make it work? Where is the documentation for the rule of placing image? Thanks again!

yshen · ‎02-10-2022

I use the following to define an icon, to display on my dashboard: eval coldImg = "/weatherAssets/apps/ics_analysis/lowTemp.png" in the Simple XML for the dashboard. Here is the path for the image: /opt/splunk/etc/apps/ics_analysis/weatherAssets/lowTemp.pngwhere ics_analysis is the name of the app and weatherAssets is the folder for the icons. It used to display, when I had the following: eval coldImg = "https://image.flaticon.com/icons/png/512/1312/1312331.png" but now it only shows a broken image icon. What could be wrong? How can I debug the problem? It's frustrating that I don't know how to find out the error message to the issue. Do I have to restart the Splunk server, or bump my dashboard? (I just did reload the web page.) Thanks for your help!

yshen · ‎01-28-2022

Thanks for the solution! Here is a minimum working example: <form> <label>Select between Current and Forecast Weather</label> <fieldset submitButton="false"> <input type="radio" id="forecast" token="forecastToken" searchWhenChanged="true"> <label></label> <choice value="0">Current</choice> <choice value="1">Forecast</choice> <default>0</default> <change> <condition value="0"> <set token="showCurrent">1</set> <unset token="showForecast"></unset> </condition> <condition value="1"> <set token="showForecast">1</set> <unset token="showCurrent"></unset> </condition> </change> <initialValue>0</initialValue> </input> </fieldset> <row> <panel depends="$showCurrent$"> <single> <title>Weather</title> <search depends="$showCurrent$"> <query>| makeresults | eval weather="Current"</query> <earliest>-7d@h</earliest> <latest>now</latest> </search> <option name="drilldown">none</option> </single> </panel> <panel depends="$showForecast$"> <single> <title>Weather</title> <search depends="$showForecast$"> <query>| makeresults | eval weather="Forecast"</query> <earliest>-7d@h</earliest> <latest>now</latest> </search> <option name="drilldown">none</option> </single> </panel> </row> </form>

yshen · ‎01-28-2022

As follows if <condition-based-on-token-value> then query_1 else query_2where query_1 and query_2 may be a series of statements producing different sets of data. Given a search query can be embedded in a panel, it might achieve the equivalent results, if there were a way to have conditional panel selection: if <condition-based-on-token-value> then panel_with_query_1 else panel_with_query_2 (The above idea is inspired by @mmccul ) but I don't know if it's possible to have such panel selection mechanism with Simple XML of Splunk?Or alternatively, if I could control the visibility of a panel based token value, then I might also achieve the panel selection mechanism: define two panels with visibility control by the token value the controls are mutually exclusive, so that only one panel will be shown I'd appreciate some pointers or examples. (edited)

yshen · ‎08-13-2021

For the record, I ran into a case, when I run into the error message. It was when I needed to replace an existing csv file for lookup. I forgot to delete the existing one, and hoping the new file will override the existing one. It turned out that Splunk just complained without clearer indication of my offense. It would have been more helpful with more concrete error diagnose.

yshen · ‎06-24-2021

I want to compute the change in temperature for each location in a given interval, say, 15 minutes, or 30 minutes. I figure that streamstats might capture the temperature value at the beginning of such time interval, using time_window to specify the interval length. But, however, the following example surprises me. The temperature readings for Pleasonton are collected every 15 minutes, thus the following query: | makeresults | eval _raw="time_ Location Temperature 2021-08-23T03:04:05.000-0700 Pleasonton 185 2021-08-23T03:04:20.000-0700 Pleasonton 86 2021-08-23T03:04:35.000-0700 Pleasonton 87 2021-08-23T03:04:50.000-0700 Pleasonton 89" | multikv forceheader=1 | eval _time=strptime(time_,"%Y-%m-%dT%H:%M:%S.%3N%z") | fields _time Location Temperature | sort _time | streamstats earliest(Temperature) as previous_temp earliest(_time) as previous_time by Location time_window=5m | convert ctime(previous_time) I’d expect the following, as with the interval 5 minutes from an event, there is no other event, but the current one. _time Location Temperature _raw previous_temp previous_time 2021-08-23 03:04:05 Pleasonton 185 2021-08-23T03:04:05.000-0700 Pleasonton 185 185 08/23/2021 03:04:05.000000 2021-08-23 03:04:20 Pleasonton 86 2021-08-23T03:04:20.000-0700 Pleasonton 86 86 08/23/2021 03:04:20.000000 2021-08-23 03:04:35 Pleasonton 87 2021-08-23T03:04:35.000-0700 Pleasonton 87 87 08/23/2021 03:04:35.000000 2021-08-23 03:04:50 Pleasonton 89 2021-08-23T03:04:50.000-0700 Pleasonton 89 89 08/23/2021 03:04:50.000000 but this is actually what I get: _time Location Temperature _raw previous_temp previous_time 2021-08-23 03:04:05 Pleasonton 185 2021-08-23T03:04:05.000-0700 Pleasonton 185 185 08/23/2021 03:04:05.000000 2021-08-23 03:04:20 Pleasonton 86 2021-08-23T03:04:20.000-0700 Pleasonton 86 185 08/23/2021 03:04:05.000000 2021-08-23 03:04:35 Pleasonton 87 2021-08-23T03:04:35.000-0700 Pleasonton 87 185 08/23/2021 03:04:05.000000 2021-08-23 03:04:50 Pleasonton 89 2021-08-23T03:04:50.000-0700 Pleasonton 89 185 08/23/2021 03:04:05.000000 All taking the earliest event's temperature, which is beyond 5 minutes from any subsequent events.How can I query to get the temperature at the beginning of the time period?

yshen · ‎04-23-2021

I've understood the example of how to display an icon and how to make it disappear. Now I need to figure out how to implement the drill-down on the icon displayed. Any suggestion would be appreciated. The more I study it, the more I feel that this is a generic requirement of placing single value visualization on a schematic diagram against a feature value, at the position specific to the feature value. Therefore, I raised an "idea" here: https://ideas.splunk.com/ideas/EID-I-945 Please review, comment, and support, if you agree. Thanks!

yshen · ‎04-21-2021

I've studied service_now.xml which is a possible example for my requirements of dynamically overlaying icons. I understand that using CSS I control the location of the overlay over a background picture displayed by HTML. Next, I guess that I need to use JavaScript to update the related CSS style for the overlay to make it visible or not depending on the value of query output. I guess that I might be able to assign the query output to a token, and JavaScript code might be able to access the assigned value in a token. I should be able to use jQuery to retrieve the CSS style and update it. Now, the last question, how does my JavaScript code integrated with my dashboard definition in service_now.xml? I'm studying the example of appserver/static/scripts/advanced_model.js it seems that it's not mentioned in any of the dashboard definition. Is it by convention that any JavaScript code in the appserver/static/scripts/ would automatically integrate with app? Furthermore, what would be triggering event for my new function to be called to determine the visibility of my icon? It's not like a user event (mouse click on some UI object), but just data update. Thanks for your help with some pointer!

yshen · ‎04-19-2021

Thanks! I've able to locate the relevant file: $SPLUNK/etc/apps/itsy_bitsy_app default/data/ui/views/service_flow.xml :: this is the definition of the dashboard. and I'm isolating the row element in the xml that contains the html code to overlay the icons.

yshen · ‎04-19-2021

@bowesmana Thanks for the pointer! The "Service Flow" seems very close to what I need. Next, could you give me a pointer of a tutorial how to set up an app with the customization like Service Flow? Thanks again!

yshen · ‎04-18-2021

For example, on a railroad schematic diagram, based on query data output? By “dynamically”, I’d like to show an icon (of alarm) at a position when there is a data value exceeding a threshold corresponding to the position on the schematic diagram. If the corresponding value is below the threshold, then, no icon should be shown. Essentially, I want to show an alarm at the that position of the diagram. It would be similar to an example of the room occupancy with “Splunk Dashboards (beta)” (app/splunk-dashboard-app/example-hub-workplace-readiness-detail), except that I only want to show those with the occupancy higher than a threshold, not those rooms with occupancy less than the threshold. For the static icon display, it is similar to Splunk’s visualization of “choropleth map”, but the coordinates have no geolocation bearing. Rather I just want to place icons of my choosing on arbitrary coordinates on the schematic diagram, based on the data value whether exceeding a threshold. I have explored to use “Splunk Dashboards (beta)”, but it does not permit to dynamically display and hide such alarm. I wonder what would be a possible approach? I heard that Splunk’s custom renderer might be able Or, I should just export the data from Splunk to do the special visualization external to Splunk? Is there any example of “custom renderer” overlaying visualization of query data on top of a schematic diagram, with dynamic behavior of showing and hiding? Some pointers and examples would be appreciated. Thanks in advance!

yshen · ‎03-20-2021

@gcusello Thanks for the pointer. My problem is a real one that I need for some field troubleshooting. Maybe, my framing of my problem could be simplified. But it's what I can do at the moment. Maybe, once I know the solution, then the problem statement might be simplified. The extraction of the fields of DEVICE, ATTRIBUTE, etc. has already been done in the respective sourcetype. Your suggestion of using transaction with startwith and endwith might work sometimes. But I eventually need to identify those event groups that have subsets of the expected events. For example, Certain group may only has event of "SOR_RESTRICT_STATUS.STATE" so the transaciton may need to startwith "SOR_RESTRICT_STATUS.STATE" as well. Likewise, some other subset may only have SOR_A_STATUS.STATE OR SOR_B_STATUS.STATE so the transaction may need also to endwith SOR_A_STATUS.STATE OR SOR_B_STATUS.STATE by this school of thought, the eventual transaction definition would be like: transaction DEVICE startswith="SOR_A_STATUS.STATE OR SOR_B_STATUS.STATE OR SOR_RESTRICT_STATUS.STATE" endswith="SOR_A_STATUS.STATE OR SOR_B_STATUS.STATE OR SOR_RESTRICT_STATUS.STATE" I have tried how it would up with the same condition for startwith and endwith I am concerned that it would lose the expected differentiation for the transactions. Maybe a question to experts.

yshen · ‎03-19-2021

Here is the data for illustration: (To facilitate experiment, I provide below the query snippet to recreate the data in Splunk query.) | makeresults | eval _raw= "Date Time DEVICE ATTRIBUTE STATE TagExpected 2021-03-19 11:56:22.449 K30 SOR_B_STATUS.STATE SOR_FAILED 1 2021-03-19 12:16:17.564 K30 SOR_A_STATUS.STATE SOR_FAILED 1 2021-03-19 12:17:55.191 K30 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_ON 1 2021-03-19 12:21:16.659 K30 SOR_A_STATUS.STATE SOR_FAILED 2 2021-03-19 12:32:42.247 K30 SOR_B_STATUS.STATE SOR_FAILED 2 2021-03-19 12:51:21.456 A60 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_ON 2 2021-03-19 12:51:52.949 A60 SOR_A_STATUS.STATE SOR_FAILED 1 2021-03-19 12:54:01.077 A60 SOR_B_STATUS.STATE SOR_FAILED 1 2021-03-19 13:01:26.367 A60 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_ON 1 2021-03-19 13:01:26.818 K30 SOR_A_STATUS.STATE SOR_FAILED 3 2021-03-19 13:02:41.142 K30 SOR_B_STATUS.STATE SOR_FAILED 3 2021-03-19 13:08:19.694 A60 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_ON 2 2021-03-19 13:09:14.433 K30 SOR_B_STATUS.STATE SOR_FAILED 4 2021-03-19 13:10:19.149 W34 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_OFF 1 2021-03-19 13:16:12.847 A60 SOR_B_STATUS.STATE SOR_FAILED 3 2021-03-19 13:24:59.420 A60 SOR_A_STATUS.STATE SOR_FAILED 3 2021-03-19 13:24:59.870 A60 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_ON 3 2021-03-19 13:25:48.068 A60 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_OFF 2021-03-19 13:35:47.614 A60 SOR_A_STATUS.STATE SOR_FAILED 4 2021-03-19 13:38:19.632 A90 SOR_B_STATUS.STATE SOR_FAILED 1 2021-03-19 13:46:10.118 R20 SOR_B_STATUS.STATE SOR_FAILED 1 2021-03-19 13:50:30.328 R50 SOR_A_STATUS.STATE SOR_FAILED 1 2021-03-19 13:54:58.831 W20 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_ON 1 2021-03-19 13:55:30.622 W20 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_OFF 2021-03-19 13:56:38.060 A60 SOR_A_STATUS.STATE SOR_FAILED 5 2021-03-19 14:02:19.102 K30 SOR_B_STATUS.STATE SOR_FAILED 5 2021-03-19 14:08:51.212 R50 SOR_A_STATUS.STATE SOR_FAILED 2 2021-03-19 14:09:47.657 R20 SOR_B_STATUS.STATE SOR_FAILED 2 2021-03-19 14:11:10.387 C30 SOR_B_STATUS.STATE SOR_FAILED 1 2021-03-19 15:01:15.315 C30 SOR_B_STATUS.STATE SOR_FAILED 2 2021-03-19 15:02:33.670 R65 SOR_A_STATUS.STATE SOR_FAILED 1 2021-03-19 15:06:56.258 C50 SOR_B_STATUS.STATE SOR_FAILED 1 2021-03-19 15:09:32.583 R50 SOR_A_STATUS.STATE SOR_FAILED 3 2021-03-19 15:09:33.484 R50 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_ON 3 2021-03-19 15:09:40.240 R50 SOR_RESTRICT_STATUS.STATE SOR_SPEED_RESTRICT_OFF 2021-03-19 15:36:17.104 A90 SOR_B_STATUS.STATE SOR_FAILED 1" | multikv forceheader=1 | eval combined=Date." ".Time | eval _time=strptime('combined', "%Y-%m-%d %H:%M:%S.%Q") | fields _time DEVICE ATTRIBUTE STATE TagExpected In the above records, I’d like to tag or group the events by the following rules: For a device value, e.g. K30, I’d like to tag the events with DEVICE value K30 with first occurrences of SOR_B_STATUS.STATE or SOR_A_STATUS.STATE (their occurrences do not matter) and SOR_RESTRICT_STATUS.STATE after with the same tag value, to be one group. For the same DEVICE value, next occurrences of SOR_B_STATUS.STATE or SOR_A_STATUS.STATE (their occurrences do not matter) and SOR_RESTRICT_STATUS.STATE after I’d group them to be tag 2, to be a different group, etc. Subset of such event group will also be considered to be separate group. The events in a group does not need to be adjacent in time. I don’t insist on the tag value, any mechanism to group the events would be fine. I feel that using streamstats or transaction in some fashion might solve the problem. But I need help to wrap around my brain to work it help. Really appreciate that you could give me some help or hint. Thanks in advance, and have nice weekends!

yshen · ‎02-23-2021

It took me at least 5 hours of experiment to make sure that I'm not making a mistake. It's really frustrated and disappointed by such mystery and arbitrary behavior without any hint of error message! A great company can do better. I wish customers should not always be treated as a victim of hidden secrete! So the moral of the story, a base search must explicitly specify all the fields that will be used by the searches using the base search by command fields or table.

yshen · ‎01-15-2021

Amazing! Powerful, and sophisticated! Thanks a million!

Posts	46
Solutions	1
Karma Given	41
Karma Received	3
Member Since	‎06-18-2020

Online Status	Offline
Date Last Visited	‎04-07-2022 02:46 PM

Fields defined by a sourcetype not being shown in ...

How to make local image accessible to dashboard?

If it is possible to have query/panel with conditi...

How to compute the field value a time interval ear...

How can I dynamically overlay graphic objects (ico...

How to tag or group events by multiple possible va...

How to create a list of literal values of strings ...

Can I use free Splunk instance to run JavaScript c...

How to do compound query with where clause?

Adding new sourcetype and input result in no effec...

Re: Fields vs table vs nothing?

Re: Fields defined by a sourcetype not being shown...

Fields defined by a sourcetype not being shown in ...

Re: How to make local image accessible to dashboar...

Re: How to make local image accessible to dashboar...

Re: How to make local image accessible to dashboar...

How to make local image accessible to dashboard?

Re: If it is possible to have query/panel with con...

If it is possible to have query/panel with conditi...

Re: Splunk could not update permissions for resour...

How to compute the field value a time interval ear...

Re: How can I dynamically overlay graphic objects ...

Re: How can I dynamically overlay graphic objects ...

Re: How can I dynamically overlay graphic objects ...

Re: How can I dynamically overlay graphic objects ...

How can I dynamically overlay graphic objects (ico...

Re: How to tag or group events by multiple possibl...

How to tag or group events by multiple possible va...

Re: Dashboard Base Search is not working for all p...

Re: How to create a list of literal values of stri...