About rogueraider

rogueraider · ‎08-23-2020

I finally found a solution! The key problem I was not grouping the results correctly into the transaction event. To do this I created the `car_group` variable to make the transaction enclose the other events. Once this common grouping had been created I was able to carry out `stats` as normal. @Richfez thanks for putting me on the right path. (index=first ON_TASK OR OFF_TASK) OR (index=second source=stream:netflow src_ip=172.16* OR dest_ip=172.16*) | rex "task\s(?<task_id>[0-9]*)" | rex "reg\s(?<car_rego>ABC-XX[CDEF])" | rename sum(bytes_in) as data_in | eval car_group=case( cidrmatch("172.16.1.1/32", src_ip) OR cidrmatch("172.16.1.1/32", dest_ip) OR car_rego="ABC-XXD", "ABC-XXD", cidrmatch("172.16.1.2/32", src_ip) OR cidrmatch("172.16.1.2/32", dest_ip) OR car_rego="ABC-XXF", "ABC-XXF", cidrmatch("172.16.1.3/32", src_ip) OR cidrmatch("172.16.1.3/32", dest_ip) OR car_rego="ABC-XXC", "ABC-XXC", cidrmatch("172.16.1.4/32", src_ip) OR cidrmatch("172.16.1.4/32", dest_ip) OR car_rego="ABC-XXE", "ABC-XXE") | transaction car_group task_id startswith="ON_TASK" endswith="OFF_TASK" maxevents=-1 maxspan=4h | eval finishtime=strftime(duration + _time, "%F %H:%M:%S") | eval starttime=strftime(_time, "%F %H:%M:%S") | eval tasked_time=tostring(duration, "duration") | stats sum(data_in) as bytes_transferred by car_group task_id starttime finishtime tasked_time | eval megabytes = bytes_transferred * 0.000001 | table car_group task_id starttime finishtime tasked_time megabytes bytes_transferred

rogueraider · ‎08-21-2020

Thanks for responding @Richfez I took your approach because it seems to have much more potential than my approach of many searches. (index=lighthouse ON_TASK OR OFF_TASK) OR (index=main source=stream:netflow) | rex "task\s(?<task_id>[0-9]*)" | rex "reg\s(?<car_rego>ABC-XX[CDEF])" | rex "(?<task_mode>O(N|FF)_TASK)" | rename sum(bytes_in) as data_in | transaction maxspan=4h keeporphans=1 car_rego, task_id startswith="ON_TASK" endswith="OFF_TASK" | eval finishtime=duration + _time | where aircraft_rego="VH-XNE" | table _time car_rego task_id data_in finishtime The key changes: `rename sum(bytes_in) as data_in` The field `sum(bytes_in)` is renamed to `data_in` because it's automatically generated by the netflow stream and causes problems when trying to work with it later. `keeporphans=1` This was an important discovery because if it is not set I only get the transaction events returned. If the netflow results are not available, the `data_in` field disappears. Now I get all the relevant results listed and populating into a table like so. _time car_rego task_id data_in finishtime 2020-08-21 01:00:01 ABC-XXC 1111 020-08-21 01:00:59 2020-08-21 01:00:02 50 2020-08-21 02:00:01 50 2020-08-21 05:00:01 ABC-XXC 1112 020-08-21 05:30:00 2020-08-21 05:00:02 50 2020-08-21 05:00:03 50 2020-08-21 05:00:06 50 There are obviously blank cells because the supporting events don't have those fields associated with them. What I want to get to is this. Note that the `finishtime` field is used to exclude some fields in the summing process. _time car_rego task_id data_total finishtime 2020-08-21 01:00:01 ABC-XXC 1111 50 020-08-21 01:00:59 2020-08-21 05:00:01 ABC-XXC 1112 150 020-08-21 05:30:00 Do you have any pointers on how to achieve this?

rogueraider · ‎08-19-2020

Goal: To get a table summing the amount of data transferred between specified time ranges based on a transaction. Sample Data: 12:00 01/01/2020 Task 1020 Started 13:00 01/01/2020 Task 1020 Finished 14:00 02/01/2020 Task 3020 Started 15:00 02/01/2020 Task 3020 Finished 12:00 01/01/2020 Data Sent 50 12:01 01/01/2020 Data Sent 50 12:02 01/01/2020 Data Sent 50 14:10 02/01/2020 Data Sent 50 14:11 02/01/2020 Data Sent 50 14:12 02/01/2020 Data Sent 50 Desired Outcome: Task 1020 Start 12:00 01/01/2020 Finish 13:00 01/01/2020 Data Sent 150 Task 3020 Start 14:00 02/01/2020 Finish 15:00 02/01/2020 Data Sent 150 Approach: This question requires two searches to be carried out. One to find the task start and finish times and a second to find and sum the data that was sent between those times. I am really struggling with the nested search aspect of this. I can get a transaction search to produce the start and finish times quite easily, but I don't know how to feed those results in to a second search to calculate the amount of data sent. Best Search index=main "Started" OR "Finished" | rex "Task\s(?<task_id>[0-9]*)" | transaction task_id startswith="Started" endswith="Finished" | eval latest = _time + duration | eval earliest = _time | fields earliest latest | format "multisearch " "[ search source=stream:netflow" "" "| stats sum(DataSent) as bytes_transferred ]" "" "" This produces a bunch of subsearches that run a single search for each task time range My problem is that I don't know to pass the identifying "task_id" value into the subsearches. My initial thought was to use something like "eval task_id=task_id" but because of the limits on the format command, I can't specify which fields to appear where. I would appreciate anyone who could help with any ideas on how to approach this problem.

rogueraider · ‎09-18-2019

I have looked at these and have used the examples to build my own maps, and thanks to this app I am getting some really exciting results!! However I don't understand how to set groups of markers to be rendered with paths, some to be rendered with markers and some to be rendered with heatmap. The examples show how you can turn on clusering, heatmap and paths on all points globally. However, I want to be able to have some marker groups only rendered as heatmap, some only rendered as clusters and some only rendered as paths. So far as I can see, heatmap and paths can only be globally enabled for all tracks. Is there maybe a table field that I can set that tells maps how to render it? Similar to the markerVisible field? I appreciate your help, it is cool to see this project coming together.

rogueraider · ‎09-17-2019

Is it possible with this app to have some of the markers rendered as paths and others are rendered as just markers? My data contains static features, cell phone towers, and moving features, cars. Ideally I would like to show a line path showing the car's history, and individual markers showing the phone towers.

rogueraider · ‎08-22-2019

Does anyone know if the latest version of this app (3.1.1) is eventually going to be available for splunk cloud? I am keen to use the playback feature that has been included in the more recent versions. From what I can see version 3.0.2 is the current version available for splunk cloud.

rogueraider · ‎08-15-2019

I found a trap with the playback feature that took me a day to figure out. Posting here to help others. Problem My data was rendering the track correctly, the playback markers where appearing, playback timer and slider were responding to the Start and Stop button, but the markers would not move. Solution My table of data was ordered from youngest to oldest which is how splunk search works by default. By adding | reverse to my search the data was ordered from oldest to youngest. After that the markers moved 🙂

Posts	7
Solutions	0
Karma Given	3
Karma Received	0
Member Since	‎08-15-2019

Online Status	Offline
Date Last Visited	‎08-24-2020 02:38 AM

Search combine transaction with summing

Mix Path and Cluster Maps

Splunk Cloud Version Support

Playback Markers Not Moving

Re: Search combine transaction with summing

Re: Search combine transaction with summing

Search combine transaction with summing

Re: Mix Path and Cluster Maps

Mix Path and Cluster Maps

Splunk Cloud Version Support

Playback Markers Not Moving