About andrewtrobec

andrewtrobec · ‎03-08-2017

Thanks @woodcock I've been experimenting but I can't get it to work. I've added PREAMBLE_REGEX = ^Job\.Description.*|String.* (which works on https://regex101.com/) and HEADER_FIELD_LINE_NUMBER = 1 but it doesn't seem to be working. I am performing a manual import, selecting MY_SOURCETYPE which is defined in my props.conf as follows: [MY_SOURCETYPE] AUTO_KV_JSON = 1 DATETIME_CONFIG = FIELD_DELIMITER = , HEADER_FIELD_LINE_NUMBER = 1 INDEXED_EXTRACTIONS = csv KV_MODE = none NO_BINARY_CHECK = true PREAMBLE_REGEX = ^Job\.Description.*|String.* SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = RunStart category = Structured description = Comma-separated value format. Set header and other settings in "Delimited Settings" disabled = false pulldown_type = true Are there any other configurations that I should be aware of? Best regards, Andrew

andrewtrobec · ‎03-07-2017

Thanks, I'll take a look. One doubt: will this allow me to read the first line as the headers and only ignore the second and third lines?

andrewtrobec · ‎03-07-2017

Hello, I am trying to index a csv log file that looks like this: Description,NumJobWaitEvents,ReturnCode,RunEnd,RunStart,ScheduledStartTime,Status Job.Description,Job.NumJobWaitEvents,Job.ReturnCode,Job.RunEnd,Job.RunStart,Job.ScheduledStartTime,Job.Status String,Integer,Integer,DateTime,DateTime,DateTime,enum.JobStatus Auto Start,0,null,"2017/03/05 06:03:39,441","2017/03/05 06:01:39,269","2017/03/05 06:01:39,065",Completed Auto Start,0,null,"2017/03/05 06:09:04,493","2017/03/05 06:06:23,915","2017/03/05 06:06:23,743",Completed AG43_542_TINA_CODE_AGB - Checking,1,null,"2017/03/05 06:32:18,908","2017/03/05 06:23:15,148","2017/03/05 06:23:14,822",Completed DATA SANITY CHECK,0,null,"2017/03/05 09:02:23,997","2017/03/05 09:00:44,073","2017/03/05 09:00:42,959",Completed The first line always contains the header, the second and third lines always contain object and type information, and the log data always starts from the fourth line. When I index the file as it is, it only indexes the first two lines even though there are thousands. My question is: how can I skip the second and third lines so I can index the actual log data? Thank you and best regards, Andrew

andrewtrobec · ‎02-28-2017

Thanks for the follow-up. I have found a working solution and updated with the information. Thank you for your support!

andrewtrobec · ‎02-28-2017

In one of my other questions I was given the suggestion by @rjthibod to work with the chart and append functions (I had since found a solution using join which I was told isn't the most efficient way of doing things). The following code works perfectly: index=my_index | chart count as EnterCount over _time span=1d | append [ search index=my_index | eval _time = strptime(Exit, "%Y-%m-%d %H:%M:%S") | chart count as ExitCount over _time span=1d ] ] | timechart span=1d latest(*) as * | fillnull value=0 Thanks to @DalJeanis @woodcock and @niketnilay for their valuable input, I very much appreciate it.

andrewtrobec · ‎02-28-2017

Thank you so much, this works perfectly! I have one additional query: I see that tstats is fast for index-time fields and works in this particular case. What is the best approach, though, if the appended search has to apply filters and perform more advanced operations? Here is a similar scenario: index=my_index | search Category="A") | timechart span=1d count as EnterCount | join type=left _time [ search index=my_index | search (Category="A") | eval _time = strptime(Exit, "%Y-%m-%d %H:%M:%S") | timechart span=1d count as ExitCount ] | fillnull value=0 How would the join be replaced with append in this case? Best regards, Andrew

andrewtrobec · ‎02-28-2017

This works, I had to add an eval to set _time to Closed , but the structure is perfect. In the meantime I found another way of doing it: index=my_index | eval _time = strptime(Closed, "%Y-%m-%d %H:%M:%S") | timechart span=1w count by Category | join type=left _time [ search index=my_index | timechart span=1w count as inbound ] (all I did was replace appendcols with a join type=left _time ) In my solution the join type means that the leftmost search will determine which values to show, whereas yours will show instances of the rightmost search where there is no match with the leftmost search. Thank you very much! Regards, Andrew

andrewtrobec · ‎02-28-2017

Thanks for the response! I need appendcols because I am putting two different metrics into the same table. Since they both have the same range I can easily do this, but for some reason they are being matched incorrectly. Regards, Andrew

andrewtrobec · ‎02-28-2017

Hello all, I have an index of events, each of which has an enter and exit timestamp where _time is associated to the former. My first search is very simple and shows the enter daily frequency: index=my_index | timechart span=1d count as EnterCount This produces the following table: _time,EnterCount 2016-08-08,0 2016-08-09,16 2016-08-10,20 2016-08-11,23 2016-08-12,13 2016-08-13,2 2016-08-14,0 2016-08-15,0 2016-08-16,11 I then have another search that shows the exit daily frequency: index=my_index | eval _time = strptime(Exit, "%Y-%m-%d %H:%M:%S") | timechart span=1d count as ExitCount This produces the following table: _time,ExitCount 2016-08-08,1 2016-08-09,4 2016-08-10,9 2016-08-11,16 2016-08-12,4 2016-08-13,0 2016-08-14,0 2016-08-15,0 2016-08-16,4 In order to chart this relationship between enter and exit , I use appendcols to put them together: index=my_index | timechart span=1d count as EnterCount | appendcols [ search index=my_index | eval _time = strptime(Exit, "%Y-%m-%d %H:%M:%S") | timechart span=1d count as ExitCount ] The resulting table, however, displays incorrect values for ExitCount by bringing the results forward by 3 days: _time,EnterCount,ExitCount 2016-08-08,0,16 2016-08-09,16,4 2016-08-10,20,0 2016-08-11,23,0 2016-08-12,13,0 2016-08-13,2,4 2016-08-14,0,10 2016-08-15,0,13 2016-08-16,11,9 Why does this happen, and what can I do to ensure that the correct association is being made? Thank you and best regards, Andrew

andrewtrobec · ‎02-27-2017

Hello and thanks for the suggestion. In my case I am tracking items that have an open date and a closed date, with _time set to the open date. What I want to do is show how many items are closed per week while also showing how many items are opened. The appendcols command seems ideal, and I can avoid calling an external script to process the data. Unless there is another way? Best regards, Andrew

andrewtrobec · ‎02-27-2017

Thanks for the response! Here is the SPL: index=my_index | eval _time = strptime(Closed, "%Y-%m-%d %H:%M:%S") | timechart span=1w count by Category | appendcols [ search index=my_index | timechart span=1w count as inbound ] This will generate a table that looks like this: _time,cat_a,cat_b,cat_c,cat_d,inbound 2016-08-04,0,3,1,2,1 2016-08-11,0,3,8,0,0 2016-08-18,0,22,10,12,36 2016-08-25,0,15,42,7,70 2016-09-01,0,39,56,12,137 2016-09-08,0,29,61,5,98 2016-09-15,0,29,65,7,242 2016-09-22,0,22,24,2,219 2016-09-29,0,16,13,2,228 2016-10-06,0,16,4,4,231 2016-10-13,0,6,6,2,256 2016-10-20,0,5,17,3,211 When formatting the chart, the first thing I notice is that when I try to make Inbound as an overlay, the list of fields does not prepopulate as it usually does (I have to type it all in manually): The final result is the following, but it is not interactive: I hope this helps to clarify the situation! Thank you and best regards, Andrew

andrewtrobec · ‎02-27-2017

Hello All, Currently using Splunk 6.5.1. As the question implies, I have a search that uses the appendcols command to add an additional column to a timechart making sure to use the same interval so it aligns correctly. I've added this new column as an overlay to the chart, but keeping it on the same axis. I've noticed that the interactive nature of the chart is lost so I cannot highlight any values. The only way to solve it is to remove the appendcols command. Is this a known issue, or is there a way to fix it? Thank you and best regards, Andrew

andrewtrobec · ‎02-22-2017

Thanks Martin, perfect response. Side question: do you know of any step-by-step tutorials on how to configure the new custom alert actions in the same way the "Run a script" feature works? Regards, Andrew

andrewtrobec · ‎02-22-2017

Hello, I am using Splunk 6.5.1 and I am working with alert trigger actions that run scripts. The documentation here states that this functionality has been officially deprecated. Does this mean that I should stop using it because it will disappear from the platform altogether? Regards, Andrew

andrewtrobec · ‎02-02-2017

Hello! I am using the bucket function to split my data into buckets of 2: bucket field_name span=2 The buckets that it creates are 0-2, 2-4, 4-6, 6-8, and so on where the upper value of a given bucket is the same as the lower value of the next bucket. The default logic for categorizing the value is >= lower value and < upper value. It also means that the value "2" will be put into the "2-4" bucket rather than the "0-2" bucket. What I would like to know is: is there a way to specify that whole numbers in my data that correspond to a bucket limit be sorted into the lower bucket? In effect making the logic > lower value and <= upper value. Thank you! Andrew

andrewtrobec · ‎12-14-2016

I agree that the multiselect provides these capabilities, but here are the issues I see: When I filter by typing a string, I can only select one of the results at a time. So if I have to filter 10 results that start with ABC, I have to type ABC and add them individually 10 times. The more values I add, the larger the box gets, so if I have more than 20, it starts to push everything downwards and generally looks ugly. I was hoping I could combine the best of the checkboxes and the best of the multiselect into a single input type. I guess this isn't possible. Thanks for your input!

andrewtrobec · ‎12-14-2016

Hello and thanks for the reply. This is an option yes, but when multiple values are selected, it displays them one after the other and creates the same usability issue. Also, values have to be selected one at a time and that's a pain. I suppose I don't have to use a checkbox, but I need something that will be more compact when a large number of values have been selected, as well as something that allows me to select multiple values easily. Regards, Andrew

andrewtrobec · ‎12-14-2016

Hello, I have a checkbox token that has 100 values. The usability problem is that the list is displayed it its entirety which pushes the visualization panels down, and makes the dashboard quite ugly. Is it possible to have the checkbox list be like a dropdown with a vertical scrollbar so that the user scrolls through and checks their desired values? Thank you and best regards, Andrew

andrewtrobec · ‎12-07-2016

I know this thread is old, but I'm trying to the same thing and am stuck. I've followed the instructions but something doesn't make sense to me. Here are my configurations: props.conf [mysourcetype] LOOKUP-sector = sectorlookup "Lookup Field" OUTPUT Sector transforms.conf [sectorlookup] filename = L_Sectors.csv match_type = WILDCARD("Lookup Field") L_Sectors.csv "Lookup Field","Sector" "A1-A2-A3*","Sector1" "B1-B2-B3-B4*","Sector2" "C1-C2-C3*","Sector3" "D1-D2-D3-D4-D5*","Sector4" My question is: how do I structure the lookup command? Right now I have * | lookup L_Sectors.csv "Lookup Field" OUTPUT Sector | table "Lookup Field", Sector but I'm not getting results. Am I doing the search correctly?

andrewtrobec · ‎12-07-2016

I've followed the instructions but something doesn't make sense to me. Here are my configurations: props.conf [mysourcetype] LOOKUP-sector = sectorlookup "Lookup Field" OUTPUT Sector transforms.conf [sectorlookup] filename = L_Sectors.csv match_type = WILDCARD("Lookup Field") L_Sectors.csv Lookup Field,Sector A1-A2-A3*,Sector1 B1-B2-B3-B4*,Sector2 C1-C2-C3*,Sector3 D1-D2-D3-D4-D5*,Sector4 My question is: how do I structure the lookup command? Right now I have * | lookup L_Sectors.csv "Lookup Field" OUTPUT Sector | table "Lookup Field", Sector but I'm not getting results. Am I doing the search correctly?

andrewtrobec · ‎12-07-2016

Hello sundareshr, thanks for the reply. The data above are the fields that I need to use for the lookup, source is in the events while target is in the lookup. To give a clearer example, ID_Field is what I need match in the lookup. The events looks like: ID_Field,S_Field1,S_Field2 A1-A2-A3-A4-A5,S_Field1Value1,S_Field2Value1 B1-B2-B3-B4-B5,S_Field1Value1,S_Field2Value2 C1-C2-C3-C4,S_Field1Value2,S_Field2Value3 D1-D2-D3-D4-D5-D6,S_Field1Value2,S_Field2Value2 While the lookup looks like: ID_Field,T_Field1 A1-A2-A3,T_Field1Value1 B1-B2-B3-B4,T_Field1Value2 C1-C2-C3,T_Field1Value3 D1-D2-D3-D4-D5,T_Field1Value4 I need to create: ID_Field,S_Field1,S_Field2,T_Field1 A1-A2-A3-A4-A5,S_Field1Value1,S_Field2Value1,T_Field1Value1 B1-B2-B3-B4-B5,S_Field1Value1,S_Field2Value2,T_Field1Value2 C1-C2-C3-C4,S_Field1Value2,S_Field2Value3,T_Field1Value3 D1-D2-D3-D4-D5-D6,S_Field1Value2,S_Field2Value2,T_Field1Value4 Under normal circumstances this would be easy, but in my case ID_Field in the lookup is the root of ID_Field in the events. Hope this helps to clarify! Regards, Andrew

andrewtrobec · ‎12-07-2016

Hello all, I have to use a lookup to get data but the problem is that the source field for the lookup is longer than the target field, but in all cases the target field is the root of the source field. Examples: Source,Target A1-A2-A3-A4-A5,A1-A2-A3 B1-B2-B3-B4-B5,B1-B2-B3-B4 C1-C2-C3-C4,C1-C2-C3 D1-D2-D3-D4-D5-D6,D1-D2-D3-D4-D5 In all cases the keywords are separated by - , but there is no fixed number of - to define the root. Basically I need to say something like "Where the target field is the root of the source field, return desired columns" Is this possible through the lookup command? Thank you and best regards, Andrew

andrewtrobec · ‎12-01-2016

Thanks for the answer. It's not quite what I'm looking for but it's given me something to work with. In reality the given_day should be the the _time value for a particular entry. What I'm looking for is: Given the _time of an entry, how many objects were opened on or before that time, and how many of those same objects have been closed on or before that time? Some pseudo-code: FOR EVERY ENTRY BEFORE _time INCLUDE IF [Opened] < [_time] INCLUDE IF [Closed] < [_time] What makes it difficult is that I have to be able to search through all previous data to make my calculation, and I am struggling to figure out how to do that. Regards, Andrew

andrewtrobec · ‎12-01-2016

Hello, I am trying to determine the number of entries that have a field date that is before or equal to _time . My data set is: _time, Opened, Closed 2016-06-01 00:20:21,2016-06-01 00:20:21,2016-06-02 02:18:23 2016-06-01 00:37:16,2016-06-01 00:37:16,2016-06-02 02:25:14 2016-06-01 01:12:07,2016-06-01 01:12:07,2016-06-03 02:44:41 2016-06-01 01:17:27,2016-06-01 01:17:27,2016-06-03 03:00:08 2016-06-02 14:01:28,2016-06-02 14:01:28,2016-06-03 02:02:50 2016-06-02 14:04:06,2016-06-02 14:04:06,2016-06-03 02:23:52 2016-06-02 14:08:53,2016-06-02 14:08:53,2016-06-04 02:05:21 2016-06-03 18:46:25,2016-06-03 18:46:25,2016-06-04 02:47:58 2016-06-03 18:47:18,2016-06-03 18:47:18,2016-06-04 02:39:18 (during import _time was created based on Opened) My goal is to get a table that will tell me for a given day, how many entries have an Opened value equal to or before that day along with the number of entries that have a Closed value equal to or before that day (but only for the entries that have already been counted due to their Opened value): _time,openedCount,closedCount 2016-06-01,4,0 2016-06-02,7,2 2016-06-03,9,4 I have tried as much as I can to use the join command, but I'm not having much luck. Right now I am experimenting with the append command to see whether I can achieve my goal. I'm opening this question in the meantime to see whether I can get any pointers... Thank you! Andrew

andrewtrobec · ‎11-29-2016

Hello, I don't think this will work since the "Name" and "Team" fields are part of the lookup, not the index. Also, would this not limit the final output to only the resources that have charged hours? Thank you for taking the time to consider my issue! Best regards, Andrew

Posts	336
Solutions	7
Karma Given	124
Karma Received	48
Member Since	‎11-09-2016

Online Status	Offline
Date Last Visited	‎08-04-2025 06:01 AM

When will emojis in dashboard dark mode work prope...

Why is my LINE_BREAKER parameter not breaking prop...

Why does user with "power" role only see the "Sear...

Why is a sourcetype not being applied to a summary...

Why does strptime not parse timestamps to nanoseco...

Can Azure AD and Microsoft Entra be configured tog...

How do I access the data in the maintenance_calend...

Why is _time in my JSON payload parsed correctly w...

What is the procedure in 2023 to get license expir...

Why is one indexed field only giving me a multival...

Re: Can I skip specific lines while indexing data?

Re: Can I skip specific lines while indexing data?

Can I skip specific lines while indexing data?

Re: Why does appendcols cause a search to produce ...

Re: Why does appendcols cause a search to produce ...

Re: Why does my chart lose interactivity with a se...

Re: Why does my chart lose interactivity with a se...

Re: Why does appendcols cause a search to produce ...

Why does appendcols cause a search to produce diff...

Re: Why does my chart lose interactivity with a se...

Re: Why does my chart lose interactivity with a se...

Why does my chart lose interactivity with a search...

Re: Will the "Run a script" trigger action for ale...

Will the "Run a script" trigger action for alerts ...

How do I assign whole numbers to the lower bucket ...

Re: Is it possible to add a vertical scrollbar to ...

Re: Is it possible to add a vertical scrollbar to ...

Is it possible to add a vertical scrollbar to my c...

Re: Can we use wildcard characters in a lookup tab...

Re: How can I use the lookup command where the tar...

Re: How can I use the lookup command where the tar...

How can I use the lookup command where the target ...

Re: How do I perform count logic on all entries fo...

How do I perform count logic on all entries for a ...

Re: How can I join a search to a lookup to see whe...

Are you a member of the Splunk Community?