I am running a single instance (i.e. everything on one box). I have updated the local props.conf again and seen no change in the indexed data.  Here is the current output from btool: C:\Program Files\Splunk\bin>splunk btool --debug props list aws:s3:csv C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf [aws:s3:csv] C:\Program Files\Splunk\etc\system\default\props.conf ADD_EXTRA_TIME_FIELDS = True C:\Program Files\Splunk\etc\system\default\props.conf ANNOTATE_PUNCT = True C:\Program Files\Splunk\etc\system\default\props.conf AUTO_KV_JSON = true C:\Program Files\Splunk\etc\system\default\props.conf BREAK_ONLY_BEFORE = C:\Program Files\Splunk\etc\system\default\props.conf BREAK_ONLY_BEFORE_DATE = True C:\Program Files\Splunk\etc\system\default\props.conf CHARSET = AUTO C:\Program Files\Splunk\etc\apps\Splunk_TA_aws\default\props.conf DATETIME_CONFIG = CURRENT C:\Program Files\Splunk\etc\system\default\props.conf DEPTH_LIMIT = 1000 C:\Program Files\Splunk\etc\system\default\props.conf DETERMINE_TIMESTAMP_DATE_WITH_SYSTEM_TIME = false C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf EVENT_BREAKER = [\r\n]+ C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf EVENT_BREAKER_ENABLE = true C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf FIELD_DELIMITER = \t C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf HEADER_FIELD_DELIMITER = \t C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf HEADER_FIELD_LINE_NUMBER = 1 C:\Program Files\Splunk\etc\system\default\props.conf HEADER_MODE = C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf INDEXED_EXTRACTIONS = TSV C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf KV_MODE = multi C:\Program Files\Splunk\etc\system\default\props.conf LB_CHUNK_BREAKER_TRUNCATE = 2000000 C:\Program Files\Splunk\etc\system\default\props.conf LEARN_MODEL = true C:\Program Files\Splunk\etc\system\default\props.conf LEARN_SOURCETYPE = true C:\Program Files\Splunk\etc\apps\Splunk_TA_aws\default\props.conf LINE_BREAKER = [\r\n]+ C:\Program Files\Splunk\etc\system\default\props.conf LINE_BREAKER_LOOKBEHIND = 100 C:\Program Files\Splunk\etc\system\default\props.conf MATCH_LIMIT = 100000 C:\Program Files\Splunk\etc\system\default\props.conf MAX_DAYS_AGO = 2000 C:\Program Files\Splunk\etc\system\default\props.conf MAX_DAYS_HENCE = 2 C:\Program Files\Splunk\etc\system\default\props.conf MAX_DIFF_SECS_AGO = 3600 C:\Program Files\Splunk\etc\system\default\props.conf MAX_DIFF_SECS_HENCE = 604800 C:\Program Files\Splunk\etc\system\default\props.conf MAX_EVENTS = 256 C:\Program Files\Splunk\etc\system\default\props.conf MAX_TIMESTAMP_LOOKAHEAD = 128 C:\Program Files\Splunk\etc\system\default\props.conf MUST_BREAK_AFTER = C:\Program Files\Splunk\etc\system\default\props.conf MUST_NOT_BREAK_AFTER = C:\Program Files\Splunk\etc\system\default\props.conf MUST_NOT_BREAK_BEFORE = C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION = indexing C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-all = full C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-inner = inner C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-outer = outer C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-raw = none C:\Program Files\Splunk\etc\system\default\props.conf SEGMENTATION-standard = standard C:\Program Files\Splunk\etc\apps\Splunk_TA_aws\default\props.conf SHOULD_LINEMERGE = false C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf TIMESTAMP_FIELDS = timestamp C:\Program Files\Splunk\etc\apps\new_app_for_s3_data\local\props.conf TIME_FORMAT = %s C:\Program Files\Splunk\etc\system\default\props.conf TRANSFORMS = C:\Program Files\Splunk\etc\apps\Splunk_TA_aws\default\props.conf TRUNCATE = 8388608 C:\Program Files\Splunk\etc\system\default\props.conf detect_trailing_nulls = auto C:\Program Files\Splunk\etc\system\default\props.conf maxDist = 100 C:\Program Files\Splunk\etc\system\default\props.conf priority = C:\Program Files\Splunk\etc\system\default\props.conf sourcetype = C:\Program Files\Splunk\etc\system\default\props.conf termFrequencyWeightedDist = false C:\Program Files\Splunk\etc\system\default\props.conf unarchive_cmd_start_mode = shell   ... View more

I should also mention that changing the sourcetype to anything other than aws:s3 or aws:s3:csv results in no data being indexed at all. ... View more

Here is the props.conf stanza from the TA's default directory that applies to the source type that is specified in the documentation: ########################### ### CSV ### ########################### [aws:s3:csv] DATETIME_CONFIG = CURRENT TIME_FORMAT = %Y-%m-%dT%H:%M:%S%Z SHOULD_LINEMERGE = false LINE_BREAKER = [\r\n]+ TRUNCATE = 8388608 EVENT_BREAKER_ENABLE = true EVENT_BREAKER = [\r\n]+ KV_MODE = json I tried adding a props.conf into the local directory for the TA but it seems to be ignored because the data ends up indexed exactly the same after adding the new file and then restarting Splunk. This is the contents of the local props.conf that I tried: [aws:s3:csv] TIME_FORMAT = %s HEADER_FIELD_LINE_NUMBER = 1 INDEXED_EXTRACTIONS = TSV TIMESTAMP_FIELDS = timestamp ... View more

You could try something like this: sourcetype="whatever" Guid="*" | eval time=_time | search [search sourcetype="whatever" NOT Guid="*" | eval time=strptime(substr(_raw,1,18)) | rename host AS HostName | fields time,HostName] Of course, this assumes that the stacktrace events will have the exact same time stamp as the typical log entry you are interested in. It also assumes all typical events have a value in Guid field and that none of the stacktrace events have the Guid field. ... View more

You could eval a new field that concatenates your 18 fields into one delimited string, use makemv on the new field, and then mvcount to create a flag for rows/records that only have a single value: your_current_search_here ... | eval concatField=Filename."|".user."|".service."|".error | makemv delim="|" concatField | eval count=mvcount(concatField) | where count>1 | fields - concatField,count I haven't tested this exact example but I've done similar things before with no problem. ... View more

You need to add the value of Counter into your second stats: | stats values(Counter) AS Counter avg(Counter) AS "Avg" stdev(Counter) AS "Stdev" by hrmarker,weekday Then you can add an eval and a where clause to see your desired results: | eval OverNorm=if(Counter>(Avg+Stdev),"yes","no") | where OverNorm="yes" Alternatively, you could just do the comparison in the where clause and not have a flag for where it's over or not. ... View more

First off, I'll use to indicate that character. If you see something like convertToMB(someField) , then someone has set up a search macro that converts the value of someField into megabytes. I don't believe there is any other use, in Splunk, for the character. ... View more

Go to Pivot>Manage Data Models> name of Data Model ( or Pivot> name of Data Model > Edit Objects ) then click the Edit link next to the field you want. The Attributes column will tell you generally how the field was created. Clicking Edit will give you the details of the eval, regex, rename/transform, lookup, etc that is being used to create that field. I believe your role would have to be explicitly given the ability to edit Data Models. If you don't see the buttons to manage/edit Data Models, you won't be able to see what I'm talking about with contacting an admin. ... View more

You won't be able to change the <label></label> value that populates the form/dashboard title using Simple XML. However, you can change the value that populates the title of a panel if you convert to using a form search. Here's a working example: <form> <label>Splunk Answers (125114) Example</label> <fieldset autoRun="false" submitButton="false"> <input type="text" token="name"> <label>Command:</label> <default>*</default> <suffix>*</suffix> </input> <input type="dropdown" token="begin" searchWhenChanged="true"> <label>Choose Start Date:</label> <populatingSearch fieldForLabel="day" fieldForValue="day"> sourcetype="syslog" | bucket span=1d _time AS day | dedup day | sort -day | convert timeformat="%B %d %Y" ctime(day) </populatingSearch> <default></default> </input> </fieldset> <row> <table> <title>syslog "COMMANDS" since $begin$</title> <searchString>sourcetype="syslog" [|gentimes start=-1 | eval earliest=strptime("$begin$","%B %d %Y") | fields earliest] | search COMMAND="$name$"</searchString> <option name="count">5</option> <option name="wrap">true</option> <option name="dataOverlayMode">none</option> <option name="drilldown">row</option> </table> </row> </form> Basically, I created a dropdown that replaces the time picker and allows you to choose the start date for your form. You can use the time picker and it's $earliest$ token instead but the value placed into $earliest$ will be something like -30d@d which isn't helpful as the date to be displayed in the panel title. If you need to specify a start time different from some day at midnight, you'll just have to be sure you also modify the strptime() parameters. Also, it would be simple to add a second dropdown to specify the ending date/time. Alertantively, you could add a time picker and use the $earliest$ and $latest$ tokens created by the time picker to set bounds on the date dropdown. But the bottom line is that you can't change the title of the form/dashboard using Simple XML. ... View more

The tstats command is used by the Pivot functionality of Splunk 6. That being the case, you probably can't find "User", "Bytes", Duration", or "Threats" in your sourcetype because they are created by the Data Model that was used by Pivot during the creation of the dashboard you are trying to "drill" into. When building a Data Model in Slunk 6, you can create new fields using regular expressions, eval, lookups, etc. Also, during the configuration of the Data Model, you can choose display different field names than the ones in the data (i.e. the username field can be renamed to User). Regarding your statement about finding the exact search, in this case, the search you posted is the exact search. Since it's using the Pivot and Data Model, you won't see the "behind-the-scenes" regexes, lookups, and/or evals used to create the fields you can't find in the data. Your ability to see the logic behind each field would depend on your user role and the Data Model permissions. ... View more

Try adding a value between the <default> tags. I don't know your real search string so I had to come up with my own. This form search works in Splunk 6: <form> <label>Search</label> <searchTemplate>sourcetype="syslog" | search COMMAND = $name$ </searchTemplate> <fieldset autoRun="true" submitButton="true"> <input type="text" token="name"> <label>Search</label> <default>./</default> <suffix>*</suffix> </input> </fieldset> <row> <table> <title>overview</title> <earliestTime>@d</earliestTime> <latestTime>now</latestTime> <option name="showPager">true</option> <option name="count">20</option> <option name="displayRowNumbers">true</option> </table> </row> </form> The only changes I made were the search string (so there would be results to test) and the addition of a default value. If you don't need a default value, you should remove the default tags entirely. The above form works just fine without a default (it just waits for the user to enter something in the input box and click Search ). ... View more

I can't say for sure but it might be that line and area charts are well suited for showing trends over time. However, if your goal is to create a dashboard panel using Pivot, you can create a bar or column graph in Pivot (which allows you to choose the X-axis field), save the search as a dashboard panel, and then change the visualization to line/area using the panel editor in the dashboard. ... View more

Try using two case statements. The first case will identify the combo numbers based on the products that need to be purchased together. The second case checks to make sure that all products were indeed purchased. If all products were purchased, the order is marked compliant. Anything other than "fully compliant" is marked as non-compliant: The main search piepline: index="prodmon" source="Orders" Defining the product combinations (I've added one that may not exist just to show a larger case statement): | eval combo_number=case((isnotnull(P1_FLAG) AND isnotnull(P2_FLAG) AND isnotnull(P3_FLAG)),1,(isnotnull(P4_FLAG) AND isnotnull(P5_FLAG) AND isnotnull(P6_FLAG)),2) Checking for comliance/non-compliance: | eval compliance=case((combo_number==1 AND (P1_FLAG=="Y" AND P2_FLAG=="Y" AND P3_FLAG=="Y")),"Compliant Orders: Combo 1",(combo_number==1 AND 1==1),"Non-compliant Orders: Combo 1",(combo_number==2 AND (P4_FLAG=="Y" AND P5_FLAG=="Y" AND P6_FLAG=="Y")),"Compliant Orders: Combo 2",(combo_number==2 AND 1==1),"Non-compliant Orders: Combo 2") Getting the counts: | stats c by compliance This should get you what you want without having to specifically define every possible combination of flags for every product combo. You only have to define what constitutes a compliant combo. The 1==1 takes care of anything that isn't compliant. ... View more

The "c" is just shorthand for "count". Similarly, dc() is shorthand for distinct_count(). So, " | stats c by mdm_user,error_code " and " | stats count by mdm_user,error_code " would produce the same results except that the count columns would be named "c" and "count" respectively. If you pipe the results of this first stats command into something else, you just need to make sure you use either "c" or "count" depending on what you used in the stats table. ... View more

@appleman, please see my updated answer above. ... View more

@vikas_gopal Please see my updated/edited response above. ... View more

The drilldown in this XML is sending the earliest and latest times based on the time range chosen by the user through the time picker. If that is the time range you are trying to send to your drilldown, you should first change the default section of the time picker to: <default> <earliest>@d</earliest> <latest>now</latest> </default> Otherwise it will default to "All Time" because the syntax above is incorrect (for Splunk 6 at least). Also, is this the full XML of the view? You mention a timechart but there isn't one in this XML. -----UPDATE STARTS HERE----- Since you are already sending other tokens to a drilldown form search, you can send the _time value as well. Just change your drilldown link to this: <![CDATA[/app/search/mdm_log_search1?form.incomingToken=$row.command$&form.timestamp=$row._time$&earliest=$earliest$&latest=$latest$]]> I added the &form.timestamp=$row._time$ section to your link. On the drilldown target, you will have <input type="text" token="timestamp" /> to receive the second incoming token. Here's a quick working example of a Main Form Search and the Drilldown Form Search showing how to send the exact time stamp to a drilldown form search: <form> <label>Main Form Search XML</label> <description/> <fieldset autoRun="true" submitButton="false"> <input type="time" searchWhenChanged="true"> <default> <earliestTime>@d</earliestTime> <latestTime>now</latestTime> </default> </input> </fieldset> <row> <table> <searchString>index=_internal | table _time,clientip</searchString> <earliestTime>-4h@m</earliestTime> <latestTime>now</latestTime> <drilldown target="_blank"> <link> <![CDATA[/app/search/drilldown_landing?form.timestamp=$row._time$]]> </link> </drilldown> </table> </row> </form> <form> <label>Drilldown Form Search Landing</label> <description/> <fieldset autoRun="true" submitButton="false"> <input type="time" searchWhenChanged="true"> <default/> </input> <input type="text" token="timestamp"/> </fieldset> <row> <event> <searchString>index=_internal _time=[| gentimes start=-1 | eval x=strptime("$timestamp$","%FT%H:%M:%S.%3N%z") | fields x | rename x AS search]</searchString> <earliestTime>0</earliestTime> <latestTime>now</latestTime> </event> </row> </form> Just make sure you use the proper time format string in strptime() in the drilldown form search. Otherwise, you'll get no results. Also, keep in mind that sending an exact time stamp like this will only get you events with that exact time stamp. ... View more

So, is it correct to say that the RequestID is unique per transaction but the TransactionID is not? Is the UniqueID you mention something that actually exists in the events or something you need to create on-the-fly? Is there something in either one or both of the log types that always indicates the beginning and end of the transaction? Is it possible for you to share a snippet of each of the log sources? Even if it is scrubbed, it might be easier to troubleshoot with an example of the data at this point. ... View more

Does that mean that RequestID is also used by multiple transactions? ... View more

Are there multiple "copies" of the same transaction in the xyz.log source or is the TransactionID being recycled/reused by multiple transactions? Also, is the beginning of a transaction always in abc.log and the end always in xyz.log ? ... View more