Here's my full btool dump for the sourcetype if it helps: [telemetry1] ANNOTATE_PUNCT = True AUTO_KV_JSON = true BREAK_ONLY_BEFORE = BREAK_ONLY_BEFORE_DATE = True CHARSET = UTF-8 DATETIME_CONFIG = EVAL-lat = substr(latitude, 1, 2).".".substr(latitude, 3, 7) EVAL-lon = substr(longitude, 1, 3).".".substr(longitude, 4, 7) FIELD_NAMES = calendar_day, scheduled_time, actual_arrival_time, actual_depart_time, adherence_seconds, time_point_id, time_point_name, block_stop_order, vehicle_num, geo_node, latitude, longitude, schd_distance, route_abbr, is_layover, route_name, isrevenue, revenue_id HEADER_MODE = INDEXED_EXTRACTIONS = csv KV_MODE = none LEARN_MODEL = true LEARN_SOURCETYPE = true LINE_BREAKER_LOOKBEHIND = 100 MATCH_LIMIT = 100000 MAX_DAYS_AGO = 2000 MAX_DAYS_HENCE = 2 MAX_DIFF_SECS_AGO = 3600 MAX_DIFF_SECS_HENCE = 604800 MAX_EVENTS = 256 MAX_TIMESTAMP_LOOKAHEAD = 128 MUST_BREAK_AFTER = MUST_NOT_BREAK_AFTER = MUST_NOT_BREAK_BEFORE = NO_BINARY_CHECK = true SEGMENTATION = indexing SEGMENTATION-all = full SEGMENTATION-inner = inner SEGMENTATION-outer = outer SEGMENTATION-raw = none SEGMENTATION-standard = standard SHOULD_LINEMERGE = false TIMESTAMP_FIELDS = calendar_day, actual_depart_time TIME_FORMAT = %d-%b-%y %I:%M:%S %p TRANSFORMS = TRUNCATE = 10000 category = Structured description = MARTA AVL Import detect_trailing_nulls = false disabled = false maxDist = 100 priority = pulldown_type = true sourcetype = ... View more

This is all (original file, sa-eventgen, conf files etc) on a single instance. The extractions don't happen on the streaming events from the eventgen like they do on the file input and that seems to be the root of the issue - even though I'm using the same sourcetype and event format (minus the header row that was in the original csv file). I'm stumped! I'm wondering if indexed field extractions occur after timestamping on streaming data for some reason, which was why I was headed down the path of treating the events as raw data and using time_format or datetime.xml. Any other thoughts let me know and thanks so much for the help! ... View more

Thats what I thought too, its how I was originally approaching, but couldn't get it to work so I was looking for another route. Here's my current props: [telemetry1] CHARSET=UTF-8 INDEXED_EXTRACTIONS = csv KV_MODE = none SHOULD_LINEMERGE = false pulldown_type = true category = Custom FIELD_NAMES = calendar_day, scheduled_time, actual_arrival_time, actual_depart_time, adherence_seconds, time_point_id TIMESTAMP_FIELDS = calendar_day, actual_depart_time TIME_FORMAT = %d-%b-%y %I:%M:%S %p But ts recognition is failing and Splunk is just using index time. SA-eventgen is the datasource if that makes a difference, events are coming in over STDOUT. I'll keep poking, but any other thoughts are appreciated - thanks! ... View more

We typically don't recommend extracting new fields and indexing them, but it is possible to configure this on the indexer, see http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Configureindex-timefieldextraction for more information. But I think based on your question, you are still trying to do something that is very possible at search time. Remember that your search runs against the _raw of your event, so in order to find 1234 in an event, you can just use an asterix wildcard on both sides of 1234 which will filter to any events that contain that set of characters, with any other characters on the left and right. This may be oversimplifying what you are trying to accomplish though. If you run a search time extraction using rex http://docs.splunk.com/Documentation/Splunk/6.3.0/SearchReference/Rex, new fields you create from the content of Value will be available once you run the search. Any search after the next pipe would be able to use the extracted fields as if they were in the data originally. Benefit of this is that you can use your filter constraint to only run the rex against Tags that match your multivalue rule. Let me know if this solves for you - thanks! ... View more

Great question! There is a Kepware Explorer app available on Splunkbase https://splunkbase.splunk.com/app/2851/ that demonstrates how to do extractions like that at search time, which would be configured on the search head or indexer. There is a good example for extracting components of the Tag etc and how those components can be turned into new fields. Always more than one way to skin a cat though, so let me know and we can work on specifics of your environment. assuming that you run a Universal Forwarder on the same machine as the Kepserver, all documentation on extract transform etc would apply once data hits the UF. ... View more

Here you go: as of 6.1! http://blogs.splunk.com/2014/05/08/17682/ ... View more