I have a log that has multiple fields and values and each event has a different set of fields and values. To handle that, I'm using a transforms stanza with a REGEX to separately extract the field and value at search time. The transform seems to be working as expected as Splunk shows all of my fields on the left side with all of their values. But that's where it stops working.
When I try to actually use one of the extracted fields in search, I get very odd behavior. If I do a search with field=value, I get no results, even if I use Splunk's built-in extraction from the field list on the left side to construct my search string. However, if I add an asterisk (*) to the end of the field=value search, then I get results. This makes me think my REGEX is extracting a bit more than it should, but I can't see any extra characters or non-printable ones.
Here is a sample event that is being extracted correctly:
2014-03-12 11:26:32,389 INFO SSID:AA87309DKj9911FFFFACDD [pool-10251-thread-1] SERVICE_KEY=5688 SERVICE=myService INDEX_POS=0 APPLICATION_ID==APPID~ACCOUNT_NUM==123456789~CUST_SUBTYPE==R~CUST_TYPE==I~ENV_CODE==ENV~MARKET_CODE==123~OPERATOR_ID==123456~ORIGIN_SYSTEM==APP~PSUBMKTGRP_ROW_COUNT==12~RUN_DATE==20140312~SUBMKT_SUB_MARKET_CODE==ABC~TRANSACTION_MODE==O~
And here is my stanza from transforms:
REGEX = ([A-Z0-9_]*?)==([^~]*?)~
FORMAT = $1::$2
In this case, Splunk properly pulls out all the field names (APPLICATION_ID, ACCOUNT_NUM, CUST_SUBTYPE, etc), and the values are also correct as the left side list of fields shows. But if my search is something like APPLICATION_ID=APPID, I'll get no results. However, simply making the search APPLICATION_ID=APPID* will work.
Because Splunk is able to properly extract field names and values in the left side in verbose mode, but then fails in search mode, this makes me think this could be a bug in Splunk. And potentially it's related to my data having double equal signs. The reason for the double equal signs is to prevent Splunk from trying to auto extract since in some cases these fields can contain an equal sign as part of the value.
Hopefully that's enough information for someone to give me some pointers. Thanks.
... View more