I'm having a field that is being specifically indexed (and not extracted during search time).
The following configuration is used:
[mysourcetype] TRANSFORMS-bla = indexex
[indexex] REGEX = ^(\d+) FORMAT = MYFIELD::$1 WRITE_META = true
[MYFIELD] INDEXED = true
This all works fine in normal indexes. However, I'm creating a summary index that contains the same field, like this:
index=myindex *searchsomething* | table something MYFIELD
This is also indexed fine, however I cannot search by MYFIELD directly:
yields 0 results. This works though:
index=summaryindex | search MYFIELD
I tried adding an indexed field for the stash sourcetype but it didn't help.
Is this an issue due to the field being indexed (and fields.conf applied globally)?
Do I have to change the field name (searching through summaryindex for other fields works, it doesn't work only for this field).
Have a look at the following blog post, which probably describes your problem and its solution.
Yeah since you put
INDEXED = true I'm pretty sure there's no good workaround other than renaming the field in the summary index.
I'm still not sure I understand why you really need to have it as an indexed field - did you actually try to have it extracted in search-time and compare performance? My guess is performance would be just as good, and possibly even better, with a search-time extracted field.
What I showed is the original log. In the summary index, it looks like this:
2012-07-11 summaryindexstuff BLA=11, SOMETHING=12, MYFIELD=10203
I needed to index it separately because it's a subtoken and I'm having a huge amount of logs. The logs are ugly and not space (or anything) delimited - they just contain concatenated numbers/letters.
That first part works great, it's fast etc. But I guess I made a mistake not renaming this one field in summary index. In example below, search works fine for other fields (i.e. BLA and SOMETHING), just doesn't work for MYFIELD.
No I mean what does the event in the summary index look like? Or is that it?
Also, not that it takes us closer to a solution, but why are you making this an indexed field rather than something that's extracted at search-time? It's very rarely a good idea.
I extract it - for example, I have a log such as this:
Where this is actually couple of variables so I extract them:
REGEX = ^[^\s]+\s(.)(....) FORMAT = SOMEFIELD::$1 MYFIELD::$2 WRITE_META = true
I used index time extraction here - and I create the same field that I use later in summary. I guess this is the problem, I could rename the field but since I already have tons of summary-indexed stuff I was wondering if there is a way to get Splunk to recognize this.
I presume Splunk tries to find that same field as indexed field and fails.
Wait, if you're summary indexing, could you show what the summary indexing events look like? I imagine they should have MYFIELD=somevalue in there, no?
Unfortunately this doesn't help - I have MYFIELD in fields.conf as indexed (since I have to extract it separately, it's a subtoken).
So I cannot add MYFIELD INDEXED = false there as fields.conf appears to be global 😕