I'm having a field that is being specifically indexed (and not extracted during search time).
The following configuration is used:
[mysourcetype] TRANSFORMS-bla = indexex
[indexex] REGEX = ^(\d+) FORMAT = MYFIELD::$1 WRITE_META = true
[MYFIELD] INDEXED = true
This all works fine in normal indexes. However, I'm creating a summary index that contains the same field, like this:
index=myindex *searchsomething* | table something MYFIELD
This is also indexed fine, however I cannot search by MYFIELD directly:
yields 0 results. This works though:
index=summaryindex | search MYFIELD
I tried adding an indexed field for the stash sourcetype but it didn't help.
Is this an issue due to the field being indexed (and fields.conf applied globally)?
Do I have to change the field name (searching through summaryindex for other fields works, it doesn't work only for this field).
Unfortunately this doesn't help - I have MYFIELD in fields.conf as indexed (since I have to extract it separately, it's a subtoken).
So I cannot add MYFIELD INDEXED = false there as fields.conf appears to be global 😕
Wait, if you're summary indexing, could you show what the summary indexing events look like? I imagine they should have MYFIELD=somevalue in there, no?
I extract it - for example, I have a log such as this:
Where this is actually couple of variables so I extract them:
REGEX = ^[^\s]+\s(.)(....) FORMAT = SOMEFIELD::$1 MYFIELD::$2 WRITE_META = true
I used index time extraction here - and I create the same field that I use later in summary. I guess this is the problem, I could rename the field but since I already have tons of summary-indexed stuff I was wondering if there is a way to get Splunk to recognize this.
I presume Splunk tries to find that same field as indexed field and fails.
No I mean what does the event in the summary index look like? Or is that it?
Also, not that it takes us closer to a solution, but why are you making this an indexed field rather than something that's extracted at search-time? It's very rarely a good idea.
What I showed is the original log. In the summary index, it looks like this:
2012-07-11 summaryindexstuff BLA=11, SOMETHING=12, MYFIELD=10203
I needed to index it separately because it's a subtoken and I'm having a huge amount of logs. The logs are ugly and not space (or anything) delimited - they just contain concatenated numbers/letters.
That first part works great, it's fast etc. But I guess I made a mistake not renaming this one field in summary index. In example below, search works fine for other fields (i.e. BLA and SOMETHING), just doesn't work for MYFIELD.
Yeah since you put
INDEXED = true I'm pretty sure there's no good workaround other than renaming the field in the summary index.
I'm still not sure I understand why you really need to have it as an indexed field - did you actually try to have it extracted in search-time and compare performance? My guess is performance would be just as good, and possibly even better, with a search-time extracted field.