I have this most wired situation, where I use inputs.conf on the UF:
[monitor://C:\Users\xxx\OneDrive - xxx\xxx\Sources\On-Board\Splunk\test\eManager]
disabled = 0
index = main
sourcetype = el:PoC:eManager
On the HF (before Indexers) I use:
props.conf
# For eManager PoC
[el:PoC:eManager]
INDEXED_EXTRACTIONS=JSON
TIMESTAMP_FIELDS=timestampUtc
TZ = UTC
SHOULD_LINEMERGE = false
AUTO_KV_JSON = false
KV_MODE = none
TRANSFORMS-sourcetype = change-eManagerSourcetype
transforms.conf
[change-eManagerSourcetype]
SOURCE_KEY = MetaData:Sourcetype
REGEX = (.*?)
FORMAT = sourcetype::el:eManager
DEST_KEY = MetaData:Sourcetype
Data get ingested, and it all look ok - EXCEPT when using this search:
index=main source=*_8.*
| rename _indextime as iTime
| foreach *Time
[
| eval <<FIELD>>=strftime(<<FIELD>>,"%Y-%m-%d %H:%M:%S") ]
| stats latest(_time) AS _time count BY index sourcetype
I get this result:
index sourcetype _time count
main el:eManager 2024-07-02 19:26:36.000 363
main el:eOperator 2024-06-06 14:02:02.986 198
And when adding sourcetype="el:eManager" or just sourcetype="*" I get this:
index sourcetype _time count
main el:eOperator 2024-06-06 14:02:02.986 198
It like sourcetype is kind-of hidden, but not hidden after rename in transforms from: "el:PoC:eManager" to "el:eManager".
I can search by index and source and show it, but not use sourcetype anymore in a direct search.
Can anyone explain please?
@BTrust- I don't see any issues with configuration or search that could be causing this issue.
Do you have any sourcetype related configuration on the search head? (ex. rename in props.conf).
If not, then I don't see any other issues with this. You can raise a Support Ticket with Splunk in that case.
I hope this helps~!!
Hi @VatsalJagani ,
Thanks for your input, and sorry for my late response.
Actually we finally figured it out, and I frankly never saw this before, but I think/hope it makes sense.
The issue was was reuse of events from same log file, from where several events was copied and pasted into several test log source files with different names, but the same event was pasted to more that one file.
Of cause this would cause duplicated events, but from different source files, why I'd expected this to work right while building and testing a new parser (props and transforms), but NO
After we made sure the same lines from the core log file was only used once in a new test log file, all was good again.
Lessons learned😋
Glad you figured it out!!!