Getting Data In

Sourcetype is visible, but not searchable after rename in transforms

BTrust
Explorer

I have this most wired situation, where I use inputs.conf on the UF:

 

 

 

[monitor://C:\Users\xxx\OneDrive - xxx\xxx\Sources\On-Board\Splunk\test\eManager]
disabled = 0
index = main
sourcetype = el:PoC:eManager

 

 

 

 

On the HF (before Indexers) I use:
props.conf

 

 

 

# For eManager PoC
[el:PoC:eManager]
INDEXED_EXTRACTIONS=JSON
TIMESTAMP_FIELDS=timestampUtc
TZ = UTC
SHOULD_LINEMERGE = false
AUTO_KV_JSON = false
KV_MODE = none
TRANSFORMS-sourcetype = change-eManagerSourcetype

 

 

 

transforms.conf

 

 

 

[change-eManagerSourcetype]
SOURCE_KEY = MetaData:Sourcetype
REGEX = (.*?)
FORMAT = sourcetype::el:eManager
DEST_KEY = MetaData:Sourcetype

 

 

 

 

Data get ingested, and it all look ok - EXCEPT when using  this search:

 

 

 

index=main source=*_8.* 
|  rename _indextime as iTime
|  foreach *Time 
    [
    |  eval <<FIELD>>=strftime(<<FIELD>>,"%Y-%m-%d %H:%M:%S") ]
    |  stats latest(_time) AS _time count  BY index sourcetype

 

 

 

I get this result:

 

 

 

index	sourcetype	_time	count
main	el:eManager	2024-07-02 19:26:36.000	363
main	el:eOperator	2024-06-06 14:02:02.986	198

 

 

 

And when adding sourcetype="el:eManager" or just sourcetype="*" I get this:

 

 

 

index	sourcetype	_time	count
main	el:eOperator	2024-06-06 14:02:02.986	198

 

 

 

 

It like sourcetype is kind-of hidden, but not hidden after rename in transforms from: "el:PoC:eManager" to "el:eManager".
I can search by index and source and show it, but not use sourcetype anymore in a direct search.

Can anyone explain please?

Tags (3)
0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@BTrust- I don't see any issues with configuration or search that could be causing this issue.

Do you have any sourcetype related configuration on the search head? (ex. rename in props.conf).

If not, then I don't see any other issues with this. You can raise a Support Ticket with Splunk in that case.

 

I hope this helps~!!

BTrust
Explorer

Hi @VatsalJagani ,

Thanks for your input, and sorry for my late response.


Actually we finally figured it out, and I frankly never saw this before, but I think/hope it makes sense.

The issue was was reuse of events from same log file, from where several events was copied and pasted into several test log source files with different names, but the same event was pasted to more that one file.

Of cause this would cause duplicated events, but from different source files, why I'd expected this to work right while building and testing a new parser (props and transforms), but NO

After we made sure the same lines from the core log file was only used once in a new test log file, all was good again.

Lessons learned😋

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

Glad you figured it out!!!

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...