Getting Data In

Sourcetype is visible, but not searchable after rename in transforms

BTrust
Path Finder

I have this most wired situation, where I use inputs.conf on the UF:

 

 

 

[monitor://C:\Users\xxx\OneDrive - xxx\xxx\Sources\On-Board\Splunk\test\eManager]
disabled = 0
index = main
sourcetype = el:PoC:eManager

 

 

 

 

On the HF (before Indexers) I use:
props.conf

 

 

 

# For eManager PoC
[el:PoC:eManager]
INDEXED_EXTRACTIONS=JSON
TIMESTAMP_FIELDS=timestampUtc
TZ = UTC
SHOULD_LINEMERGE = false
AUTO_KV_JSON = false
KV_MODE = none
TRANSFORMS-sourcetype = change-eManagerSourcetype

 

 

 

transforms.conf

 

 

 

[change-eManagerSourcetype]
SOURCE_KEY = MetaData:Sourcetype
REGEX = (.*?)
FORMAT = sourcetype::el:eManager
DEST_KEY = MetaData:Sourcetype

 

 

 

 

Data get ingested, and it all look ok - EXCEPT when using  this search:

 

 

 

index=main source=*_8.* 
|  rename _indextime as iTime
|  foreach *Time 
    [
    |  eval <<FIELD>>=strftime(<<FIELD>>,"%Y-%m-%d %H:%M:%S") ]
    |  stats latest(_time) AS _time count  BY index sourcetype

 

 

 

I get this result:

 

 

 

index	sourcetype	_time	count
main	el:eManager	2024-07-02 19:26:36.000	363
main	el:eOperator	2024-06-06 14:02:02.986	198

 

 

 

And when adding sourcetype="el:eManager" or just sourcetype="*" I get this:

 

 

 

index	sourcetype	_time	count
main	el:eOperator	2024-06-06 14:02:02.986	198

 

 

 

 

It like sourcetype is kind-of hidden, but not hidden after rename in transforms from: "el:PoC:eManager" to "el:eManager".
I can search by index and source and show it, but not use sourcetype anymore in a direct search.

Can anyone explain please?

Tags (3)
0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@BTrust- I don't see any issues with configuration or search that could be causing this issue.

Do you have any sourcetype related configuration on the search head? (ex. rename in props.conf).

If not, then I don't see any other issues with this. You can raise a Support Ticket with Splunk in that case.

 

I hope this helps~!!

BTrust
Path Finder

Hi @VatsalJagani ,

Thanks for your input, and sorry for my late response.


Actually we finally figured it out, and I frankly never saw this before, but I think/hope it makes sense.

The issue was was reuse of events from same log file, from where several events was copied and pasted into several test log source files with different names, but the same event was pasted to more that one file.

Of cause this would cause duplicated events, but from different source files, why I'd expected this to work right while building and testing a new parser (props and transforms), but NO

After we made sure the same lines from the core log file was only used once in a new test log file, all was good again.

Lessons learned😋

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

Glad you figured it out!!!

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

What Is Splunk? Here’s What You Can Do with Splunk

Hey Splunk Community, we know you know Splunk. You likely leverage its unparalleled ability to ingest, index, ...

Level Up Your .conf25: Splunk Arcade Comes to Boston

With .conf25 right around the corner in Boston, there’s a lot to look forward to — inspiring keynotes, ...

Manual Instrumentation with Splunk Observability Cloud: How to Instrument Frontend ...

Although it might seem daunting, as we’ve seen in this series, manual instrumentation can be straightforward ...