Splunk Dev

Error with keyname conflicting with source. Can I retrieve the source key value?

prosenjit2707
Explorer

Unfortunately, I have been indexing the events which have a key named "source" and splunk by default treat the key "source" as the source of the events.

Now, when I am trying to retrieve the values from key "source", it is providing me the event source.

Is there any way to retrieve the source key values from the events instead event sources(directories) or it is a bug/conflicts!

Can anyone help me in this situation, how can I get the values without using regex/rex cmds?

0 Karma

prosenjit2707
Explorer

This is the optimum way that I can think of, as I do not have admin rights to change the config files.

| rex "source=(?P\"?[^,]+)" | eval Source = lower(Source1)|chart count by Source

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

You have multiple options I can see

  1. Use TERM in your search: <yourSearch> TERM(source=yoursourcevaluefromevent)
  2. Use rex to extract source under a different name: | rex "source=(?<mySource>\w+)\s"
  3. Do a SEDCMD on ingest to change the field in your events to a different value

prosenjit2707
Explorer

I tried but TERM is not working in this scenario as I have events like below:

07/31/2013 15:38:18, field1=v1 field2=v2 field3=v3 07/31/2013 15:38:48, field1=v4 source=abcdef field2=v5 field3=v6

This will not work as source=abcdef has "=" as it is a minor break from the below reference:
https://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Search/UseCASEandTERMtomatchphrases

0 Karma

blacknight659
Explorer

Maybe I am missing it, but is this what you are looking for?

source="=abcdef"

0 Karma

prosenjit2707
Explorer

Thank you for the response. But I am looking for all the possible values that comes under "source", but not from the default source (i.e. source=/opt/splunk/abc/xyz/log1.txt)

0 Karma

blacknight659
Explorer

Ok, so you have two fields named "source" ?

You are trying to search this data? NOT modify it at index time?

If you are tying to search against the data, you will need to Eval or Rex it. Ultimately, you need get a little fancy to capture the values there. Ultimately, correcting this at index time is the best solution.

If it were me, I would extract everything with the rex below, then search for all sources that do not have \ (since most sources have the \ to the directory of the log).

| rex field=source "(?<newSourceName>[\w\W]*)"
| search newSourceName!="*\*"
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Good point, although - interestingly enough - it works for me (V7).

I think your best bet for solving this at search time is to create an inline field extraction for the affected sourcetype(s) that pulls out the values and renames the key. Simply upper-casing the first letter should be least intrusive.
This allows you to not only search for the values, but also run reporting/statistics.
Example in props.conf:
[yoursourcetype]
EXTRACT-Source = source=(?[^[\s]+)

You can also do it via the UI (Settings->Fields->Field Extractions). Make sure it is visible in the appropriate app context (Permissions).

Cleanest would be to either change it at the log source or apply a SEDCMD at index time to modify _raw.

0 Karma

prosenjit2707
Explorer

Thanks for your response, Yes. I agreed with your option to change at props.conf itself. I tried in the field extractions, I did not got the proper values.

Sometime, I have events with source="abcdef" (i.e. values enclosed in inverted comma).

0 Karma

DalJeanis
Legend

@ssievert - Ooooo, I like TERM(). I haz a new toy! Is Chrissmass!

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...