Splunk Search

Does each Splunk event have a unique identifier?

araitz
Splunk Employee
Splunk Employee

I would like to tag some specific events to group them together for incident response and forensics purposes. Is this possible with Splunk?

1 Solution

Stephen_Sorkin
Splunk Employee
Splunk Employee

No, it is not yet.

Each event does have a unique id, the tuple (splunk_server, index, _cd), but "_cd" is not searchable (only filterable). You could use lookup tables to map this to a tag or key.

When we make _cd searchable, that will allow searching on the tags or groups.

View solution in original post

Lowell
Super Champion

So there's been some movement on this question in the recent weeks, so I'll drop in and give some commentary. Hopefully this helps someone. Bottom line, there still isn't a really good answer to this, as far as I know. Most of when I hear this question asked, it's because of a misunderstanding of what Splunk does and how it works. If you really need a feature like this, open a splunk support ticket and explain your use case (that's how new features like this get introduced.) Otherwise, enjoy the following weeds:

Before going much further let me point out that Sorkin's answer was from before Splunk indexer clustering, and therefore won't work reliably in on clustered environments today. The answer with the next highest points (mikaelbje) shows an id based on a hash of the event's raw text. But there's no fundamental guarantee of uniqueness for _raw (e.g., the same message could repeat multiple time per second, or data could be ingested twice, ...), and more importantly, there's no fast way to search on it. If you're trying to pick out one event out of a few thousand, then this is probably acceptable, but it does not scale. (This is because Splunk has to pull back every event from disk, then calculate the checksum, and then compare it. There's no fast index operations involved, therefore it will be slow.)

So there's a couple issues that index clustering bring to the mix that changes things dramatically. First, with bucket replication, there's no guarantee that the same indexer will always be returning the same event in the future, it could easily be handled off to another peer who has a replicated copy. (Also note that splunk_server represents the current serverName of that indexer, which may be different from index time.) So Splunk introduced the _bkt field (around the same time as indexer clustering, if I remember correctly.) And it always returns the same bucket name, even if the hostname changes (because it uses a GUID not a hostname). This works even if the original host is decommissioned. (Yes, that still breaks if the index name changes.)

So the modern equivalent of (splunk_server, index, _cd) tuple, is now (_bkt, _cd).

And fortunately, _btk is always available and consistent, even if you're not on a cluster. (Although in that case the GUID just represents the current GUID of the server, not the GUID embedded in the bucket's name.) The _btk field is composition of (1) index name, (2) simple bucket id ( or "local id", a simple incrementing integer), and (3) GUID of the initial bucket creator. And, as always, _cd is a combination of (1) bucket id (integer only), and (2) the internal event number for that bucket. (If you look under the covers, this is the id that Splunk uses to "delete", aka hide, events. Notr: It's unclear to me if the event id (stored in_cd) is perstient across a bucket rebuild (if thawed after being frozen, or if passed through exporttool/importtool for some other reasons.) Again, it's probably not best to rely on this mechanism.)

But as of Splunk 6.5, you can't search on "_cd" in the base search, and while searching for "_bkt" in the base search works, according to LISPY, it tries to find it as a raw string, which sure doesn't look efficient. (Also makes me wonder if fields.conf is setup wrong.... another research problem for another day...)

Bottom line, if this is a feature that you actually need, file an enhancement request. If you just need something in Splunk with a unique and consistent key, take a look at the KV store--they say it's magic!

tcgerhard
Explorer

My use case is that I want to select an event from a list and display more details about the event in another panel via drilldown. Some sort of ID (and ideally, indexed) would make it straightforward.

0 Karma

thisissplunk
Builder

I've found that you can at least access _cd for a stats if you do a rename first:

| rename _cd as unique_id
| stats count by unique_id

If you want to search on that data, you can do this:

index=awesome sourcetype=woah
rename _cd as unique_id
| search unique_id=9320:49207386

0 Karma

mikaelbje
Motivator

Update for Splunk 6.2.1.

_cd is still not searchable after 5 years. I suggest using the following method which calculates a hash based on the raw event:

your search here | eval id=md5(_raw) | id="VALUE_YOU_ARE_LOOKING_FOR"

Stephen_Sorkin
Splunk Employee
Splunk Employee

No, it is not yet.

Each event does have a unique id, the tuple (splunk_server, index, _cd), but "_cd" is not searchable (only filterable). You could use lookup tables to map this to a tag or key.

When we make _cd searchable, that will allow searching on the tags or groups.

koshyk
Super Champion

I feel it is unfair to downvote an answer after 7+ years. (may be at that time, it was the only solution.).

christianhuber
Path Finder

I downvoted this post because there is a newer and better answer

0 Karma

dangsecurity
Engager

Why didn't you link to the "newer and better answer" so others can benefit as well?

pembleton
Path Finder

any update?

0 Karma

deloach
Engager

Any updates on being able to tag specific events or time frame when this functionality might be available?

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...