Getting Data In

Does Cribl always produce cooked data?

LovingSplunk
Path Finder

I realize in the new system I take care of, that all the Windows wineventlogs are being streamed to Splunk via Cribl, and it seems that the data comes cooked.

Is data always "cooked" when passing through Cribl? Specifically, if the logs are coming from a Windows Universal Forwarder (UF) and passing through Cribl before hitting the Indexers, does Cribl inherently "cook" the data, or is this a configuration choice within the Cribl Pipeline?

How do I maintain control over field extractions? In a traditional setup, we rely heavily on the Splunk Add-on for Microsoft Windows (Windows TA) for index-time and search-time extractions. If the data is being transformed in Cribl, does that bypass the TA’s logic?

Labels (2)
Tags (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

It depends. Cribl can handle outputting to Splunk in one of several ways.

It can do raw event to /services/collector/raw - this way the event is ingested as raw and goes through all normal event processing except line breaking IIRC

It can do HEC to /services/collector/event - this way the event goes directly into typingQueue (I'm not sure if you can tell Cribl to use the ?auto_extract_timestamp=true parameter to push the events earlier in the pipeline.

And it can do s2s (either directly or over HTTP via /services/collector/s2s) in which case it sends data as cooked and parsed (not just cooked!).

Anyway, it has nothing to do with search-time extractions. Search-time extractions happen - as the name says - during searching so it doesn't matter how the data was ingested (unless they rely on something that should have happened during indexing, like sourcetype rewrite).

LovingSplunk
Path Finder

Thank you @PickleRick, I'm completely new to Cribl, and I'm in charge of this system that utilizes it. How can I figure out how Cribl interacts with Splunk? most likely, directly to the indexers.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Do you mean that you want to check how your specific setup is configured? You have to check your pipelines or whatever they are called in Cribl (I don't remember; I don't use Cribl on a daily basis) and see for yourself.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

If Cribl is sending to the HEC event endpoint then the data is treated as cooked by Splunk and index-time props will be skipped.  If the data is sent to the raw endpoint then it is uncooked and normal pipeline processing applies.

---
If this reply helps you, Karma would be appreciated.
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Unlocking Unified Insights: New Gigamon Federated Search App for Splunk

In today’s data-heavy environment, organizations are caught in a data distribution dilemma. As data volumes ...

GA: New Data Management App in Splunk Platform

Streamlining Data Management: Introducing a unified experience in Splunk Managing data at scale shouldn’t feel ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...