Can I use field extractions and lookups to extend the power of Splunk Cloud?

[adukes_splunk]

Since I can't edit .conf files in Splunk Cloud, how can I get more granular insights from my data?

[adukes_splunk · Accepted solution]

The Splunk Product Best Practices team provided this response. Read more about How Crowdsourcing is Shaping the Future of Splunk Best Practices.

Unlock the power of the platform!

The variability of your data is unlimited, so your tools should be too. Splunk Cloud’s out-of-the-box features provide everything you need to get started searching and gaining insights to your data. But when those insights lead to deeper questions, Splunk Cloud gives you the flexibility to extend its base capabilities using lookups and field extractions.

How lookups and field extractions help you extend the power of Splunk

Splunk Cloud customers do not have the ability to directly edit .conf files. However, the Splunk field extractor enables you to create new, custom fields to build search-time field extractions that are associated with specific source types through the Splunk Web user interface.

Lookups match field-value combinations in your event data with field-value combinations in external lookup tables. If Splunk software finds those field-value combinations in your lookup table, it appends the corresponding field-value combinations from the table to the events in your search.

Splunk determines configuration priorities based on factors such as the current user and current app (scope) and alpha-numeric name sorting (lexicographical naming). This enables to tune your data's source type and increase the performance of indexing and searching.

Things to know

Fields are the building blocks of Splunk searches, reports, and data models. When you run a search on your event data, Splunk software looks for fields in that data. Fields appear in event data as searchable name-value pairings.
You can build field extractions with the field extractor to build search-time field extractions. After you run a search, fields extracted for that search are listed in the fields sidebar. You can create custom field extractions to define which fields are extracted and when Splunk software extracts fields.
Lookups can reference fields that are added to events by field extractions, field aliases, and calculated fields. They cannot reference event types and tags. Lookups are extremely powerful in that they can augment existing data with information that isn't stored in Splunk. You can create lookups in Splunk Web through the Settings pages for lookups.

• Automatic lookups: Applies a lookup to all searches at search time. After you define an automatic lookup for a lookup definition, you do not need to manually invoke it in searches with the lookup command.
• btool: A command line tool that can help you troubleshoot configuration file issues or see what values are being used by your Splunk Universal Forwarders
• Field extractor: A utility that helps you to dynamically create custom fields.
• Inline field extraction: The regular expression is in props.conf file. This enables you to use one regular expression to extract multiple fields or create a new field by configuring an extraction.
• Lookup table files: Files that contain a lookup table. A standard lookup pulls fields out of this table and adds them to your events when corresponding fields in the table are matched in your events.
• Lookup definitions: Part of a lookup configuration file that provides a lookup name and a path to find the lookup table.
• Precedence: The order in which Splunk prioritizes configuration settings based on which directory it's in local, app, or system default.
• Source type: A source type determines how Splunk Enterprise formats the data during the indexing process.

Things to do

• Troubleshoot an existing source type. Find an especially important source type and resolve data quality issues to make sure it's set up for success.
• Define and tune timestamps. Review at the timestamps in your data. Configure timestamp recognition to make sure Splunk doesn't waste time trying to figure out the right date-time stamp to use
• Define and tune event breaks. Multi-line events? Bet you have some! Figuring out what's mutli-line can be taxing on the indexers. Set the segmentation for event data to optimize your source types with what you've learned about .conf files.
• Create a new source type. Create a source type using the Source types management page.
• Watch the Splunk Cloud Tutorial. Watch the Splunk Cloud Tutorial to see how to set up Splunk Cloud and get data in using a Universal Forwarder.
• Think about a learning path. Review courses for Splunk Cloud customers from Splunk Education.
• Leverage the power of lookups. Lookups make it easy to add context and create correlations with your data. For example, you can use a geospatial lookup to turn a series of IP addresses into geographical locations. Learn more about lookups and how they can enhance your search experience.