Splunk Love: Sandeep’s Splunk Search Proficiency Puts Patterns Toward Peak Performance Engineering

[bjennewein]

About Sandeep

Sandeep was among the first to reach out to us after we launched Splunk Love in December of 2021, and it was his ingenuity regarding Splunk Search that immediately impressed us. He works for an American multinational information services and technology company in the area of performance engineering and testing, and his contributions are critical to ensuring the uptime, stability, and reliability of his clients’ apps and digital experiences.

In his own words:

“I am a Performance Tester & Engineer, for the last one Jupiter year. Nearly half of it, my journey is with Splunk, as a key tool: Spelunking through the server logs, learning each day more ways to interpret the data from a huge pile, into a more human understanding. I am overjoyed, sluicing the gold bearing gravel of data from the banks of the Servers. :)”

Some of Sandeep’s Love for Splunk

Sandeep’s love for Splunk is rooted in both Splunk’s capabilities and in the efficiency and excellence it can bring to performance engineering and application performance management. It’s not uncommon for performance engineers to have to rigorously test applications and digital experiences with conditions different from a production or live environment, placing different aspects of the app’s infrastructure, services, or experiences under varying degrees of demand or use in order to confidently assure its stability and operability in similar live conditions. One objective of this critical work is to isolate and remediate errors if and when they arise from such circumstances, and this is where Splunk becomes critical, and where Sandeep’s use of Splunk becomes truly novel.

You may have guessed by now that Sandeep and his team ingest collections of data from all aspects of the applications they test into Splunk, and it probably comes as no surprise that Splunk helps Sandeep and his team search for, investigate, and remediate errors they do find. All of this is part and parcel for using Splunk for operations and for application performance monitoring. But what’s not so routine is how Sandeep has managed to orchestrate his use of Splunk search to take error identification and investigation down from an hour to mere minutes.

In Sandeep’s line-of-work, errors and issues often fall into similar patterns, but these patterns differ from those that might present themselves in a live environment, so traditional pattern detection tactics might lack both the data and the data context to increase the efficiency of performance engineers’ testing and remediation efforts. This is where Sandeep’s use of Splunk Search really impressed me. He and his team are able to recognize the patterns in errors easily, and when they do, they are able to create a search string that accounts for the pattern and returns what matches or what doesn’t. And when a novel error presents itself, his resulting dashboard makes it easy to obtain the error details, investigate, and get the valuable feedback needed to the right teams.

The end result of this clever work is more efficient testing, to the tune of an hour’s worth of work taking minutes. Now you might be thinking, “but what happens when an error doesn’t fall into a pattern accounted for by Sandeep’s search queries?” Well… much like AI/ML learn from detecting patterns, so does Sandeep and his search patterns. Investigating non-comforming errors is easy with Splunk, and when previously unaccounted for errors fit a pattern, too, he merely adapts his query. The result is an entire team efficiently investigating and remediating the anomalous instead of the mundane. Exceptional and inspiring work, Sandeep and team!

What’s next for Sandeep…

Sandeep shared that he’s recently begun exploring a trial of Splunk’s Observability Cloud, to better understand how some of these newer capabilities might further enhance his and his team’s work and impact. Good on you, Sandeep! Rumor also has it he is a writer! Perhaps he’ll share some of his work here some day soon!

If you’d like to connect with Sandeep, he is @Sandeep_prabha in our Splunk Community.

Keep the Splunk Love Coming, Nerds!

- Bryan Jennewein, Sr. Director Splunk Community

Want to snag yourself some limelight like Sandeep? Check out our Splunk Love program and submit your interest today!