Join the Conversation
- Find Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Re: Performance issue
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Performance issue
I have 24 months of data from Jan 2017 to Nov 2018
I have count of 23900000 for year 2017
27900000 for year 2018
I need all required columns given in below query.
Index=test sourcetype=historical Period="Jan-17 | table rft period expenses ID transaction balance region area country
If i run above command to fetch one month of data , it takes more than 30 minutes.
how do i improve performance, I cannot try creating lookup & summary Indexing as data is huge , do we have any other option to expedite search query to run fast.Thanks
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Splunk is about matching and aggregation: turning haystacks into needles. It is not a haystack delivery system. You don't really need all umpty-kazillion events, right? You need to report something that you calculate or isolate about them right? Tell us what that thing is and we can help you get there. If you need haystacks moved and presented, then Splunk is the wrong tool.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@rakesh44 - change table to fields and test again. table is not a streaming command, so it's causing all the data to be sent to the search head.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi DalJeanis,
I have view data in tabular format to check & compare each events & hence i could not fields command.Do we have any alternate option for table commands.
Thanks
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Daljeanis,
I have checked with fields command not that much improvement, which i expected .
Thanks
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@rakesh44 check out David Veuve 's .Conf Session for using Search Acceleration mechanisms:
https://conf.splunk.com/conf-online.html?search=FN120994#/
| makeresults | eval message= "Happy Splunking!!!"
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@niketnilay , I dont used any reporting commands & hence it will not allow me to use report acceleration
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think @adonio answer at Can we improve on a standard index=index_name sourcetype=*prod | stats query? can be applied in your case.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That Information is not useful in my case
I need below columns in results
index=test sourcetype=historical Period="Jan-17 | table rft period expenses ID transaction balance region area country
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
Splunk Community Badges!
How to find the worst searches in your Splunk environment and how to fix them
Share Your Feedback: On Admin Config Service (ACS)!
