Join the Conversation
- Find Answers
- :
- Splunk Products
- :
- Splunk Enterprise
- :
- Re: Oneshot and Kvstore update
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Need to upload the contents of a CSV that exceeds the size allowed in our web.conf. Will modify this as a last resort, but.....
Ultimately need to populate a Kvstore with the contents of a 100+MB CSV. Investigated Oneshot, but don't see how to ingest the file contents into a Kvstore, only an index. Any suggestions would be welcome.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Why do you need it in a kvstore? Can you not simply deploy it as a file-based lookup?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Why do you need it in a kvstore? Can you not simply deploy it as a file-based lookup?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I can, and did with multiple smaller files. My concern was performance when accessing a 200k+ row file, which will be leveraged regularly (asset library). If not a big deal (haven't been there yet) then am fine with using file-based.
If so, the question becomes: how can I populate a file-based lookup without modifying my web.conf, and without having to breakup the file into smaller chunks for ingestion?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There is no limit if you are using lookup; only if you are using inputlookup. In the super-majority of cases when people think that they should be using inputlookup, they really should be using lookup. What exactly are you needing to do and we can probably make it use lookup and you will be golden.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
To the question of "what exactly are you needing"? At this point, only to make the data available. TBD how the community of users will "use" it (so an unknown at this point). Get your point (I think) about using lookup for a refined search vs. inputlookup for a simple "listing"
So, I use a lookup table, the question is similar. Is there a better way to upload the file, avoiding the modification of web.conf (which is currently blocking the upload)?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You just put the file where it needs to be via any means available. I am not aware that the GUI-based "upload a lookup file" has any size limit (I am pretty sure that it does not). Give it a try.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It's the simple things in life that always get in the way 😕
Yes, the gui has a limit, that keeps the users out of trouble... but as you stated, why don't I simply UPLOAD it via the CLI to the same location as the others.... Wait a minute, I recall something about some forests and trees....Thanks for the "nudge"
And yes, scp to each of the WFE's worked just fine
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I was thinking Deployment Server, but that works!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have done millions of long rows with DS and also app install (lookups built into an app).
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
Announcing Modern Navigation: A New Era of Splunk User Experience
Modernize your Splunk Apps – Introducing Python 3.13 in Splunk
Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...
