topic Re: Search History disappeared ( Splunk Enterprise 8.0.1) in Splunk Enterprise

Why did my search history disappear( Splunk Enterprise 8.0.1)?

olgademo — Fri, 22 Jul 2022 16:08:16 GMT

Lost my Search History twice: on Jan 02 - but it came back, and on Jan 03, and it was not recovered since. I checked that I am in the right app, and set "All Time".
- |history returns recent 30 searches, mostly from file loading or UI, not my own
- index=_internal user=* sourcetype=splunkd_ui_access | dedup q | table _time, q | eval q=urldecode(q) returns 30 searches, not my custom ones except one ( ?!)

Installed Splunk Enterprise 8.0.1 at the end of Dec ; Search history was there every time I logged in except the hicckup on Jan 02 and full disappearance on Jan 03.
Thanks!

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

ivanreis — Wed, 30 Sep 2020 03:35:40 GMT

The history information is being saved under $SPLUNK_HOME/etc/users/youruser/search/history on a csv file. Please login at splunk search head using cli and check if you have a csv file under the history folder at $SPLUNK_HOME/etc/users/youruser/search/history. If you did not see the file under this path, it means the history is already gone. A possible alternative to recover it, if you ran a backup of etc folders before you ran the upgrade.

if you are running on a search head cluster, it is possible that your history is not being properly replicated -> https://answers.splunk.com/answers/391876/is-there-any-way-to-get-splunk-to-replicate-search.html

Please see other search history topics that maybe can help you ->https://answers.splunk.com/topics/search-history.html

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

olgademo — Mon, 06 Jan 2020 01:18:40 GMT

Hi ivanreis, thanks for your advice. I located the file you mentioned. It had recent UI:Dashboard and today's UI:Search lines, but nothing from Jan 1, 2 or December. Those searches are not in there, and I did not delete them. I don't think I am running on the search head cluster, unless it is a default mode.
Can it be a bug?

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

ivanreis — Mon, 06 Jan 2020 02:17:50 GMT

I did not see any history issue reported on this version 8.0.0. I am not really sure if this history file is cleanup from time to time. I am not really sure if it can be a bug. Maybe you should open a ticket at splunk support for investigation. Create a diag file and attach to your case running $Splunk_Home/bin/splunk diag

Run this command at cli to check if you are running on a cluster environment. If you are not you are receiving the message below. the admin id is required in order to get this information.
$Splunk_Home/bin/splunk show shcluster-status
Your session is invalid. Please login.
Splunk username: admin
Password:

Encountered some errors while trying to obtain shcluster status.
Search Head Clustering is not enabled on this node. REST endpoint is not available

If you see this information is valid, please vote to my answer. thanks

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

dkozinn — Wed, 30 Sep 2020 03:33:26 GMT

I've got a similar issue with a clean install of 8.0.1 under Ubuntu, though in my case no history shows up at all on the summary page and I get nothing at all back from |history.

In my case, there is a .csv file in $SPLUNK_HOME/etc/users/my_user/search/history and new searches get appended to that.

Everything is running on a single machine, no clustering.

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

forloop — Mon, 15 Jun 2020 19:20:12 GMT

I have the same exact issue. 8.0.4.1

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

BDein — Fri, 22 Jul 2022 15:04:11 GMT

Did you ever get a solution to this?

I'm running a single instance on Mac, and have same issue, but the csv is there, but it doesn't show up in "Search History" nor using | history

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

dkozinn — Fri, 22 Jul 2022 15:12:39 GMT

I'd forgotten about this post, I did get an resolution: For devtest instances, it turns out that a number of functions don't work properly if you've changed the name of the user to anything other than "admin". Once I changed it back to admin not only did history start working, but an issue I'd had with mail not being delivered got fixed as well.

Let me know if that fixes the issue for you.

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

BDein — Fri, 22 Jul 2022 15:20:51 GMT

So basically what you're telling is, that non-standard usernames can cause this issue, or what exactly are you describing here?

I've struggled with this for more than a year now, and got no usable answer anywhere...

my username in Splunk is nilsjul, but the history lookup file is named: "BDs-MacBook-Pro-2019-idx.csv"

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

dkozinn — Fri, 22 Jul 2022 15:29:12 GMT

For the license I have (Dev/Test), I can only have one user, which by default is "admin". I'd changed that, and from the information that I got, the user name not being "admin" on a Dev/Test instance is what causes the problems.

Note that this has nothing to do with the account on the machine that it's running on; This is the user that you provision within Splunk itself (Settings->Users).

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

BDein — Fri, 22 Jul 2022 15:34:32 GMT

Thanks for the info, but it still doesn't make much sense, unless Splunk has made some weird hardcode on the admin account for test/dev licenses.

Thanks again for your input:-)

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

BDein — Fri, 22 Jul 2022 15:49:46 GMT

For info, it actually worked as soon as I renamed my userid in etc/passwd👍

Re: Search History disappeared ( Splunk Enterprise 8.0.1)

dkozinn — Fri, 22 Jul 2022 16:01:15 GMT

Interesting that you did that. My instance of Splunk runs on a small Ubuntu server (no GUI) and I access from any number of different systems, but always using "admin" as a user ID when I have to log in.