topic Re: Importing Data From One index to my Splunk Enterprise in Getting Data In

Importing Data From One index to my Splunk Enterprise

ivialex — Fri, 11 Oct 2019 21:41:35 GMT

Hi guys,

I am trying to import data from an index provided by the instructor of a Splunk training course.

Follow the steps below:

To Import Course Example Data:

Navigate to Settings—>Indexes—>New Index
Create a new index with the desired name
Save the new index
Use file transfer program to transfer the four folders into new index folder within the Splunk OS
    *Nix: /opt/splunk/var/lib/splunk/INDEX_NAME
Search imported data by searching just this index

The file mentioned above has the four folders: colddb, datamodel_summary, db and thaweddb.

After copying all the above files, skipping copying the .bucketManifest and CreationTime files.

The next step I did was restart no splunk.

This procedure did not work. The current size of my index was 0B.

That is, it seems that my Splunk Enterprise (Indexer) did not recognize the index data copied and provided by the instructor.

What can it be?

Re: Importing Data From One index to my Splunk Enterprise

anthonymelita — Fri, 11 Oct 2019 23:30:29 GMT

Did you make sure the files have the same permissions? For example owned by the splunk user.

Re: Importing Data From One index to my Splunk Enterprise

richgalloway — Sat, 12 Oct 2019 00:00:53 GMT

Have you contacted the instructor?

Re: Importing Data From One index to my Splunk Enterprise

gcusello — Wed, 30 Sep 2020 02:31:52 GMT

Hi ivialex,
did you created indexes.conf before restart Splunk?
the correct procedure should be:

create an indexes.conf or add to an existing one the information about the new index: [sample] homePath = $SPLUNK_DB\sample\db coldPath = $SPLUNK_DB\sample\colddb thawedPath = $SPLUNK_DB\sample\thaweddb
create a folder in $SPLUNK_HOME/var/lib/splunk/my_index or in your $SPLUNK_DB
copy the four subfolders under my_index
give the same grants and ownership of the other indexes
restart Splunk

Bye.
Giuseppe

Re: Importing Data From One index to my Splunk Enterprise

woodcock — Sat, 12 Oct 2019 17:00:39 GMT

You realize that INDEX_NAME is a placeholder, right? You have to substitute INDEX_NAME text for the actual name of the index that you created from the GUI.

Re: Importing Data From One index to my Splunk Enterprise

ivialex — Sat, 12 Oct 2019 19:54:37 GMT

Hi @anthonymelita . I checked and I'll try to import and start with the admin user. I create the index, after I stop my service in Windows. Then, I delete all folder inside my index. After I copy the four new folder and start the service. But, it didn't work too.

Re: Importing Data From One index to my Splunk Enterprise

ivialex — Sat, 12 Oct 2019 19:55:48 GMT

Hi @richgalloway . Yes, I send an email to my instrutor. He reply my asks and I'll try his instructions.

Re: Importing Data From One index to my Splunk Enterprise

ivialex — Wed, 30 Sep 2020 02:36:24 GMT

Hi @gcusello ,

I tried to follow your instructions as bellow:

index definitions

[pluralsight_generating_tailored_searches_splunk]
homePath = $SPLUNK_DB\pluralsight_generating_tailored_searches_splunk\db
coldPath = $SPLUNK_DB\pluralsight_generating_tailored_searches_splunk\colddb
thawedPath = $SPLUNK_DB\pluralsight_generating_tailored_searches_splunk\thaweddb
maxDataSize = 100

And yet, it doesn't start splunk service on my windows.

Re: Importing Data From One index to my Splunk Enterprise

ivialex — Sun, 13 Oct 2019 01:07:39 GMT

Hi @woodcock ,

My INDEX_NAME is in this path in my windows machine: C:\Program Files\Splunk\var\lib\splunk\

And this index folder is the same name that I created in my GUI Splunk Enterprise.

Re: Importing Data From One index to my Splunk Enterprise

gcusello — Wed, 30 Sep 2020 02:32:04 GMT

Hi ivialex,
you can see the value of $SPLUNK_DB variable in $SPLUNK_HOME\etc\splunk-launch.conf
usually is commented.
If it's commented you can replace $SPLUNK_DB with $SPLUNK_HOME\var\lib\splunk

Then, don'r use maxDataSize = 100 because in this way you could delete some data.

When you try to restart windows services, use the cmd window with administration grants, in this way you can see if there's any problem.

Bye.
Giuseppe

Re: Importing Data From One index to my Splunk Enterprise

ivialex — Wed, 30 Sep 2020 02:36:32 GMT

Hi @gcusello ,

My local indexes.conf as bellow:

[pluralsight_generating_tailored_searches_splunk]
homePath =
$SPLUNK_DB\pluralsight_generating_tailored_searches_splunk\db
coldPath =
$SPLUNK_DB\pluralsight_generating_tailored_searches_splunk\colddb
thawedPath =
$SPLUNK_DB\pluralsight_generating_tailored_searches_splunk\thaweddb

My splunk-launch.conf as bellow:

Version 7.3.2

Modify the following line to suit the location of your Splunk install.

If unset, Splunk will use the parent of the directory containing the splunk

CLI executable.

SPLUNK_HOME=C:\Program Files\Splunk

By default, Splunk stores its indexes under SPLUNK_HOME in the

var\lib\splunk subdirectory. This can be overridden

here:

SPLUNK_DB=$SPLUNK_HOME\var\lib\splunk

Splunkd service name SPLUNK_SERVER_NAME=Splunkd

Splunkweb service name SPLUNK_WEB_NAME=splunkweb

The result of the using the cmd window with administration grants as bellow:

C:\Program Files\Splunk\bin>splunk
start --accept-license

Splunk> The Notorious B.I.G. D.A.T.A.

Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Done.
Checking critical directories... Done
Checking indexes...
(skipping validation of index paths because not running as
LocalSystem)
Validated: _audit _internal _introspection _telemetry _thefishbucket edureka_access_combined_wcookie
history main
pluralsight_generating_tailored_searches_splunk
summary
Done
Checking filesystem compatibility... Done
Checking conf files for problems...
Done
Checking default conf files for edits...
Validating installed files against hashes from 'C:\Program
Files\Splunk\splunk-7.3.2-c60db69f8e32-windows-64-manifest'
All installed files intact.
Done All preliminary checks passed.

Starting splunk server daemon
(splunkd)...

Splunkd: Starting (pid 12628)

Timed out waiting for splunkd to
start.

C:\Program Files\Splunk\bin>

And it didn't work fine. My instrutor send me the .csv file to import data. I believe that is conflict between data system because are diferrent operate system.
Then I will try to install Splunk on a Linux for example, on a virtual machine and try the same procedure to see if this problem is due to having exported the data on an operating system (Linux or Mac) and trying to import on a Windows.

Re: Importing Data From One index to my Splunk Enterprise

gcusello — Mon, 14 Oct 2019 07:05:10 GMT

Hi ivialex,
this means that the $SPLUNK_DB is the default one.

Please, check you indexes.conf files, probably you have your index in more than one file.

Ciao.
Giuseppe