Splunk Search

Import a text data with a index

gsfiorese_au
Engager

Hello everyone, I'm starting with the development in splunk...

Each time a new database or datatable and created on the server, I have a service that generates a text file with the respective Info thereof, for example:

[hive_APOLLO_APOLLO_DATA]
vix.description = (hive orc) edb exit nbn database
vix.input.1.path = /data/tdc/prd/corp/base/edb/APOLLO/...
vix.provider = bud_hive_orc
vix.input.1.splitter.hive.dbname = APOLLO
vix.input.1.splitter.hive.tablename = APOLLO_DATA
vix.input.1.splitter.hive.fileformat = orc

[hive_APOLLO_EPI_IV_CLI]
vix.description = (hive orc) edb exit nbn database
vix.input.1.path = /data/tdc/prd/corp/base/edb/APOLLO/...
vix.provider = bud_hive_orc
vix.input.1.splitter.hive.dbname = APOLLO
vix.input.1.splitter.hive.tablename = EPI_IV_CLI
vix.input.1.splitter.hive.fileformat = orc

[hive_APOLLO_EPI_IV_NBN_CLI]
vix.description = (hive orc) edb exit nbn database
vix.input.1.path = /data/tdc/prd/corp/base/edb/APOLLO/...
vix.provider = bud_hive_orc
vix.input.1.splitter.hive.dbname = APOLLO
vix.input.1.splitter.hive.tablename = EPI_IV_NBN_CLI
vix.input.1.splitter.hive.fileformat = orc

...

Once the data is imported into splunk the need to show the dashboard the entire information for the file.

alt text

In this case when I click one of the results the system does the search and returns the information from block concerning the same, however, the index generated and generally for the file (main):

alt text

What do I need and the index is generated based on information contained in square brackets ([hive_APOLLO_TEST_PARTITION]) so when I run the search system bring something like this:

Main View
   DB1
      DB1.TBL1 (virtual.index.1) -> links to Child View (virtual.index.1)

 Child View (virtual.index.1)
 <Search results for splunk search: index=virtual.index.1>

 DB1=APOLLO
 TBL1=APOLLO_DATA
 virtual.index.1=hive_APOLLO_APOLLO_DATA

Sorry for the long text, someone help me?

Thank you all!

0 Karma
1 Solution

woodcock
Esteemed Legend

Your clarification was excellent; try this search:

search $database$ $datatable$ sourcetype="my_data" | stats values(vix_input_1_splitter_hive_tablename) BY vix_input_1_splitter_hive_dbname | rename vix_input_1_splitter_hive_dbname AS "Database Name" | rename vix_input_1_splitter_hive_tablename AS "Datatable Name"

View solution in original post

woodcock
Esteemed Legend

Your clarification was excellent; try this search:

search $database$ $datatable$ sourcetype="my_data" | stats values(vix_input_1_splitter_hive_tablename) BY vix_input_1_splitter_hive_dbname | rename vix_input_1_splitter_hive_dbname AS "Database Name" | rename vix_input_1_splitter_hive_tablename AS "Datatable Name"

gsfiorese_au
Engager

Thank you!! Woodcock!

0 Karma

woodcock
Esteemed Legend

I am having a great deal of trouble understanding very much of your request and it would help if you added more detail to your question (you can re-edit it). I think that I understand this statement:

Once the data is imported into splunk the need to show the dashboard the entire information for the file.

To do this, you can use a search like this:

... | reverse | stats list(_raw) by source

This will reconstitute every file almost exactly the way the file was when it was indexed. Hopefully I am understanding that part of your question and it helps you.

0 Karma

gsfiorese_au
Engager

Hi Woodcock,

Thanks for the reply and I'm sorry if I was not clear, if it is not uncomfortable try to explain better now.

Every day the txt file (myfile.txt) is imported into splunk and the data are shown through the dashboard below:

As I have many informations (1 for n) the visualization is compromised and too long, and many pages are generated and this complicates the search.

Apollo >> Table_BlaBla_1
Apollo >> Table_BlaBla_2
Apollo >> Table_BlaBla_3

Gendesk >> Table_Genx_1
Gendesk >> Table_Genx_2
Gendesk >> Table_Genx_3
Gendesk >> Table_Genx_4

I need the header once appears, and the dependencies come down (something like a tree view)

Apollo

Table_BlaBla_1
Table_BlaBla_2
Table_BlaBla_3

Gendesk

Table_Genx_1
Table_Genx_2
Table_Genx_3
Table_Genx_4

Below is the code I used to create the dashboard, if his reach to help me or anyone else will be grateful.

0 Karma

gsfiorese_au
Engager

Sorry I forget my code for dashboard:

<form>
  <label>Databases and Datatables - Search</label>
  <description>List of Databases and Datatables from HIVE</description>
  <fieldset autoRun="true" submitButton="true">    
    <input type="text" token="database">
      <label>Enter an Database Name</label>
      <default></default>
      <prefix>vix_input_1_splitter_hive_dbname=</prefix>
      <sufix></sufix>
    </input>
    <input type="text" token="datatable">
      <label>Enter an Database Name</label>
      <default></default>
      <prefix>vix_input_1_splitter_hive_tablename=</prefix>
      <sufix></sufix>
    </input>
  </fieldset>
  <row>
    <table>
       <title>Databases and Datatables list found in the file:</title>
       <search>
       <query>
         search $database$ $datatable$ sourcetype="my_data"
         | table vix_input_1_splitter_hive_dbname vix_input_1_splitter_hive_tablename 
         | rename vix_input_1_splitter_hive_dbname AS "Database Name" 
         | rename vix_input_1_splitter_hive_tablename AS "Datatable Name"           
       </query>
       </search>   
       <!--earliestTime>-60m@m</earliestTime-->
       <latestTime>now</latestTime>
       <option name="drilldown">row</option>
       <option name="count">10</option>
     </table>
  </row>
</form>

Tks,

Gabriel

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...