Splunk Search
Highlighted

Import a text data with a index

Engager

Hello everyone, I'm starting with the development in splunk...

Each time a new database or datatable and created on the server, I have a service that generates a text file with the respective Info thereof, for example:

[hive_APOLLO_APOLLO_DATA]
vix.description = (hive orc) edb exit nbn database
vix.input.1.path = /data/tdc/prd/corp/base/edb/APOLLO/...
vix.provider = bud_hive_orc
vix.input.1.splitter.hive.dbname = APOLLO
vix.input.1.splitter.hive.tablename = APOLLO_DATA
vix.input.1.splitter.hive.fileformat = orc

[hive_APOLLO_EPI_IV_CLI]
vix.description = (hive orc) edb exit nbn database
vix.input.1.path = /data/tdc/prd/corp/base/edb/APOLLO/...
vix.provider = bud_hive_orc
vix.input.1.splitter.hive.dbname = APOLLO
vix.input.1.splitter.hive.tablename = EPI_IV_CLI
vix.input.1.splitter.hive.fileformat = orc

[hive_APOLLO_EPI_IV_NBN_CLI]
vix.description = (hive orc) edb exit nbn database
vix.input.1.path = /data/tdc/prd/corp/base/edb/APOLLO/...
vix.provider = bud_hive_orc
vix.input.1.splitter.hive.dbname = APOLLO
vix.input.1.splitter.hive.tablename = EPI_IV_NBN_CLI
vix.input.1.splitter.hive.fileformat = orc

...

Once the data is imported into splunk the need to show the dashboard the entire information for the file.

alt text

In this case when I click one of the results the system does the search and returns the information from block concerning the same, however, the index generated and generally for the file (main):

alt text

What do I need and the index is generated based on information contained in square brackets ([hiveAPOLLOTEST_PARTITION]) so when I run the search system bring something like this:

Main View
   DB1
      DB1.TBL1 (virtual.index.1) -> links to Child View (virtual.index.1)

 Child View (virtual.index.1)
 <Search results for splunk search: index=virtual.index.1>

 DB1=APOLLO
 TBL1=APOLLO_DATA
 virtual.index.1=hive_APOLLO_APOLLO_DATA

Sorry for the long text, someone help me?

Thank you all!

0 Karma
Highlighted

Re: Import a text data with a index

Esteemed Legend

I am having a great deal of trouble understanding very much of your request and it would help if you added more detail to your question (you can re-edit it). I think that I understand this statement:

Once the data is imported into splunk the need to show the dashboard the entire information for the file.

To do this, you can use a search like this:

... | reverse | stats list(_raw) by source

This will reconstitute every file almost exactly the way the file was when it was indexed. Hopefully I am understanding that part of your question and it helps you.

0 Karma
Highlighted

Re: Import a text data with a index

Engager

Hi Woodcock,

Thanks for the reply and I'm sorry if I was not clear, if it is not uncomfortable try to explain better now.

Every day the txt file (myfile.txt) is imported into splunk and the data are shown through the dashboard below:

As I have many informations (1 for n) the visualization is compromised and too long, and many pages are generated and this complicates the search.

Apollo >> TableBlaBla1
Apollo >> TableBlaBla2
Apollo >> TableBlaBla3

Gendesk >> TableGenx1
Gendesk >> TableGenx2
Gendesk >> TableGenx3
Gendesk >> TableGenx4

I need the header once appears, and the dependencies come down (something like a tree view)

Apollo

TableBlaBla1
TableBlaBla2
TableBlaBla3

Gendesk

TableGenx1
TableGenx2
TableGenx3
TableGenx4

Below is the code I used to create the dashboard, if his reach to help me or anyone else will be grateful.

0 Karma
Highlighted

Re: Import a text data with a index

Engager

Sorry I forget my code for dashboard:

<form>
  <label>Databases and Datatables - Search</label>
  <description>List of Databases and Datatables from HIVE</description>
  <fieldset autoRun="true" submitButton="true">    
    <input type="text" token="database">
      <label>Enter an Database Name</label>
      <default></default>
      <prefix>vix_input_1_splitter_hive_dbname=</prefix>
      <sufix></sufix>
    </input>
    <input type="text" token="datatable">
      <label>Enter an Database Name</label>
      <default></default>
      <prefix>vix_input_1_splitter_hive_tablename=</prefix>
      <sufix></sufix>
    </input>
  </fieldset>
  <row>
    <table>
       <title>Databases and Datatables list found in the file:</title>
       <search>
       <query>
         search $database$ $datatable$ sourcetype="my_data"
         | table vix_input_1_splitter_hive_dbname vix_input_1_splitter_hive_tablename 
         | rename vix_input_1_splitter_hive_dbname AS "Database Name" 
         | rename vix_input_1_splitter_hive_tablename AS "Datatable Name"           
       </query>
       </search>   
       <!--earliestTime>-60m@m</earliestTime-->
       <latestTime>now</latestTime>
       <option name="drilldown">row</option>
       <option name="count">10</option>
     </table>
  </row>
</form>

Tks,

Gabriel

0 Karma
Highlighted

Re: Import a text data with a index

Esteemed Legend

Your clarification was excellent; try this search:

search $database$ $datatable$ sourcetype="my_data" | stats values(vix_input_1_splitter_hive_tablename) BY vix_input_1_splitter_hive_dbname | rename vix_input_1_splitter_hive_dbname AS "Database Name" | rename vix_input_1_splitter_hive_tablename AS "Datatable Name"

View solution in original post

Highlighted

Re: Import a text data with a index

Engager

Thank you!! Woodcock!

0 Karma