topic Re: How to insert data from kv store to index with customized _time coloumn in Getting Data In

How to insert data from kv store to index with customized _time coloumn

jitendragupta — Tue, 29 Sep 2020 17:43:51 GMT

I am inserting data from kv store to Index but in index it is taking insertion time by default in _time column but I want my custom time same as from_date column in kv store.
How can I achieve this?

Re: How to insert data from kv store to index with customized _time coloumn

493669 — Tue, 16 Jan 2018 14:02:18 GMT

Hi @jitendragupta,
can you try below:

  | inputlookup kv_demo | head 10 | eval _time = from_date | table myid name _time| collect index= demoindex

Re: How to insert data from kv store to index with customized _time coloumn

mayurr98 — Tue, 16 Jan 2018 14:12:58 GMT

hey @jitendragupta

in your eval command you have assigned from_date to _time but in table you are using from_date
You should change that to _time instead. Also you want demoindex to index only 10 events? Or you are just trying this out?

| inputlookup kv_demo | head 10 | eval _time = from_date | table _time myid name | collect index=demoindex

This will store data into summary index called demoindex
If you directly run this command you will get error Received event for unconfigured/disabled/deleted index=demoindex with source="..and so on
So first create and index called demoindex and then run this query

let me know if this helps!

Re: How to insert data from kv store to index with customized _time coloumn

jitendragupta — Tue, 29 Sep 2020 17:40:29 GMT

Even after correcting search query as you said, i am not able to get my custom time in _time Column of index.

_time column is storing data insertion time by default.
I want to copy my custom time from from_date column of my kv store.
As u can see in this image from_date and _time are different:

Re: How to insert data from kv store to index with customized _time coloumn

jitendragupta — Wed, 17 Jan 2018 09:27:37 GMT

Image URL:

https://drive.google.com/open?id=1UU4E02SJmthdBvLChUxDUvtZ9ZBr8Egl

Re: How to insert data from kv store to index with customized _time coloumn

mayurr98 — Wed, 17 Jan 2018 09:39:28 GMT

okay i got it ! can you give me sample values from from_date field? i want to know the format of values are they in epoch time?

Re: How to insert data from kv store to index with customized _time coloumn

mayurr98 — Wed, 17 Jan 2018 09:45:12 GMT

run this

 | inputlookup kv_demo | head 10 | eval _time=strptime(from_date,"%d/%m/%Y") | table _time myid name | collect index=demoindex

let me know if this helps !

Re: How to insert data from kv store to index with customized _time coloumn

jitendragupta — Wed, 17 Jan 2018 09:46:15 GMT

Any format like dd/mm/yy for date and hh:mm:ss for time is ok with me. But the main thing which I am expecting is, from_date column should copy to _time column.

If this is possible than than only I can proceed with my work.

Re: How to insert data from kv store to index with customized _time coloumn

493669 — Wed, 17 Jan 2018 09:52:18 GMT

here you have to convert your time to epoch(in seconds) so that Time format to get understood by splunk
so try below it will work:

 | inputlookup kv_demo | head 10 | eval _time = from_date |eval _time=strptime(_time,"%d/%m/%Y")| table myid name _time| collect index= demoindex

using strptime time command it will parse your time field in format "%d/%m/%Y" and converts it in epoch then by table command you are entering required field to get collected in index.
Hope this helps you.
Cheers.

Re: How to insert data from kv store to index with customized _time coloumn

mayurr98 — Tue, 29 Sep 2020 17:44:16 GMT

it wont add anything specific if you have from_date="dd/mm/yy hh:mm:ss"
then you have to make change | eval _time=strptime(from_date,"%d/%m/%Y")
you have to give the format of from_date value

I can see from the screenshot that your from_date has only dd/mm/yy so accordingly i have changed my query