topic Re: Timestamps for two different field in Getting Data In

Timestamps for two different field

johnwyane — Mon, 28 Sep 2020 14:42:42 GMT

Hi,
I met one log file that have two timestamps on different field.
The first one is the exported time by program on the first line. However, it's not the real time of the syslog. The second one will be in each line of the syslog but only have the hour time without year,month and day. For the following sample log. Supposedly I want to have the timestamps format like 20130901 15-3-0-126 , any way to do this? Please advise. Thank you very much!

FILE

|service_request-success-15-3-0-126-196|||||#0(nocausecode)|ril3_cause|||||||||||||||||||||||||466-01-9772-1-0-32482|||466015122237199-3962050817-0124210040450917-886938657363||||||||||||0||||466015122237199-3962050817-0124210040450917-8869386573635-211.77.228.233-114.140.116.65-undefined|||wcdma|0||ms||data|||#0(bearer successful)|||||||

|service_request-success-15-3-0-137-140|||||#0(nocausecode)|ril3_cause|||||||||||||||||||||||||466-01-9113-1-0-25658|||466015801456696-4280957185-3556660525560234-886989688406||||||||||||0||||466015801456696-4280957185-3556660525560234-8869896884065-211.77.232.34-110.26.113.23-undefined|||wcdma|0||ms||data|||#0(bearer successful)|||||||

Re: Timestamps for two different field

tincupchalice — Mon, 28 Sep 2020 14:42:48 GMT

I did not see the date in the event, just the source file so to answer this I used the following...



*

| eval file="/home/f65351/EBM/A20130901.1600+0800-20130901.1615+0800_10_ebs.1158"

| eval fname=mvindex(split(file, "/"), -1)

| eval date=substr(mvindex(split(fname, "."), 0),2)

| eval blah="|service_request-success-15-3-0-137-140|||||#0(nocausecode)|ril3_cause|||||||||||||||||||||||||466-01-9113-1-0-25658|||466015801456696-4280957185-3556660525560234-886989688406||||||||||||0||||466015801456696-4280957185-3556660525560234-8869896884065-211.77.232.34-110.26.113.23-undefined|||wcdma|0||ms||data|||#0(bearer  successful)|||||||"

| rex field=blah "service_request\-success\-(?\d{1,2})\-(?\d{1,2})\-(?\d{1,2})\-(?\d{1,3})\-(?\d{1,3})"

| eval ts=date . " " . hour . "-" . min . "-" . sec . "-" . msec

| eval uts = ts . usec

| eval numts=strptime(uts, "%Y%m%d %H-%M-%S-%f")

| eval sqlts=strftime(numts, "%Y-%m-%d %H:%M:%S.%f")

| stats count by date hour min sec msec usec ts uts numts sqlts

Now you will replace file with source and blah with _raw.
ts I believe answers your question.
uts adds what I think is the extra microseconds.
numts gives you an epoch number for that exact time.
sqlts is how you can now use that number for other types of applications (like sql).

Re: Timestamps for two different field

johnwyane — Mon, 09 Sep 2013 06:46:33 GMT

Hi,
This is a magnificent answer. I spent lots of time to study it. However,everything is work find until rex field . So I didn't get the final answer since I don't have any ideas to modify your answer. But still many thanks.

Also,Point is from 1 to 10 ??

Re: Timestamps for two different field

tincupchalice — Mon, 09 Sep 2013 13:04:20 GMT

I just realized that the rex field got screwed up in formatting. I fixed it above by adding a \ where appropriate.
I don't know how the points work to be honest 😛