Timestamps for two different field

[johnwyane]

Hi,
I met one log file that have two timestamps on different field.
The first one is the exported time by program on the first line. However, it's not the real time of the syslog. The second one will be in each line of the syslog but only have the hour time without year,month and day. For the following sample log. Supposedly I want to have the timestamps format like 20130901 15-3-0-126 , any way to do this? Please advise. Thank you very much!

FILE

Input file=/home/f65351/EBM/A20130901.1600+0800-20130901.1615+0800_10_ebs.1158
header|l_header|activation_type|apn|attach_type|bearers|cause_code|cause_prot_type|ci|combined_tau_type|cs_fallback_service_type|deactivation_trigger|deconnect_pdn_type|default_bearer_id|detach_trigger|detach_type|dropped_pdp|duration|eci|ggsn|handover_node_role|handover_rat_change_type

|service_request-success-15-3-0-126-196|||||#0(nocausecode)|ril3_cause|||||||||||||||||||||||||466-01-9772-1-0-32482|||466015122237199-3962050817-0124210040450917-886938657363||||||||||||0||||466015122237199-3962050817-0124210040450917-8869386573635-211.77.228.233-114.140.116.65-undefined|||wcdma|0||ms||data|||#0(bearer successful)|||||||

|service_request-success-15-3-0-137-140|||||#0(nocausecode)|ril3_cause|||||||||||||||||||||||||466-01-9113-1-0-25658|||466015801456696-4280957185-3556660525560234-886989688406||||||||||||0||||466015801456696-4280957185-3556660525560234-8869896884065-211.77.232.34-110.26.113.23-undefined|||wcdma|0||ms||data|||#0(bearer successful)|||||||

[tincupchalice · Accepted solution]

I did not see the date in the event, just the source file so to answer this I used the following...

*

| eval file="/home/f65351/EBM/A20130901.1600+0800-20130901.1615+0800_10_ebs.1158"

| eval fname=mvindex(split(file, "/"), -1)

| eval date=substr(mvindex(split(fname, "."), 0),2)

| eval blah="|service_request-success-15-3-0-137-140|||||#0(nocausecode)|ril3_cause|||||||||||||||||||||||||466-01-9113-1-0-25658|||466015801456696-4280957185-3556660525560234-886989688406||||||||||||0||||466015801456696-4280957185-3556660525560234-8869896884065-211.77.232.34-110.26.113.23-undefined|||wcdma|0||ms||data|||#0(bearer  successful)|||||||"

| rex field=blah "service_request\-success\-(?\d{1,2})\-(?\d{1,2})\-(?\d{1,2})\-(?\d{1,3})\-(?\d{1,3})"

| eval ts=date . " " . hour . "-" . min . "-" . sec . "-" . msec

| eval uts = ts . usec

| eval numts=strptime(uts, "%Y%m%d %H-%M-%S-%f")

| eval sqlts=strftime(numts, "%Y-%m-%d %H:%M:%S.%f")

| stats count by date hour min sec msec usec ts uts numts sqlts

Now you will replace file with source and blah with _raw.
ts I believe answers your question.
uts adds what I think is the extra microseconds.
numts gives you an epoch number for that exact time.
sqlts is how you can now use that number for other types of applications (like sql).

[tincupchalice]

I just realized that the rex field got screwed up in formatting. I fixed it above by adding a \ where appropriate.
I don't know how the points work to be honest 😛

[johnwyane]

Hi,
This is a magnificent answer. I spent lots of time to study it. However,everything is work find until rex field . So I didn't get the final answer since I don't have any ideas to modify your answer. But still many thanks.

Also,Point is from 1 to 10 ??