topic Re: How to build a dashboard to show extracted email details from a log file? in Dashboards & Visualizations

How to build a dashboard to show extracted email details from a log file?

moiezuddin — Tue, 10 Mar 2015 06:30:59 GMT

Please help to create a dashboard for log file. /opt/www/logs/nbcucentral/nbcucentral.log to extract users sso, last name, and email address

examples of this log file. /opt/www/logs/nbcucentral/nbcucentral.log

[09/03/2015] [06:57:57.510] [INFO ] [FILE: com.nbcu.registration.VerifyMailServlet.doPost() IP:- 10.99.145.82] [206456577] [User details Lipsit, Christopher :: Chri.Lipsit@bolfchannel.com]
[09/03/2015] [06:28:42.976] [INFO ] [FILE: com.nbcu.registration.VerifyMailServlet.doPost() IP:- 10.99.145.145] [206457105] [User details Pelfrey, John :: John.Pelf@bolfchannel.com]
[09/03/2015] [05:13:50.242] [INFO ] [FILE: com.nbcu.registration.VerifyMailServlet.doPost() IP:- 3.161.145.238] [206453165] [User details Douguet, Juliette :: Julie.Dou@bcun.com

Re: How to build a dashboard to show extracted email details from a log file?

satishsdange — Tue, 10 Mar 2015 08:35:35 GMT

Could you please try below -

index=test | rex "(?P\w+.\w+@\w+.\w+)" | rex "details (?P\w+)" | table Name, Email

Similarly you can extract SSO as well. You will find an option to save it as report/dashboard on right side (above time range picker).

Re: How to build a dashboard to show extracted email details from a log file?

satishsdange — Tue, 10 Mar 2015 08:38:03 GMT

Please append w with slash.

Re: How to build a dashboard to show extracted email details from a log file?

moiezuddin — Tue, 10 Mar 2015 08:53:36 GMT

It doesnot work

source="/opt/www/logs/nbcucentral/nbcucentral.log"| rex "(?Pw+.w+@w+.w+)" | rex "details (?Pw+)" | table Name Email

showing error

Error in 'rex' command: Encountered the following error while compiling the regex '(?Pw+.w+@w+.w+)': Regex: unrecognized character after (?P
The search job has failed due to an error. You may be able view the job in the Job Inspector

Re: How to build a dashboard to show extracted email details from a log file?

satishsdange — Tue, 10 Mar 2015 09:11:48 GMT

Did you use backslash before w?

Re: How to build a dashboard to show extracted email details from a log file?

moiezuddin — Tue, 10 Mar 2015 09:34:57 GMT

Its not working , can you please come up with another example , thanks for quick response

Re: How to build a dashboard to show extracted email details from a log file?

ppablo — Tue, 10 Mar 2015 21:11:48 GMT

Hi @moiezuddin

I just edited @satishdange's answer to properly show all the characters for the regular expression in the search. The backslashes \ were not displaying properly. Your search above didn't include the backslashes. Can you try the search now with the correctly syntax and see if you still get that error?

Re: How to build a dashboard to show extracted email details from a log file?

moiezuddin — Wed, 11 Mar 2015 07:01:42 GMT

Thanks for your effort , but still its not working.
I extracted new fields and tested and named the fields as per my requirement.
Now its showing the results
source="/opt/www/logs/nbcucentral/nbcucentral.log" LastName=* OR Email=* OR SSO=* | table SSO,Email,LastName
Can you please let me know how to get top 20 results of the above query.

Re: How to build a dashboard to show extracted email details from a log file?

satishsdange — Wed, 11 Mar 2015 07:18:27 GMT

Please try this

"your source" | rex "(?P<Email>\w+.\w+@\w+.\w+)" | rex "details (?P<Name>\w+)" | stats count by Name, Email | Head 20

Re: How to build a dashboard to show extracted email details from a log file?

moiezuddin — Wed, 11 Mar 2015 09:28:49 GMT

Thank you very much for your help.
Result showing in the dashboard is excellent
One thing is missing SSO
SSO is a field for getting userid"S
so how can i add SSO field in the QUERY

Re: How to build a dashboard to show extracted email details from a log file?

markthompson — Wed, 11 Mar 2015 10:27:14 GMT

What do you mean add it?

If you want to extract more, you can add another rex, if you already have it as a field, then add it after Email.

Re: How to build a dashboard to show extracted email details from a log file?

moiezuddin — Wed, 11 Mar 2015 11:11:03 GMT

Here i added SSO field

SSO field need to show Userid like 2065554822
But sso field showing name of the user not his userid .

What i need to do ? Kindly assist

Re: How to build a dashboard to show extracted email details from a log file?

moiezuddin — Wed, 11 Mar 2015 11:14:25 GMT

I did like this

source="/opt/www/logs/nbcucentral/nbcucentral.log" | rex "(?P\w+.\w+@\w+.\w+)" | rex "(?i)^(?:[^\\[]*\\[){5}(?P[^\\]]+)" | rex "details (?P\w+)" | stats count by Name, SSO, Email | Head 20

Its working showing the results exactly thank you very much for your time .
Great work boss .. 🙂

Re: How to build a dashboard to show extracted email details from a log file?

satishsdange — Wed, 11 Mar 2015 11:20:39 GMT

 "your source" | rex "(?P<Email>\w+.\w+@\w+.\w+)" | rex "details (?P<Name>\w+)" | rex "(?<User_ID>\d{9})" | stats count by Name, Email, User_ID | Head 20

Re: How to build a dashboard to show extracted email details from a log file?

moiezuddin — Thu, 19 Mar 2015 07:03:27 GMT

Hi,

Can you help me to add 2 more fields it the above query please
Fields are jobTitle, orgName, userType

Thanks