I have a log files where it contains duplicates like "json from session" log duplicates .. so the log which contains this "json from session" , that kind of user should be eliminated from count of unique users.....
Example logs are.........
Log 1 Mar 3, 2012 9:34:00 AM context log
Info: AuthProfile :: lastname="Dilshan",firstname="tilak",siteid=>"IND123G"......
Log 2 Mar 3, 2012 9:34:00 AM context log
Info: access ....
Log 3 Mar 3, 2012 9:34:00 AM context log
Info : transaction ............
Log 4 Mar 3, 2012 9:34:00 AM context log
Info : Authenticat : retrieved non-empty json: {lastName ........
Log 5 Mar 3, 2012 9:34:00 AM context log
Info : Authenticat : json from session= lastname="Dilshan",firstname="tilak",siteid="IND123G"
Log 7 Mar 4, 2012 10:12:34 AM context log
Info : action ee.........
Log 8 Mar 4, 2012 10:12:34 AM context log
Info : AuthProfile :: lastname="Micheal",firstname="John",siteid=>"AUS123G"......
Log 9 Mar 4, 2012 10:12:34 AM context log
Info: access ....
Log 10 Mar 4, 2012 10:12:34 AM context log
Info : transaction ............
Log 11 Mar 4, 2012 10:12:34 AM context log
Info : Authenticat : retrieved non-empty json: {lastName ........
Log 12 Mar 5, 2012 10:12:34 AM context log
Info : transaction processing ..............
So like this i have N number of logs in which i have to identify unique users without duplicates like..............from AuthProfile ... Unique user "John" and count is 1 .....
but not Dilshan..it contains ( json from session ) So , it is duplicate... So , it must be eliminated ....how to make this ,,,can u guide me ...plz .............
This should work for you:
index="your_index_name" sourcetype="your_source_type_name" AND AuthProfile|eval user=firstname+""+lastname|stats dc(user) as Distinct_User
This should work for you:
index="your_index_name" sourcetype="your_source_type_name" AND AuthProfile|eval user=firstname+""+lastname|stats dc(user) as Distinct_User
yes i want user count without having "json from session" entry...
The way your question is worded, it seems to me that Ipolo's answer is correct.
Perhaps we would understand it better if you described the result you want, rather than the logic.
Are you saying
- Count the number of unique users (based on user name) BUT
- DO NOT COUNT any users who have a "json from session" entry
this logic is also looks cool but not working , so I want a logic
like this for example
search AuthProfile(log 1) where the line from it (log 5) not json from session ... if this condition satisfies
then count user .... else leave that user
all my logs files contains AuthProfile along with some "json from session" log repeated exactly at each 5th log from AuthProfile ....for duplicates..... and some does not contain the json at the 5th log which is the original ...which i need to predict in as per the above logic ...