The event contains a 'before' and 'after' list of permissions and users SIDs, I can get splunk to extract the entire 'before' list and the entire 'after' list but only as single events. but i need to break it down to list  to indivudal Permission and SID   This it the entire event: 2020-12-07 22:45:51.123 91046 SUCCESS  Domain\User  Archive Permissions Archive 133481FD9531D0347BBCE92FFF45B4FE11110000evaultcol < Archive ArchiveID= " 133481FD9531D0347vaultcol " ArchiveName= " Last , First ">< OldManualSD > 😧 ( A ;; CCDCLCSWRPWPDT ;;; S-1-5-21-299502267-1960408961-839522115-10875 )( A ;; CCSW ;;; S-1-5-21-299502267-1960408961-839522115-2406856 )( A ;; CCSW ;;; S-1-5-21-299502267-1960408961-839522115-2406857 ) < /OldManualSD >< NewManualSD > 😧 ( A ;; CCDCLCSWRPWPDT ;;; S-1-5-21-299502267-1960408961-839522115-10875 )( A ;; CCSW ;;; S-1-5-21-299502267-1960408961-839522115-2406856 )( A ;; CCSW ;;; S-1-5-21-299502267-1960408961-839522115-2406857 ) ( A ;; CCDCSWRPDT ;;; S-1-5-21-299502267-1960408961-839522115-3949157 )< /NewManualSD >< /Archive > ServerName The 'before' list is between the  < OldManualSD > and <\OldManualSD> tags, the 'after' list is between the <NewManualSD> and </NewManualSD> tags The Permissions field is between the ;; and ;;; delimiters and is followed by the SID. There is a varying number of permsissons/SIDs in each event   Can get part way there; ex_OldManual_GP and ex_NewManual_GP fields extract from the "Info" field and the contain the before and after, but trying to get a second extraction based off ex_OldManual_GP and ex_NewManual_GP always fails    from the event above, I would like: OldManual = A;;CCDCLCSWRPWPDT;;;S-1-5-21-299502367-1960408961-839522117-10475 OldManual = A;;CCSW;;;S-1-5-21-299502367-1960408961-839522117-2406456 OldManual = A;;CCSW;;;S-1-5-21-299502367-1960408961-839522117-2406457 NewManual =  A;;CCDCLCSWRPWPDT;;;S-1-5-21-299502367-1960408961-839522117-10875 NewManual = A;;CCSW;;;S-1-5-21-299502367-1960408961-839522117-2406456 NewManual =  A;;CCSW;;;S-1-5-21-299502367-1960408961-839522117-2406457 NewManua l= A;;CCDCSWRPDT;;;S-1-5-21-299502367-1960408961-839522117-3949147 Any ideas?   my transforms.conf file: [ex_fields_extract] FIELDS = "AuditDate","AuditID","Status","UserName","CategoryName","SubCategoryName","ObjectID","Vault","info","MachineName" DELIMS = "\t" [ex_OldManual_GP] SOURCE_KEY = info REGEX=\>(<OldManualSD>D:)((?P<OldManual_GP>.*))(<\/OldManualSD>) [ex_NewManual_GP] SOURCE_KEY = info REGEX=\>(<NewManualSD>D:)((?P<NewManual_GP>.*))(<\/NewManualSD>) [ex_OldManual_MV] SOURCE_KEY = OldManual_GP REGEX=;;(?P<perm>\w+);;;* MV_ADD=true [ex_NewManual_MV] SOURCE_KEY = NewManual_GP REGEX=(?<NewManual>[^,]+),* MV_ADD=true   my props.conf file [exlogs] REPORT-ex_fields = ex_fields_extract REPORT-mvalue = ex_OldManual_MV, ex_NewManual_MV, ex_NewManual_GP, ex_OldManual_GP SHOULD_LINEMERGE = false   ... View more

We have recently turned on journaling within MS Exchange which basically sends a copy of every item to a journaling mail box. We know the email address the process uses and this appears in the message tracking logs as additional emails - in short journaling has doubled our Splunk licence usage!! We would like to exclude the journaling email address from being indexed. Exchange can't turn off message tracking for certain email addresses We are using UF on the Exchange servers with a load balanced intermediate level of Heavy forwarders. We have tried to apply the exclusion based on this answer - Answer 289736 - how to exclude a sourcetype from being indexed, but using a regex that picks up the journaling email address. The address we want to exclude is: journal@ev.local We've tried it on the Intermediate Heavy forwarders and on the index servers with no effect. The config we have applied is: props.conf [MSExchange:2013:MessageTracking] TRANSFORMS-JournalRemoval = JournalRemoval transforms.conf [JournalRemoval] REGEX =.*journal@ev.* DEST_KEY = queue FORMAT = nullqueue Any ideas why this might not be working? Thanks ... View more

We have layer of servers that act as a internet facing, intermediate forwarding layer providing an extra layer of separation to the hosts that may be connecting via the internet. The hosts are configured via a deployment server with these using the built in load balancing but we cannot find a way of identifying which server a received event has connected through. Ideally we would add a meta tag or a new field with the server name to the event as is passes through an intermediate server, but this does not appear to be possible (unless some one can tell me differently) We can get rough stats from metrics.log for the throughput and volume of data that is being forwarded, but is there any way of identifying what intermediate server an individual event ? ... View more

Owing to the way exchange outputs log files, for some reason we get two versions of the cs_username field username eg employeebob or Email address eg Bob.Employee@work.com Both versions exist in the active directory lookup file we have as "sAMAccountName" and "mail" and I want to get an output field of "Email Address". I can get lookup files to work on either version during a search, but not on both at the same time in the same search. Is there a way of running two lookups on the same file in the same search against the same field? Was looking at the "if" and "where" options, but they don't appear to work. Also, I tried to set two lookups in the same search..... index=msexchange sourcetype="MSWindows:2008R2:IIS" WebApplication="Microsoft-Server-ActiveSync" Cmd=Sync | lookup User_Info mail AS cs_username OUTPUT l AS Location, title AS Title, department AS Department, mail AS "Email Address" | lookup User_Info sAMAccountName AS cs_username OUTPUT l AS Location, title AS Title, department AS Department, mail AS "Email Address" ... View more

I have an access log from a document system that includes a username and the type of action that was carried out on the document. The documents are all uniquely named and there are thousands of them. There are currently 5 actions a user can carry out on a file (lets call them A, B, C, D and E) and each log entry will include one these. The user can do more than one action be document per day, and the user can do the same action multiple times per day on a single document. All the actions are Want I would like to display on a single line of a table is: UserName Total_Docs_Accessed Total_Actions "Number Of Docs With Action A", "Number Of Docs With Action B", "Number Of Docs With Action C" etc Some times there will be a ZERO count for some of the actions I have been able to get this to work with one line for each action for each user, but that gives up to 5 lines per user, and I have thousands of users. I've tried join'ing, eval'ing, stat'ing and nothing seem to work where there is more than one user. Thanks Below is a simplified example of the log file Time, User, File, Action 01/01/16 10:28 User1 File1.doc A 01/01/16 10:29 User1 File2.doc A 01/01/16 10:30 User1 File2.doc C 01/01/16 10:32 User1 File2.doc B 01/01/16 10:34 User1 File3.doc E 01/01/16 10:35 User1 File1.doc A 01/01/16 10:36 User2 File4.doc A 01/01/16 10:37 User3 File1.doc D 01/01/16 10:38 User4 File5.doc D 01/01/16 10:39 User4 File5.doc A This should give a table thus: Username, Total Docs, Total Actions, Action A, Action B, Action C, Action D, Action E User1, 3, 6, 3, 1, 1, 0 1 User2, 1, 1, 0, 0, 0, 0, 0 User3, 1, 1, 0, 0, 0, 1, 0 User4, 1, 1, 1, 0, 0, 1, 0 ... View more

I'm running a summary index tracking event counts from a number of servers for each day. I'm trying to put up a dashboard that tracks the number in events by day and by servers over the past 30 days. The search works fine, but when I try to chart this, the date format is all messed up because it only sorts on the day field of the time stamp. index=summary source="summary-search earliest=-30d@d latest=@d | convert timeformat="%d/%m/%y" ctime(info_max_time) AS Date | chart count over Date by orig_host info_max_time is an epoch timestamp, I've tried various | sort options (_time, Date, _indextime, info_max_time) after the chart command that all work in tables but not for charts. Managers like pretty graphs and the US date format confuses them 🙂 ... View more

I have the transaction command running successfully to identify where users log on and log off from our HDI environment using two different log files that end up in the same index, but with different sourcetypes (LogOn or LogOff) One feature of the HDI is they can move location and take there session with them. This results in the IP address for the log off being different from the log on IP address. I'm trying to work out how to identify the user that have moved location during a session The _raw is the same for both LogOn and LogOff: Date, time, server, username, IPaddress The transaction works on the Username and the two different sourcetypes. This works fine and a typical transaction is: 2015/11/03, 23:23:00, ServerA, bobjones, 10.10.10.10 2015/11/03, 20:20:00, ServerA, bobjones, 10.10.10.10 IPAddress=10.10.10.10 | server=ServerA | sourcetype=LogOn sourcetype=LogOff |Username=bobjones If Bob decides to move location during the day, his transaction look like this: 2015/11/02, 23:23:00, ServerA, bobjones, 10.10.10.20 2015/11/02, 20:20:00, ServerA, bobjones, 10.10.10.10 IPAddress=10.10.10.10 IPAddress=10.10.10.20 | server=ServerA | sourcetype=LogOn sourcetype=LogOff |Username=bobjones The transaction is correct, but how can I report only on users who have different IP addresses within a transaction so I can post it to a dashboard? Tried distinct counts to no avail because I need to dc within the transaction, not on the _raw. Thanks ... View more