We get a weekly ingest of a data set for our vulnerability management. Each line contains a unique value matching a vulnerability with a server I want to be able to report on: a. how many new vulnerabilities are in this weeks report compared to last week and b. how many vulnerabilities have been fixed (so are not reported) in this weeks list compared to last week I'm looking for splunk to tell me whats new and whats missing week by week but also track these over the long term.  Cant seem to get any meaningful results with a 'set diff' search   Any help gratefully received!! ... View more

The event contains a 'before' and 'after' list of permissions and users SIDs, I can get splunk to extract the entire 'before' list and the entire 'after' list but only as single events. but i need to break it down to list  to indivudal Permission and SID   This it the entire event: 2020-12-07 22:45:51.123 91046 SUCCESS  Domain\User  Archive Permissions Archive 133481FD9531D0347BBCE92FFF45B4FE11110000evaultcol < Archive ArchiveID= " 133481FD9531D0347vaultcol " ArchiveName= " Last , First ">< OldManualSD > 😧 ( A ;; CCDCLCSWRPWPDT ;;; S-1-5-21-299502267-1960408961-839522115-10875 )( A ;; CCSW ;;; S-1-5-21-299502267-1960408961-839522115-2406856 )( A ;; CCSW ;;; S-1-5-21-299502267-1960408961-839522115-2406857 ) < /OldManualSD >< NewManualSD > 😧 ( A ;; CCDCLCSWRPWPDT ;;; S-1-5-21-299502267-1960408961-839522115-10875 )( A ;; CCSW ;;; S-1-5-21-299502267-1960408961-839522115-2406856 )( A ;; CCSW ;;; S-1-5-21-299502267-1960408961-839522115-2406857 ) ( A ;; CCDCSWRPDT ;;; S-1-5-21-299502267-1960408961-839522115-3949157 )< /NewManualSD >< /Archive > ServerName The 'before' list is between the  < OldManualSD > and <\OldManualSD> tags, the 'after' list is between the <NewManualSD> and </NewManualSD> tags The Permissions field is between the ;; and ;;; delimiters and is followed by the SID. There is a varying number of permsissons/SIDs in each event   Can get part way there; ex_OldManual_GP and ex_NewManual_GP fields extract from the "Info" field and the contain the before and after, but trying to get a second extraction based off ex_OldManual_GP and ex_NewManual_GP always fails    from the event above, I would like: OldManual = A;;CCDCLCSWRPWPDT;;;S-1-5-21-299502367-1960408961-839522117-10475 OldManual = A;;CCSW;;;S-1-5-21-299502367-1960408961-839522117-2406456 OldManual = A;;CCSW;;;S-1-5-21-299502367-1960408961-839522117-2406457 NewManual =  A;;CCDCLCSWRPWPDT;;;S-1-5-21-299502367-1960408961-839522117-10875 NewManual = A;;CCSW;;;S-1-5-21-299502367-1960408961-839522117-2406456 NewManual =  A;;CCSW;;;S-1-5-21-299502367-1960408961-839522117-2406457 NewManua l= A;;CCDCSWRPDT;;;S-1-5-21-299502367-1960408961-839522117-3949147 Any ideas?   my transforms.conf file: [ex_fields_extract] FIELDS = "AuditDate","AuditID","Status","UserName","CategoryName","SubCategoryName","ObjectID","Vault","info","MachineName" DELIMS = "\t" [ex_OldManual_GP] SOURCE_KEY = info REGEX=\>(<OldManualSD>D:)((?P<OldManual_GP>.*))(<\/OldManualSD>) [ex_NewManual_GP] SOURCE_KEY = info REGEX=\>(<NewManualSD>D:)((?P<NewManual_GP>.*))(<\/NewManualSD>) [ex_OldManual_MV] SOURCE_KEY = OldManual_GP REGEX=;;(?P<perm>\w+);;;* MV_ADD=true [ex_NewManual_MV] SOURCE_KEY = NewManual_GP REGEX=(?<NewManual>[^,]+),* MV_ADD=true   my props.conf file [exlogs] REPORT-ex_fields = ex_fields_extract REPORT-mvalue = ex_OldManual_MV, ex_NewManual_MV, ex_NewManual_GP, ex_OldManual_GP SHOULD_LINEMERGE = false   ... View more

We have recently turned on journaling within MS Exchange which basically sends a copy of every item to a journaling mail box. We know the email address the process uses and this appears in the message tracking logs as additional emails - in short journaling has doubled our Splunk licence usage!! We would like to exclude the journaling email address from being indexed. Exchange can't turn off message tracking for certain email addresses We are using UF on the Exchange servers with a load balanced intermediate level of Heavy forwarders. We have tried to apply the exclusion based on this answer - Answer 289736 - how to exclude a sourcetype from being indexed, but using a regex that picks up the journaling email address. The address we want to exclude is: journal@ev.local We've tried it on the Intermediate Heavy forwarders and on the index servers with no effect. The config we have applied is: props.conf [MSExchange:2013:MessageTracking] TRANSFORMS-JournalRemoval = JournalRemoval transforms.conf [JournalRemoval] REGEX =.*journal@ev.* DEST_KEY = queue FORMAT = nullqueue Any ideas why this might not be working? Thanks ... View more

We have layer of servers that act as a internet facing, intermediate forwarding layer providing an extra layer of separation to the hosts that may be connecting via the internet. The hosts are configured via a deployment server with these using the built in load balancing but we cannot find a way of identifying which server a received event has connected through. Ideally we would add a meta tag or a new field with the server name to the event as is passes through an intermediate server, but this does not appear to be possible (unless some one can tell me differently) We can get rough stats from metrics.log for the throughput and volume of data that is being forwarded, but is there any way of identifying what intermediate server an individual event ? ... View more

Owing to the way exchange outputs log files, for some reason we get two versions of the cs_username field username eg employeebob or Email address eg Bob.Employee@work.com Both versions exist in the active directory lookup file we have as "sAMAccountName" and "mail" and I want to get an output field of "Email Address". I can get lookup files to work on either version during a search, but not on both at the same time in the same search. Is there a way of running two lookups on the same file in the same search against the same field? Was looking at the "if" and "where" options, but they don't appear to work. Also, I tried to set two lookups in the same search..... index=msexchange sourcetype="MSWindows:2008R2:IIS" WebApplication="Microsoft-Server-ActiveSync" Cmd=Sync | lookup User_Info mail AS cs_username OUTPUT l AS Location, title AS Title, department AS Department, mail AS "Email Address" | lookup User_Info sAMAccountName AS cs_username OUTPUT l AS Location, title AS Title, department AS Department, mail AS "Email Address" ... View more

I have an access log from a document system that includes a username and the type of action that was carried out on the document. The documents are all uniquely named and there are thousands of them. There are currently 5 actions a user can carry out on a file (lets call them A, B, C, D and E) and each log entry will include one these. The user can do more than one action be document per day, and the user can do the same action multiple times per day on a single document. All the actions are Want I would like to display on a single line of a table is: UserName Total_Docs_Accessed Total_Actions "Number Of Docs With Action A", "Number Of Docs With Action B", "Number Of Docs With Action C" etc Some times there will be a ZERO count for some of the actions I have been able to get this to work with one line for each action for each user, but that gives up to 5 lines per user, and I have thousands of users. I've tried join'ing, eval'ing, stat'ing and nothing seem to work where there is more than one user. Thanks Below is a simplified example of the log file Time, User, File, Action 01/01/16 10:28 User1 File1.doc A 01/01/16 10:29 User1 File2.doc A 01/01/16 10:30 User1 File2.doc C 01/01/16 10:32 User1 File2.doc B 01/01/16 10:34 User1 File3.doc E 01/01/16 10:35 User1 File1.doc A 01/01/16 10:36 User2 File4.doc A 01/01/16 10:37 User3 File1.doc D 01/01/16 10:38 User4 File5.doc D 01/01/16 10:39 User4 File5.doc A This should give a table thus: Username, Total Docs, Total Actions, Action A, Action B, Action C, Action D, Action E User1, 3, 6, 3, 1, 1, 0 1 User2, 1, 1, 0, 0, 0, 0, 0 User3, 1, 1, 0, 0, 0, 1, 0 User4, 1, 1, 1, 0, 0, 1, 0 ... View more