About user12345a_2

user12345a_2 · ‎09-07-2017

So I've been running the following Powershell script for the past several months without issues. It downloads up to 5 million messages and runs in about 40 seconds. I run it via task scheduler every 10 minutes. It's proven very useful to our Incident Response to have the message trace logs in Splunk. ... <SESSION CONNECTION STUFF> ... [DateTime]$DateEnd = Get-Date -format g $DateEnd = $DateEnd.ToUniversalTime() [DateTime]$DateStart = $DateEnd.Addminutes(-10) $Outfile = "c:\O365\logs\MessageTrace_" + (get-date -Format "MM-dd-yyy-hh-mm-ss") + ".csv" $FoundCount = 0 For($i = 1; $i -le 1000; $i++) # Maximum allowed pages is 1000 { $Messages = Get-MessageTrace -StartDate $DateStart -EndDate $DateEnd -PageSize 5000 -Page $i If($Messages.count -gt 0) { $Status = $Messages[-1].Received.ToString("MM/dd/yyyy HH:mm") + " - " + $Messages[0].Received.ToString("MM/dd/yyyy HH:mm") + " [" + ("{0:N0}" -f ($i*5000)) + " Searched | " + $FoundCount + " Found]" Write-Progress -activity "Checking Messages (Up to 5 Million)..." -status $Status $Entries = $Messages | Select Received, SenderAddress, RecipientAddress, Subject, Status, FromIP, Size, MessageId $Entries | Export-Csv $Outfile -NoTypeInformation -Append $FoundCount += $Entries.Count } Else { Break } } Write-Host $FoundCount "Entries Found & Logged In" $Outfile # (Get-Content $Outfile) | Foreach-Object {$_ -replace '"', ""} | out-file -FilePath $Outfile -Force -Encoding ascii ################################################### # Delete all Files in C:\O365\ older than 2 days # ################################################### $Path = "C:\O365\logs" $Daysback = "-2" $CurrentDate = Get-Date $DatetoDelete = $CurrentDate.AddDays($Daysback) Get-ChildItem $Path | Where-Object { $_.LastWriteTime -lt $DatetoDelete } | Remove-Item

user12345a_2 · ‎01-11-2017

So I have the following search: search host="MY_IP_LIST" index="test" earliest="1/5/2017:00:00:01" latest="1/5/2017:13:00:00"| table src_ip which generates a list of IPs for which I need to search for username's against. Normally, for a single IP (e.g. 192.168.0.1) I can run: sourcetype=WinEventLog* 192.168.0.1 "EventCode=4624" | table src_ip, user|dedup src_ip, user and I get a list of users that have logged into that system. How do I use the results of the IP list from my first search to generate a table with IPs and usernames? I've been trying subsearches but to-date I've been unsuccessful. Thank you in advance for the help.

user12345a_2 · ‎11-03-2016

Hello. At the moment my organization uses MS Exchange on-premise. We index our Message Tracking Logs for our Information Security team who use that information in searches / panels for message recipient lists, etc. We are moving our on-premise Exchange to Office 365. Will the Splunk-Add on for MS Cloud Services enable me to give them similar information to what they used to get from the Message Track. Logs? Thanks in advance for any advice / pointers.

user12345a_2 · ‎08-30-2016

Good morning. So I have some TomCat logs of the format below that are parsing correctly in my lab but not in my production environment. The lab is using a completed test/sample log and the production environment logs are using "live" logs. Both lab/production are running 6.3.1, although I tried 6.4.3. in the lab and it still indexed properly with no change. 2016-08-12 11:50:10,241 INFO [org.jasig.cas.services.DefaultServicesManagerImpl] - Reloading registered services. 2016-08-12 11:50:16,410 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - Beginning ticket cleanup... 2016-08-12 11:50:16,410 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed. 2016-08-12 11:51:03,355 INFO [org.jasig.inspektr.audit.support.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN ============================================================= WHO: cas9tg11 WHAT: Supplied credentials: [cas9tg11] ACTION: AUTHENTICATION_SUCCESS APPLICATION: CAS WHEN: Fri Aug 12 11:51:03 CDT 2016 CLIENT IP ADDRESS: 192.168.14.38 SERVER IP ADDRESS: cas.idm.mycompany.com ============================================================= 2016-08-12 11:51:05,039 INFO [org.jasig.inspektr.audit.support.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN ============================================================= WHO: audit:unknown WHAT: TGT-57-998bHKceoKhWacmcNYg7o2669zcRaWfgtrhy67eH3EO-cas.idm.mycompany.com ACTION: TICKET_GRANTING_TICKET_CREATED APPLICATION: CAS WHEN: Fri Aug 12 11:51:05 CDT 2016 CLIENT IP ADDRESS: 192.168.14.38 SERVER IP ADDRESS: cas.idm.mycompany.com ============================================================= My inputs.conf looks like: [monitor:///usr/share/tomcat8_1/logs/cas-*.log] disabled = false sourcetype=cas_audit index=app_idmcas However while the multi-line logs look fine in my test lab, in production the multi-line logs split at the "WHEN" line: e.g. 8/30/16 7:39:49.560 AM 2016-08-30 07:39:49,560 INFO [org.jasig.inspektr.audit.support.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN ============================================================= WHO: 12fesm WHAT: Supplied credentials: [12fesm] ACTION: AUTHENTICATION_SUCCESS APPLICATION: CAS host = idm-cas2.idm.mycompany.com source = /usr/share/tomcat8_1/logs/cas-2016-08-30-07-1.log sourcetype = cas_audit 8/30/16 7:39:49.000 AM WHEN: Tue Aug 30 07:39:49 CDT 2016 CLIENT IP ADDRESS: 192.168.136.105 SERVER IP ADDRESS: cas.idm.mycompany.com ============================================================= host = idm-cas2.idm.mycompany.com source = /usr/share/tomcat8_1/logs/cas-2016-08-30-07-1.log sourcetype = cas_audit So I ran the btool command on my indexers to compare props.conf variable settings and they are the same: /opt/splunk/etc/system/local/props.conf [cas_audit] /opt/splunk/etc/system/default/props.conf ANNOTATE_PUNCT = True /opt/splunk/etc/system/default/props.conf AUTO_KV_JSON = true /opt/splunk/etc/system/local/props.conf BREAK_ONLY_BEFORE = \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3} /opt/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE_DATE = True /opt/splunk/etc/system/default/props.conf CHARSET = AUTO /opt/splunk/etc/system/default/props.conf DATETIME_CONFIG = /etc/datetime.xml /opt/splunk/etc/system/default/props.conf HEADER_MODE = /opt/splunk/etc/system/default/props.conf LEARN_SOURCETYPE = true /opt/splunk/etc/system/default/props.conf LINE_BREAKER_LOOKBEHIND = 100 /opt/splunk/etc/system/default/props.conf MAX_DAYS_AGO = 2000 /opt/splunk/etc/system/default/props.conf MAX_DAYS_HENCE = 2 /opt/splunk/etc/system/default/props.conf MAX_DIFF_SECS_AGO = 3600 /opt/splunk/etc/system/default/props.conf MAX_DIFF_SECS_HENCE = 604800 /opt/splunk/etc/system/default/props.conf MAX_EVENTS = 256 /opt/splunk/etc/system/local/props.conf MAX_TIMESTAMP_LOOKAHEAD = 30 /opt/splunk/etc/system/default/props.conf MUST_BREAK_AFTER = /opt/splunk/etc/system/default/props.conf MUST_NOT_BREAK_AFTER = /opt/splunk/etc/system/default/props.conf MUST_NOT_BREAK_BEFORE = /opt/splunk/etc/system/default/props.conf SEGMENTATION = indexing /opt/splunk/etc/system/default/props.conf SEGMENTATION-all = full /opt/splunk/etc/system/default/props.conf SEGMENTATION-inner = inner /opt/splunk/etc/system/default/props.conf SEGMENTATION-outer = outer /opt/splunk/etc/system/default/props.conf SEGMENTATION-raw = none /opt/splunk/etc/system/default/props.conf SEGMENTATION-standard = standard /opt/splunk/etc/system/default/props.conf SHOULD_LINEMERGE = True /opt/splunk/etc/system/default/props.conf TRANSFORMS = /opt/splunk/etc/system/default/props.conf TRUNCATE = 10000 /opt/splunk/etc/system/default/props.conf detect_trailing_nulls = auto /opt/splunk/etc/system/default/props.conf maxDist = 100 /opt/splunk/etc/system/default/props.conf priority = /opt/splunk/etc/system/default/props.conf sourcetype = I've tried various other settings for "BREAK_ONLY_BEFORE" and changed "BREAK_ONLY_BEFORE_DATE = false" and adjusted the "MAX_TIMESTAMP_LOOKAHEAD = 128", none of these changed seemed to affect anything. Any ideas what I could try next? Thank you in advance for any assistance.

user12345a_2 · ‎08-30-2016

So I tried multiple variations of the BREAK_ONLY_BEFORE variable and other props.conf settings with no effect. I ended up sending this to a friend who ran it through his VM lab and without any props.conf and it imported beautifully. I was using a Windows 7 UF client and switch to a *NIX UF client and it fixed the problem in my VM lab. Thanks for the help.

user12345a_2 · ‎08-19-2016

supabuck, Thank you for the help. So is there anyway to make this work given that the server where these reside has a UF not a HF? Can the props.conf changes be made on the indexer? Thank you.

user12345a_2 · ‎08-18-2016

Hello, I'm trying to get some Tomcat Catalina logs to import correctly. Manually importing the files works fine, but isn't an option beyond the test I'm running. So, when I select "Add Data" from main dashboard, select "upload files from my computer", select my file and choose the log file, set the sourcetype as "application->catalina" I get the expected results: Splunk Result 1: 2016-08-12 11:51:21,391 INFO [org.jasig.inspektr.audit.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN ============================================================= WHO: audit:unknown WHAT: ${ticket} ACTION: SERVICE_TICKET_VALIDATE_FAILED APPLICATION: CAS WHEN: Fri Aug 12 12:51:21 CDT 2016 CLIENT IP ADDRESS: 123.45.6.78 SERVER IP ADDRESS: myserver.mydomain.com ============================================================ host = indexer1 source = cas-81216.log sourcetype = catalina However, when I set up a forwarder and set up my inputs.conf as follows: [monitor://C:\test_dir\] index=my_test_index sourcetype=catalina (I've tried "Catalina", and leaving this line out as well, none produced the desired results) Every line (including the ======='s) is a separate Splunk result. Splunk Result 1: 2016-08-12 11:51:21,391 INFO [org.jasig.inspektr.audit.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN Splunk Result 2: ============================================================= ...etc. So how do I get my forwarded results to not break on each line? Thank you in advance for any help.

user12345a_2 · ‎07-25-2016

We do something similar with our firewalls, tracking the number of sessions that are active. Adding timechart count(connect_id) for us charts the number of action connections over a given period of time and shows the peak during the day. eventtype=vpn_session_event | transaction fields="user" maxspan=12h30m connected=f startswith="*connection established*" | search eventtype!=cisco_vpn_end | dedup user | **timechart count(user)**

user12345a_2 · ‎07-22-2016

Perfect, just what I needed, thanks!

user12345a_2 · ‎07-22-2016

user12345a_2 · ‎06-08-2016

So I have the following search/report that I run daily: index=os_linux NOT root tag=authentication NOT tag=failure | stats count by host, user, eventtype, src_ip | search eventtype=*authentication* | eval dest_count=host+":"+src_ip+"("+count+")" | stats values(dest_count) AS Daily by user, eventtype which generates output similar to the following: user eventtype Daily ---- --------- ----- user1 sshd_authentication system5.xyz.com:192.168.1.2(1) user2 sshd_authentication system12.xyz.com:192.168.1.42(2) system24.xyz.com:192.168.1.29(12) user3 sshd_authentication system15.xyz.com:192.168.1.24(7) I want to modify this to add a column to give me the 30-day average for each user on the respective system. A sample output would be (or something similar): user eventtype Daily Avg ---- --------- ----- ----- user1 sshd_authentication system5.xyz.com:192.168.1.2(1) 5 user2 sshd_authentication system12.xyz.com:192.168.1.42(2) 3 system24.xyz.com:192.168.1.29(42) 1 user3 sshd_authentication system15.xyz.com:192.168.1.24(7) 7 I'm having trouble getting this to work. Can anyone offer any suggestions on what the best way would be to accomplish this? Thank you.

Posts	11
Solutions	2
Karma Given	0
Karma Received	3
Member Since	‎05-23-2016

Online Status	Offline
Date Last Visited	‎06-05-2020 02:04 AM

How do I pull data from a subsearch?

Splunk Add-on for Microsoft Cloud Services: How do...

Why is my production environment breaking logs dif...

Why are my Catalina logs not line breaking as expe...

How can I list full table entries when an extracte...

How do I edit my search to find the monthly averag...

Re: Splunk Add-on for Microsoft Cloud Services: Ho...