Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
- Find Answers
- :
- About user12345a_2
user12345a_2
Explorer
Member since:
05-23-2016
06-05-2020
Community Statistics
| Posts | 11 |
| Solutions | 2 |
| Karma Given | 0 |
| Karma Received | 3 |
| Member Since | 05-23-2016 |
Activity Feed
- Got Karma for Splunk Add-on for Microsoft Cloud Services: How do I get Message Tracking Logs from Exchange Online into Splunk?. 06-05-2020 12:48 AM
- Got Karma for Splunk Add-on for Microsoft Cloud Services: How do I get Message Tracking Logs from Exchange Online into Splunk?. 06-05-2020 12:48 AM
- Got Karma for Why is my production environment breaking logs differently than my lab?. 06-05-2020 12:48 AM
- Posted Re: Splunk Add-on for Microsoft Cloud Services: How do I get Message Tracking Logs from Exchange Online into Splunk? on All Apps and Add-ons. 09-07-2017 11:32 AM
- Posted How do I pull data from a subsearch? on Splunk Search. 01-11-2017 01:08 PM
- Tagged How do I pull data from a subsearch? on Splunk Search. 01-11-2017 01:08 PM
- Tagged How do I pull data from a subsearch? on Splunk Search. 01-11-2017 01:08 PM
- Posted Splunk Add-on for Microsoft Cloud Services: How do I get Message Tracking Logs from Exchange Online into Splunk? on All Apps and Add-ons. 11-03-2016 10:43 AM
- Tagged Splunk Add-on for Microsoft Cloud Services: How do I get Message Tracking Logs from Exchange Online into Splunk? on All Apps and Add-ons. 11-03-2016 10:43 AM
- Posted Why is my production environment breaking logs differently than my lab? on Getting Data In. 08-30-2016 07:13 AM
- Tagged Why is my production environment breaking logs differently than my lab? on Getting Data In. 08-30-2016 07:13 AM
- Tagged Why is my production environment breaking logs differently than my lab? on Getting Data In. 08-30-2016 07:13 AM
- Tagged Why is my production environment breaking logs differently than my lab? on Getting Data In. 08-30-2016 07:13 AM
- Posted Re: Why are my Catalina logs not line breaking as expected when I set up a directory monitor in inputs.conf? on Getting Data In. 08-30-2016 06:23 AM
- Posted Re: Why are my Catalina logs not line breaking as expected when I set up a directory monitor in inputs.conf? on Getting Data In. 08-19-2016 12:23 PM
- Posted Why are my Catalina logs not line breaking as expected when I set up a directory monitor in inputs.conf? on Getting Data In. 08-18-2016 12:40 PM
- Tagged Why are my Catalina logs not line breaking as expected when I set up a directory monitor in inputs.conf? on Getting Data In. 08-18-2016 12:40 PM
- Tagged Why are my Catalina logs not line breaking as expected when I set up a directory monitor in inputs.conf? on Getting Data In. 08-18-2016 12:40 PM
- Tagged Why are my Catalina logs not line breaking as expected when I set up a directory monitor in inputs.conf? on Getting Data In. 08-18-2016 12:40 PM
- Tagged Why are my Catalina logs not line breaking as expected when I set up a directory monitor in inputs.conf? on Getting Data In. 08-18-2016 12:40 PM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 2 | |||
| 1 | |||
| 0 | |||
| 0 | |||
| 0 |
09-07-2017
11:32 AM
So I've been running the following Powershell script for the past several months without issues. It downloads up to 5 million messages and runs in about 40 seconds. I run it via task scheduler every 10 minutes. It's proven very useful to our Incident Response to have the message trace logs in Splunk.
...
<SESSION CONNECTION STUFF>
...
[DateTime]$DateEnd = Get-Date -format g
$DateEnd = $DateEnd.ToUniversalTime()
[DateTime]$DateStart = $DateEnd.Addminutes(-10)
$Outfile = "c:\O365\logs\MessageTrace_" + (get-date -Format "MM-dd-yyy-hh-mm-ss") + ".csv"
$FoundCount = 0
For($i = 1; $i -le 1000; $i++) # Maximum allowed pages is 1000
{
$Messages = Get-MessageTrace -StartDate $DateStart -EndDate $DateEnd -PageSize 5000 -Page $i
If($Messages.count -gt 0)
{
$Status = $Messages[-1].Received.ToString("MM/dd/yyyy HH:mm") + " - " + $Messages[0].Received.ToString("MM/dd/yyyy HH:mm") + " [" + ("{0:N0}" -f ($i*5000)) + " Searched | " + $FoundCount + " Found]"
Write-Progress -activity "Checking Messages (Up to 5 Million)..." -status $Status
$Entries = $Messages | Select Received, SenderAddress, RecipientAddress, Subject, Status, FromIP, Size, MessageId
$Entries | Export-Csv $Outfile -NoTypeInformation -Append
$FoundCount += $Entries.Count
}
Else
{
Break
}
}
Write-Host $FoundCount "Entries Found & Logged In" $Outfile
# (Get-Content $Outfile) | Foreach-Object {$_ -replace '"', ""} | out-file -FilePath $Outfile -Force -Encoding ascii
###################################################
# Delete all Files in C:\O365\ older than 2 days #
###################################################
$Path = "C:\O365\logs"
$Daysback = "-2"
$CurrentDate = Get-Date
$DatetoDelete = $CurrentDate.AddDays($Daysback)
Get-ChildItem $Path | Where-Object { $_.LastWriteTime -lt $DatetoDelete } | Remove-Item
... View more
01-11-2017
01:08 PM
So I have the following search:
search host="MY_IP_LIST" index="test" earliest="1/5/2017:00:00:01" latest="1/5/2017:13:00:00"| table src_ip
which generates a list of IPs for which I need to search for username's against.
Normally, for a single IP (e.g. 192.168.0.1) I can run:
sourcetype=WinEventLog* 192.168.0.1 "EventCode=4624" | table src_ip, user|dedup src_ip, user
and I get a list of users that have logged into that system.
How do I use the results of the IP list from my first search to generate a table with IPs and usernames? I've been trying subsearches but to-date I've been unsuccessful.
Thank you in advance for the help.
... View more
11-03-2016
10:43 AM
2 Karma
Hello.
At the moment my organization uses MS Exchange on-premise. We index our Message Tracking Logs for our Information Security team who use that information in searches / panels for message recipient lists, etc. We are moving our on-premise Exchange to Office 365. Will the Splunk-Add on for MS Cloud Services enable me to give them similar information to what they used to get from the Message Track. Logs?
Thanks in advance for any advice / pointers.
... View more
08-30-2016
07:13 AM
1 Karma
Good morning. So I have some TomCat logs of the format below that are parsing correctly in my lab but not in my production environment. The lab is using a completed test/sample log and the production environment logs are using "live" logs.
Both lab/production are running 6.3.1, although I tried 6.4.3. in the lab and it still indexed properly with no change.
2016-08-12 11:50:10,241 INFO [org.jasig.cas.services.DefaultServicesManagerImpl] - Reloading registered services.
2016-08-12 11:50:16,410 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - Beginning ticket cleanup...
2016-08-12 11:50:16,410 INFO [org.jasig.cas.ticket.registry.DefaultTicketRegistry] - 0 expired tickets found and removed.
2016-08-12 11:51:03,355 INFO [org.jasig.inspektr.audit.support.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN
=============================================================
WHO: cas9tg11
WHAT: Supplied credentials: [cas9tg11]
ACTION: AUTHENTICATION_SUCCESS
APPLICATION: CAS
WHEN: Fri Aug 12 11:51:03 CDT 2016
CLIENT IP ADDRESS: 192.168.14.38
SERVER IP ADDRESS: cas.idm.mycompany.com
=============================================================
2016-08-12 11:51:05,039 INFO [org.jasig.inspektr.audit.support.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN
=============================================================
WHO: audit:unknown
WHAT: TGT-57-998bHKceoKhWacmcNYg7o2669zcRaWfgtrhy67eH3EO-cas.idm.mycompany.com
ACTION: TICKET_GRANTING_TICKET_CREATED
APPLICATION: CAS
WHEN: Fri Aug 12 11:51:05 CDT 2016
CLIENT IP ADDRESS: 192.168.14.38
SERVER IP ADDRESS: cas.idm.mycompany.com
=============================================================
My inputs.conf looks like:
[monitor:///usr/share/tomcat8_1/logs/cas-*.log]
disabled = false
sourcetype=cas_audit
index=app_idmcas
However while the multi-line logs look fine in my test lab, in production the multi-line logs split at the "WHEN" line:
e.g.
8/30/16 7:39:49.560 AM
2016-08-30 07:39:49,560 INFO [org.jasig.inspektr.audit.support.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN
=============================================================
WHO: 12fesm
WHAT: Supplied credentials: [12fesm]
ACTION: AUTHENTICATION_SUCCESS
APPLICATION: CAS
host = idm-cas2.idm.mycompany.com source = /usr/share/tomcat8_1/logs/cas-2016-08-30-07-1.log sourcetype = cas_audit
8/30/16 7:39:49.000 AM
WHEN: Tue Aug 30 07:39:49 CDT 2016
CLIENT IP ADDRESS: 192.168.136.105
SERVER IP ADDRESS: cas.idm.mycompany.com
=============================================================
host = idm-cas2.idm.mycompany.com source = /usr/share/tomcat8_1/logs/cas-2016-08-30-07-1.log sourcetype = cas_audit
So I ran the btool command on my indexers to compare props.conf variable settings and they are the same:
/opt/splunk/etc/system/local/props.conf [cas_audit]
/opt/splunk/etc/system/default/props.conf ANNOTATE_PUNCT = True
/opt/splunk/etc/system/default/props.conf AUTO_KV_JSON = true
/opt/splunk/etc/system/local/props.conf BREAK_ONLY_BEFORE = \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}
/opt/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE_DATE = True
/opt/splunk/etc/system/default/props.conf CHARSET = AUTO
/opt/splunk/etc/system/default/props.conf DATETIME_CONFIG = /etc/datetime.xml
/opt/splunk/etc/system/default/props.conf HEADER_MODE =
/opt/splunk/etc/system/default/props.conf LEARN_SOURCETYPE = true
/opt/splunk/etc/system/default/props.conf LINE_BREAKER_LOOKBEHIND = 100
/opt/splunk/etc/system/default/props.conf MAX_DAYS_AGO = 2000
/opt/splunk/etc/system/default/props.conf MAX_DAYS_HENCE = 2
/opt/splunk/etc/system/default/props.conf MAX_DIFF_SECS_AGO = 3600
/opt/splunk/etc/system/default/props.conf MAX_DIFF_SECS_HENCE = 604800
/opt/splunk/etc/system/default/props.conf MAX_EVENTS = 256
/opt/splunk/etc/system/local/props.conf MAX_TIMESTAMP_LOOKAHEAD = 30
/opt/splunk/etc/system/default/props.conf MUST_BREAK_AFTER =
/opt/splunk/etc/system/default/props.conf MUST_NOT_BREAK_AFTER =
/opt/splunk/etc/system/default/props.conf MUST_NOT_BREAK_BEFORE =
/opt/splunk/etc/system/default/props.conf SEGMENTATION = indexing
/opt/splunk/etc/system/default/props.conf SEGMENTATION-all = full
/opt/splunk/etc/system/default/props.conf SEGMENTATION-inner = inner
/opt/splunk/etc/system/default/props.conf SEGMENTATION-outer = outer
/opt/splunk/etc/system/default/props.conf SEGMENTATION-raw = none
/opt/splunk/etc/system/default/props.conf SEGMENTATION-standard = standard
/opt/splunk/etc/system/default/props.conf SHOULD_LINEMERGE = True
/opt/splunk/etc/system/default/props.conf TRANSFORMS =
/opt/splunk/etc/system/default/props.conf TRUNCATE = 10000
/opt/splunk/etc/system/default/props.conf detect_trailing_nulls = auto
/opt/splunk/etc/system/default/props.conf maxDist = 100
/opt/splunk/etc/system/default/props.conf priority =
/opt/splunk/etc/system/default/props.conf sourcetype =
I've tried various other settings for "BREAK_ONLY_BEFORE" and changed "BREAK_ONLY_BEFORE_DATE = false" and adjusted the "MAX_TIMESTAMP_LOOKAHEAD = 128", none of these changed seemed to affect anything.
Any ideas what I could try next? Thank you in advance for any assistance.
... View more
08-30-2016
06:23 AM
So I tried multiple variations of the BREAK_ONLY_BEFORE variable and other props.conf settings with no effect. I ended up sending this to a friend who ran it through his VM lab and without any props.conf and it imported beautifully. I was using a Windows 7 UF client and switch to a *NIX UF client and it fixed the problem in my VM lab. Thanks for the help.
... View more
08-19-2016
12:23 PM
supabuck,
Thank you for the help. So is there anyway to make this work given that the server where these reside has a UF not a HF? Can the props.conf changes be made on the indexer?
Thank you.
... View more
08-18-2016
12:40 PM
Hello,
I'm trying to get some Tomcat Catalina logs to import correctly. Manually importing the files works fine, but isn't an option beyond the test I'm running. So, when I select "Add Data" from main dashboard, select "upload files from my computer", select my file and choose the log file, set the sourcetype as "application->catalina" I get the expected results:
Splunk Result 1:
2016-08-12 11:51:21,391 INFO [org.jasig.inspektr.audit.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN
=============================================================
WHO: audit:unknown
WHAT: ${ticket}
ACTION: SERVICE_TICKET_VALIDATE_FAILED
APPLICATION: CAS
WHEN: Fri Aug 12 12:51:21 CDT 2016
CLIENT IP ADDRESS: 123.45.6.78
SERVER IP ADDRESS: myserver.mydomain.com
============================================================
host = indexer1 source = cas-81216.log sourcetype = catalina
However, when I set up a forwarder and set up my inputs.conf as follows:
[monitor://C:\test_dir\]
index=my_test_index
sourcetype=catalina (I've tried "Catalina", and leaving this line out as well, none produced the desired results)
Every line (including the ======='s) is a separate Splunk result.
Splunk Result 1:
2016-08-12 11:51:21,391 INFO [org.jasig.inspektr.audit.Slf5jLoggingAuditTrailManager] - Audit trail record BEGIN
Splunk Result 2:
=============================================================
...etc.
So how do I get my forwarded results to not break on each line?
Thank you in advance for any help.
... View more
07-25-2016
05:43 AM
We do something similar with our firewalls, tracking the number of sessions that are active. Adding timechart count(connect_id) for us charts the number of action connections over a given period of time and shows the peak during the day.
eventtype=vpn_session_event
| transaction fields="user" maxspan=12h30m connected=f startswith="*connection established*"
| search eventtype!=cisco_vpn_end
| dedup user
| **timechart count(user)**
... View more
07-22-2016
12:37 PM
Perfect, just what I needed, thanks!
... View more
07-22-2016
10:18 AM
Good morning. So I have a search which generates a list of recipients for a particular message subject.
The search is as follows:
index="exchange*" message_subject="test message subject" | rex field=recipient_address "(?<my_recip>[a-zA-Z0-9._%+-]+)@(?<my_todomain>[a-zA-Z0-9.+]+[\.(a-zA-Z)+])" max_match=0| convert ctime(_time) as my_date timeformat=%m-%d-%y | convert ctime(_time) as my_hour timeformat=%H:%M | eval full_email=my_recip."@".my_todomain | table my_date, my_hour, message_subject, my_recip, my_todomain, full_email, sender_address | dedup sender_address, my_recip
In a given scenario, if user1 sends 3 messages:
Message 1: To: [email protected]
Message 2: To: [email protected]
Message 3: To: [email protected], [email protected], [email protected]
The query returns the following results:
my_date | my_hour | message_subject | my_recip | my_todomain | full_email | sender_address
7-22-16 | 10:43 | test message | userA | this.org | [email protected] | [email protected]
7-22-16 | 10:44 | test message | userB | this.org | [email protected] | [email protected]
7-22-16 | 10:45 | test message | userC | this.org | [email protected] | [email protected]
userD
userE
What I’d like the query to return is:
my_date | my_hour | message_subject | my_recip | my_todomain | full_email | sender_address
7-22-16 | 10:43 | test message | userA | this.org | [email protected] | [email protected]
7-22-16 | 10:44 | test message | userB | this.org | [email protected] | [email protected]
7-22-16 | 10:45 | test message | userC | this.org | [email protected] | [email protected]
7-22-16 | 10:45 | test message | userD | this.org | [email protected] | [email protected]
7-22-16 | 10:45 | test message | userE | this.org | [email protected] | [email protected]
Is there an easy way to do this? Thanks in advance for any assistance.
... View more
06-08-2016
05:14 AM
So I have the following search/report that I run daily:
index=os_linux NOT root tag=authentication NOT tag=failure | stats count by host, user, eventtype, src_ip | search eventtype=*authentication* | eval dest_count=host+":"+src_ip+"("+count+")" | stats values(dest_count) AS Daily by user, eventtype
which generates output similar to the following:
user eventtype Daily
---- --------- -----
user1 sshd_authentication system5.xyz.com:192.168.1.2(1)
user2 sshd_authentication system12.xyz.com:192.168.1.42(2)
system24.xyz.com:192.168.1.29(12)
user3 sshd_authentication system15.xyz.com:192.168.1.24(7)
I want to modify this to add a column to give me the 30-day average for each user on the respective system. A sample output would be (or something similar):
user eventtype Daily Avg
---- --------- ----- -----
user1 sshd_authentication system5.xyz.com:192.168.1.2(1) 5
user2 sshd_authentication system12.xyz.com:192.168.1.42(2) 3
system24.xyz.com:192.168.1.29(42) 1
user3 sshd_authentication system15.xyz.com:192.168.1.24(7) 7
I'm having trouble getting this to work. Can anyone offer any suggestions on what the best way would be to accomplish this? Thank you.
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
06-05-2020
02:04 AM
|
