Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
- Find Answers
- :
- About jedatt01
jedatt01
Builder
Member since:
04-22-2011
06-05-2020
Community Statistics
| Posts | 167 |
| Solutions | 7 |
| Karma Given | 14 |
| Karma Received | 40 |
| Member Since | 04-22-2011 |
Activity Feed
- Got Karma for How to exclude specific time ranges in search results. 01-10-2024 01:45 AM
- Got Karma for Can I use kvstore as a web application backend?. 06-05-2020 12:49 AM
- Karma Re: Potential bug in R Analytics App for gwobben. 06-05-2020 12:48 AM
- Karma Re: How can I determine if a field is an integer (whole number) for woodcock. 06-05-2020 12:48 AM
- Got Karma for Re: How can I add empty time buckets to my table?. 06-05-2020 12:48 AM
- Karma Re: Where to find dashboard source files in a Splunk 5.x distributed search environment on Unix? for dolfantimmy. 06-05-2020 12:47 AM
- Karma Re: comparing earliest of subsearch and main search for acharlieh. 06-05-2020 12:47 AM
- Karma Re: Sideview Utils: How to use $foo$ to select chart type from a pulldown? for sideview. 06-05-2020 12:47 AM
- Karma Re: How to use inputlookup with realtime search for martin_mueller. 06-05-2020 12:47 AM
- Karma Re: How to count how many times a field value has changed for a stream of events over a period of time? for Runals. 06-05-2020 12:47 AM
- Karma Re: How do I handle fields with no value or a blank space in a rex field extraction so they show up as null? for somesoni2. 06-05-2020 12:47 AM
- Karma Re: Can you specify timezone in a REST API search? for woodcock. 06-05-2020 12:47 AM
- Got Karma for Re: Timestamp problem with DB Connect. 06-05-2020 12:47 AM
- Got Karma for Re: How to get more than 100 rows in web framework using Django binding?. 06-05-2020 12:47 AM
- Got Karma for Re: Timestamp problem with DB Connect. 06-05-2020 12:47 AM
- Got Karma for Simple XML: How to keep dashboard from auto running. 06-05-2020 12:47 AM
- Got Karma for HTML: How to create multiple charts in a single panel on the same row?. 06-05-2020 12:47 AM
- Got Karma for HTML: Make panel clickable for hyperlinking. 06-05-2020 12:47 AM
- Got Karma for Re: HTML: Make panel clickable for hyperlinking. 06-05-2020 12:47 AM
- Got Karma for How to extract string in a field that spans multiple lines?. 06-05-2020 12:47 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 1 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
05-09-2014
10:52 AM
I was picking my brain trying to figure out how this worked. Is it because every event goes through every transform? Then the last transform it goes through is where the event will end up?
... View more
05-09-2014
09:14 AM
1 Karma
I have a transform that I need help writing a regex for. It has two conditions.
It needs to match the value in this field <BusinessDomainID>*</BusinessDomainID>
It needs to not match this exact string <LogEventTypeCode>SEC_EVENT</LogEventTypeCode>
Here's what I have so far which meets the requirements for condition #1 and works correctly
REGEX=(?m)\<BusinessDomainId\>(BusinessDomainId1|BusinessDomainId2|BusinessDomainId3)\</BusinessDomainId\>
How can I add condition #2 in there so it will not match if it sees <LogEventTypeCode>SEC_EVENT</LogEventTypeCode> in the same event?
Here's some raw events.
This one matches condition 1 and 2
<ELLogInputMessage>
<Header>
<LogEventTypeCode>SEC_EVENT</LogEventTypeCode>
<LogSeverityCode>CRITICAL</LogSeverityCode>
<LogEventDateTime>2014-05-06T23:59:59.9999999-05:00</LogEventDateTime>
</Header>
<SourceInformation>
<EAPMId>1</EAPMId>
<HostMachineName>HostMachineName3</HostMachineName>
<HostEnvironmentName>HostEnvironmentName3</HostEnvironmentName>
<ComponentId>ComponentId3</ComponentId>
<ComponentName>ComponentName3</ComponentName>
<ApplicationEventCorrelationId>ApplicationEventCorrelationId3</ApplicationEventCorrelationId>
<UserId>UserId1</UserId>
<UserSrc>UserSrc1</UserSrc>
<BusinessDomainId>BusinessDomainId1</BusinessDomainId>
<BusinessDomainName>BusinessDomainName1</BusinessDomainName>
</SourceInformation>
<DataAccessInformation>
<DataCompId>DataCompId2</DataCompId>
<TypeOfAccess>VIEW</TypeOfAccess>
<SubjectOfInterest>
<SubjectId>SubjectId13</SubjectId>
<SubjectName>SubjectName13</SubjectName>
<SubjectDomainName>SubjectDomainName3</SubjectDomainName>
</SubjectOfInterest>
<AccessDateTime>2014-05-06T23:59:59.9999999-05:00</AccessDateTime>
</DataAccessInformation>
<DetailedLogInformation>anyType</DetailedLogInformation>
</ELLogInputMessage>
This one matches only condition 1
The only difference here is this line <LogEventTypeCode>APP_EVENT</LogEventTypeCode>
<ELLogInputMessage>
<Header>
<LogEventTypeCode>APP_EVENT</LogEventTypeCode>
<LogSeverityCode>CRITICAL</LogSeverityCode>
<LogEventDateTime>2014-05-06T23:59:59.9999999-05:00</LogEventDateTime>
</Header>
<SourceInformation>
<EAPMId>1</EAPMId>
<HostMachineName>HostMachineName3</HostMachineName>
<HostEnvironmentName>HostEnvironmentName3</HostEnvironmentName>
<ComponentId>ComponentId3</ComponentId>
<ComponentName>ComponentName3</ComponentName>
<ApplicationEventCorrelationId>ApplicationEventCorrelationId3</ApplicationEventCorrelationId>
<UserId>UserId1</UserId>
<UserSrc>UserSrc1</UserSrc>
<BusinessDomainId>BusinessDomainId1</BusinessDomainId>
<BusinessDomainName>BusinessDomainName1</BusinessDomainName>
</SourceInformation>
<DataAccessInformation>
<DataCompId>DataCompId2</DataCompId>
<TypeOfAccess>VIEW</TypeOfAccess>
<SubjectOfInterest>
<SubjectId>SubjectId13</SubjectId>
<SubjectName>SubjectName13</SubjectName>
<SubjectDomainName>SubjectDomainName3</SubjectDomainName>
</SubjectOfInterest>
<AccessDateTime>2014-05-06T23:59:59.9999999-05:00</AccessDateTime>
</DataAccessInformation>
<DetailedLogInformation>anyType</DetailedLogInformation>
</ELLogInputMessage>
... View more
- Tags:
- regex
- transforms.conf
05-09-2014
08:53 AM
I had a thought, in my regex
REGEX=(?m)<BusinessDomainId>(BusinessDomainId1|BusinessDomainId2|BusinessDomainId3)</BusinessDomainId>
is there a way to add a condition for it not to match if it sees the string SEC_EVENT in the same event?
... View more
05-09-2014
08:34 AM
I have a single xml source that I need to go multiple indexes. To complicate things, I need multiple transforms to make sure all events get to the appropriate index. Here are my requirements:
If any events contains the string <LogEventTypeCode>SEC_EVENT</LogEventTypeCode> it needs to be routed to an index called Security
All other events that do not contain <LogEventTypeCode>SEC_EVENT</LogEventTypeCode> need to be routed to appropriate index based on another field <BusinessDomainId>"businessdomain"</BusinessDomainId>
Requirement #2 already works, and what i'm seeing that requirement #1 only works if it has value for <BusinessDomainId>"businessdomain"</BusinessDomainId> that I do not have an index created for.
Props.conf
[mq]
BREAK_ONLY_BEFORE = \<ELLogInputMessage>
KV_MODE=XML
SHOULD_LINEMERGE = true
MAX_EVENTS = 50000
TIME_PREFIX = <LogEventDateTime>
pulldown_type = 1
TRANSFORMS-route = Security, BusinessDomainId
Transforms.conf
[Security]
SOURCE_KEY = _raw
DEST_KEY = _MetaData:Index
REGEX=<LogEventTypeCode>PI_EVENT</LogEventTypeCode>
FORMAT=Security
[BusinessDomainId]
SOURCE_KEY = _raw
DEST_KEY =_MetaData:Index
REGEX=(?m)\<BusinessDomainId\>(BusinessDomainId1|BusinessDomainId2|BusinessDomainId3)\</BusinessDomainId\>
FORMAT=$1
Sample Event
<ELLogInputMessage>
<Header>
<LogEventTypeCode>PI_EVENT</LogEventTypeCode>
<LogSeverityCode>CRITICAL</LogSeverityCode>
<LogEventDateTime>2014-05-06T23:19:59.9999999-05:00</LogEventDateTime>
</Header>
<SourceInformation>
<EAPMId>3</EAPMId>
<HostMachineName>HostMachineName3</HostMachineName>
<HostEnvironmentName>HostEnvironmentName3</HostEnvironmentName>
<ComponentId>ComponentId3</ComponentId>
<ComponentName>ComponentName3</ComponentName>
<ApplicationEventCorrelationId>ApplicationEventCorrelationId3</ApplicationEventCorrelationId>
<UserId>UserId33</UserId>
<UserSrc>UserSrc33</UserSrc>
<BusinessDomainId>BusinessDomainId33</BusinessDomainId>
<BusinessDomainName>BusinessDomainName33</BusinessDomainName>
</SourceInformation>
<DataAccessInformation>
<DataCompId>DataCompId33</DataCompId>
<TypeOfAccess>VIEW</TypeOfAccess>
<SubjectOfInterest>
<SubjectId>SubjectId32</SubjectId>
<SubjectName>SubjectName32</SubjectName>
<SubjectDomainName>SubjectDomainName32</SubjectDomainName>
</SubjectOfInterest>
<AccessDateTime>2014-05-06T23:19:59.9999999-05:00</AccessDateTime>
</DataAccessInformation>
<DetailedLogInformation>anyType</DetailedLogInformation>
</ELLogInputMessage>
... View more
- Tags:
- transforms.conf
05-06-2014
07:42 AM
when you have an input that is true json or xml you can have the view of the events in a tree structure where you hit the + sign to see nested information.
... View more
04-29-2014
09:45 AM
That extracted all the fields, thanks! Wish there was a way to get a pull down view but this will do the trick if that's not possible.
... View more
04-29-2014
08:04 AM
Yes, I tried using KV_MODE = XML but it is not picking up. Is the only way to do individual field extraction?
... View more
04-29-2014
06:20 AM
I have a datasource that reads in events in XML format. Could someone please help me build a props.conf that will extract all fields and show the events in treeview. Sample event below
Mon Apr 28 16:45:57 EDT 2014 name="TOPIC_msg_received" event_id="ID:404040" msg_dest="SplunkTopic" msg_body="<?xml version="1.0" encoding="utf-8"?><ELLogInputLayout xmlns="http://www.test.com/1"><ELLogInputMessage> <Header> <LogEventTypeCode>ERROR</LogEventTypeCode> <LogSeverityCode>CRITICAL</LogSeverityCode> <LogEventDateTime>9999-12-31T23:59:59.9999999-05:00</LogEventDateTime> </Header> <SourceInformation> <EAPMId>2</EAPMId> <HostMachineName>HostMachineName3</HostMachineName> <HostEnvironmentName>HostEnvironmentName3</HostEnvironmentName> <ComponentId>ComponentId3</ComponentId> <ComponentName>ComponentName3</ComponentName> <ApplicationEventCorrelationId>ApplicationEventCorrelationId3</ApplicationEventCorrelationId> <UserId>UserId3</UserId> <UserSrc>UserSrc3</UserSrc> <BusinessDomainId>BusinessDomainId3</BusinessDomainId> <BusinessDomainName>BusinessDomainName3</BusinessDomainName> </SourceInformation> <ErrorInformation> <ErrorCode>ErrorCode3</ErrorCode> <ErrorDescription>ErrorDescription3</ErrorDescription><DetailedErrorInformation>DetailedErrorInformation3</DetailedErrorInformation> </ErrorInformation> <DetailedLogInformation>anyType</DetailedLogInformation> </ELLogInputMessage></ELLogInputLayout>"
... View more
- Tags:
- extraction
- xml
04-01-2014
08:19 AM
This is a single server splunk deployment. The events come directly into the splunk server via webservice.
... View more
04-01-2014
08:07 AM
What about data that has not already been indexed? I'm not concerned about the data that is already there.
... View more
04-01-2014
06:48 AM
I have a modular input that collects data from a webservice. The events are not collected in realtime so to get the true timestamp I have to extract that from the time field on each event instead of using when splunk consumes it as the time stamp.
The problem with this is the event time is in UTC and my server is in US/Eastern time (UTC-5). When I search for the events they show 5 hours ahead. This causes problems when using relative search times because no data shows up.
How can I use props.conf or other method to make the events show up in Splunk as US/Eastern time so my searches work correctly? My current props.conf is below. I've tried to change the TZ= setting but it makes no difference. Please help!
[test]
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^{
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3Q
TZ=UTC
KV_MODE=json
TRUNCATE=15000
Here's what the raw event logs like from splunkd.log
{
'tot': 86,
'epoch': 1396352800,
'tos': 85,
'sid': 318,
'browsertype': IE7,
'type': 'txtest',
'sname': New York, NY - Verizon,
'ttime': 2014-04-01 07:46:40.433,
'tpf': 0,
'rtime': 5954,
'nbyte': 729580,
'tof': 0,
'mid': 14247945,
'tps': 3, 'tpt': 3
}
... View more
- Tags:
- props.conf
03-28-2014
02:02 PM
It works! thanks for hanging in there with me
... View more
03-28-2014
12:43 PM
1 Karma
Let me try to rephrase. My lookup table has dates and times that span from Jan 2014 through Dec 2014. When I search with the all time range results come up properly. When I search a relative time range (Month to date) it says no results are displayed. I removed dates from my lookup table that were outside of my month to date range and ran the search again and it was successful. So, I have concluded that the search will not complete if the lookup table is referencing dates/times that are outside of my search time.
... View more
03-28-2014
11:41 AM
Martin, I found the root of the problem. My lookup file has dates for the entire year. When I do a relative lookup those dates aren't applicable. But, when using all time they are. Can I make some modifications to the search to account for this?
... View more
03-28-2014
09:38 AM
This is a single server splunk deployment. Not sure what I can check
... View more
03-28-2014
07:24 AM
Ok, almost there. That worked for all time, but does not work if i'm using a relative time (ex. Month to date) is there any way around that?
... View more
03-28-2014
07:14 AM
Looks like it's not happy with the where NOT, Splunk says its not supported. I'm on Splunk 6
... View more
03-28-2014
04:22 AM
Are you saying if I was to convert all time values in my lookup to epoch using fields start and end this logic should work?
... View more
03-27-2014
10:29 AM
I am collecting statistics from an application and am trying to find a way to exclude search results from statistics that occur during a predetermined yearly outage window. I've loaded the outage window details in a lookup table and was wondering how I could use the lookup command inline to exclude from my search results. Lookup table fields and example below.
Date Starttime Endtime
1/10/2014 3:00 AM 5:00 AM
1/24/2014 3:00 AM 7:00 AM
....................................
... View more
- Tags:
- lookup
03-26-2014
08:33 AM
That gets me closer. Now how can I specify a relative day, like every friday?
... View more
03-26-2014
07:50 AM
4 Karma
I have a datasource which contains availability statistics from an application. I also have a predetermined maintenance schedule that occurs every two weeks on the same day and time. How can I use search to exclude the date/time ranges for my maintenance schedule in my search results?
... View more
- Tags:
- datetime
02-11-2014
10:27 AM
well, turns out your first solution did work! I had some problems with my splunk instance and had to reinstall. Just to be thorough, i'll add that you need to import the subprocess module at the top of the wrapper script for this to work. Thanks for the help!
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
06-05-2020
02:02 AM
|
Karma given to
