Splunk Search
Highlighted

How do I search who is exporting aggregate data from my log files?

New Member

I am fairly new to Splunk and hoping someone could help with this. I have Index log files loaded onto Splunk, so to begin, I am searching keywords such as "Export" and the server name which works fine. Now I would want to know who is exporting aggregate data, so looking at a log file individually (without Splunk) I can see the sql script, and if the script contains a group by clause, then I can assume this is an aggregated export. Is there anyway I could do this in Splunk?

0 Karma
Highlighted

Re: How do I search who is exporting aggregate data from my log files?

Champion

I think some sample log entries would be helpful. For example, does adding "group by" to your search bring back entries you want or are the log files more complicated than that?

But most likely, splunk can get at the data you want, yes. We just need to better understand what that data looks like.

0 Karma
Highlighted

Re: How do I search who is exporting aggregate data from my log files?

New Member

Hi maclep thanks for replying. The log files are complicated, they are generated from a SAS server. Nothing comes back if i search for the term "Group By"

0 Karma
Highlighted

Re: How do I search who is exporting aggregate data from my log files?

Champion

Understood. Are you able to post some of the sample log entries, of course masking anything private? We'll need some idea what the data looks, otherwise we really won't know how to help you search it.

0 Karma
Highlighted

Re: How do I search who is exporting aggregate data from my log files?

New Member

Do you mean from Splunk or directly from the log files?

0 Karma
Highlighted

Re: How do I search who is exporting aggregate data from my log files?

Champion

either should be fine, I think. Are your events being parsed correctly in Splunk? Meaning, do the log files look broken up correctly in Splunk? Or do you have say multiple events in Splunk that you would consider to be one entry in the log file?

0 Karma
Highlighted

Re: How do I search who is exporting aggregate data from my log files?

New Member

Some of the log file code (generally there is a lot more than this):

2016-03-15 11:25:48,364 [Main] INFO  SAS.EG.App [(null)] - Starting SEGuide
2016-03-15 11:25:48,614 [Main] INFO  SAS.EG.App [(null)] - Version: File:             C:\
InternalName:     SEGuide.exe
OriginalFilename: SEGuide.exe
FileVersion:      7.100.1.2711
FileDescription:  SAS Enterprise Guide 7.1
Product:          SAS Enterprise Guide 7.1
ProductVersion:   7.11 (7.100.1.2711)
Debug:            False
Patched:          False
PreRelease:       False
PrivateBuild:     False
SpecialBuild:     False
Language:         Language Neutral


Does that help? Not sure if i could upload a print screen of results from Splunk?
0 Karma
Highlighted

Re: How do I search who is exporting aggregate data from my log files?

Champion

Sorry for the slow response, I don't have a lot of time to spend out here while at work.

What about an example where the script data is included? Is it similar but with a script section?

0 Karma
Highlighted

Re: How do I search who is exporting aggregate data from my log files?

New Member

Hi below is the logfile containing the sql script:

2016-06-29 15:45:45,822 [14] DEBUG SAS.EG.JobManagement.WorkspaceJob [(null)] - (Id=1) OnExecuting() - getting page setup
2016-06-29 15:45:45,822 [14] DEBUG JobSpy [(null)] - Log for job [1] on server [SASApp] at [*******]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [t11                                                          The SAS System                             ***********]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [t ]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 1          ;*';*";*/;quit;run;]************
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 2          OPTIONS PAGENO=MIN;]*********
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 3          %LET _CLIENTTASKLABEL*********
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 4          %LET _C********************
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 5          %LET _CLIENTPROJ**************
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 6          %LET _CLIENTPROJE***************
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 7          ]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 8          ODS _ALL_ CLOSE;]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 9          OPTIONS DEV=ACTIVEX;]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 10         GOPTIONS XPIXELS=0 YPIXELS=0;]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 11         FILENAME EGSR TEMP;]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 12         ODS tagsets.sasreport13(ID=EGSR) FILE=EGSR]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 13             STYLE=HtmlBlue]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 14             STYLESHEET=(URL="file:///R****************")]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 15             NOGTITLE]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 16             NOGFOOTNOTE]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 17             GPATH=&sasworklocation]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 18             ENCODING=UTF8]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 19             options(rolap="on")]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 20         ;]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [n NOTE: Writing TAGSETS.SASREPORT13(EGSR) Body file: EGSR]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 21         ]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 22         GOPTIONS ACCESSIBLE;]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 23         %_eg_conditional_dropds(WORK.MAT);]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 24         ]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 25         PROC SQL;]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 26            CREATE TABLE WORK.MAT AS]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 27            SELECT t1.DELSTAT_1,]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 28                   t1.DELMETH_D,]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 29                   /* COUNT************ */]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 30                     (COUNT(t1.*********)) AS COUNT_of_********]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 31               FROM **************]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 32               WHERE ********* = 1]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 33               GROUP BY t1.********,]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 34                        t1.********]
2016-06-29 15:45:45,838 [14] DEBUG JobSpy [(null)] - [s 35               ORDER BY t1.******,]
0 Karma
Highlighted

Re: How do I search who is exporting aggregate data from my log files?

Champion

That does look like a fun log file! I do see group by in there. Is this log in Splunk? But you can find it by searching for "group by"?

I understand that if those are all separate events, getting just a group by event back may not be too helpful. But I wonder if maybe it's worth putting all of those script events into one large event before indexing? Not sure how easy/feasible that would be, but could be worth a shot.

Also, you mentioned wanting to know which user ran that. Is that in the log anywhere too?

0 Karma