Tracked deleting files in PowerShell
Hello, Habr! The theme of my post has already been raised here, but I have something to add.
When our file storage exchanged third terabytes, increasingly, our Department began to receive requests to find out who deleted an important document or an entire folder of documents. Often it happens because somebody's malicious intent. Backups are good, but the country must know its heroes. But the milk is twice tastier when we can write it in PowerShell.
Yet I understood, I decided to write for colleagues, but then I thought that might be useful to someone else. The material is mixed. Someone will find a ready solution, someone will come in handy a few obvious methods of work with PowerShell or the task scheduler, and someone will check on the performance of your scripts.
In the process of finding a solution to the problem read article authored Deks. Decided to take it, but some things did not satisfy me.
the
A short discourse: If auditing is enabled the file system at the time of deletion of the file in the security log are created two events, with codes 4663 and, then, 4660. The first records the attempted access request for deletion, the user information and the path to the removed file, and the second one captures the fact of removal. Events have a unique identifier EventRecordID that is different per unit of these two events.
The following is the original script that collects information about deleted files and users, to remove them.
the
Use the command Measure-Command has received the following:
the
Too much, on the secondary FS will be longer. Immediately it is not like a ten-pipe, so to start I have it structured:
the
Managed to reduce the number of storeys of the pipe and to remove enums, Foreach and at the same time make your code more readable, but the effect is not given, the difference is within the error:
the
Had to think a little head. What operations take the most time? It would be possible to stick a dozen Measure-Command, but in this case it is evident that the most time is spent on queries to the log (it's not a fast process even in the MMC) and a recurring conversion to XML (in addition, in the case of EventRecordID it is optional). Let's try to do both at once, and at the same time to exclude the intermediate variables:
the
But this is the result. Acceleration is almost two times!
Glad, and that's enough. Three minutes is better than five, but how best to run the script? Times per hour? So may escape the record, which appear simultaneously with the start of the script. To make a request not for an hour, and for 65 minutes? Then recording can be repeated. Yes, and then seek a record of the desired file among the thousands of logs — mutor. To write once a day? The log rotation will forget half of it. You need something more reliable. In the comments to the article Deks someone talking about the app on dotnet working in the service mode, but it's, you know, one of those "There are 14 competing standards"...
In Windows task scheduler you can create a trigger on the event in the system log. Here it is:
Great! The script will run exactly at the moment, and our magazine will be sozdavala in real time! But our joy will be incomplete if we will not be able to determine what kind of event we need to record at the time of launch. We need a trick. We have them! Short googling showed that the trigger Event planner can transfer the executable file information about the event. But this is, to put it mildly, not obvious. The sequence of actions is this:
But we don't want to aticipate all this with a mouse on 20 servers, right? You want to automate. Unfortunately, PowerShell is not omnipotent, and the New-ScheduledTaskTrigger not yet know how to create triggers Event. Therefore, we apply the cheat code and create a task using the COM object (that is quite often necessary to resort to COM, although the regular cmdlets know more and more c each new version of PS):
the
Must allow running multiple instances and, I think, is to disallow manual start and set the time limit:
the
Create a trigger of type 0 (Event). Then you set the XML query to obtain the desired events. The XML request can be obtained from the MMC console "event Log" and select the necessary settings and switch to the tab "XML":
the
The main trick: specify the variable to pass to the script.
the
In fact, a description of the command being executed:
the
And — fly!
the
Back to the script for logging. Now we don't need to get all events, and need to get one, but still passed as the argument. For this, we would add headers, converting the script to a cmdlet with parameters. Pile up — will make it possible to change the path to the log "on the fly", maybe useful:
the
Then there is a caveat: up until this point we've got events with Get-WinEvent cmdlet and the filter parameter FilterHashtable. It understands a limited set of attributes, which doesn't include the EventRecordID. So the filter we will be using option -FilterXml, we now know how it is!
the
Now we no longer need the enumeration Foreach-Object, as processed only one event. Not two, because the event ID 4660 is only used to initiate the script, it is useful information in itself does not carry.
Remember, in the beginning I wanted to allow users without my participation of updatesite? So, in case if you deleted a file in the documents folder of any Department — written log also to the root folder of the Department.
the
Well, the pieces of sliced, left to put it all together and even a little bit to optimize. Get something like this:
the
Left to put the script in a convenient place and run the key Install.
Now the employees of any Department can real-time to see who, what and when removed from their directory. Note that I did not consider here the right of access log files (so that villain couldn't remove) and rotate them. Structure and access rights to the directories on our filer pull on separate article, and rotation to some extent, will complicate the search for the desired string.
— the Finest reference on regular expressions
— Tutorial on creating a task linked to an event
— a description of the scripting API scheduler
UPD: In the final script had a typo, after line 41 was unnecessary gravis. For the detection thanks to the reader Habra Ruslan Sultanov.
Article based on information from habrahabr.ru
When our file storage exchanged third terabytes, increasingly, our Department began to receive requests to find out who deleted an important document or an entire folder of documents. Often it happens because somebody's malicious intent. Backups are good, but the country must know its heroes. But the milk is twice tastier when we can write it in PowerShell.
Yet I understood, I decided to write for colleagues, but then I thought that might be useful to someone else. The material is mixed. Someone will find a ready solution, someone will come in handy a few obvious methods of work with PowerShell or the task scheduler, and someone will check on the performance of your scripts.
In the process of finding a solution to the problem read article authored Deks. Decided to take it, but some things did not satisfy me.
the
-
the
- first, the generation time of the report four hours on a 2-terabyte storage, which simultaneously employs around 200 people, made up of about five minutes. And despite the fact that once we have in the logs are not written. This is less than Deks but more than hotelos would, because... the
- secondly, all the same should be implemented on twenty servers, it is much less efficient than the main. the
- and thirdly, raised questions launch schedule report generation. the
- And fourth, we wanted to exclude yourself from the process of delivering collected information to end consumers (read: to automate, to me this question is no longer called).
But the thinking of Deks I liked...
A short discourse: If auditing is enabled the file system at the time of deletion of the file in the security log are created two events, with codes 4663 and, then, 4660. The first records the attempted access request for deletion, the user information and the path to the removed file, and the second one captures the fact of removal. Events have a unique identifier EventRecordID that is different per unit of these two events.
The following is the original script that collects information about deleted files and users, to remove them.
the
$time = (get-date) - (new-timespan -min 240)
$Events = Get-WinEvent -FilterHashtable @{LogName="Security";ID=4660;StartTime=$time} | Select TimeCreated,@{n="Record";e={([xml]$_.ToXml()).Event.System.EventRecordID}} |sort Record
$BodyL = ""
$TimeSpan = new-TimeSpan -sec 1
foreach($event in $events){
$PrevEvent = $Event.Entry
$PrevEvent = $PrevEvent - 1
$TimeEvent = $Event.TimeCreated
$TimeEventEnd = $TimeEvent+$TimeSpan
$TimeEventStart = $TimeEvent- (new-timespan -sec 1)
$Body = Get-WinEvent -FilterHashtable @{LogName="Security";ID=4663;StartTime=$TimeEventStart;EndTime=$TimeEventEnd} |where {([xml]$_.ToXml()).Event.System.EventRecordID -match "$PrevEvent"}|where{ ([xml]$_.ToXml()).Event.EventData.Data |where {$_.name-eq "ObjectName"}|where {($_.'#text') -notmatch ".*tmp"} |where {($_.'#text') -notmatch ".*~lock*"}|where {($_.'#text') -notmatch ".*~$*"}} |select TimeCreated, @{n="Fail";e={([xml]$_.ToXml()).Event.EventData.Data | ? {$_.Name-eq "ObjectName"} | %{$_.'#text'}}},@{n="Polzovatel";e={([xml]$_.ToXml()).Event.EventData.Data | ? {$_.Name-eq "SubjectUserName"} | %{$_.'#text'}}}
if ($Body -match ".*Secret*"){
$BodyL=$BodyL+$Body.TimeCreated+"`t"+$Body.Fail+"`t"+$Body.Polzovatel+"`n"
}
}
$Month = $Time.Month
$Year = $Time.Year
$name = "DeletedFiles-"+$Month+"-"+$Year+".txt"
$Outfile = "\serverServerLogFilesDeletedFileslog"+$name
$BodyL | out-file $Outfile -appendUse the command Measure-Command has received the following:
the
Measure-Command {
...
} | Select-Object TotalSeconds | Format-List
...
TotalSeconds : 313,6251476Too much, on the secondary FS will be longer. Immediately it is not like a ten-pipe, so to start I have it structured:
the
Get-WinEvent -FilterHashtable @{
LogName="Security";ID=4663;StartTime=$TimeEventStart;EndTime=$TimeEventEnd
} `
| Where-Object {([xml]$_.ToXml()).Event.System.EventRecordID -match "$PrevEvent"} `
| Where-Object {([xml]$_.ToXml()).Event.EventData.Data `
| Where-Object {$_.name-eq "ObjectName"} `
| Where-Object {($_.'#text') -notmatch ".*tmp"} `
| Where-Object {($_.'#text') -notmatch ".*~lock*"} `
| Where-Object {($_.'#text') -notmatch ".*~$*"}
}
| Select-Object TimeCreated,
@{
n="Fail";
e={([xml]$_.ToXml()).Event.EventData.Data `
| Where-Object {$_.Name-eq "ObjectName"} `
| ForEach-Object {$_.'#text'}
}
},
@{
n="Polzovatel";
e={([xml]$_.ToXml()).Event.EventData.Data `
| Where-Object {$_.Name-eq "SubjectUserName"} `
}
}Managed to reduce the number of storeys of the pipe and to remove enums, Foreach and at the same time make your code more readable, but the effect is not given, the difference is within the error:
the
Measure-Command {
$time = (Get-Date) - (New-TimeSpan -min 240)
$Events = Get-WinEvent -FilterHashtable @{LogName="Security";ID=4660;StartTime=$time}`
| Select TimeCreated,@{n="EventID";e={([xml]$_.ToXml()).Event.System.EventRecordID}}`
| Sort-Object EventID
$DeletedFiles = @()
$TimeSpan = new-TimeSpan -sec 1
foreach($Event in $Events){
$PrevEvent = $Event.EventID
$PrevEvent = $PrevEvent - 1
$TimeEvent = $Event.TimeCreated
$TimeEventEnd = $TimeEvent+$TimeSpan
$TimeEventStart = $TimeEvent- (New-TimeSpan -sec 1)
$DeletedFiles += Get-WinEvent -FilterHashtable @{LogName="Security";ID=4663;StartTime=$TimeEventStart;EndTime=$TimeEventEnd} `
| Where-Object {`
([xml]$_.ToXml()).Event.System.EventRecordID -match "$PrevEvent" `
-and (([xml]$_.ToXml()).Event.EventData.Data `
| where {$_.name-eq "ObjectName"}).'#text' `
-notmatch ".*tmp$|.*~lock$|.*~$*"
} `
| Select-Object TimeCreated,
@{n="FilePath";e={
(([xml]$_.ToXml()).Event.EventData.Data `
| Where-Object {$_.Name-eq "ObjectName"}).'#text'
}
},
@{n="UserName";e={
(([xml]$_.ToXml()).Event.EventData.Data `
| Where-Object {$_.Name-eq "SubjectUserName"}).'#text'
}
} `
}
} | Select-Object TotalSeconds | Format-List
$DeletedFiles | Format-Table UserName,FilePath -AutoSize
...
TotalSeconds : 302,6915627Had to think a little head. What operations take the most time? It would be possible to stick a dozen Measure-Command, but in this case it is evident that the most time is spent on queries to the log (it's not a fast process even in the MMC) and a recurring conversion to XML (in addition, in the case of EventRecordID it is optional). Let's try to do both at once, and at the same time to exclude the intermediate variables:
the
Measure-Command {
$time = (Get-Date) - (New-TimeSpan -min 240)
$Events = Get-WinEvent -FilterHashtable @{LogName="Security";ID=4660,4663;StartTime=$time}`
| Select TimeCreated,ID,RecordID,@{n="EventXML";e={([xml]$_.ToXml()).Event.EventData.Data}}`
| Sort-Object RecordID
$DeletedFiles = @()
foreach($Event in ($Events | Where-Object {$_.Id -EQ 4660})){
$DeletedFiles += $Events `
| Where-Object {`
$_.Id -eq 4663 `
-and $_.RecordID -eq ($Event.RecordID - 1) `
-and ($_.EventXML | where Name -eq "ObjectName").'#text"
-notmatch ".*tmp$|.*~lock$|.*~$"
} `
| Select-Object `
@{n="RecordID";e={$Event.RecordID}}, TimeCreated,
@{n="ObjectName";e={($_.EventXML | where Name -eq "ObjectName").'#text'}},
@{n="UserName";e={($_.EventXML | where Name -eq "SubjectUserName").'#text'}}
}
} | Select-Object TotalSeconds | Format-List
$DeletedFiles | Sort-Object UserName,TimeDeleted | Format-Table -AutoSize -HideTableHeaders
...
TotalSeconds : 167,7099384But this is the result. Acceleration is almost two times!
Automate
Glad, and that's enough. Three minutes is better than five, but how best to run the script? Times per hour? So may escape the record, which appear simultaneously with the start of the script. To make a request not for an hour, and for 65 minutes? Then recording can be repeated. Yes, and then seek a record of the desired file among the thousands of logs — mutor. To write once a day? The log rotation will forget half of it. You need something more reliable. In the comments to the article Deks someone talking about the app on dotnet working in the service mode, but it's, you know, one of those "There are 14 competing standards"...
In Windows task scheduler you can create a trigger on the event in the system log. Here it is:
Great! The script will run exactly at the moment, and our magazine will be sozdavala in real time! But our joy will be incomplete if we will not be able to determine what kind of event we need to record at the time of launch. We need a trick. We have them! Short googling showed that the trigger Event planner can transfer the executable file information about the event. But this is, to put it mildly, not obvious. The sequence of actions is this:
-
the
- Create a task with trigger of type "Event"; the
- to Export the task in XML format (via the MMC); the
- to Add a branch "EventTrigger" new branch "ValueQueries" with elements describing variables:
the<EventTrigger> ... <ValueQueries> <Value name="eventRecordID" > Event/System/EventRecordID</Value> </ValueQueries> </EventTrigger>
where "eventRecordID" — the name of the variable that can be passed to the script, and "Event/System/EventRecordID" element of schema in Windows journal, which can be found at the link below. In this case, the element with the unique number of the event.
But we don't want to aticipate all this with a mouse on 20 servers, right? You want to automate. Unfortunately, PowerShell is not omnipotent, and the New-ScheduledTaskTrigger not yet know how to create triggers Event. Therefore, we apply the cheat code and create a task using the COM object (that is quite often necessary to resort to COM, although the regular cmdlets know more and more c each new version of PS):
the
$scheduler = New-Object -ComObject "Schedule.Service"
$scheduler.Connect("localhost")
$rootFolder = $scheduler.GetFolder("\")
$taskDefinition = $scheduler.NewTask(0)Must allow running multiple instances and, I think, is to disallow manual start and set the time limit:
the
$taskDefinition.Settings.Enabled = $True
$taskDefinition.Settings.Hidden = $False
$taskDefinition.Principal.RunLevel = 0 # 0 - normal privileges, 1 - elevated privileges
$taskDefinition.Settings.MultipleInstances = $True
$taskDefinition.Settings.AllowDemandStart = $False
$taskDefinition.Settings.ExecutionTimeLimit = "PT5M"Create a trigger of type 0 (Event). Then you set the XML query to obtain the desired events. The XML request can be obtained from the MMC console "event Log" and select the necessary settings and switch to the tab "XML":
the
$Trigger = $taskDefinition.Triggers.Create(0)
$Trigger.Subscription = '<QueryList>
<Query Id="0" Path="Security">
<Select Path="Security">
*[System[Provider[@Name="Microsoft-Windows-Security-Auditing"] and EventID=4660]]
</Select>
</Query>
</QueryList>'The main trick: specify the variable to pass to the script.
the
$Trigger.ValueQueries.Create("eventRecordID", "Event/System/EventRecordID")In fact, a description of the command being executed:
the
$Action = $taskDefinition.Actions.Create(0)
$Action.Path = 'PowerShell.exe'
$Action.WorkingDirectory = 'C:\Temp'
$Action.Arguments = '.\ParseDeleted.ps1 $(eventRecordID) C:\Temp\DeletionLog.log'And — fly!
the
$rootFolder.RegisterTaskDefinition("Log Deleted Files", $taskDefinition, 6, 'SYSTEM', $null, 5)"the Concept has changed"
Back to the script for logging. Now we don't need to get all events, and need to get one, but still passed as the argument. For this, we would add headers, converting the script to a cmdlet with parameters. Pile up — will make it possible to change the path to the log "on the fly", maybe useful:
the
[CmdletBinding()]
Param(
[Parameter(Mandatory=$True,Position=1)]$RecordID,
[Parameter(Mandatory=$False,Position=2)]$LogPath = "C:\DeletedFiles.log"
)Then there is a caveat: up until this point we've got events with Get-WinEvent cmdlet and the filter parameter FilterHashtable. It understands a limited set of attributes, which doesn't include the EventRecordID. So the filter we will be using option -FilterXml, we now know how it is!
the
$XmlQuery="<QueryList>
<Query Id='0' Path='Security'>
<Select Path='Security' > *[System[(EventID=4663) and (EventRecordID=$($RecordID - 1))]]</Select>
</Query>
</QueryList>"
$Event = Get-WinEvent -FilterXml $XmlQuery `
| Select TimeCreated,ID,RecordID,@{n="EventXML";e={([xml]$_.ToXml()).Event.EventData.Data}}`Now we no longer need the enumeration Foreach-Object, as processed only one event. Not two, because the event ID 4660 is only used to initiate the script, it is useful information in itself does not carry.
Remember, in the beginning I wanted to allow users without my participation of updatesite? So, in case if you deleted a file in the documents folder of any Department — written log also to the root folder of the Department.
the
$EventLine = ""
if (($Event.EventXML | where Name -eq "ObjectName").'#text' -notmatch ".*tmp$|.*~lock$|.*~$"){
$EventLine += "$($Event.TimeCreated)`t"
$EventLine += "$($Event.RecordID)`t"
$EventLine += ($Event.EventXML | where Name -eq "SubjectUserName").'#text' + "`t"
$EventLine += ($ObjectName = ($Event.EventXML | where Name -eq "ObjectName").'#text')
if ($ObjectName -match "Documents\Unit"){
$OULogPath = $ObjectName `
-replace "(.*Documents\\Units\\[^\\]*\\)(.*)",'$1\DeletedFiles.log'
if (!(Test-Path $OULogPath)){
"DeletionDate'tEventID'tUserName'tObjectPath"| Out-File -FilePath $OULogPath
}
$EventLine | Out-File -FilePath $OULogPath -Append
}
if (!(Test-Path $LogPath)){
"DeletionDate'tEventID'tUserName'tObjectPath" | Out-File -FilePath $LogPath }
$EventLine | Out-File -FilePath $LogPath -Append
}Final cmdlet
Well, the pieces of sliced, left to put it all together and even a little bit to optimize. Get something like this:
the
[CmdletBinding()]
Param(
[Parameter(Mandatory=$True,Position=1,ParameterSetName='logEvent')][int]$RecordID,
[Parameter(Mandatory=$False,Position=2,ParameterSetName='logEvent')]
[string]$LogPath = "$PSScriptRoot\DeletedFiles.log",
[Parameter(ParameterSetName='install')][switch]$Install
)
if ($Install) {
$service = New-Object -ComObject "Schedule.Service"
$service.Connect("localhost")
$rootFolder = $service.GetFolder("\")
$taskDefinition = $service.NewTask(0)
$taskDefinition.Settings.Enabled = $True
$taskDefinition.Settings.Hidden = $False
$taskDefinition.Settings.MultipleInstances = $True
$taskDefinition.Settings.AllowDemandStart = $False
$taskDefinition.Settings.ExecutionTimeLimit = "PT5M"
$taskDefinition.Principal.RunLevel = 0
$trigger = $taskDefinition.Triggers.Create(0)
$trigger.Subscription = '
<QueryList>
<Query Id="0" Path="Security">
<Select Path="Security">
*[System[Provider[@Name="Microsoft-Windows-Security-Auditing"] and EventID=4660]]
</Select>
</Query>
</QueryList>'
$trigger.ValueQueries.Create("eventRecordID", "Event/System/EventRecordID")
$Action = $taskDefinition.Actions.Create(0)
$Action.Path = 'PowerShell.exe'
$Action.WorkingDirectory = $PSScriptRoot
$Action.Arguments = '.\' + $MyInvocation.MyCommand.Name + '$(eventRecordID) ' + $LogPath
$rootFolder.RegisterTaskDefinition("Log Deleted Files", $taskDefinition, 6, 'SYSTEM', $null, 5)
} else {
$XmlQuery="<QueryList>
<Query Id='0' Path='Security'>
<Select Path='Security' > *[System[(EventID=4663) and (EventRecordID=$($RecordID - 1))]]</Select>
</Query>
</QueryList>"
$Event = Get-WinEvent -FilterXml $XmlQuery `
| Select TimeCreated,ID,RecordID,@{n="EventXML";e={([xml]$_.ToXml()).Event.EventData.Data}}
if (($ObjectName = ($Event.EventXML | where Name -eq "ObjectName").'#text') `
-notmatch ".*tmp$|.*~lock$|.*~$"){
$EventLine = "$($Event.TimeCreated)`t" + "$($Event.RecordID)`t" `
+ ($Event.EventXML | where Name -eq "SubjectUserName").'#text' + "`t" `
+ $ObjectName
if ($ObjectName -match ".*Documents\\Units\\[^\\]*\\"){
$OULogPath = $Matches[0] + '\DeletedFiles.log'
if (!(Test-Path $OULogPath)){
"DeletionDate'tEventID'tUserName'tObjectPath"| Out-File -FilePath $OULogPath
}
$EventLine | Out-File -FilePath $OULogPath -Append
}
if (!(Test-Path $LogPath)){
"DeletionDate'tEventID'tUserName'tObjectPath" | Out-File -FilePath $LogPath }
$EventLine | Out-File -FilePath $LogPath -Append
}
}Left to put the script in a convenient place and run the key Install.
Now the employees of any Department can real-time to see who, what and when removed from their directory. Note that I did not consider here the right of access log files (so that villain couldn't remove) and rotate them. Structure and access rights to the directories on our filer pull on separate article, and rotation to some extent, will complicate the search for the desired string.
materials Used:
— the Finest reference on regular expressions
— Tutorial on creating a task linked to an event
— a description of the scripting API scheduler
UPD: In the final script had a typo, after line 41 was unnecessary gravis. For the detection thanks to the reader Habra Ruslan Sultanov.
Комментарии
Отправить комментарий