Folder copy with logging in Powershell, and a bit about scripts in general

Problem

I have been trying for some time to find an easy method for keeping my scripts up to date on my servers. I could of course use robocopy or something like that, but I wanted something written in PS. I figured I would learn something along the way, and I also had hopes that this would be fairly easy to accomplish. It would seem I was wrong on the being easy part, or perhaps I have over-engineered it slightly Smilefjes som blunker.

Not wanting to re-invent the wheel, I summoned the powers of the closest search engine to get up with some samples I could build on. I am a bit prejudiced towards scripts from the web in general, as I usually find most scripts longer than a few lines have some logical bugs in them. Scripts are, in general, easy to get started with, but it is very difficult to produce robust scripts. I have debugged countless VB and Powershell scripts (both my own and those of others) who were working fine in the lab, and perhaps also in production, but suddenly they cease to function as expected. Usually this is caused by some simple logical error appearing due to changes in the environment the script is running in, but from time to time you come across some obscure scripting engine bug you have to program around. And of course you have the pure idiotic fails, such as creating vb-scripts requiring “On error resume next” at the top of the script. Those are usually doomed by design and can take days to debug. Since I am fairly proficient in C# I usually just write a small utility .exe instead, thus circumventing many of the problems altogether. Once you have spent 4 hours debugging an error caused by a misspelled variable name in the middle of a 200 line script, you start dreaming about the wonders of explicit variable declaration and intellisense. Anyways, I think that is enough ranting for one post.Smilefjes

Research

After some searching I came across this marvelous script by Brandon Shell: http://bsonposh.com/archives/231. It describes a fairly robust method for syncing two folders, using MD5 to check if the files are equal. I was only interested in doing a one way copy, so I made some changes and rewrote it as a function. I found it was limited to local paths only, but a small change enabled me to use UNC paths as well.

##Copy folder contents with log
##Inspired by http://bsonposh.com/archives/231
function copyWithLog
{
	Param($Source,$Destination)

	#Check if destination exists
	if(!(Test-Path -Path $Destination -PathType Container)) 
	{
	   $null = New-Item $Destination -Type Directory -Force | Out-Null
	}
	$log = join-path $Destination "copy.log"
	Add-Content -force $log -value ("`n`n[$(get-date)] Copy started from  $Source to $Destination") 
	# Getting Files/Folders from Source and Destination
	 $SrcEntries = Get-ChildItem $Source -Recurse
	 $DesEntries = Get-ChildItem $Destination -Recurse

	# Parsing the folders and Files from Collections
	 $Srcfolders = $SrcEntries | Where-Object{$_.PSIsContainer}
	 $SrcFiles = $SrcEntries | Where-Object{!$_.PSIsContainer}
	 $Desfolders = $DesEntries | Where-Object{$_.PSIsContainer}
	 $DesFiles = $DesEntries | Where-Object{!$_.PSIsContainer}

	# Checking for Folders that are in Source, but not in Destination
	foreach($folder in $Srcfolders)
	{
	 $DesFolder = $folder.Fullname -replace [regex]::Escape($Source),$Destination
		 if($DesFolder -ne ""){
			 if(!(test-path $DesFolder))
			 {
				 Write-Host "Folder $DesFolder is missing. Adding..."
				 Add-Content -force $log -value ("Folder $DesFolder is missing. Adding...")
				 new-Item $DesFolder -type Directory | out-Null
			 }
		}
	}

	# Checking for Files that are in the Source, but not in Destination
	 foreach($entry in $SrcFiles)
	 {
		 $SrcFullname = $entry.fullname
		 $SrcName = $entry.Name
		 $DesFile = $SrcFullname -replace [regex]::Escape($Source),$Destination
		 if(test-Path $Desfile)
		 {
			 $SrcMD5 = Get-FileMD5 $SrcFullname
			 $DesMD5 = Get-FileMD5 $DesFile
			 If($srcMD5 -ne $desMD5)
			 {
				 Write-Host "The files MD5's are different. Updating $DesFile..."
				 Add-Content -force $log -value ("The files MD5's are different. Updating $DesFile...")
				 copy-Item -path $SrcFullName -dest $DesFile -force
			 }
		 }
		 else
		 {
			 Write-Host "$Desfile missing. Copying from $SrcFullname"
			 Add-Content -force $log -value ("$Desfile missing. Copying from $SrcFullname")
			 copy-Item -path $SrcFullName -dest $DesFile -force
		 }
	 }	

   . goToMenu
} 

 ##Get MD5
 function Get-FileMD5 {
    Param([string]$file)
    $mode = [System.IO.FileMode]("open")
    $access = [System.IO.FileAccess]("Read")
    $md5 = New-Object System.Security.Cryptography.MD5CryptoServiceProvider
    $fs = New-Object System.IO.FileStream($file,$mode,$access)
    $Hash = $md5.ComputeHash($fs)
    $fs.Close()
    [string]$Hash = $Hash
    Return $Hash
 }

The following change had to be done to enable the use of UNC paths:

Original:

$SrcFolderPath = $source -replace “\\”,”\\” -replace “\:”,”\:”
$DesFolder = $folder.Fullname -replace $SrcFolderPath,$Destination

New:

$DesFolder = $folder.Fullname -replace [regex]::Escape($Source),$Destination

I don’t know what is wrong with the original as I don’t know all that much about regex, but a plain text comparison works nicely. ([regex]::Escape($Variable) forces a plain text comparison). This function enables me to copy a set of files from folder A to folder B. It is recursive, so all subfolders and files get copied as well. I the target file exists, it checks if the MD5 hashes are equal. If not, the file is overwritten by the source. All changes are logged to a file called copy.log in the target folder. I thought about scheduling it, but I usually need to update manually as I make changes so I decided to add the function to a manual script instead. Then I had an idea, what if I could update the Sysinternals Suite using the same function? It is available as a webdav share at live.sysinternals.com, and I spend quite a bit of time updating it on my servers. Then it struck me that some of my servers are not allowed to connect directly to the internet, but I could solve this by by creating a local repository for those servers, and then updating the repository from a workstation.

After that thought I came down with a trace of featureitis and added a couple of bonus functions: Creating a process explorer shortcut, enabling the telnet client feature and adding the sysinternals folder to the path.

Solution

The script is fairly long, over 250 lines, but I think it should be pretty self-explanatory with the comments and the menu.

image

You can download the finished script here

Author: DizzyBadger

SQL Server DBA, Cluster expert, Principal Analyst

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.