PowerShell - Not outputting file as scheduled task

I have the following ps script. Shown below. When run interactively (via ps shell or call via a batch file) it outputs both the .csv and .xlsx file. When run via a scheduled task it will only output the .csv file and not the .xlsx file. I know it is not permission related as it works fine interactively (outputs both xls and csv file).

The excel call in the script is working when called via a scheduled task because it is shows up in Task Manager.

The job runs with admin level account and has the "Run with highest privileges" toggled. It is also set to run whether a user is logged in our not.

Param(
$csvFile = "e:\csvexport\export.csv",
$path = "e:\csvexport\ad-export.xlsx"
)
Set-ExecutionPolicy RemoteSigned
import-module activedirectory

If (Test-Path e:\csvexport\ad-export.xlsx){
Remove-Item e:\csvexport\ad-export.xlsx -Recurse
}

If (Test-Path e:\csvexport\export.csv){
Remove-Item e:\csvexport\export.csv -Recurse
}
Get-ADUser -Filter * -SearchBase "OUR OU INFO" -Properties EmailAddress,OfficePhone,MobilePhone | Export-Csv -NoTypeInformation -Path e:\csvexport\export.csv

$processes = Import-Csv -Path $csvFile
$excel = New-Object -ComObject excel.application
$excel.visible = $false
$workbook = $excel.workbooks.add()
$excel.cells.item(1,1) = "First Name"
$excel.cells.item(1,2) = "Last Name"
$excel.cells.item(1,3) = "E-Mail Address"
$excel.cells.item(1,4) = "Office Phone"
$excel.cells.item(1,5) = "Mobile Phone"
$excel.cells.item(1,6) = "Login Name"
$excel.cells.item(1,7) = "AD Object Type"
$excel.cells.item(1,8) = "AD Account Enabled"
$excel.cells.item(1,9) = "AD Tree Location"
$i = 2
foreach($process in $processes)
{
$excel.cells.item($i,1) = $process.GivenName
$excel.cells.item($i,2) = $process.Surname
$excel.cells.item($i,3) = $process.EmailAddress
$excel.cells.item($i,4) = $process.OfficePhone
$excel.cells.item($i,5) = $process.MobilePhone
$excel.cells.item($i,6) = $process.SamAccountName
$excel.cells.item($i,7) = $process.ObjectClass
$excel.cells.item($i,8) = $process.Enabled
$excel.cells.item($i,9) = $process.DistinguishedName
$i++
} #end foreach process
$workbook.saveas($path)
$Excel.Quit()
Remove-Variable -Name excel
[gc]::collect()
[gc]::WaitForPendingFinalizers()