Having issues with Basic ACL in powershell

I am having issues removing NTFS Privileges on a Windows 2012 server (I want to make learn how use it in command line mode before I roll over my servers to windows 2012 Core)

This should be simple, but I cannot figure out what I'm doing wrong. Its probably obvious but im blind to the solution.

I have a folder 'D:\Finance' that has the basic ACL's assigned to it. I want to leave administrative/System/Owner users and groups as is but want to only give members of the group TestDNS\Finance privileges.

Here is the default ACL

PS D:\ Get-ACL \Finance | Format-List
Directory: D:\Finance

Path   : Microsoft.PowerShell.Core\FileSystem::D:\Finance        
Owner  : NT AUTHORITY\SYSTEM
Group  : NT AUTHORITY\SYSTEM
Access : Everyone Allow ReadAndExecute, Synchronize
         Creator Owner Allow 268435456
         NT Authority\System Allow FullControl
         BUILTIN\Administrators Allow FullControl
         BUILTIN\Users Allow AppendData
         BUILTIN\Users Allow CreateFiles
         BUILTIN\Users Allow ReadAnExecute, Synchronize

Audit  :
Sddl   : O:SYG:SYD:(A;;0x1200a9;;;WD)(A;OICIIO;GA;;;CO)(A;OICI;
         FA;;;SY)(A;OICI;FA;;;BA)(A;CI;LC;;BU)(A;CIIO;DC;;;BU)(A;
         OICI;0x1200a9;;;BU)

The first thing I did was save to a variable and remove inheritance:

PS D:\> $acl = Get-Acl D:\Finance
PS D:\> $acl.SetAccessRuleProtection($true,$true)
PS D:\> Set-ACL D:\Finance $acl
PS D:\> $acl = Get-Acl D:\Finance

That appears to have worked, if I run

PS D:\> $acl.getaclaccessrules($true,$true,[system.security.principal.ntaccount])

it shows that none of the finsystem have the "IsInherited" property set to true. But that is where it stops working.

I can add new permissions, and remove those, but I cannot remove any of the preset permissions.

one of the results I get is

FileSystemRights   : CreateFiles, AppendData
AccessControleType : Allow
IdentityReference  : BUILTIN\Users
IsInheritded       : False
InheritanceFlags   : ContainerInherit
PropagationFlags   : None

I tried removing the ACLs a couple of ways:

PS D:\> $ACLrule = new-object system.security.accesscontrol.FileSystemAccessRule("BUILTIN\USERS", "CreateFiles, AppendData",,,"Allow")
PS D:\> $Acl.removeaccessrule($ACLrule)

and

PS D:\> $ACLrule = new-object system.security.accesscontrol.FileSystemAccessRule("BUILTIN\USERS", "CreateFiles, AppendData","ContainerInherit","None","Allow")
PS D:\> $Acl.removeaccessrule($ACLrule)

Wen I perform either of those commands, and check the $Acl it is still there. If somebody can help me out, point out what exactly I'm doing wrong, It would help me out a lot.

I have been banging my head away on this for 2 days, and its no clearer then It was when I first came across the problem. I'm certain it is something simple I'm overlooking, but I cannot see it..