Using FileVaultMaster.keychain key recovery on FileVault2 Lion FDE

Update Sep 2012: apple has finally released some filevault doco that describes the architecture and this process.

In OS X 10.7 Lion, Apple introduced full disk encryption in the form of FileVault 2. Apple provides a means to recover encrypted disks using 24 character recovery codes like this:

There is a more corporate version that uses asymmetric keys in a similar way to FileVault 1 (see 'Using FileVault Master Keychain'). First, create the FileVaultMaster keychain

$ sudo certtool y c k=/Library/Keychains/FileVaultMaster.keychain
...0 certificates found
...0 CRLs found

Then add a public/private key pair. The process seems to require that the labels and/or the CN of the certificate begin with the string 'FileVault Recovery Key'.

$ sudo certtool c k=/Library/Keychains/FileVaultMaster.keychain o=/Library/Keychains/FileVaultMaster.cer
Enter key and certificate label: FileVault Recovery Key (myhost.mydomain)        

Please specify parameters for the key pair you will generate.

  r  RSA
  d  DSA
  f  FEE
  e  ECDSA

Select key algorithm by letter: r

Valid key sizes for RSA are 512..2048; default is 512
Enter key size in bits or CR for default: 2048

You have selected algorithm RSA, key size 2048 bits.
OK (y/anything)? y
Enter cert/key usage (s=signing, b=signing AND encrypting, d(derive AND sign): b
...Generating key pair...

Please specify the algorithm with which your certificate will be signed.

  5  RSA with MD5
  s  RSA with SHA1

Select signature algorithm by letter: s

You have selected algorithm RSA with SHA1.
OK (y/anything)? y
...creating certificate...

You will now specify the various components of the certificate's
Relative Distinguished Name (RDN). An RDN has a number of 
components, all of which are optional, but at least one of 
which must be present. 

Note that if you are creating a certificate for use in an 
SSL/TLS server, the Common Name component of the RDN must match
exactly the host name of the server. This must not be an IP
address, but the actual domain name, e.g. www.apple.com.

Entering a CR for a given RDN component results in no value for
that component.

Common Name       (e.g, www.apple.com) : FileVault Recovery Key (myhost.mydomain)
Country           (e.g, US) : 
Organization      (e.g, Apple Computer, Inc.) : 
Organization Unit (e.g, Apple Data Security) : 
State/Province    (e.g, California) : 
Email Address     (e.g, johngalt@rand.com) : 

You have specified:
  Common Name       : FileVault Recovery Key (myhost.mydomain)
Is this OK (y/anything)? y
Wrote 765 bytes of CSR to /Library/Keychains/FileVaultMaster.cer
..cert stored in Keychain.

You can view the contents of the keychain with dump-keychain

$ sudo security dump-keychain /Library/Keychains/FileVaultMaster.keychain 
keychain: "/Library/Keychains/FileVaultMaster.keychain"
class: 0x00000010 
attributes:
    0x00000000 =0x00000010 
    0x00000001 =0x46696C655661756C74205265636F76657279204B6579202863383A32613A31343A34333A30353A33352900  "FileVault Recovery Key (myhost.mydomain)\000"
    0x00000002 =
    0x00000003 =0x00000001 
    0x00000004 =0x00000000 
    0x00000005 =0x00000000 
    0x00000006 =0xC2BF20DAFFB583D6D0B61DFB8AB67A09F73CE879  "\302\277 \332\377\265\203\326\320\266\035\373\212\266z\011\367<\350y"
    0x00000007 =
    0x00000008 =0x7B38373139316361322D306663392D313164342D383439612D3030303530326235323132327D00  "{87191ca2-0fc9-11d4-849a-000502b52122}\000"
    0x00000009 =0x0000002A  "\000\000\000*"
    0x0000000A =0x00000800 
    0x0000000B =0x00000800 
    0x0000000C =0x0000000000000000 
    0x0000000D =0x0000000000000000 
    0x0000000E =0x00000001 
    0x0000000F =0x00000001 
    0x00000010 =0x00000001 
    0x00000011 =0x00000000 
    0x00000012 =0x00000000 
    0x00000013 =0x00000001 
    0x00000014 =0x00000000 
    0x00000015 =0x00000001 
    0x00000016 =0x00000000 
    0x00000017 =0x00000000 
    0x00000018 =0x00000000 
    0x00000019 =0x00000000 
    0x0000001A =0x00000001 
keychain: "/Library/Keychains/FileVaultMaster.keychain"
class: 0x80001000 
attributes:
    "alis"="FileVault Recovery Key (myhost.mydomain)"
    "cenc"=0x00000003 
    "ctyp"=0x00000001 
    "hpky"=0xC2BF20DAFFB583D6D0B61DFB8AB67A09F73CE879  "\302\277 \332\377\265\203\326\320\266\035\373\212\266z\011\367<\350y"
    "issu"=0x30353133303106035504030C2A46696C655661756C74205265636F76657279204B6579202863383A32613A31343A34333A30353A333529  "051301\006\003U\004\003\014*FileVault Recovery Key (myhost.mydomain)"
    "labl"="FileVault Recovery Key (myhost.mydomain)"
    "skid"=
    "snbr"=0x659E5B0A  "e\236[\012"
    "subj"=0x30353133303106035504030C2A46696C655661756C74205265636F76657279204B6579202863383A32613A31343A34333A30353A333529  "051301\006\003U\004\003\014*FileVault Recovery Key (myhost.mydomain)"

Copy the keychain somewhere for escrow/backup:

$ cp /Library/Keychains/FileVaultMaster.keychain /Volumes/usbdisk/

Delete the private key out of the keychain using the 'Keychain Access' GUI. This must be done, or FileVault2 will refuse to recognise the keychain.

$ sudo /Applications/Utilities/Keychain\ Access.app/Contents/MacOS/Keychain\ Access 

You should now see only the public cert:

$ sudo security dump-keychain /Library/Keychains/FileVaultMaster.keychain 
keychain: "/Library/Keychains/FileVaultMaster.keychain"
class: 0x80001000 
attributes:
    "alis"="FileVault Recovery Key (myhost.mydomain)"
    "cenc"=0x00000003 
    "ctyp"=0x00000001 
    "hpky"=0xC2BF20DAFFB583D6D0B61DFB8AB67A09F73CE879  "\302\277 \332\377\265\203\326\320\266\035\373\212\266z\011\367<\350y"
    "issu"=0x30353133303106035504030C2A46696C655661756C74205265636F76657279204B6579202863383A32613A31343A34333A30353A333529  "051301\006\003U\004\003\014*FileVault Recovery Key (myhost.mydomain)"
    "labl"="FileVault Recovery Key (myhost.mydomain)"
    "skid"=
    "snbr"=0x659E5B0A  "e\236[\012"
    "subj"=0x30353133303106035504030C2A46696C655661756C74205265636F76657279204B6579202863383A32613A31343A34333A30353A333529  "051301\006\003U\004\003\014*FileVault Recovery Key (myhost.mydomain)"

For recovery you want to be able to mount the encrypted disk. Boot into a lion CD/USB disk/netboot image (you need lion since it has a new version of diskutil with the corestorage commands).

-bash-3.2# diskutil cs list
CoreStorage logical volume groups (1 found)
|
+-- Logical Volume Group 0483D899-2A53-4113-B8AA-6F11416AEB4C
    =========================================================
    Name:         os
    Sequence:     1
    Free Space:   0 B (0 B)
    |
    +-< Physical Volume 6E70D8B2-BBE0-42F9-BBAD-F472882FC6C2
    |   ----------------------------------------------------
    |   Index:    0
    |   Disk:     disk0s2
    |   Status:   Online
    |   Size:     108549537792 B (108.5 GB)
    |
    +-> Logical Volume Family 7021584E-3C31-46FC-AB8C-DAFFBF660818
        ----------------------------------------------------------
        Sequence:               8
        Encryption Status:      Locked
        Encryption Type:        AES-XTS
        Encryption Context:     Present
        Conversion Status:      Complete
        Has Encrypted Extents:  Yes
        Conversion Direction:   -none-
        |
        +-> Logical Volume 18878163-252F-49B3-B4BF-D923D30CB797
            ---------------------------------------------------
            Disk:               -none-
            Status:             Locked
            Sequence:           4
            Size (Total):       108230766592 B (108.2 GB)
            Size (Converted):   -none-
            Revertible:         Yes (unlock and decryption required)
            LV Name:            os
            Content Hint:       Apple_HFS

Unlock the encrypted logical volume:

-bash-3.2# diskutil coreStorage unlockVolume 18878163-252F-49B3-B4BF-D923D30CB797 -recoveryKeyChain /Volumes/usbdisk/FileVaultMaster.keychain
Started CoreStorage operation
Logical Volume successfully unlocked
Logical Volume successfully attached as disk14
Logical Volume successfully mounted as /Volumes/os
Core Storage disk: disk14
Finished CoreStorage operation
-bash-3.2# ls /Volumes/os/
.DS_Store       .Trashes        .fseventsd      Applications    Network         Users           bin             dev             home            net             sbin            usr
.Spotlight-V100 .file           .vol            Library         System          Volumes         cores           etc             mach_kernel     private         tmp             var

Is this Mac OS X app compiled with ASLR?

To determine if an OS X Mach-o binary is compiled with ASLR, look for the Position Independent Executable (PIE) flag in the Mach header:

$ otool -hv MobileSafari 

MobileSafari:
Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
   MH_MAGIC     ARM         V7  0x00     EXECUTE    40       4560   NOUNDEFS DYLDLINK TWOLEVEL PIE

Python: add an entry to an dict of arrays or create one if it doesn't exist

I frequently am working with dictionaries of arrays and want to add an element to an existing array, or create one if it doesn't exist. There is a neat, if not particularly easy to understand, way to do this with setdefault:

result = {}
for thisthing in alist:
   result.setdefault(thisthing.somekey, []).append(thisthing.somevalue)

So you will end up with something like:

{'key1':[value1, value2], 'key2': [value3]}