Fintek IR receiver and XBox remote to control MythTV: fail

So I'd gotten the Fintek IR receiver to work previously under ubuntu, but the upgrade to precise broke it. Little did I know that things had changed drastically when LIRC was built into the kernel.

Mythtv has new LIRC configuration instructions, and I spent a fair bit of time following a helpful forum post. After spending a number of hours screwing with it, I'm documenting what I have and giving up, maybe I'll try again later.

Lets start from the beginning. The kernel recognises the device (hurrah!):

$ lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 002: ID 1934:5168 Feature Integration Technology Inc. (Fintek) F71610A or F71612A Consumer Infrared Receiver/Transceiver

If you stop lirc, you can get keycodes out of ir-keytable:

$ sudo ir-keytable -t --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00
Testing events. Please, press CTRL-C to abort.
1356325906.087938: event MSC: scancode = 1f3f
1356325906.087941: event sync
1356325906.201936: event MSC: scancode = 800f7422
1356325906.201942: event sync
1356325907.147952: event MSC: scancode = 1f3f
1356325907.147955: event sync
1356325907.253948: event MSC: scancode = 1f3f
1356325907.253951: event sync

I switched to using the xbox remote, and found someone had already posted a complete keytab file, here it is for reference should anyone else want to save some time:

# table xbox, type: UNKNOWN
0x800f7400 KEY_NUMERIC_0
0x800f7401 KEY_NUMERIC_1
0x800f7402 KEY_NUMERIC_2
0x800f7403 KEY_NUMERIC_3
0x800f7404 KEY_NUMERIC_4
0x800f7405 KEY_NUMERIC_5
0x800f7406 KEY_NUMERIC_6
0x800f7407 KEY_NUMERIC_7
0x800f7408 KEY_NUMERIC_8
0x800f7409 KEY_NUMERIC_9
0x800f740a KEY_DELETE
0x800f740b KEY_ENTER
0x800f740c KEY_SLEEP
0x800f740d KEY_MEDIA
0x800f740e KEY_MUTE
0x800f740f KEY_INFO
0x800f7410 KEY_VOLUMEUP
0x800f7411 KEY_VOLUMEDOWN
0x800f7412 KEY_CHANNELUP
0x800f7413 KEY_CHANNELDOWN
0x800f7414 KEY_FASTFORWARD
0x800f7415 KEY_REWIND
0x800f7416 KEY_PLAY
0x800f7417 KEY_RECORD
0x800f7418 KEY_PAUSE
0x800f7419 KEY_STOP
0x800f741a KEY_NEXT
0x800f741b KEY_PREVIOUS
0x800f741c KEY_NUMERIC_POUND
0x800f741d KEY_NUMERIC_STAR
0x800f741e KEY_UP
0x800f741f KEY_DOWN
0x800f7420 KEY_LEFT
0x800f7421 KEY_RIGHT
0x800f7422 KEY_OK
0x800f7423 KEY_EXIT
0x800f7424 KEY_DVD
0x800f744f KEY_EPG
0x800f7427 KEY_ZOOM
0x800f7432 KEY_MODE
0x800f7433 KEY_PRESENTATION
0x800f7428 KEY_EJECTCD
0x800f743a KEY_BRIGHTNESSUP
0x800f7446 KEY_TV
0x800f7447 KEY_AUDIO
0x800f7448 KEY_PVR
0x800f7449 KEY_CAMERA
0x800f744a KEY_VIDEO
0x800f744c KEY_LANGUAGE
0x800f7451 KEY_TITLE
0x800f744e KEY_PRINT
0x800f7450 KEY_RADIO
0x800f745a KEY_SUBTITLE
0x800f7425 KEY_RED
0x800f7466 KEY_GREEN
0x800f7426 KEY_YELLOW
0x800f7468 KEY_BLUE
0x800f7465 KEY_POWER2
0x800f746e KEY_PLAYPAUSE
0x800f746f KEY_PLAYER
0x800f7480 KEY_BRIGHTNESSDOWN
0x800f7481 KEY_PLAYPAUSE

I wrote the table to the system keytable (not sure what that error means):

$ sudo ir-keytable -c -w /etc/xbox_remote_keymap --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00
Read xbox table
Old keytable cleared
Wrote 62 keycode(s) to driver
���/protocol: No such file or directory
Couldn't change the IR protocols

Check it is stored OK:

$ sudo ir-keytable -r --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00
[snip same output]
Enabled protocols: RC-5

And now, key events are recognised:

$ sudo ir-keytable -t --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00Testing events. Please, press CTRL-C to abort.
1356326333.173697: event MSC: scancode = 800f7422
1356326333.173702: event key down: KEY_OK (0x0160)
1356326333.173703: event sync
1356326333.201701: event key up: KEY_OK (0x0160)
1356326333.201704: event MSC: scancode = 1f3f
1356326333.201705: event sync
1356326333.280699: event MSC: scancode = 800f7422
1356326333.280703: event key down: KEY_OK (0x0160)
1356326333.280704: event sync
1356326333.530999: event key up: KEY_OK (0x0160)
1356326333.531001: event sync
1356326337.414740: event MSC: scancode = 800f7410
1356326337.414747: event key down: KEY_VOLUMEUP (0x0073)
1356326337.414748: event sync
1356326337.522744: event MSC: scancode = 800f7410
1356326337.522746: event sync
1356326337.771005: event key up: KEY_VOLUMEUP (0x0073)
1356326337.771007: event sync

I changed my lirc config over to devinput as suggested in the mythtv instructions, but if I start LIRC back up, I don't see any output in irw, and the button presses don't do anything. If I leave lirc off, lots of functionality (such as back on the remote = Esc in mythtv) doesn't work since lirc isn't doing that key mapping.

I'm running lirc 0.9:

$ lircd --version
lircd 0.9.0

I suspect the problem is in the error message from the write operation. Even if you tell it to send events to LIRC specifically:

$ sudo ir-keytable -c -p LIRC,RC-5 -w /etc/xbox_remote_keymap --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00
Read xbox table
Old keytable cleared
Wrote 62 keycode(s) to driver
���/protocol: No such file or directory
Couldn't change the IR protocols

It still only lists RC-5 as an 'enabled protocol'. And yet, this seems to indicate it is enabled:

$ cat /sys/class/rc/rc1/protocols 
[rc-5] [nec] [rc-6] [jvc] [sony] [mce_kbd] [lirc]

Python sqlite3

Some notes on using sqlite3 from python. First, lets create a table in an in-memory database:

import sqlite3

DB_TABLE_NAME = 'cache'
DB_TABLE_COLUMN_DEFS = 'event blob'

conn = sqlite3.connect(':memory:')
cur = conn.cursor()

sql = 'CREATE TABLE %(table_name)s (%(column_defs)s)' % {
      'table_name': DB_TABLE_NAME,
      'column_defs': DB_TABLE_COLUMN_DEFS}

cur.execute(sql)

Next we want to insert multiple values at once. You can use executemany to avoid needing a ? for every single value, although the format it requires is quite clumsy. A list of values for insertion would look like this:

[('test',), ('test',), ('test',), ('test',), ('test',)]

Neither the questionmark or named parameter substitution seems to work for table names, so you'll need to stick those in using normal string interpolation. The actual insert statement is:

VALUES = [('test',)] * 58
sql = 'INSERT INTO %(table_name)s VALUES (?)' % {'table_name':
                                                  DB_TABLE_NAME}
cur.executemany(sql, VALUES)

This function will delete a list of ids by converting the list to the necessary array of tuples first:

def DeleteRows(id_list):
  """Deletes row from database.

  Args:
    id_list: list of row ids.
  """
  sql = 'DELETE FROM %s WHERE rowid IN (?)' % DB_TABLE_NAME

  # sqlite needs a sequence like [(34,), (38,)]
  id_tuples = []
  for rowid in id_list:
    id_tuples.append((rowid,))

  cur.executemany(sql, id_tuples)

Test code for this function looks like the following. Using executemany for select will get you an error, so I just spell out the necessary ?'s.

id_list = [8, 51, 52, 58]
select_list = [8, 51, 34, 38, 52, 58]
DeleteRows(id_list)
sql = 'SELECT rowid FROM %(table_name)s WHERE rowid IN (?,?,?,?,?,?)' % {
    'table_name': DB_TABLE_NAME}
result = cur.execute(sql, tuple(select_list))
assert(result.fetchall() == [(34,), (38,)])

Python: else clause on a for loop

From the "huh, I didn't know you could do that" files:

In [41]: list=[1,2,3]

In [42]: for x in list:
   ....:     print x
   ....: else:
   ....:     print '%s was last' % x
   ....:     
1
2
3
3 was last

It turns out you can have an else clause on loop statements, it will execute as long as the loop exits cleanly (i.e. doesn't break). This includes if 'list' is empty. There's an interesting discussion about its merits here.

Facebook BigMac Ruxcon presentation

Nice security presentation from Facebook. Of interest to the mac security world: basic persistence (launcha/launchd etc.) logged and white/black listed (see slide 66).

Ubuntu Unity/Compiz disable workspace switching animation

Disabling the horrible compiz workspace switching animation in Ubuntu precise is doable, if not particularly easy:

sudo apt-get install compizconfig-settings-manager

• Open Unity dash > type "compiz" > run "CompizConfig Settings Manager"
• on the right panel click on Desktop Wall
• in the Viewport Switching tab set "Wall Sliding Duration" to 0
• in the Viewport Switch Preview set "Switch Target Preview Visibility Time" to 0

Sophail: Tavis and Sophos

Tavis continues his damning analysis of Sophos AV.  The latest full disclosure post has a good summary, and includes links to sophail1 and sophail2 papers, and a very well-commented working exploit for Sophos on the Mac.  If you use Sophos you should seriously consider the benefits it is providing vs. the increased attack surface it brings.

Sophos' latest official response is quite different from the tone of previous interactions with Tavis.

Micosoft Windows testing VMs

Microsoft has recently provided a set of VMs for testing, which is great. The set includes images for XP, Vista, and 7. They are built for MS Virtual PC, but I'm told unpacking the .exe with 7zip and running them in Virtual Box works, but you'll need to deal with a shorter activation expiry (3 days) due to the 'hardware change'.

[SOLVED] dependency problems prevent configuration of libc6-dev

I recently descended into endlessly broken apt and dpkg hell. Usually this is enough to get out of the woods:

sudo dpkg --configure -a
sudo apt-get -f install

but not this time. I also tried cleaning out the cache and re-downloading the packages, but no dice. Until this is fixed you can't install or remove any other packages. Here's a few things I tried and the errors I was getting:

$ sudo dpkg --configure -a
dpkg: dependency problems prevent configuration of libc6-dev:
 libc6-dev depends on libc6 (= 2.15-0ubuntu10.2); however:
  Version of libc6 on system is 2.15-0ubuntu10.3.
 libc6-dev depends on libc-dev-bin (= 2.15-0ubuntu10.2); however:
  Version of libc-dev-bin on system is 2.15-0ubuntu10.3.
dpkg: error processing libc6-dev (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 libc6-dev

$ sudo dpkg -r libc6-dev
dpkg: dependency problems prevent removal of libc6-dev:
 libgnutls-dev depends on libc6-dev | libc-dev; however:
  Package libc6-dev is to be removed.
  Package libc-dev is not installed.
  Package libc6-dev which provides libc-dev is to be removed.

 [snip]

 libgcrypt11-dev depends on libc6-dev | libc-dev; however:
  Package libc6-dev is to be removed.
  Package libc-dev is not installed.
  Package libc6-dev which provides libc-dev is to be removed.
dpkg: error processing libc6-dev (--remove):
 dependency problems - not removing
Errors were encountered while processing:
 libc6-dev

$ sudo apt-get remove libc-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'libc6-dev' instead of 'libc-dev'
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
 libc6-dev : Depends: libc6 (= 2.15-0ubuntu10.2) but 2.15-0ubuntu10.3 is to be installed
             Depends: libc-dev-bin (= 2.15-0ubuntu10.2) but 2.15-0ubuntu10.3 is to be installed
E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).

$ sudo apt-get -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  libutouch-grail1 libutouch-evemu1 libutouch-frame1 libutouch-geis1
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  libc6-dev
Suggested packages:
  glibc-doc
The following packages will be upgraded:
  libc6-dev
1 upgraded, 0 newly installed, 0 to remove and 156 not upgraded.
1 not fully installed or removed.
Need to get 0 B/5,100 kB of archives.
After this operation, 2,048 B disk space will be freed.
Do you want to continue [Y/n]? 
dpkg: dependency problems prevent configuration of libc6-dev:
 libc6-dev depends on libc6 (= 2.15-0ubuntu10.2); however:
  Version of libc6 on system is 2.15-0ubuntu10.3.
 libc6-dev depends on libc-dev-bin (= 2.15-0ubuntu10.2); however:
  Version of libc-dev-bin on system is 2.15-0ubuntu10.3.
dpkg: error processing libc6-dev (--configure):
 dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
                                                                                                          Errors were encountered while processing:
 libc6-dev
E: Sub-process /usr/bin/dpkg returned an error code (1)

The solution was to put the offending package on hold (run as root):

# echo "libc6-dev hold" | dpkg --set-selections

then run:

# apt-get -f install

which removed a bunch of packages, including the offending broken one. I could then apt-get update and upgrade as normal. Hooray!

List of browser cache directories on OS X

Quick and dirty list of browser cache directories on OS X, I didn't spend much time checking this was complete, but posting here for future reference:

  ~/Library/Application Support/Google/Chrome,
  ~/Library/Application Support/Google/Chrome\ Canary,
  ~/Library/Caches/Google/Chrome,
  ~/Library/Caches/Google/Chrome Canary/,
  ~/Library/Caches/Metadata/Safari/History/,
  ~/Library/Caches/Firefox,
  ~/Library/Application Support/Firefox,

Appending to an array in a plist

Handy commandline to append an item to an array in a plist:

/usr/libexec/PlistBuddy -c "Add :SomeArray: string 'newstring'" /Library/Preferences/com.something.plist

If you want to operate on a specific item, you can specify a 0-based array offset, this deletes the first element in the array:

/usr/libexec/PlistBuddy -c "Delete :SomeArray:0 string" /Library/Preferences/com.something.plist

Chrome enterprise policy controls

Chrome comes with a bunch of enterprise controls. These enable you to, amongst other things, whitelist/blacklist/force-install extensions. Also, if you want to install extensions automatically, but allow users to disable or remove them, there is an alternate way to get them installed.

Minimal OpenVPN setup: ubuntu server, OS X client

There are some good instructions for setting OpenVPN up quickly on ubuntu here, and I've also covered it previously on this blog. Because of a bug I was getting this error from pkitool when building the ca:

The correct version should have a comment that says: easy-rsa version 2.x

The solution was just a symlink, no need to mess with the vars file:

ln -s openssl-1.0.0.cnf openssl.cnf

To route all traffic through the VPN uncomment this line in the server.conf:

push "redirect-gateway def1 bypass-dhcp"

And then you'll want to add a NAT so that traffic comes back to the right clients, this iptables config also allows some services through:

# Generated by iptables-save v1.4.12 on Wed Oct 17 21:32:01 2012
*nat
:PREROUTING ACCEPT [293:18619]
:INPUT ACCEPT [3:148]
:OUTPUT ACCEPT [6:508]
:POSTROUTING ACCEPT [6:508]
-A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE
COMMIT
# Completed on Wed Oct 17 21:32:01 2012
# Generated by iptables-save v1.4.12 on Wed Oct 17 21:32:01 2012
*filter
:INPUT DROP [1:42]
:FORWARD ACCEPT [571:142706]
:OUTPUT ACCEPT [589:168958]
-A INPUT -i lo -j ACCEPT
-A INPUT -i eth+ -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i eth+ -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -i eth+ -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -i eth+ -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -i eth+ -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i eth+ -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i eth+ -p udp -m udp --dport 1194 -j ACCEPT
-A INPUT -s 10.8.0.0/24 -i eth+ -j ACCEPT
-A INPUT -s 10.8.0.0/24 -i tun0 -j ACCEPT
-A INPUT -j LOG
COMMIT
# Completed on Wed Oct 17 21:32:01 2012

To ssh to the server while the VPN is active, use:

ssh 10.8.0.1

On the client side (a mac), first install tunnelblick. You can use it to create an example config, which is a directory where you dump the ca and client cert. The most important bit in the config you need to set is:

remote myserver.com 1194

OS X Packaging (Luggage) Tutorial: hello world

Apple ships a GUI utility called PackageMaker to help you create .pkg files for deployment. It isn't included in newer versions of XCode, you'll need to download it, it's in the 'Auxiliary tools for XCode' package.

The problem is that reproducing builds with many different project collaborators is a PITA, enter The Luggage.

Here's a hello world.

Download git and clone and install the luggage.

git clone https://github.com/unixorn/luggage
cd luggage
make bootstrap_files

Write a simple Makefile (this assumes you have copied PackageMaker.app Auxiliary tools into /Applications):

include /usr/local/share/luggage/luggage.make
TITLE=Hello_World
REVERSE_DOMAIN=com.testing.something
PAYLOAD=pack-usr-local-bin-hello_world
PACKAGEMAKER=/Applications/PackageMaker.app/Contents/MacOS/PackageMaker

Create a dummy file to install, and build the dmg:

touch hello_world
make dmg

You'll get a Hello_World-20121016.dmg containing Hello_World-20121016.pkg, that when installed will create /usr/local/bin/hello_world

Registered mime-types and default handlers on OS X

Which application is the default handler for file type .blah on OS X? Turns out this is actually a pretty complicated question.

Launch Services keeps a database, into which applications can register themselves as mime handlers, using CFBundleDocumentTypes and CFBundleURLTypes in Info.plist in their application Contents, which are parsed periodically (not sure when). Here is part of Safari's:

$defaults read /Applications/Safari.app/Contents/Info.plist 
{
    "Application-Group" = "dot-mac";
    BuildMachineOSBuild = 12A251;
    CFBundleDevelopmentRegion = English;
    CFBundleDocumentTypes =     (
                {
            CFBundleTypeExtensions =             (
                css
            );
            CFBundleTypeIconFile = "document.icns";
            CFBundleTypeMIMETypes =             (
                "text/css"
            );
            CFBundleTypeName = "CSS style sheet";
            CFBundleTypeRole = Viewer;
            NSDocumentClass = BrowserDocument;
        },
                {
            CFBundleTypeExtensions =             (
                pdf
            );
            CFBundleTypeIconFile = "document.icns";
            CFBundleTypeMIMETypes =             (
                "application/pdf"
            );
            CFBundleTypeName = "PDF document";
            CFBundleTypeRole = Viewer;
            NSDocumentClass = BrowserDocument;
        },
[snip]

Also as an aside, safari has a list of file types that it will automatically open with the registered mime handler because they are considered 'safe'. The list is stored in:

/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/System

Where:

• LSRiskCategorySafe: Safari will automatically open these files after download
• LSRiskCategoryNeutral: Not auto-opened, no warnings
• LSRiskCategoryUnsafeExecutable: Warning displayed when opened by the user.
• LSRiskCategoryMayContainUnsafeExecutable: e.g. zip files. This will trigger a warning if Safari can't determine that the contents are safe or neutral

Back to the LaunchServices database. You can use API calls to register a mime-type, and there is also a helper utility called lsregister that provides a simple interface to the database. This command will dump the contents of the database, there are also options to force a registration:
There is also an app (RCDefaultApp) that presents a more usable grapical frontend to the data.

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -dump | less

The user also has some control over mime-type registration, and that information is stored in:

$ defaults read ~/Library/Preferences/com.apple.LaunchServices
{
    LSHandlers =     (
                {
            LSHandlerRoleAll = "com.google.chrome";
            LSHandlerURLScheme = http;
        },
                {
            LSHandlerRoleAll = "com.google.chrome";
            LSHandlerURLScheme = https;
        },
                {
            LSHandlerContentType = "public.html";
            LSHandlerRoleViewer = "com.google.chrome";
        },
                {
            LSHandlerContentType = "public.url";
            LSHandlerRoleViewer = "com.google.chrome";
        }
    );
}

Reading the output of the database query and the plist above, it is difficult to determine which handler will fire for certain files where multiple handlers are registered. Apple provides some documentation, about how the mime handler is chosen for files and URLs in the case of multiple handers, but the flow chart ends with:

If two or more candidate applications remain after all of the foregoing criteria have been applied, Launch Services chooses one of the remaining applications in an unspecified manner.

The quickest way to check is to use 'open', which according to the man page:

opens a file (or a directory or URL), just as if you had double-clicked the file's icon.

So, creating a dummy css file and running open like this should pop up safari:

open blah.css

Find out a file's mime type with:

$ mdls -name kMDItemContentType -name kMDItemContentTypeTree test.dmg 
kMDItemContentType     = "com.apple.disk-image-udif"
kMDItemContentTypeTree = (
    "com.apple.disk-image-udif",
    "com.apple.disk-image",
    "public.archive",
    "public.data",
    "public.item",
    "public.disk-image"
)

Disable OS X auto-login after Filevault 2 unlock

Interesting nugget from the munki-dev list, a description of how to stop the auto-login that occurs after FileVault 2 unlock (I haven't tested this yet). You essentially disable the credential forwarding from the disk unlock window to the login window.

Edit /etc/authorization
Find the "system.login.console" array.
Find the "mechanisms" array within this.
Remove the line "builtin:forward-login,privileged".
Save and reboot.

Extract and view application signing certs on OS X

To view the certs used to sign executables on OS X binaries, first dump out the cert signing chain:

$ codesign -d --extract-certificates /Applications/Utilities/Adobe\ Flash\ Player\ Install\ Manager.app/
Executable=/Applications/Utilities/Adobe Flash Player Install Manager.app/Contents/MacOS/Adobe Flash Player Install Manager

This will give you all the certs in the embedded cert chain in ASN.1 DER format, with codesign0 being the leaf:

$ ls codesign*
codesign0   codesign1   codesign2   codesign3

Then you can use openssl to look at the attributes in a super-ugly format:

$ openssl asn1parse -in codesign0 -inform DER    
    0:d=0  hl=4 l=1302 cons: SEQUENCE          
    4:d=1  hl=4 l=1022 cons: SEQUENCE          
    8:d=2  hl=2 l=   3 cons: cont [ 0 ]        
   10:d=3  hl=2 l=   1 prim: INTEGER           :02
   13:d=2  hl=2 l=  16 prim: INTEGER           :15E5AC0A487063718E39DA52301A0488
   31:d=2  hl=2 l=  13 cons: SEQUENCE          
   33:d=3  hl=2 l=   9 prim: OBJECT            :sha1WithRSAEncryption
   44:d=3  hl=2 l=   0 prim: NULL              
   46:d=2  hl=3 l= 180 cons: SEQUENCE          
   49:d=3  hl=2 l=  11 cons: SET               
   51:d=4  hl=2 l=   9 cons: SEQUENCE          
   53:d=5  hl=2 l=   3 prim: OBJECT            :countryName
   58:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :US
   62:d=3  hl=2 l=  23 cons: SET               
   64:d=4  hl=2 l=  21 cons: SEQUENCE          
   66:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
   71:d=5  hl=2 l=  14 prim: PRINTABLESTRING   :VeriSign, Inc.
   87:d=3  hl=2 l=  31 cons: SET               
   89:d=4  hl=2 l=  29 cons: SEQUENCE          
   91:d=5  hl=2 l=   3 prim: OBJECT            :organizationalUnitName
   96:d=5  hl=2 l=  22 prim: PRINTABLESTRING   :VeriSign Trust Network
  120:d=3  hl=2 l=  59 cons: SET               
  122:d=4  hl=2 l=  57 cons: SEQUENCE          
  124:d=5  hl=2 l=   3 prim: OBJECT            :organizationalUnitName
  129:d=5  hl=2 l=  50 prim: PRINTABLESTRING   :Terms of use at https://www.verisign.com/rpa (c)10
  181:d=3  hl=2 l=  46 cons: SET               
  183:d=4  hl=2 l=  44 cons: SEQUENCE          
  185:d=5  hl=2 l=   3 prim: OBJECT            :commonName
  190:d=5  hl=2 l=  37 prim: PRINTABLESTRING   :VeriSign Class 3 Code Signing 2010 CA
  229:d=2  hl=2 l=  30 cons: SEQUENCE          
  231:d=3  hl=2 l=  13 prim: UTCTIME           :101215000000Z
  246:d=3  hl=2 l=  13 prim: UTCTIME           :121214235959Z
  261:d=2  hl=3 l= 221 cons: SEQUENCE          
  264:d=3  hl=2 l=  11 cons: SET               
  266:d=4  hl=2 l=   9 cons: SEQUENCE          
  268:d=5  hl=2 l=   3 prim: OBJECT            :countryName
  273:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :US
  277:d=3  hl=2 l=  19 cons: SET               
  279:d=4  hl=2 l=  17 cons: SEQUENCE          
  281:d=5  hl=2 l=   3 prim: OBJECT            :stateOrProvinceName
  286:d=5  hl=2 l=  10 prim: PRINTABLESTRING   :California
  298:d=3  hl=2 l=  17 cons: SET               
  300:d=4  hl=2 l=  15 cons: SEQUENCE          
  302:d=5  hl=2 l=   3 prim: OBJECT            :localityName
  307:d=5  hl=2 l=   8 prim: PRINTABLESTRING   :San Jose
  317:d=3  hl=2 l=  35 cons: SET               
  319:d=4  hl=2 l=  33 cons: SEQUENCE          
  321:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
  326:d=5  hl=2 l=  26 prim: T61STRING         :Adobe Systems Incorporated
  354:d=3  hl=2 l=  28 cons: SET               
  356:d=4  hl=2 l=  26 cons: SEQUENCE          
  358:d=5  hl=2 l=   3 prim: OBJECT            :organizationalUnitName
  363:d=5  hl=2 l=  19 prim: T61STRING         :Information Systems
  384:d=3  hl=2 l=  62 cons: SET               
  386:d=4  hl=2 l=  60 cons: SEQUENCE          
  388:d=5  hl=2 l=   3 prim: OBJECT            :organizationalUnitName
  393:d=5  hl=2 l=  53 prim: PRINTABLESTRING   :Digital ID Class 3 - Microsoft Software Validation v2
  448:d=3  hl=2 l=  35 cons: SET               
  450:d=4  hl=2 l=  33 cons: SEQUENCE          
  452:d=5  hl=2 l=   3 prim: OBJECT            :commonName
  457:d=5  hl=2 l=  26 prim: T61STRING         :Adobe Systems Incorporated
  485:d=2  hl=3 l= 159 cons: SEQUENCE          
  488:d=3  hl=2 l=  13 cons: SEQUENCE          
  490:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
  501:d=4  hl=2 l=   0 prim: NULL              
  503:d=3  hl=3 l= 141 prim: BIT STRING        
  647:d=2  hl=4 l= 379 cons: cont [ 3 ]        
  651:d=3  hl=4 l= 375 cons: SEQUENCE          
  655:d=4  hl=2 l=   9 cons: SEQUENCE          
  657:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Basic Constraints
  662:d=5  hl=2 l=   2 prim: OCTET STRING      [HEX DUMP]:3000
  666:d=4  hl=2 l=  14 cons: SEQUENCE          
  668:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Key Usage
  673:d=5  hl=2 l=   1 prim: BOOLEAN           :255
  676:d=5  hl=2 l=   4 prim: OCTET STRING      [HEX DUMP]:03020780
  682:d=4  hl=2 l=  64 cons: SEQUENCE          
  684:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 CRL Distribution Points
  689:d=5  hl=2 l=  57 prim: OCTET STRING      [HEX DUMP]:30373035A033A031862F687474703A2F2F637363332D323031302D63726C2E766572697369676E2E636F6D2F435343332D323031302E63726C
  748:d=4  hl=2 l=  68 cons: SEQUENCE          
  750:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Certificate Policies
  755:d=5  hl=2 l=  61 prim: OCTET STRING      [HEX DUMP]:303B3039060B6086480186F84501071703302A302806082B06010505070201161C68747470733A2F2F7777772E766572697369676E2E636F6D2F727061
  818:d=4  hl=2 l=  19 cons: SEQUENCE          
  820:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Extended Key Usage
  825:d=5  hl=2 l=  12 prim: OCTET STRING      [HEX DUMP]:300A06082B06010505070303
  839:d=4  hl=2 l= 113 cons: SEQUENCE          
  841:d=5  hl=2 l=   8 prim: OBJECT            :Authority Information Access
  851:d=5  hl=2 l= 101 prim: OCTET STRING      [HEX DUMP]:3063302406082B060105050730018618687474703A2F2F6F6373702E766572697369676E2E636F6D303B06082B06010505073002862F687474703A2F2F637363332D323031302D6169612E766572697369676E2E636F6D2F435343332D323031302E636572
  954:d=4  hl=2 l=  31 cons: SEQUENCE          
  956:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Authority Key Identifier
  961:d=5  hl=2 l=  24 prim: OCTET STRING      [HEX DUMP]:30168014CF99A9EA7B26F44BC98E8FD7F00526EFE3D2A79D
  987:d=4  hl=2 l=  17 cons: SEQUENCE          
  989:d=5  hl=2 l=   9 prim: OBJECT            :Netscape Cert Type
 1000:d=5  hl=2 l=   4 prim: OCTET STRING      [HEX DUMP]:03020410
 1006:d=4  hl=2 l=  22 cons: SEQUENCE          
 1008:d=5  hl=2 l=  10 prim: OBJECT            :1.3.6.1.4.1.311.2.1.27
 1020:d=5  hl=2 l=   8 prim: OCTET STRING      [HEX DUMP]:30060101000101FF
 1030:d=1  hl=2 l=  13 cons: SEQUENCE          
 1032:d=2  hl=2 l=   9 prim: OBJECT            :sha1WithRSAEncryption
 1043:d=2  hl=2 l=   0 prim: NULL              
 1045:d=1  hl=4 l= 257 prim: BIT STRING      

Disable Captive Network Support in OS X

iOS4+ and OS X (10.7+) Devices have a feature called Captive Network Support, which when you connect to an access point tries to download:

http://www.apple.com/library/test/success.html

to see if the device is connected to the internet. If it doesn't get the success response it assumes you are behind a captive portal and pops a webkit window so you can do the portal dance.  This is mostly useful if you are using thick-client apps, since if you're using a browser you're going to see the portal page as soon as you go anywhere.

To disable it, set this preference:

sudo defaults write /Library/Preferences/SystemConfiguration/com.apple.captive.control Active -boolean false

Prevent OS X from sleeping while ssh connection is active

As of mountain lion, you can stop a mac from sleeping while there is an active ssh session:

caffeinate -i ssh yourhost.something.com

This will prevent idle sleeps, other options are available.

FileVault2 destroyfvkeyonstandby

In their filevault2 doco apple describes the destroyfvkeyonstandby setting:

...the FileVault key is stored in EFI to transparently come out of standby mode. Organizations especially sensitive to a high-attack environment, or potentially exposed to full device access when the device is in standby mode, should mitigate this risk by destroying the FileVault key in firmware. Doing so doesn’t destroy the use of FileVault, but simply requires the user to enter the password in order for the system to come out of standby mode.

To destroy the filevault key on standby:

# pmset destroyfvkeyonstandby 1

And check the setting with:

# pmset -g

Adding items to your start menu on Vista when access has been 'disabled'

As a windows user it is handy to have some programs start on login, like your mail client, web browser, chat client etc. so it is nice to have write access to the 'Startup' folder in your start menu. Write access is often disabled, so here is a possible work around.

First you need to be able to see your start menu. Try mapping your C$ share:

Open explorer and tap the 'Alt' key to bring up the old menu system.
Tools -> Map Network Drive...
\\127.0.0.1\C$

Browse to the share you mounted. Hopefully your start menu is under something like:

Z:\Users\\Start Menu

OR

Z:\Users\\AppData\Roaming\Microsoft\Windows\Start Menu

Create a shortcut to the program you want to start in your 'Startup' folder and you should be good to go.

Running an unsigned app with Gatekeeper set to 'only apps from the Mac App Store and identified developers'

So with Gatekeeper set to 'Mac App store and identified developers', how do you actually run an app that isn't signed? If you double-click you'll see something like this:

As explained previously you can delete the HFS+ attribute or run it from the commandline, but for regular users that doesn't work too well. The solution is Ctrl-Click and 'Open'.

OS X: get a list of running launchd jobs

Here's a nice easy way to get a list of running launchd jobs using PyObjC to talk to the ServiceManagement API:

from ServiceManagement import SMCopyAllJobDictionaries

print SMCopyAllJobDictionaries('kSMDomainSystemLaunchd')

So how does this compare to launchctl? On a first look its great, it provides way more information about each launchd job.

        {
        Label = "com.apple.systemprofiler";
        LastExitStatus = 0;
        LimitLoadToSessionType = Aqua;
        MachServices =         {
            "com.apple.systemprofiler" = 0;
        };
        OnDemand = 1;
        Program = "/Applications/Utilities/System Information.app/Contents/MacOS/System Information";
        TimeOut = 30;
    },

But wait, I'm getting different results:

$ sudo launchctl list | grep airport
- 0 com.apple.airport.updateprefs
- 0 com.apple.airportd
$ sudo python print_daemons.py | grep airport
$

So what is launchctl doing different? Why am I missing airportd and other services? I spent some time reading the source, but that didn't help. I finally realised this was a execution context issue.

$ sudo /usr/libexec/StartupItemContext python print_daemons.py | grep airport
        Label = "com.apple.airportd";
            "com.apple.airportd" = 0;
        Program = "/usr/libexec/airportd";
        Label = "com.apple.airport.updateprefs";
            "com.apple.airport.updateprefs" = 0;
            "com.apple.airport.wps" = 0;

The launchd jobs are running in the System context (there are a bunch of others, see table 1 here), and can't see jobs running in other contexts. This also means that if we use StartupItemContext as above to run in the System context, we're going to miss jobs that have set LimitLoadToSessionType to something else, as with the systemprofiler job, which only runs in the Aqua (GUI) context.

$ sudo /usr/libexec/StartupItemContext python print_daemons.py | grep systemprofiler
$ sudo python temp.py | grep systemprofiler
        Label = "com.apple.systemprofiler";
            "com.apple.systemprofiler" = 0;

git: undo changes, revert uncommitted changes

I mostly love git, but I can never remember the syntax for undoing and reverting things, which is partly because I don't use it that often, and partly because the syntax is un-intuitive. I'll update this post as I hit the various scenarios.

Merge broken by conflicts

Often I'll pull in changes from the stable repo, forgetting I've made some uncommitted local changes, which results in a conflict. To drop all of your local uncommitted changes:

git reset --hard HEAD

plistbuddy and defaults on OS X

I use defaults to read most plists on OS X, but it has an annoying habit of being very chatty in the logs. If you want to write code to query for a value that may or may not be present, it ends up being very noisy. e.g. even if you suppress stderr like this:

defaults read /Library/Preferences/com.apple.CrashReporter.plist DOESNT_EXIST 2> /dev/null

For some reason the defaults tool thinks the failure to read the value is really important and writes a log line like:

defaults[34157]: The domain/default pair of (com.apple.CrashReporter.plist, DOESNT_EXIST) does not exist

So if you don't want the spurious logs, you can use plist buddy to do the same thing:

/usr/local/bin/PlistBuddy -c Print:DOESNT_EXIST /Library/Preferences/com.apple.CrashReporter.plist

Mox: stub out a python builtin method like 'exit' or 'open'

Occasionally you need to stub out a part of core python for a test to work. In my case it was 'exit', another example is 'open'. This is what your test code will look like:

import __builtin__

m = mox.Mox()
m.StubOutWithMock(__builtin__, 'exit')
exit(my_module.EXIT_ERROR)

m.ReplayAll()
my_module.myfunction()
m.VerifyAll()

JAVA CA store on OS X

To get the list of CAs trusted by Java on OS X:

keytool -v -list -keystore /System/Library/Java/Support/CoreDeploy.bundle/Contents/Home/lib/security/cacerts

The default password is 'changeit'.

Ruby foo: flatten a hash into a string

Flatten a ruby hash into a string. In this case the comma is the key value delimiter and the semicolon is the entry delimiter.

irb(main):016:0> a={'sdfs'=>['aaa','bbb'], 'ppp'=>['aa','a']}
=> {"ppp"=>["aa", "a"], "sdfs"=>["aaa", "bbb"]}
=> ["ppp:aa,a", "sdfs:aaa,bbb"]
irb(main):018:0> b=a.map { |k, v| "#{k}:#{v.join(',')}" }.join(";")
=> "ppp:aa,a;sdfs:aaa,bbb"

Ruby Dir.glob to provides some of the python os.walk functionality

The ruby Dir.glob command is pretty nice. It provides some of what os.walk provides for python and is a reasonable choice for working with files and directories. Double * gets you recursion. Quick and dirty way to list user home directories and user-level global environment file:

Dir.glob("/Users/**/.MacOSX/environment")

More comprehensive is to use dscl, but it is more work, first get a list of users:

dscl . -list /Users

Then iterate over each of those:

dscl . -read /Users/myuser NFSHomeDirectory

To get a rough list of Applications and their plists (note getting a list of everything installed is more complicated):

Dir.glob("/Applications/**/**app/Contents/Info.plist")

Ruby string manipulation: struct, split, join

Every time I write ruby it seems like I have to re-learn everything. To shortcut the process for next time here is a snippet that demonstrates some string handling by parsing the output of lsof (OS X format). Ruby structs are the equivalent of python namedtuple, very handy for splitting up strings in a sane way.

def parse_lsof(lsof_out)
  net_listener = Struct.new(:command, :pid, :user, :fd, :type, :device, :size, :node, :name, :status)
  output = []

  # strip header line
  lsof_lines = lsof_out.split("\n")[1..-1]

  lsof_lines.each {
    |line|

    line_array = line.split("\s")
    listener_o = net_listener.new(*line_array)
    output.push("#{listener_o.command},#{listener_o.user},#{listener_o.type},#{listener_o.node},#{listener_o.name}")
  }
  return output.join("    ")
end

Adjust log level of python logger in a library (pyactiveresource)

My logging was being polluted by pyactiveresource that was writing a lot of useless logs at the 'INFO' level. I wanted my own logging set at INFO, so how can you change pyactiveresource? As described in the python doco, you can access the logger with getLogger and adjust the level like this:

  noisy_logger = logging.getLogger('pyactiveresource')
  noisy_logger.setLevel(logging.WARN)

Then just set your own logger as normal, something like:

  syslog = logging.handlers.SysLogHandler('/var/run/syslog')
  syslog.setFormatter(logging.Formatter('%(name)s: %(message)s'))
  syslog.setLevel(logging.INFO)
  logging.getLogger().addHandler(syslog)

Mac OS X pkg, bom files, package installation and utilities

List all installed packages:

pkgutil --pkgs --volume /

What files does this package install?

pkgutil --files com.vmware.fusion

Verify file permissions and owner/group is the same as listed in the package BOM in /private/var/db/receipts/ (actually calls repair_packages). See the end of this post for how to verify the file content.

pkgutil --verify com.vmware.fusion

There isn't any uninstall functionality on OS X, this command just drops the metadata out of the receipts directory, which is handy if you want to re-install:

pkgutil --forget com.vmware.fusion

pkgutil has a code signing check feature, but it looks like it is broken or doesn't work how I expected it to:

$ pkgutil --check-signature com.apple.pkg.Safari
Package does not exist: com.apple.pkg.Safari
$ pkgutil --check-signature /Applications/Safari.app/
Package "Safari":
   Status: no signature
$ codesign -d -vvv /Applications/Safari.app/
Executable=/Applications/Safari.app/Contents/MacOS/Safari
Identifier=com.apple.Safari
Format=bundle with Mach-O universal (i386 x86_64)
CodeDirectory v=20100 size=185 flags=0x0(none) hashes=3+3 location=embedded
Hash type=sha1 size=20
CDHash=66615ae53cb89ac254e4efc6d3eb2f93fa6a4a85
Signature size=4064
Authority=Software Signing
Authority=Apple Code Signing Certification Authority
Authority=Apple Root CA
Info.plist entries=37
Sealed Resources rules=11 files=393
Internal requirements count=1 size=108

PKG files

PKG files are just XAR files that contain a bunch of metadata files and the app itself inside a zipped cpio.

To extract the files on OS X, use (note tempdir will be created and can't exist prior to running the command):

pkgutil --expand my.pkg tempdir

Here is a manual unpacking sequence (if you just want metadata use the pkgutil commands to avoid all this unpacking). tar understands xar on OS X but on linux you'll need to install the xar package and use 'xar -xf helloworld.pkg':

$ tar zxvf helloworld.pkg 
x com.totallylegit.helloworld.pkg
x com.totallylegit.helloworld.pkg/Bom
x com.totallylegit.helloworld.pkg/Payload
x com.totallylegit.helloworld.pkg/PackageInfo
x Distribution

$ cd com.totallylegit.helloworld.pkg/
$ ls
Bom  PackageInfo Payload
$ mv Payload Payload.gz
$ gunzip Payload
$ cpio -iv < Payload
.
./helloworld.app
./helloworld.app/Contents
./helloworld.app/Contents/Info.plist
./helloworld.app/Contents/MacOS
./helloworld.app/Contents/MacOS/helloworld
./helloworld.app/Contents/PkgInfo
./helloworld.app/Contents/Resources
./helloworld.app/Contents/Resources/en.lproj
./helloworld.app/Contents/Resources/en.lproj/Credits.rtf
./helloworld.app/Contents/Resources/en.lproj/InfoPlist.strings
./helloworld.app/Contents/Resources/en.lproj/MainMenu.nib
87 blocks

BOM files

Use lsbom to inspect bill of materials files (note apple says "The files and directories where receipts are stored are subject to change. Always use pkgutil to query or modify them"). These options are more readable than the defaults:

$ lsbom -p MUGsf /private/var/db/receipts/com.vmware.fusion.bom
drwxr-xr-x  root admin  .
drwxrwxr-x  root admin  ./Applications
-rwxr-xr-x  root admin 82 ./Applications/._VMware Fusion.app
[snip]

While pkgutil/repair_packages could check file content hasn't changed by calculating the CRC32 and comparing that with the one in the BOM, it doesn't. It would be fairly simple to do. The BOM content for a simple package looks like this (path, octal mode, UID/GID, size, CRC32):

$ lsbom Bom 
. 0 0/0
./helloworld.app 40755 0/0
./helloworld.app/Contents 40755 0/0
./helloworld.app/Contents/Info.plist 100644 0/0 1429 3178638052
./helloworld.app/Contents/MacOS 40755 0/0
./helloworld.app/Contents/MacOS/helloworld 100755 0/0 13488 1649029420
./helloworld.app/Contents/PkgInfo 100644 0/0 8 742937289
./helloworld.app/Contents/Resources 40755 0/0
./helloworld.app/Contents/Resources/en.lproj 40755 0/0
./helloworld.app/Contents/Resources/en.lproj/Credits.rtf 100644 0/0 436 2072274354
./helloworld.app/Contents/Resources/en.lproj/InfoPlist.strings 100644 0/0 92 2769355950
./helloworld.app/Contents/Resources/en.lproj/MainMenu.nib 100644 0/0 27463 4224082203

and the checksum can be calculated with cksum which prints the size in bytes and the CRC32:

$ cksum helloworld 
1649029420 13488 helloworld

Installation

On installation the package app directory is copied to the installation dir (usually /Applications/) and the BOM is written with a plist of installation metadata to the Receipts directory:

$ ls -1 /private/var/db/receipts/com.vmware*
/private/var/db/receipts/com.vmware.fusion.bom
/private/var/db/receipts/com.vmware.fusion.plist

Gatekeeper, XProtect and the Quarantine attribute

Apps can opt-in to Gatekeeper and Xprotect protection by adding LSFileQuarantineEnabled to their Contents/Info.plist. This means that any files created by that app will get tagged with the apple quarantine HFS+ extended attribute. For example, here's what chrome looks like when downloaded with Safari:

$ ls -l@eOd Downloads/googlechrome.dmg 
-rw-r--r--@ 1 user  group  - 39937778 Apr 16 11:31 Downloads/googlechrome.dmg
 com.apple.diskimages.fsck       20 
 com.apple.diskimages.recentcksum       79 
 com.apple.metadata:kMDItemDownloadedDate       53 
 com.apple.metadata:kMDItemWhereFroms      210 
 com.apple.quarantine       57 

and the same file downloaded with curl (which isn't a cocoa app and hence can't opt-in):

$ curl -o chrome_curl.dmg -L https://dl.google.com/chrome/mac/stable/GGRO/googlechrome.dmg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 38.0M  100 38.0M    0     0  15.7M      0  0:00:02  0:00:02 --:--:-- 18.2M
$ ls -l@eOd chrome_curl.dmg 
-rw-r--r--  1 user  group  - 39937627 Apr 20 14:47 chrome_curl.dmg

Looking at the contents of the extended attributes we see a bunch of interesting metadata, including the HTTP referrer in a binary plist inside 'com.apple.metadata:kMDItemWhereFroms' and the quaratine attribute in 'com.apple.quarantine'.

$ xattr -l Downloads/googlechrome.dmg 
com.apple.diskimages.fsck: |?W??\<!*
                                    ?#Z???r??u
com.apple.diskimages.recentcksum: i:325165 on 3A5356B1-31CB-32FB-AFC7-DBE23A5C1547 @ 1334601060 - CRC32:$AE7BD4AF
com.apple.metadata:kMDItemDownloadedDate:
00000000  62 70 6C 69 73 74 30 30 A1 01 33 41 B5 42 0E 78  |bplist00..3A.B.x|
00000010  17 F3 17 08 0A 00 00 00 00 00 00 01 01 00 00 00  |................|
00000020  00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 13                                   |.....|
00000035
com.apple.metadata:kMDItemWhereFroms:
00000000  62 70 6C 69 73 74 30 30 A2 01 02 5F 10 3D 68 74  |bplist00..._.=ht|
00000010  74 70 73 3A 2F 2F 64 6C 2E 67 6F 6F 67 6C 65 2E  |tps://dl.google.|
00000020  63 6F 6D 2F 63 68 72 6F 6D 65 2F 6D 61 63 2F 73  |com/chrome/mac/s|
00000030  74 61 62 6C 65 2F 47 47 52 4D 2F 67 6F 6F 67 6C  |table/GGRM/googl|
00000040  65 63 68 72 6F 6D 65 2E 64 6D 67 5F 10 61 68 74  |echrome.dmg_.aht|
00000050  74 70 73 3A 2F 2F 77 77 77 2E 67 6F 6F 67 6C 65  |tps://www.google|
00000060  2E 63 6F 6D 2F 63 68 72 6F 6D 65 3F 26 62 72 61  |.com/chrome?&bra|
00000070  6E 64 3D 43 48 4D 41 26 75 74 6D 5F 63 61 6D 70  |nd=CHMA&utm_camp|
00000080  61 69 67 6E 3D 65 6E 26 75 74 6D 5F 73 6F 75 72  |aign=en&utm_sour|
00000090  63 65 3D 65 6E 2D 68 61 2D 6E 61 2D 75 73 2D 62  |ce=en-ha-na-us-b|
000000A0  6B 26 75 74 6D 5F 6D 65 64 69 75 6D 3D 68 61 08  |k&utm_medium=ha.|
000000B0  0B 4B 00 00 00 00 00 00 01 01 00 00 00 00 00 00  |.K..............|
000000C0  00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00  |................|
000000D0  00 AF                                            |..|
000000d2
com.apple.quarantine: 0002;4f91d6f8;Safari;A89FCF40-0748-46BE-9C5E-1599A280E9D6

Breaking down this string based on the description in:

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Headers/LSQuarantine.h

• 0002: kLSQuarantineType (I think), one of:
  • kLSQuarantineTypeWebDownload
  • kLSQuarantineTypeOtherDownload
  • kLSQuarantineTypeEmailAttachment
  • kLSQuarantineTypeInstantMessageAttachment
  • kLSQuarantineTypeCalendarEventAttachment
  • kLSQuarantineTypeOtherAttachment
• 4f91d6f8: LSQuarantineTimeStamp, seconds since epoch timestamp in hex, print with 'date -r 0x4f91d6f8'
• Safari: kLSQuarantineAgentName, the name of the process that wrote the file, or whatever the process sets via the API.
• A89FCF40-0748-46BE-9C5E-1599A280E9D6: a UUID. The key (LSQuarantineEventIdentifier) used for entries in the SQLite DB.

Wait, what SQLite DB? The one here:

$ sqlite3 /Users/auser/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2
sqlite> .dump
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE LSQuarantineEvent (  LSQuarantineEventIdentifier TEXT PRIMARY KEY NOT NULL,  LSQuarantineTimeStamp REAL,  LSQuarantineAgentBundleIdentifier TEXT,  LSQuarantineAgentName TEXT,  LSQuarantineDataURLString TEXT,  LSQuarantineSenderName TEXT,  LSQuarantineSenderAddress TEXT,  LSQuarantineTypeNumber INTEGER,  LSQuarantineOriginTitle TEXT,  LSQuarantineOriginURLString TEXT,  LSQuarantineOriginAlias BLOB );
INSERT INTO "LSQuarantineEvent" VALUES('A89FCF40-0748-46BE-9C5E-1599A280E9D6',356650616.054473,'com.apple.Safari','Safari','https://dl.google.com/chrome/mac/stable/GGRO/googlechrome.dmg',NULL,NULL,0,NULL,'https://www.google.com/',NULL);
CREATE INDEX LSQuarantineEventIndex  ON LSQuarantineEvent (  LSQuarantineEventIdentifier );
COMMIT;

Note the NULL fields, which are optional fields that can be filled in by the developer who writes the file. The OS writes LSQuarantineAgentNameKey, LSQuarantineAgentBundleIdentifierKey, and LSQuarantineTimeStampKey by default. Chrome adds LSQuarantineDataURLString and LSQuarantineOriginURLString. Apple said in the Leopard Launch Services release notes:

When the Launch Services API is used to open a quarantined file and the file appears to be an application, script, or other executable file type, Launch Services will display an alert to confirm the user understands the file is some kind of application. If the file is opened, the quarantine properties are automatically cleared by Launch Services if the user has write access to the file.

So translating this to the Gatekeeper world, with the default settings Gatekeeper will block execution and fire an alert if:

1. The file is opened with Launch Services (e.g. from the finder via user action, from another cocoa app, or as a mime handler). Excludes executables/installers run from the commandline, or programmatically via POSIX APIs. 'UnsafeExecutable' may play a role here (needs checking).
2. The file has the quarantine xattr set.
3. The file is not signed by apple or an apple developer cert.

Xprotect follows similar logic, and checks files against the attributes in

/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/XProtect.plist

to determine if the file is known-bad.

Check java browser plugin enabled/disabled state and Flash version

With all the press about the Flashback trojan, OS X users would be well advised to check that their java browser plugin is disabled. You can do so by visiting this java test page. In a similar vein, check you are up-to-date with flash at the adobe site.

Determining which applications use iCloud and push notifications on OS X

Entitlements are set by the developer in Xcode at build time, and are used to control access to iCloud, Push Notifications, and the Application Sandbox. You can see the entitlements of an app using the codesign utility. Here's safari showing it is iCloud enabled for bookmark syncing via the iCloud key-value store:

$codesign -d --entitlements - /Applications/Safari.app
Executable=/Applications/Safari.app/Contents/MacOS/Safari
??qq?<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>com.apple.private.accounts.allaccounts</key>
        <true/>
        <key>com.apple.developer.ubiquity-kvstore-identifier</key>
        <string>com.apple.Safari.SyncedTabs</string>
        <key>com.apple.private.tcc.allow</key>
        <array>
                <string>kTCCServiceAddressBook</string>
        </array>
</dict>
</plist>

Here's a nasty bit of shell foo to get a list of apps:

find /Applications/ -name "*.app" -type d -exec codesign -d --entitlements - {} \; 2>&1 | grep com.apple.developer.ubiquity --before-context=3 --after-context=4

Similarly apps that use Apple push notifications will have a 'com.apple.private.aps-connection-initiate' entitlement:

$ codesign -d --entitlements - /Applications/iTunes.app/
Executable=/Applications/iTunes.app/Contents/MacOS/iTunes
??qq<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>com.apple.private.aps-connection-initiate</key>
 <true/>
</dict>
</plist>

And you should also be able to see the push notification TCP connection being held open. This is created once and used by all the apps consuming push notifications:

$ sudo lsof -iTCP 
COMMAND    PID    USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
[snip]
applepush 279 root    7u  IPv4 0xa27f1934fb5654ef      0t0  TCP hostname.myorg.com:52236->nk11p01st-courier023-bz.push.apple.com:5223 (ESTABLISHED)

Enumerating network listeners and IPC sockets on OS X

TCP listeners are easy, this gives you numeric port numbers for sockets in state LISTEN:

$ sudo lsof -P -iTCP -sTCP:listen
COMMAND PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
launchd   1 root   21u  IPv6 0xffffff8014bfb9c0      0t0  TCP localhost:631 (LISTEN)
launchd   1 root   22u  IPv4 0xffffff8014c00500      0t0  TCP localhost:631 (LISTEN)
launchd   1 root  179u  IPv6 0xffffff8014bfb600      0t0  TCP *:22 (LISTEN)
launchd   1 root  183u  IPv4 0xffffff8014bfda40      0t0  TCP *:22 (LISTEN)

And UDP is similar, but you don't get the state information (the FAQ says you should be able to specify "-sUDP:^idle" and similar, but lsof 4.84 on Lion wouldn't accept it):

$ sudo lsof -P -iUDP
COMMAND    PID           USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
launchd      1           root   13u  IPv4 0xffffff8014647dd8      0t0  UDP *:137
launchd      1           root   14u  IPv4 0xffffff8014647c60      0t0  UDP *:138
UserEvent   14           root    6u  IPv4 0xffffff801561b8d0      0t0  UDP *:*

If you are going to script these I find these commands give the fastest and most useful answers:

lsof +c15 -P -n -iUDP -u^_mdnsresponder
lsof +c15 -P -n -iTCP -sTCP:listen

You might also be interested in launchd daemons listening on UNIX domain sockets for IPC, and xinetd-style (look for the inetdCompatibility key) daemons. Here is a quick and dirty bit of python to look for enabled daemons using sockets (you will want to check /Library/LaunchDaemons too):

#! /usr/bin/python
import os
from Foundation import NSDictionary

def GetPlistKey(plist, key):
  mach_info = NSDictionary.dictionaryWithContentsOfFile_(plist)
  if mach_info:
    if key in mach_info:
      return mach_info[key]
  else:
    return None

for plist in os.listdir('/System/Library/LaunchDaemons'):
  plist_path = os.path.join('/System/Library/LaunchDaemons', plist)
  sockets = GetPlistKey(plist_path, 'Sockets')
  disabled = GetPlistKey(plist_path, 'Disabled')
  if not disabled and sockets:
    print plist_path

Secure delete 'shred' equivalent for OS X

OS X ships with srm instead of shred. Functionality is fairly similar, the default operation is a 35-pass Gutmann algorithm, which can be rather slow. Here are the timing results using the defaults and then the fast option (single random pass, then zeros) on 50M files:

$ time srm random_test

real 0m22.623s
user 0m2.847s
sys 0m0.417s

$ time srm -sz random_test2

real 0m1.527s
user 0m0.349s
sys 0m0.046s

Unless you are feeling super paranoid this is the best option:

srm -sz file_to_delete 

MCE Remote FINTEK eHome Infrared Transceiver 1934:5168 and lirc

I bought the "OEM HP MCE KIT REMOTE CONTROL/USB IR RECEIVER/EMITTER" from amazon after seeing comments it was plug and play with MythTV. Turns out not so much, at least for the version I'm running on Ubuntu lucid.

lsusb looked like this, I was expecting some sort of device manufacturer strings, so that seemed ominous:

$ lsusb
[snip]
Bus 004 Device 002: ID 1934:5168  

and in syslog:

kernel: [ 1005.516531] usb 4-2: new full speed USB device using uhci_hcd and address 2
kernel: [ 1005.691272] usb 4-2: configuration #1 chosen from 1 choice

I followed the ubuntu community doco to add my remote to the lirc src and install the new module. The patch was:

--- /usr/src/lirc-0.8.6/drivers/lirc_mceusb/lirc_mceusb.c.old 2012-04-21 14:48:41.000000000 -0700
+++ /usr/src/lirc-0.8.6/drivers/lirc_mceusb/lirc_mceusb.c 2012-04-21 15:07:57.000000000 -0700
@@ -210,6 +210,8 @@
  { USB_DEVICE(VENDOR_FINTEK, 0x0602) },
  /* Fintek eHome Infrared Transceiver (in the AOpen MP45) */
  { USB_DEVICE(VENDOR_FINTEK, 0x0702) },
+ /* HP MCE */
+ { USB_DEVICE(VENDOR_FINTEK, 0x5168) },
  /* Pinnacle Remote Kit */
  { USB_DEVICE(VENDOR_PINNACLE, 0x0225) },
  /* Elitegroup Computer Systems IR */

Remove the old stuff, build and install the new, restart lirc:

sudo dkms remove -m lirc -v 0.8.6 --all
sudo dkms add -m lirc -v 0.8.6
sudo dkms -m lirc -v 0.8.6 build
sudo dkms -m lirc -v 0.8.6 install
sudo /etc/init.d/lirc restart

And you should see a syslog line like this when you plug it in:

kernel: [ 3713.864034] lirc_mceusb[5]: FINTEK eHome Infrared Transceiver on usb4:5

The /etc/lirc/hardware.conf should look like:

#Chosen Remote Control
REMOTE="DViCO_Dual_Digital"
REMOTE_MODULES="lirc_dev lirc_mceusb"
REMOTE_DRIVER=""
REMOTE_DEVICE="/dev/lirc0"
REMOTE_LIRCD_CONF="dvico/lircd.conf.fusionHDTV"
REMOTE_LIRCD_ARGS=""

Test with 'irw' and pressing some buttons. For some reason I had to change the remote name everywhere in my ~/.lirc/mythtv and ~/.lirc/mplayer to what irw thought it was (DViCO_Utraview).

And, it works! I'm certainly not the only person struggling with this IR receiver, but apparently it is supported by the new kernel driver.

Python csv reader and namedtuple: together at last

Here is a really tasty code snippet that is a good example of the power of combining named tuples with the python csv reader.

with open(myfilename, 'rb') as f:
  LogLine = namedtuple('LogLine','fqdn,ip,user')
  for line in map(LogLine._make, csv.reader(f)):
    print 'user %s visited %s from %s' % (line.user,
                                          line.fqdn,
                                          line.ip)

What's going on here?

1. csv.reader(f) reads from the file descriptor f line-by-line and returns an array like ['user','blah.com','192.168.1.1']
2. We use map to call LogLine._make on each member of the csv iterator (i.e. each line), which creates a LogLine tuple from the array.
   

The advantages are:

• No need to split() and strip(), the csv reader figures it out for you.
• No referring to line[0], line[1] etc. forgetting which one is which, and having to change a bunch of code if you add a field or the order changes. The tuple gives you names that are human readable and the file structure is contained in one place, the tuple constructor, so changes to fields are simple.
  
• Also, if you aren't familiar with the 'with' statement, it is very nice for working with files since it replaces the 'try: read, finally: close' boilerplate code.
  

Checking client-side certificate verification with openssl

So you have some client software that claims it checks server SSL certs are issued by a valid CA, or better, by a specific CA? How do you check its claims? Set up a simple MITM attack (works for OS X and linux):

Create a self-signed cert:

openssl req -new -x509 -days 365 -nodes -out cert.pem -keyout cert.pem

Fire up the openssl server (man s_server for more options, including simple webserver functionality):

sudo openssl s_server -debug -cert cert.pem -accept 443

Edit your /etc/hosts file and add the domain you are attacking to resolve to localhost:

127.0.0.1 secure.company.com

Run your client.

ComputerName vs. LocalHostName vs. HostName on OS X

Hostnames on OS X are...complicated. Using scutil is probably the easiest way to get at them, although there are others. It looks like scutil just parses this plist:

/Library/Preferences/SystemConfiguration/preferences.plist

Note: If you want to get/set these values without shelling out to scutil, check out SCPreferencesPathGetValue and SCPreferencesPathSetValue part of the SystemConfiguration Framework called from python like this.

ComputerName

This is the computer name visible in the GUI in Preferences | Sharing | Computer Name and the scutil manpage describes it as 'The user-friendly name for the system.'

scutil --get ComputerName

Apple says:

The AppleTalk name and the default name used for SLP/DA. The Network browser in the Finder uses SMB/CIFS to find computers that provide Windows file sharing.

SCPreferencesPath: /System/System/ComputerName

LocalHostName

This appears to be only used for Bonjour-aware services on the local network.

scutil --get LocalHostName

Apple says:

The name that designates a computer on a local subnet.

and as of 10.6:

Host names that contain only one label in addition to local, for example "My-Computer.local", are resolved using Multicast DNS (Bonjour) by default. Host names that contain two or more labels in addition to local, for example "server.domain.local", are resolved using a DNS server by default.

SCPreferencesPath: /System/Network/HostNames/LocalHostName

HostName

The name associated with hostname(1) and gethostname(3). Displayed in the default terminal command prompt (user@hostname).

scutil --get HostName

Apple says:

You can’t specify this name during server setup. Server Assistant sets the host name to AUTOMATIC in /etc/hostconfig. This setting causes the server’s host name to be the first name that’s true in this list:
- The name provided by the DHCP or BootP server for the primary IP address
- The first name returned by a reverse DNS (address-to-name) query for the primary IP address
- The local hostname
- The name “localhost”

SCPreferencesPath: /System/System/HostName

Using a dictionary to populate a format string in python for human and computer-friendly error reporting

Python 2.6 has brought some interesting string formatting options to the table. I recently needed an object that could pass back error information that could be consumed by humans and other code. The new string formatting came in handy.

class ValidationError(Exception):
  """Class for reporting validation errors."""

  BAD_RECORD = 1

  def __init__(self, error, msg, **kwargs):
    """Record Validation Error.

    Requires python 2.6

    Args:
      error: ValidationError code e.g. ValidationError.BAD_RECORD
      msg: format string for human readable version of the error
      **kwargs: kwargs to populate format string
    """
    super(ValidationError, self).__init__(self)
    self.message = msg.format(**kwargs)
    self.error = error
    self.kwargs = dict(kwargs)

  def __str__(self):
    return self.message

You can use it like this:

e = ValidationError(ValidationError.BAD_RECORD,
                    'Bad record detected in {records}',
                    records=records)

So to present the info to a human:

In [18]: print e
Bad record detected in {'a': 12, 'b': 123}

and other code can use:

In [19]: e.error
Out[19]: 1

and access the relevant varaibles like:

In [20]: e.kwargs
Out[20]: {'records': {'a': 12, 'b': 123}}

Really basic wireless survey with built-in linux tools

What channel should I use for my access point? It seems like I'm always asking this question as people start new access points in my neighbourhood. Some wireless access points can pick a channel automatically, but my experience with this has been pretty poor. I suspect the algorithm is too simplistic to be of any use. A naive implementation might pick the first channel that has no other APs on it, which could be a terrible choice if the strongest interference signal is on the adjacent channel.

Ideally you want to pick the channel that gives you the best signal-to-noise ratio, but where there are heaps of APs at varying strengths in different parts of the house the choice is non-obvious. Sadly there doesn't seem to be a FOSS wireless survey tool like Fluke AirMagnet, and while you can use kismet to collect signal strength data you will have to do some analysis of the data yourself (my wireless card also didn't play well with kismet).

I decided to optimise for one location - where I use the laptop the most. So I set my AP to channel 1 and:

sudo iwlist wlan0 scanning > ch1.txt

Rinse and repeat for all the other channels (setting the channel on my AP is quick, but this might be impractical if yours is slow, requires re-auth etc.). Then compare the signal strengths:

grep --before-context=4 "myessid" ch*.txt

Pick the channel with the highest quality number (best signal strength).

Root a TMobile MyTouch 4G (HTC Glacier) to install Cyanogen

This was my first time rooting an android phone. There are most probably better ways to do this, but here are my notes for better or worse. I followed these instructions, which were pretty good and I won't reproduce them, so consider this a diff. First install the jdk from the 'default-jdk' apt package.

Install the Android SDK, including the platform-tools component to get adb. If you try to use it without root you'll get a permissions error like this:

$ android-sdk-linux/platform-tools/adb devices
List of devices attached 
???????????? no permissions

Add this UDEV rule to /etc/udev/rules.d/51-android.rules. Note this gives all users write permissions:

SUBSYSTEM=="usb", SYSFS{idVendor}=="0bb4", MODE="0666"

Plug it back in:

$ android-sdk-linux/platform-tools/adb devices
List of devices attached 
XXXXXXXXXXXX device

Change the mode on the phone to 'charge-only', and unmount the SD card if it is mounted. This allows adb and subsequent rooting tools to access /sdcard. If you have it in disk drive mode you'll get errors like these from adb:

failed to copy 'Superuser.apk' to '/sdcard//Superuser.apk': Read-only file system

To do the cyangogen install you can use the 'ROM Manager' app by ClockworkMod, available in the android market.

Ubuntu system monitor in the taskbar post-unity (Natty and Oneiric)

With the unity changes from Natty onward, the traditional Ubuntu gnome taskbar disappeared to be replaced by a set of application panel indicators. I was used to seeing my machine vitals (cpu, network, load etc.) displayed using system monitor in the taskbar. I was pleased to see that this can still be achieved by installing the 'indicator-multiload' package.

Depending on the version you have, the autostart might not work. If it doesn't, just:

mkdir ~/.config/autostart

and toggle the 'autostart' checkbox under preferences.

Broken BCM4313 "ping: sendmsg: No buffer space available"

My Broadcom wireless card was broken on Ubuntu 11.10 Oneiric Ocelot. The card connected to my AP fine, but after a short time would barf. A ping to the AP would return a few successful replies followed by:

ping: sendmsg: No buffer space available

This is a known issue, and seems to affect quite a few people.

First I tried installing the proprietary broadcom driver from the 'broadcom-sta-common' package, but that was worse - it wouldn't even recognise the interface. I tried a whole bunch of later kernels and finally found one that worked: 3.1.8.

The ubuntu mainline kernel repo makes trying new kernels easy. Just download and install these debs:

linux-headers-3.1.8-030108_3.1.8-030108.201201061759_all.deb
linux-headers-3.1.8-030108-generic-pae_3.1.8-030108.201201061759_i386.deb
linux-image-3.1.8-030108-generic_3.1.8-030108.201201061759_i386.deb

Update: spoke too soon. This is still broken on 3.1.8 it just takes longer to manifest. I'm now also seeing a bunch of these messages:

kernel: [25244.951243] ieee80211 phy0: START: tid 1 is not agg'able

Update 2012-01-30: Now I'm also seeing lots of this:

ieee80211 phy0: brcms_c_prec_enq_head: No where to go, prec == 4

I'm going to up net.core.rmem and net.core.wmem as suggested in a comment, but that's really only treating the symptoms.

Update 2012-02-10: I realised that at some point I'd put my AP into 'high bandwidth' mode, presumably 802.11n, which worked really badly. I'm guessing that was because of the crowded 2.4GHz spectrum in my area and/or poor linux driver support. When I put it back to regular 802.11g it has been rock solid ever since.

Firewire and DMA attacks on OS X

I recently spent some time testing Firewire memory access on OS X to determine the current state of vulnerability to DMA attacks. There is a lot of prior art (Uwe Hermann has a good summary), and I was particularly interested in the claims of deficiencies in the DMA defences provided by Lion (Update: it looks like this was CVE-2011-3215 fixed in 10.7.2, which agrees with my findings).

To check for firewire and thunderbolt ports programmatically you can parse the output of system_profiler:

$ system_profiler SPFireWireDataType SPThunderboltDataType
FireWire:

    FireWire Bus:

      Maximum Speed: Up to 800 Mb/sec
       ...

Thunderbolt:

    MacBook Pro:

      Vendor Name: Apple, Inc.
      Device Name: MacBook Pro
      UID: 0x0001000A19999900
       ...

I used two MacBookPros, one running 10.6.8 and one running 10.7.2 fully patched as at 2011-12-29. Hardware specs:

$ system_profiler SPHardwareDataType

Model Name: MacBook Pro
Model Identifier: MacBookPro8,2
Processor Name: Intel Core i7
Processor Speed: 2 GHz
  ...

I used the pyfw-20041111 module to do the DMA using dumpmem which reads 256MB from 0x10000000:

python demo_dumpmem.py
hexdump -C cfffffffffffffff-10000000-20000000.memdump

and I also tested using ramdump.py and the libforensic1394 libraries, with the same results.

python ./ramdump.py 1024 f
SBP-2 not used, forcing 2048 byte pages.
[<forensic1394.device.Device object at 0xfffffffff>]
Attempting to access device 0: Apple Computer, Inc. Macintosh
Wrote file ramdump.bin.

Target 10.7.2, attacked from 10.6.8.

When targeting the 10.7.2 machine I could only access memory when the screen was unlocked, in all other scenarios I just received zeros. Note I tested with and without FileVault2 enabled, and the results were the same.

Status	DMA Capture Result
Logged in, not locked	Captured
Logged in, locked	0s
Login, log out, at login screen	0s
System booted, not logged in	0s
Switch user from fast user switching menu	0s

The cleartext password for the logged in user can be trivially retrieved with:

strings memdump | grep --after-context=4 longname | grep --after-context=1 password

Target 10.6.8, attacked from 10.7.2.

10.6.8 is the complete opposite, capture is possible in all cases (note that there is a NVRAM setting that can be used to defend against this attack without requiring a firmware password).

Status	DMA Capture Result
Logged in, not locked	Captured
Logged in, locked	Captured
Login, log out, at login screen	Captured
System booted, not logged in	Captured
Switch user from fast user switching menu	Captured

Get cleartext password with:

strings memdump | grep --after-context=3 managedUser | grep --after-context=1 password

Inception and libforensic1394

Update 2013-01-29: I get regular questions about this tool, so I'm going to update this post to record my answers.

Carsten's tool inception, is talked about a lot. He has wrapped the libforensic1394 libraries with some convenience functions to make sure the firewire modules are loaded, and detect when a new firewire device is connected. He's made it a little easier to run this attack, although I'm surprised he went to the trouble of wrapping the library but didn't write the couple of extra lines to do the grep for the password string.

Despite the title of this video and its ominous soundtrack, inception does not attack via the thunderbolt interface directly (that would be interesting). You need a thunderbolt to firewire converter to use the tool since it's just using libforensic1394, which uses the IOKit firewire library, to read from a firewire device. And as stated above, assuming you have filevault2 enabled, it only works with the one fairly lame remaining attack scenario (user logged in, screen unlocked, attacker with physical access to the machine). To his credit Carsten has described a plausible attack, but if you're talking about physical access attacks I'd be more worried about Snare's bootkit.

OS X Open Firmware settings: use nvram security-mode to disable firewire DMA without a firmware password

First some background. OS X uses Open Firmware, which is similar to a PC's BIOS. The nvram utility can be used to manipulate Open Firmware settings stored in NVRAM. Print NVRAM settings with:

nvram -p

For firewire DMA the important setting is security-mode. Here is an explanation of the different nvram security modes:

The "none" mode effectively disables security. The "command" mode just restricts the commands that may be executed to "go" and "boot". Additionally, under the "command" mode, the "boot" command may not have any arguments--that is, it will only boot the device specified in the boot device variable; no other command may be entered or any settings changed unless the password is supplied. Moreover, this password protection feature also applies to booting up with the option key held down (which allows you to choose from available bootable volumes through a built-in graphical user interface). Finally, in "full" mode, the machine is completely prohibited from booting until the password is entered.

To set a firmware password (puppet recipe):

nvram security-password=mypass
nvram security-mode=command

Note that anyone with root can read the hex encoded password with nvram security-password. To remove the password ('none' is the default security mode):

nvram -d security-password
nvram security-mode=none

When a firmware password is set, Firewire DMA is disabled. This can be abused to disable firewire DMA without setting a password - just by setting security-mode to something other than 'none'. This works for example (and will take effect after a reboot):

nvram security-mode=NONE

This causes the firewire controller to put itself into secure mode:

OSString * securityModeProperty = OSDynamicCast( OSString, options->getProperty("security-mode") );
if( securityModeProperty != NULL && strncmp( "none", securityModeProperty->getCStringNoCopy(), 5 ) != 0 ) {
  // set security mode to secure/permanent
  mode = kIOFWSecurityModeSecurePermanent;
}

Gather OS X hardware and software information with system_profiler

system_profiler is a good way to gather lots of recon information about OS X hardware and software. Some examples

     system_profiler -detailLevel full
       Gives you absolutely everything

     system_profiler -listDataTypes
       Shows a list of the available data types.

     system_profiler SPSoftwareDataType SPNetworkDataType
       Generates a text report containing only software and network data.

     system_profiler SPThunderboltDataType SPFireWireDataType
       Just firewire and thunderbolt interfaces