BCM4313 wireless DHCP timing out 'ip-config-unavailable'

I recently had a fight with a Broadcom wireless driver (BCM4313 [14e4:4727]), which was eventually solved by forcing the OS to use the brcmsmac driver. The symptoms were that the WPA-PSK authentication handshake would succeed but DHCP would always timeout, I believe because the underlying 802.11 frames themselves were getting screwed up. Typical logs looked like this:

NetworkManager[948]:  (eth1): DHCPv4 request timed out.
NetworkManager[948]:  (eth1): canceled DHCP transaction, DHCP client pid 2455
NetworkManager[948]:  Activation (eth1) Stage 4 of 5 (IPv4 Configure Timeout) scheduled...
NetworkManager[948]:  Activation (eth1) Stage 4 of 5 (IPv4 Configure Timeout) started...
NetworkManager[948]:  (eth1): device state change: ip-config -> failed (reason 'ip-config-unavailable') [70 120 5]

The state of support for Broadcom wireless chips is quite complicated. Archlinux has a good summary of the mess.

HOWTO disable the GPG GUI dialog prompt

This just annoyed me enough to figure out how to stop it. Ubuntu turns on 'use-agent' by default now:

# For Ubuntu we now use-agent by default to support more automatic
# use of GPG and S/MIME encryption by GUI programs.  Depending on the
# program, users may still have to manually decide to install gnupg-agent.

Which means you see an annoying dialog anytime you need to put in a password to encrypt/decrypt. The solution is to comment out 'use-agent' in

~/.gnupg/gpg.conf

You can also hit escape to get to the regular curses prompt.

Python seconds since epoch timestamp

Getting a 'seconds since epoch' timestamp in python is easy, but there are also some wrong ways to do it, a number of which are being recommended on Stack Overflow. time.time() seems to be the best choice:

# System time since epoch for reference (--utc actually makes no difference)
In [30]: !date --utc +%s
1360802215

# Correct!
In [31]: time.time()
Out[31]: 1360802218.887309

# This one is wrong, it mixes local and UTC
In [32]: time.mktime(time.gmtime())
Out[32]: 1360831021.0

# This is also correct, but requires a whole other module
In [33]: calendar.timegm(time.gmtime())
Out[33]: 1360802225

Standard time format ISO 8601

I'm often forgetting what this standard is. Here are some code snippets to generate ISO 8601 compliant datetime strings.

Python

In [8]: import time

In [9]: time.strftime('%Y-%m-%dT%H:%M:%S%z')
Out[9]: '2013-01-09T17:58:37-0800'

Python datetime, which has dumb handling of timezones, easiest is to just get a UTC time and hard-code in the Z for Zulu:

In [39]: datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%SZ')
Out[39]: '2013-01-10T02:09:17Z'

Ruby

$ irb
irb(main):001:0> require 'time'
=> true
irb(main):002:0> Time.now.iso8601
=> "2013-01-09T17:55:01-08:00"

Bash

$ date +%Y-%m-%dT%H:%M:%S%z
2013-01-09T17:59:41-0800

As an aside, python uses the old and incredibly confusing UNIX convention for timezones (man tzset) that is seconds West of UTC (e.g. Pacific US is +28800 currently), whereas basically everything else these days uses hours and minutes East of UTC (Pacific US is -0800 currently).

Here's what it looks like:

In [46]: time.timezone
Out[46]: 28800

In [47]: time.timezone/3600
Out[47]: 8

Fintek IR receiver and XBox remote to control MythTV: fail

So I'd gotten the Fintek IR receiver to work previously under ubuntu, but the upgrade to precise broke it. Little did I know that things had changed drastically when LIRC was built into the kernel.

Mythtv has new LIRC configuration instructions, and I spent a fair bit of time following a helpful forum post. After spending a number of hours screwing with it, I'm documenting what I have and giving up, maybe I'll try again later.

Lets start from the beginning. The kernel recognises the device (hurrah!):

$ lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 002: ID 1934:5168 Feature Integration Technology Inc. (Fintek) F71610A or F71612A Consumer Infrared Receiver/Transceiver

If you stop lirc, you can get keycodes out of ir-keytable:

$ sudo ir-keytable -t --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00
Testing events. Please, press CTRL-C to abort.
1356325906.087938: event MSC: scancode = 1f3f
1356325906.087941: event sync
1356325906.201936: event MSC: scancode = 800f7422
1356325906.201942: event sync
1356325907.147952: event MSC: scancode = 1f3f
1356325907.147955: event sync
1356325907.253948: event MSC: scancode = 1f3f
1356325907.253951: event sync

I switched to using the xbox remote, and found someone had already posted a complete keytab file, here it is for reference should anyone else want to save some time:

# table xbox, type: UNKNOWN
0x800f7400 KEY_NUMERIC_0
0x800f7401 KEY_NUMERIC_1
0x800f7402 KEY_NUMERIC_2
0x800f7403 KEY_NUMERIC_3
0x800f7404 KEY_NUMERIC_4
0x800f7405 KEY_NUMERIC_5
0x800f7406 KEY_NUMERIC_6
0x800f7407 KEY_NUMERIC_7
0x800f7408 KEY_NUMERIC_8
0x800f7409 KEY_NUMERIC_9
0x800f740a KEY_DELETE
0x800f740b KEY_ENTER
0x800f740c KEY_SLEEP
0x800f740d KEY_MEDIA
0x800f740e KEY_MUTE
0x800f740f KEY_INFO
0x800f7410 KEY_VOLUMEUP
0x800f7411 KEY_VOLUMEDOWN
0x800f7412 KEY_CHANNELUP
0x800f7413 KEY_CHANNELDOWN
0x800f7414 KEY_FASTFORWARD
0x800f7415 KEY_REWIND
0x800f7416 KEY_PLAY
0x800f7417 KEY_RECORD
0x800f7418 KEY_PAUSE
0x800f7419 KEY_STOP
0x800f741a KEY_NEXT
0x800f741b KEY_PREVIOUS
0x800f741c KEY_NUMERIC_POUND
0x800f741d KEY_NUMERIC_STAR
0x800f741e KEY_UP
0x800f741f KEY_DOWN
0x800f7420 KEY_LEFT
0x800f7421 KEY_RIGHT
0x800f7422 KEY_OK
0x800f7423 KEY_EXIT
0x800f7424 KEY_DVD
0x800f744f KEY_EPG
0x800f7427 KEY_ZOOM
0x800f7432 KEY_MODE
0x800f7433 KEY_PRESENTATION
0x800f7428 KEY_EJECTCD
0x800f743a KEY_BRIGHTNESSUP
0x800f7446 KEY_TV
0x800f7447 KEY_AUDIO
0x800f7448 KEY_PVR
0x800f7449 KEY_CAMERA
0x800f744a KEY_VIDEO
0x800f744c KEY_LANGUAGE
0x800f7451 KEY_TITLE
0x800f744e KEY_PRINT
0x800f7450 KEY_RADIO
0x800f745a KEY_SUBTITLE
0x800f7425 KEY_RED
0x800f7466 KEY_GREEN
0x800f7426 KEY_YELLOW
0x800f7468 KEY_BLUE
0x800f7465 KEY_POWER2
0x800f746e KEY_PLAYPAUSE
0x800f746f KEY_PLAYER
0x800f7480 KEY_BRIGHTNESSDOWN
0x800f7481 KEY_PLAYPAUSE

I wrote the table to the system keytable (not sure what that error means):

$ sudo ir-keytable -c -w /etc/xbox_remote_keymap --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00
Read xbox table
Old keytable cleared
Wrote 62 keycode(s) to driver
���/protocol: No such file or directory
Couldn't change the IR protocols

Check it is stored OK:

$ sudo ir-keytable -r --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00
[snip same output]
Enabled protocols: RC-5

And now, key events are recognised:

$ sudo ir-keytable -t --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00Testing events. Please, press CTRL-C to abort.
1356326333.173697: event MSC: scancode = 800f7422
1356326333.173702: event key down: KEY_OK (0x0160)
1356326333.173703: event sync
1356326333.201701: event key up: KEY_OK (0x0160)
1356326333.201704: event MSC: scancode = 1f3f
1356326333.201705: event sync
1356326333.280699: event MSC: scancode = 800f7422
1356326333.280703: event key down: KEY_OK (0x0160)
1356326333.280704: event sync
1356326333.530999: event key up: KEY_OK (0x0160)
1356326333.531001: event sync
1356326337.414740: event MSC: scancode = 800f7410
1356326337.414747: event key down: KEY_VOLUMEUP (0x0073)
1356326337.414748: event sync
1356326337.522744: event MSC: scancode = 800f7410
1356326337.522746: event sync
1356326337.771005: event key up: KEY_VOLUMEUP (0x0073)
1356326337.771007: event sync

I changed my lirc config over to devinput as suggested in the mythtv instructions, but if I start LIRC back up, I don't see any output in irw, and the button presses don't do anything. If I leave lirc off, lots of functionality (such as back on the remote = Esc in mythtv) doesn't work since lirc isn't doing that key mapping.

I'm running lirc 0.9:

$ lircd --version
lircd 0.9.0

I suspect the problem is in the error message from the write operation. Even if you tell it to send events to LIRC specifically:

$ sudo ir-keytable -c -p LIRC,RC-5 -w /etc/xbox_remote_keymap --device=/dev/input/by-id/usb-FINTEK_eHome_Infrared_Transceiver_88636562727801-event-if00
Read xbox table
Old keytable cleared
Wrote 62 keycode(s) to driver
���/protocol: No such file or directory
Couldn't change the IR protocols

It still only lists RC-5 as an 'enabled protocol'. And yet, this seems to indicate it is enabled:

$ cat /sys/class/rc/rc1/protocols 
[rc-5] [nec] [rc-6] [jvc] [sony] [mce_kbd] [lirc]

Python sqlite3

Some notes on using sqlite3 from python. First, lets create a table in an in-memory database:

import sqlite3

DB_TABLE_NAME = 'cache'
DB_TABLE_COLUMN_DEFS = 'event blob'

conn = sqlite3.connect(':memory:')
cur = conn.cursor()

sql = 'CREATE TABLE %(table_name)s (%(column_defs)s)' % {
      'table_name': DB_TABLE_NAME,
      'column_defs': DB_TABLE_COLUMN_DEFS}

cur.execute(sql)

Next we want to insert multiple values at once. You can use executemany to avoid needing a ? for every single value, although the format it requires is quite clumsy. A list of values for insertion would look like this:

[('test',), ('test',), ('test',), ('test',), ('test',)]

Neither the questionmark or named parameter substitution seems to work for table names, so you'll need to stick those in using normal string interpolation. The actual insert statement is:

VALUES = [('test',)] * 58
sql = 'INSERT INTO %(table_name)s VALUES (?)' % {'table_name':
                                                  DB_TABLE_NAME}
cur.executemany(sql, VALUES)

This function will delete a list of ids by converting the list to the necessary array of tuples first:

def DeleteRows(id_list):
  """Deletes row from database.

  Args:
    id_list: list of row ids.
  """
  sql = 'DELETE FROM %s WHERE rowid IN (?)' % DB_TABLE_NAME

  # sqlite needs a sequence like [(34,), (38,)]
  id_tuples = []
  for rowid in id_list:
    id_tuples.append((rowid,))

  cur.executemany(sql, id_tuples)

Test code for this function looks like the following. Using executemany for select will get you an error, so I just spell out the necessary ?'s.

id_list = [8, 51, 52, 58]
select_list = [8, 51, 34, 38, 52, 58]
DeleteRows(id_list)
sql = 'SELECT rowid FROM %(table_name)s WHERE rowid IN (?,?,?,?,?,?)' % {
    'table_name': DB_TABLE_NAME}
result = cur.execute(sql, tuple(select_list))
assert(result.fetchall() == [(34,), (38,)])

Python: else clause on a for loop

From the "huh, I didn't know you could do that" files:

In [41]: list=[1,2,3]

In [42]: for x in list:
   ....:     print x
   ....: else:
   ....:     print '%s was last' % x
   ....:     
1
2
3
3 was last

It turns out you can have an else clause on loop statements, it will execute as long as the loop exits cleanly (i.e. doesn't break). This includes if 'list' is empty. There's an interesting discussion about its merits here.