Mounting a HP HSV300 SAN lun on Ubuntu linux

To get your HP HSV300 SAN luns mounted on linux over fiber channel, first use the HP GUI to set up your luns and present them to your host. On the host install 'multipath-tools'. You will probably see these lines in dmesg:

[    4.670129] qla2xxx 0000:0e:00.0: firmware: requesting ql2400_fw.bin
[    4.673089] qla2xxx 0000:0e:00.0: Firmware image unavailable.
[    4.673091] qla2xxx 0000:0e:00.0: Firmware images can be retrieved from: ftp://ftp.qlogic.com/outgoing/linux/firmware/.

Download the firmware from the QLogic server, and put it in '/lib/firmware'. Reload the kernel module:

rmmod qla2xxx
modprobe qla2xxx

You should get a whole lot of /dev/sd* devices created, I believe there is one for each lun times the number of paths to the storage. Put these lines in '/etc/multipath.conf':

defaults {
udev_dir        /dev
polling_interval    10
selector        "round-robin 0"
path_grouping_policy    failover
getuid_callout      "/lib/udev/scsi_id -g -u -d /dev/%n"
prio_callout        "/bin/true"
path_checker        tur
rr_min_io       100
rr_weight           uniform
failback        immediate
no_path_retry       12
user_friendly_names yes
bindings_file       "/var/lib/multipath/bindings"
}

devnode_blacklist {
devnode     "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode     "^hd[a-z][[0-9]*]"
devnode     "^cciss!c[0-9]d[0-9]*"
}

devices {
device {
vendor                  "HP"
product                 "HSV300"
path_grouping_policy    group_by_prio
getuid_callout          "/lib/udev/scsi_id -g -u -d /dev/%n"
path_checker            tur
path_selector           "round-robin 0"
prio_callout            "/sbin/mpath_prio_alua /dev/%n"
rr_weight               uniform
failback                immediate
hardware_handler        "0"
no_path_retry           18
rr_min_io               100
}
}

Restart the multipath service, and you should see your LUNs in /dev/mapper/mpath*

If you have just presented new LUNs, you might also need to reload the qla2xxx module as described above.

DViCO FusionHDTV DVB-T Dual USB Tuner Card remote control LIRC config

The remote for the DViCO Dual USB Tuner appears as part of the USB interface. It looks like this in syslog:

input: IR-receiver inside an USB DVB receiver as /devices/pci0000:00/0000:00:1d.7/usb1/1-4/input/input6

First grab the remote control config file and tack this section onto /usr/share/lirc/remotes/dvico/lircd.conf.fusionHDTV:

# Please make this file available to others
# by sending it to 
#
# this config file was automatically generated
# using lirc-0.8.0(userspace) on Mon Mar  5 16:00:35 2007
#
# contributed by: Soth
#
# brand:  DViCO FusionHDTV DVB-T Dual Digital
# model no. of remote control: Fusion MCE
# devices being controlled by this remote:
#

begin remote

name  DViCO_Dual_Digital
bits           16
eps            30
aeps          100

one             0     0
zero            0     0
pre_data_bits   16
pre_data       0x1
gap            251756
toggle_bit      0   

begin codes
#starting at the top
dtv                      0x0179
mp3                      0x0187
dvd                      0x0185
cpf                      0x016C
#outer circle clockwise from top
tvpower                  0x0164
guide                    0x016D
info                     0x0166
alttab                   0x000F
skip                     0x00A3
start                    0x001C
replay                   0x00A5
dvdmenu                  0x008B
back                     0x009E
setup                    0x008D
#inner circle
up                       0x0067
down                     0x006C
left                     0x0069
right                    0x006A
ok                       0x0160
#volume and channel
voldn                    0x0072
volup                    0x0073
chup                     0x0192
chdn                     0x0193
#keypad
camera                   0x00D4
live                     0x0182
folder                   0x0086
1                        0x0002
2                        0x0003
3                        0x0004
4                        0x0005
5                        0x0006
6                        0x0007
7                        0x0008
8                        0x0009
9                        0x000A
aspect                   0x0173
0                        0x000B
zoom                     0x0174
#play buttons
rew                      0x00A8
playpause                0x00A4
ff                       0x00D0
mute                     0x0071
stop                     0x0080
rec                      0x00A7
power                    0x0074
end codes 
end remote

Then in /home/user/.lirc/mythtv (where 'user' is the system user that runs the frontend) put your config for what you want each button to do in mythtv, here are some examples (the whole file is too big to include here):

begin
remote = DViCO_Dual_Digital
prog = mythtv
button = fastforward
config = >
repeat = 0
delay = 0
end
begin
remote = DViCO_Dual_Digital
prog = mythtv
button = rewind
config = <
repeat = 0
delay = 0
end

In my setup I have a symlink:

/home/user/.mythtv/lircrc -> ../.lirc/mythtv

Lirc-aware applications look in your ~/.lircrc, which will look something like this:

include ~/.lirc/mythtv
include ~/.lirc/mplayer
include ~/.lirc/xine
include ~/.lirc/vlc
include ~/.lirc/xmame
include ~/.lirc/xmess
include ~/.lirc/totem
include ~/.lirc/elisa

I needed to do a fair bit of work to my ~/.lirc/mplayer, so I have included it here. A full list of mplayer commands is available here:

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = playpause
config = pause
repeat = 0
delay = 0
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = back 
config = quit
repeat = 0
delay = 0
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = stop
config = quit
repeat = 0
delay = 0
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = ff
config = seek +30 
repeat = 3 
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = rew 
config = seek -30 
repeat =3 
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = right 
config = seek +30 
repeat = 3 
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = left 
config = seek -30 
repeat =3 
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = up 
config = speed_incr +.1 
repeat = 3
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = down 
config = speed_incr -.1 
repeat = 3
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = ok 
config = speed_set 1
repeat = 0
end


begin
remote = DViCO_Dual_Digital
prog = mplayer
button = volup 
config = audio_delay +.1 
repeat = 3
end

begin
remote = DViCO_Dual_Digital
prog = mplayer
button = voldown 
config = audio_delay -.1 
repeat = 3
end

Configure /etc/lirc/hardware.conf. The important parts are:

REMOTE="DViCO_Dual_Digital"
REMOTE_MODULES=""
REMOTE_DRIVER="devinput"
REMOTE_DEVICE="/dev/input/by-path/pci-0000:00:1d.7-event-ir"
REMOTE_LIRCD_CONF="dvico/lircd.conf.fusionHDTV"
REMOTE_LIRCD_ARGS=""

Restart lirc (I found you often need to restart it twice), and you should be good to go. You don't need to restart mythfrontend. The original instructions also have some good troubleshooting tips.

DViCO FusionHDTV DVB-T Dual USB Tuner Card on Linux

I'm immortalising some old notes here in case others have problems with this card. The driver is in the kernel now, so most of the below isn't required anymore, just plug and play (although you still need the firmware to get both tuners).

Drivers

I followed these instructions to get the driver source (you will need mercurial).

To update the source, cd to the directory and use:

hg pull -u http://linuxtv.org/hg/v4l-dvb

To get the USB working I also needed the dvb-usb-bluebird-01.fw firmware, which then needs to go in your kernel directory:

cp dvb-usb-bluebird-01.fw /lib/firmware/2.6.something/

Now you should have two adapters listed in /dev/dvb

Strangely enough when I upgraded to dapper, only the USB frontend would get loaded on boot. To get them both to load I had to go to the v4l-dvb directory and:

  sudo make rmmod
  sudo make insmod

DVB Utils

Follow the same instructions to get the DVB utils, then to pull changes:

  hg pull -u http://linuxtv.org/hg/dvb-apps

Scan

To test the card, try scanning the channels (this is for adapter 0, s/0/1/ for adapter 1):

  sudo scan -a 0 -v /usr/share/dvb/dvb-t/au-Adelaide | tee mychannels.conf

You can also try the other (+/- 166667 Hz) configs:

  sudo scan -a 0 -v /usr/share/dvb/dvb-t/au-Adelaide.mod | tee mychannels.conf
  sudo scan -a 0 -v /usr/share/dvb/dvb-t/au-Adelaide.mod2 | tee mychannels.conf

Tzap

Tune with tzap:

  cp mychannels.conf ~/.tzap/channels.conf
  tzap "ABC TV Canberra"

dvbstream

Once you have tuned with tzap, grab an mpeg with dvbstream and open in xine:

  dvbstream 512 650 -o > test.mpeg

The numbers are the PIDs from the tuning line, near the end:

  ABC TV:205791667:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_3_4:QAM_64:
TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:512:650:529

xine

You can also play a dvb stream directly in xine by right clicking and selecting playlist | DVB

HOWTO create an encrypted backup disk with LUKS

First partition as normal with fdisk. Then create the encrypted header:

sudo cryptsetup --verify-passphrase --verbose --hash=sha256 \
--cipher=aes-cbc-essiv:sha256 --key-size=256 luksFormat /dev/sda1

Then create the filesystem and mount the disk:

sudo cryptsetup luksOpen /dev/sda1 crypt-backup
sudo mkfs -t ext4 /dev/mapper/crypt-backup
sudo mkdir /mnt/backup
sudo mount /dev/mapper/crypt-backup /mnt/backup

Copy your backup, then unmount with:

sudo umount /mnt/backup
sudo cryptsetup luksClose crypt-backup

HOWTO get a disk UUID

udev now uses disk UUIDs for consistent mounting, so you probably see something like this is /etc/fstab:

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# /dev/sda1
UUID=27EEEEEE-CFAF-4608-8036-FFFFBAEEEEEE /          xfs     relatime,errors=remount-ro 0       1   

So how can you map the /dev/sd* structure to UUIDs? Simple:

~$ ls -lh /dev/disk/by-uuid/
total 0
lrwxrwxrwx 1 root root 10 2010-03-20 14:15 27EEEEEE-CFAF-4608-8036-FFFFBAEEEEEE -> ../../sda1

If the list is out of date you can refresh it by restarting udev:

sudo /etc/init.d/udev restart

The disktype utility can also help:

~$ sudo disktype /dev/sda

--- /dev/sda
Block device, size 298.1 GiB (320072933376 bytes)
DOS/MBR partition map
Partition 1: 298.1 GiB (320070288384 bytes, 625137282 sectors from 63)
  Type 0x83 (Linux)
  XFS file system, version 4
    Volume name ""
    UUID 27EEEEEE-CFAF-4608-8036-FFFFBAEEEEEE
    Volume size 296.6 GiB (318523899904 bytes, 77764624 blocks of 4 KiB)

And you can also use 'vol_id' (now deprecated in favour of 'blkid') to print the UUID:

~$ sudo vol_id /dev/sda1
ID_FS_USAGE=filesystem
ID_FS_TYPE=xfs
ID_FS_VERSION=
ID_FS_UUID=27EEEEEE-CFAF-4608-8036-FFFFBAEEEEEE
ID_FS_UUID_ENC=27EEEEEE-CFAF-4608-8036-FFFFBAEEEEEE
ID_FS_LABEL=
ID_FS_LABEL_ENC=
ID_FS_LABEL_SAFE=

*.microsoft.com sites return empty pages when accessed via squid

The microsoft.com webserver (which serves msdn.microsoft.com and support.microsoft.com among others) sends back a chunked HTTP request "Transfer-Coding: chunked" in response to a HTTP 1.0 GET. The Transfer-Coding header is not valid for HTTP 1.0 and the HTTP 1.1 RFC is very clear:

A server MUST NOT send transfer-codings to an HTTP/1.0 client

In firefox the page renders as empty. If you view source you can see part of the page (i.e. the first chunk) was downloaded, but the full content has been truncated.

Moving squid to HTTP 1.1 should probably fix this, except according to squid.conf it won't:

# Enables HTTP/1.1 support to clients. The HTTP/1.1
# support is still incomplete with an internal HTTP/1.0
# hop, but should work with most clients. The main
# HTTP/1.1 features missing due to this is forwarding
# of requests using chunked transfer encoding (results
# in 411) and forwarding of 1xx responses (silently
# dropped)

So the solution is to drop the accept-encoding header for the microsoft.com domain:

acl support.microsoft.com dstdomain support.microsoft.com
header_access Accept-Encoding deny support.microsoft.com

OR if you are using IE, apparently you can untick "Use HTTP1.1 through proxy connections" in IE’s Advanced internet options tab.

Managing GPS waypoints on a Garmin Etrex Vista HCx under Linux

I used to use GPSman for uploading waypoints and downloading tracks to/from my Garmin Etrex Legend. When I moved to the Vista HCx I found GPSman didn't play well with the new USB interface. I soon stumbled across QLandkarte, which is a lot more polished (at least GUI-wise) and supports the Vista HCx over USB. It is the best software I have come across for linux interaction with Garmin devices.

Django current 'active' page highlighting for navigation

It is pretty common to want to highlight the current page in your navigation bar. Unfortunately it is surprisingly hard to do with django. After reading a few blogs I decided on the following, which involves creating a custom template tag and enabling the request context processor. First create a custom template tag (don't forget the __init__.py in your templatetags directory):

#!/usr/bin/env python
#project/site/templatetags/active.py

from django import template
register = template.Library()

@register.simple_tag
def active(request, pattern):
    import re
    if re.search(pattern, request.path):
        return 'current_page_item'
    return ''

Add names to your views in urls.py so you can reference the regex in the template. The prototype is '(regular expression, Python callback function [, optional dictionary [, optional name]])':

(r'^$', 'home view', {}, 'home')

Then in your template:

{% load active %}
{% url home as home %}

<li class="{% active request home %}">Home</li>

In settings.py define the 'request' context processor (you also have to define the other defaults so you don't lose them):

TEMPLATE_CONTEXT_PROCESSORS = ('django.core.context_processors.auth',
                                'django.core.context_processors.debug',
                                'django.core.context_processors.i18n',
                                'django.core.context_processors.media',
                                'django.core.context_processors.request')

Now you just need a line in your CSS that does somehthing different for the 'current_page_item' class.

Re-implementing the python 'inspect' module (whoops)

I am working with python introspection to have plugins for a framework registered (mostly) automatically. Until I read this blog post I was writing most of it myself, based on how the unittesting framework registers tests. Now I know there is a python inspect module that does all the heavy lifting, and the unittest module should be using it :) I learnt some interesting things about python introspection along the way I thought I should record.

Here is my implmentation of

inspect.getmembers(testclasses,inspect.ismodule)

to get a list of modules (.py files) from a package (directory with __init__.py):

from types import ModuleType
list = dir(testclasses)
modules = filter(lambda x: getattr(testclasses,x).__class__ == ModuleType, list)

Similarly, I was going to get the testmethods using a filter and the following method:

def isTestMethod(attrname, theclass=theclass, prefix=settings.TEST_METHOD_PREFIX):
    """It is a test method if it is callable and starts with the prefix"""
    return attrname.startswith(prefix) and hasattr(getattr(theclass, attrname), '__call__')

which can be replaced by

inspect.getmembers(theclass,inspect.ismethod)

Using django components outside of django

Using django components (such as the model for DB abstraction) outside of django has a few gotchas, since you need to set up the environment correctly first. Most of the time you can just get away with setting DJANGO_SETTINGS_MODULE:

export DJANGO_SETTINGS_MODULE=yoursite.settings

But this blog has a great summary of the different options. Another approach worth noting is the one manage.py uses:

from django.core.management import setup_environ
from mysite import settings

setup_environ(settings)