Wireless problems on karmic - no probe response, disassociating

Adding to the litany of problems I have experienced with Ubuntu Karmic, I have been having terrible wireless problems at home. I'm experiencing disassociations every couple of minutes:

[ 7647.936113] wlan0: no probe response from AP - disassociating
[ 7649.299667] wlan0: authenticate with AP
[ 7649.336162] wlan0: autheniticated
[ 7649.336170] wlan0: associate with AP
[ 7649.436184] wlan0: RX ReassocResp from AP
[ 7649.436192] wlan0: associated
[ 7688.948079] wlan0: no probe response from AP - disassociating
[ 7696.585236] wlan0: direct probe to AP try 1
[ 7696.648183] wlan0 direct probe responded
[ 7696.648192] wlan0: authenticate with AP
[ 7696.748181] wlan0: authenticated

Turns out it is a problem with ath9x in the kernel so it isn't canonical's fault.

Installing linux-backports-modules-2.6.31 seems to have fixed the problem.

Set mp3 track names based on filename

A quick and dirty script I used to set mp3 track names based on file name for a given directory. Uses python and the id3v2 tagging tool. I didn't bother with album, artist, genre etc. because you can already use id3v2 to set them en-masse.

#!/usr/bin/env python

import glob
import os.path
import os
import re
import sys

list = glob.glob(os.path.join(sys.argv[1],"*.mp3"))
for thisfile in list:
    track = re.sub("^[0-9]+(\s|\.|\-)+","",os.path.basename(thisfile))
    track = track.replace(".mp3","")
    print("/usr/bin/id3v2 -t \"%s\" \"%s\"" % (track,thisfile))
    os.system("/usr/bin/id3v2 -t \"%s\" \"%s\"" % (track,thisfile))

Dell NICs still causing problems with incomplete arps

I ran into the same problem again, this time with a BCM 5708 NIC on a Dell 2RU Poweredge 2950. Exactly the same symptoms: the box makes ARPs, which are answered, but the arp cache shows incomplete entries for all IP addresses. Setting the entries manually didn't help - my pings then just seemed to go nowhere. iptables was not getting in the way. There appears to be some discussion about the problem but no solutions...

My setup was a bridged interface that consisted of two bonded NICs to get more bandwith throughput using Cisco port channelling on the switch. Getting rid of the bond and just using a regular bridge on one of the NICs fixed it. Might try the NIC bonding again on the next hardware refresh.

Creating an audio CD from mp3 files on linux

K3b is great for this, you can just start an audio project and drag mp3 files onto it. On ubuntu karmic you will need these packages:

k3b and libk3b6-extracodecs

CD ripping (sound-juicer) stuffed in karmic

My favourite CD ripper 'grip' has disappeared from the repository in ubuntu as of karmic. The built-in is sound-juicer, which sucks.

It comes with a shitty MP3 profile, but wouldn't even let me select it in the profile menu (yes, it was 'active'). Finally I ended up deleting all the profiles and creating my open mp3 profile, which allowed me to select it, but after I clicked extract, it always failed with 'failed to get output format'. I have the bad and ugly gstreamer packages installed, as well as lame.

I gave up and fell back to abcde, which is a command-line utility.

I edited '/etc/abcde.conf' and set:

LAMEOPTS="--preset standard"

then used 'abcde -o mp3'.

Why is it always so damn hard to rip a CD to mp3 in linux?

Howto install zabbix on ubuntu jaunty

apt-get install zabbix-server-pgsql

Get the auto-generated db password out of /etc/zabbix/zabbix_server.conf and use it when prompted in the next step:

apt-get install zabbix-frontend-php

If when you visit localhost/zabbix your browser tries to download a phtml file, you need to:

apt-get install php5-pgsql

and restart apache and the zabbix server.

If you try to import your zabbix config and you get "could not open XML input", you need to increase your "upload_max_filesize" in php.ini.

Zabbix mysql innodb backend shits itself on power outage

We had a power outage, and the mysql database that backs our zabbix install shat itself. The innodb tables couldn't recover themselves. I had to force an innodb recovery, which got me a running server, but I couldn't successfully do a mysqldump or select * into outfile because it failed on one table. I dumped everything else and tried a drop table on the corrupted table. The drop table didn't work, the mysql connection died after I hit enter and the table was still there. Eventually tried drop database with the same result. Yay.

Killed mysql and installed postgres.

Submitting malware and phishing urls to google safebrowsing and microsoft smartscreen

To submit dodgy urls, follow these instructions for firefox and these instructions for IE (requires IE 8 to be installed just to submit, thanks microsoft).

Chef - the new puppet?

I just encountered Chef, which is a 'systems integration framework' like puppet that allows you to manage configuration and much more across many machines. I've been using puppet for quite a while and love it, although as stated here, I find the puppet Domain Specific Language (DSL) very restrictive. There have been so many times I have wanted to do something reasonably simple for a fully-fledged programming language, that has either been impossible, impractical, or completely un-intuitive to accomplish with puppet. No, I don't have an example - next time it happens I'll blog.

In any case, Chef seeks to address this problem by giving you full-strength ruby, and is written by some long-term members of the puppet community. Apparently puppet has made the decision to stay away from a full programming language, hence Chef. Puppet has the momentum at the moment but this may be worth watching.

Updating a debian package

I followed this HOWTO to create a debian package update. With a debian watch file it is quite easy to grab a new upstream version. To run the 'debuild' step you will need all the build dependencies installed - these are listed in the debian/control file in the package directory.

sudo apt-get build-dep package

will install all the build dependencies for a given package.

Using the new sourceforge shell interface

Sourceforge has changed their shell access procedure again. This time it is a little complicated and unintuitive. You need to create a shell (time limited) with:

ssh -t username,projectname@shell.sourceforge.net create

Then you can copy files to your website using:

scp index.html username@web.sourceforge.net:/home/groups/p/pr/projectname/htdocs/

HOWTO configure zabbix to send emails with postfix

Under Admin|Media Types configure the email option as:

SMTP server: localhost
SMTP helo: myinternetdomain.com
SMTP email: zabbix@myinternetdomain.com

Add a user with email media notifications. It didn't seem to be enough to have this setting, I also had to add an Action. The triggers were an "AND" of:

(A)  Trigger severity >= "High"
(B)  Trigger value = "PROBLEM"

with the recovery option ticked this got me a PROBLEM email and a RECOVERY email for all triggers above "high". Without the PROBLEM condition I didn't get the recovery message I specified: instead I got two PROBLEM emails with different trigger status (what the?). Others have run into the same issue.

Install postfix and set:

myhostname = myinternetdomain.com
inet_interfaces = 127.0.0.1

You now have a postfix listening on localhost - you can test it with the 'sendmail' terminal command.

Trac error: Unsupported version control system "darcs"

I got this error after an OS upgrade:

Unsupported version control system "darcs": Can't find an appropriate component, maybe the corresponding plugin was not enabled?

To solve it I grabbed the latest darcs trac plugin:

darcs get http://darcs.arstecnica.it/trac-darcs

and installed it:

sudo python setup.py install

I then had to upgrade and resync each repository:

trac-admin /var/lib/trac/myrepo/ upgrade
trac-admin /var/lib/trac/myrepo/ resync

What's wrong with Zabbix

I have been using Zabbix for monitoring for a while now, and have been really happy so far. Today I ran into some things that are really annoying:

• You can't change the X-axis on any graph. It is always time. The time period can be adjusted using the little calendar app in the bottom left corner of the browser. I don't really like this approach. I'd like to also be able to specify time periods per screen, because what makes sense for one graph may not work for all the others.
  
• You can't have a graph of one item (e.g disk-usage) across all hosts. Suck! I want to have a graph of OS versions on the y-axis and hostnames on the x-axis. Not possible!
  
• net.tcp.listen[port] isn't supported on linux(?) - you will need to use net.tcp.port[,port] instead.
  
  

HOWTO: Monitor puppet with zabbix and do agent command testing/troubleshooting

This turned out to be really easy, but not obvious with the GUI. The GUI shows the proc.num syntax as:

proc.num[ <,user> <,state>]

but the manual has the real story:

proc.num[ <,user> <,state> <,cmdline>]

So you can use the cmdline to monitor something like this:

ruby /usr/sbin/puppetd -w 0

with:

proc.num[ruby,,,"/usr/sbin/puppetd"]

To troubleshoot agent monitoring strings, use the commandline on the box (quicker than clicking around in the GUI) like this:

zabbix_agent -t proc.num[ruby,,,"/usr/sbin/puppetd"]

Eliminating openldap '(uid) not indexed' errors

My ldap server logs were full of hundreds of these:

slapd[2921]: <= bdb_equality_candidates: (uid) not indexed

Basically one of these gets written every time a ldap search is done on a non-indexed attribute. The fix is, like most things with LDAP, completely unintuitive.

Take a look at the current indexing being done by:

sudo /usr/sbin/slapcat -n 0 -l output.ldif

and grep for olcDbIndex (mine was only indexing objectClass by default).

Create a ldif file (indexchanges.ldif) to change the indexing attribute:

dn: olcDatabase={1}hdb,cn=config
changetype: modify
replace: olcDbIndex
olcDbIndex: uid,uidNumber,gidNumber,memberUid,uniqueMember,objectClass,cn eq

And run it with:

sudo ldapmodify -f indexchanges.ldif -D cn=admin,cn=config -x -y /etc/ldap.secret

Note that as I mentioned previously ldapmodify fails if you are only listening on ldaps. Change SLAPD_SERVICES to include ldap:/// in '/etc/default/slapd', restart ldap, use ldapmodify, change back, restart ldap.

You then need to tell it to actually build those indexes (need to keep the index files owned by openldap user):

sudo /etc/init.d/slapd stop
sudo su -s /bin/bash -c slapindex openldap
sudo /etc/init.d/slapd start

MythTV and Lirc, adding pause to mplayer

Needed to add a keybinding for pausing video playback when MythTV uses mplayer, thought I might as well record how it worked here. The global lirc config (/etc/lircd.conf) points to your remote, in my case:

include /usr/share/lirc/remotes/dvico/lircd.conf.fusionHDTV

You then configure the appropriate buttons in

/home/mymythuser/.mythtv/lircrc

the pause button is as below. Most of the entries in this file will be for 'prog=mythtv' rather than mplayer:

begin
    remote = DVICO_DUAL
    prog = mplayer
    button = playpause
    config = p
    repeat = 0
    delay = 0
end

HOWTO delete a moinmoin wiki user

The MoinMoin instructions for how to delete a user, while correct, suck. First find your wiki settings file (mine is in /etc/moin/mywiki.py). In that file find your 'data_dir', mine was pointing to '/var/local/somewiki_wiki'. The user files sit in '/var/local/somewiki_wiki/user', one for each user. Grep that directory for the name you are looking for.

Delete the appropriate user file and also delete the cached username mapping file in ../cache/mywiki/user/name2id (this will be regenerated by django). Restart apache.

iptables the Ubuntu way

First, get your rules right on the commandline, some examples:

iptables -F
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT
iptables -P FORWARD DROP
iptables -A INPUT -i lo -j ACCEPT 
iptables -A INPUT -i eth+ -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT 
iptables -A INPUT -i eth+ -p tcp -m tcp --dport 22 -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

Then save to a file:

iptables-save >/etc/iptables.rules

Then in your /etc/network/interfaces file in the block for your interface:

pre-up iptables-restore < /etc/iptables.rules

If you are using network manager, you might want to put a script in dispatcher.d instead of using network/interfaces.

Another alternative is installing the iptables-persistent package, which installs a service that runs iptables-save and iptables-restore against rules in these files (for IPv4 and IPv6):

/etc/iptables/rules.v4
/etc/iptables/rules.v6

You can do iptables-save to create those files, or just get the rules the way you want then let the package do it for you:

sudo dpkg-reconfigure iptables-persistent

mdd for windows memory dumps

I have used 'dd.exe' from the Forensic Acquistion Utilities toolkit for Windows memory dumps in the past. The website now appears to be down, so I tried out mdd, an open source project. Worked a treat.

And more malware - lsass.exe

Turns out the previous post wasn't the last of it. An AV message popped up alerting me to a buffer overflow on the heap triggered by C:\windows\cursors\lsass.exe (what is a regular user supposed to do about that?). There may have been some level of rootkitting because I couldn't see the file on the commandline or with windows explorer - booting a linux live CD fixed that problem. This is a location that has been associated with sasser, and this *may* have been a variant, but not a single AV picked it up at virus total. It was using this key to persist:

 [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon]
"Shell"="Explorer.exe C:\\WINDOWS\\Cursors\\lsass.exe"

Sent to McAfee again, and a different Bangalore monkey produced another signature! Win.

Fun tracking down malware - svcchost.exe

I love being tech support. Found a little nasty called svcchost.exe (classic!) on a family windows computer. Run key in:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run
C:\Documents and Settings\username\Application Data\Microsoft\svcchost.exe

Beacons:

1 0.000000 10.1.1.9 10.1.1.1 DNS Standard query A xdemonx.selfip.org
2 0.000590 10.1.1.1 10.1.1.9 DNS Standard query response A 96.18.166.50
3 0.000895 10.1.1.9 96.18.166.50 TCP 1588 > 3085 [SYN] Seq=0 Win=16384 Len=0 MSS=1460

Reported to McAfee. They actually have some decent advice for finding runkeys:

On Windows XP systems, click START RUN, type MSCONFIG and hit ENTERClick the Startup tab.

A McAfee monkey in Bangalore produced a signature and sent back an extra.dat file in just a few minutes. Mission accomplished.

Advice on how to report an ubuntu bug

The Ubuntu wiki has a great page on how to report a bug. It includes details of how to install debugging symbol packages (I didn't know these existed, I always headed for the source and compiled a debug build), and detailed gdb instructions.

I also finally learnt a better way to find out which version you are running (usually I look at /etc/issue or /etc/apt/sources), but this:

lsb_release -rd

gives you the numerical release number. Sweet.

Restarting NFS after changes to /etc/exports

On ubuntu I was using

/etc/init.d/nfs-common restart

to restart NFS after making changes to /etc/exports. Unfortunately this doesn't work. Use "exportfs -a" to sync changes (clients don't need to re-mount either).

Zabbix monitoring tips

Random Tips

• The zabbix server comes with some great example graphs, copy these to your own templates and every host linked with the template will have them available to use as screens.
  
• You can disable an item/trigger inherited from a template for just a single host by disabling it in the hosts view (not the template).
  
• If you delete an item, triggers depending upon that item will automatically be deleted.
  
• The multi-value graphs you can create under the graphs tab are way better than the 'simple
  graph' type available in the screens menu.
  
• You can automatically add hosts discovered through automatic discovery to templates (and do many other things) using actions under the actions tab.
  

Errors

I got the following error from the zabbix agent daemon (zabbix_agentd):

Can't find shared memory for collector

Restarting didn't help, but deleting the shared memory objects did:

ipcs -ma          to list the objects
ipcrm             to delete the objects owned by the zabbix user

Restarting the zabbix daemon and killing off the defunct processes got me back into business.

Creating new server certificates with an existing CA

Creating new server certificates with an existing CA, using unencrypted server keys so you don't get prompted when apache starts:

openssl req -new -config openssl.cnf -nodes -keyout newkey.pem -out newreq.pem -days 1825
openssl ca -config openssl.cnf -policy policy_anything -out newcert.pem -infiles newreq.pem

Firefox XUL dialog box transparent by default

This took far too long to figure out. I was opening a dialog in XUL in my firefox plugin, but it was arriving with a transparent background, which made it render really weirdly. The solution is to put the following in your css for the dialog tag:

    background-color: menu !important;
    opacity: 1.0;

My guess is that the background is transparent so you can see images set by firefox themes/skins, but it is pretty annoying!

HOWTO Convert a windows vmware image to a linux qemu/kvm

This is a good HOWTO for converting windows vmware images to linux Kernel Virtual Machines (KVMs). Note that if you don't have a single monolithic vmdk disk file, you can just run qemu-img over each individual file, converting it to raw, then cat them together in order. I tried this and it worked a treat:

for i in $( ls *.vmdk ); do qemu-img convert -O raw $i ~/$i.dd; done
cat s001.vmdk.dd s002.vmdk.dd s003.vmdk.dd > monolithic_image.dd

Verify ssl certificates and keys

The following openssl commands are handy for verifying/inspecting/converting keys/certificates and inspecting CRLs:

# Check a private key
openssl rsa -in /etc/apache2/ssl/server.key -check -noout -text

# Verify this cert against this CA list
openssl verify -verbose -purpose sslserver -CAfile /etc/ssl/cacert.pem /etc/apache2/ssl/server.pem

# Display cert details
openssl x509 -in /var/cert.pem -text -noout

# What purposes does this cert have?
openssl x509 -in mycert.pem -purpose

# Connect to a server and show the certs
openssl s_client -debug -connect localhost:443 -showcerts

# Inspect a CRL
openssl crl -inform DER -in blah.crl -text -noout

# Convert DER (.crt .cert .der) to PEM

openssl x509 -inform der -in cert.cer -out cert.pem

# Inspect DER without converting

openssl x509 -inform der -in cert.cer -noout -text

All calls on Samsung F480 appearing as "unknown" number

My phone wasn't displaying any numbers when I received a call - even though I had corresponding entries in the phonebook. For the benefit of others googling for a solution to this problem: it isn't your phone. It doesn't matter if your contacts are in the phone or in the sim. Most likely you need to call your provider and make sure caller id forwarding is enabled on your service. This can happen when you change providers or start a new service.

Convert epoch timestamp (seconds since 1970) on the command line

Use this to convert epoch timestamp (seconds since 1970) on the command line. For some reason the man page doesn't reveal this secret:

date -d @1241958464

If you have the epoch seconds in hex, use this on linux:

echo $((0x4f91d6f8)) | xargs -I## date -d @##

The same thing is much easier on OS X:

date -r 0x4f91d6f8

Setting up a Samsung F480 for Gmail IMAP and SMTP

Set up one of these recently. Google's instructions are pretty good but I thought I'd document specifically for this phone:

• Enable IMAP in gmail settings
  
• Incoming settings:
  
  
  
  • IMAP4 server: imap.gmail.com
    
  • Port: 993
    
  • Security type: SSL (always)
    
  
  
• Outgoing settings:
  
  
  
  • SMTP server: smtp.gmail.com
    
  • Port: 465
    
  • Secure connection: SSL
    
  
  
• APOP login: No
  
• My address: myname@gmail.com
  
• Use SMTP auth: Yes
  
• Same as POP3/IMAP4: Yes
  

The openssl client came in handy for checking the google certificates, which for some reason are signed by their own ca:

openssl s_client -connect smtp.gmail.com:465 -showcerts

Creating certificates for ldaps on windows 2003 server using openssl CA

/usr/lib/ssl/misc/CA.sh -newca
(Set a password for the CA)

/usr/lib/ssl/misc/CA.sh -newreq
(Set a password for the key for this cert)

/usr/lib/ssl/misc/CA.sh -sign                 
openssl pkcs12 -export -in newcert.pem -inkey newkey.pem -out keyandcert.p12
(Consolidate key and cert into single file, protect with 'import password')

Then test cert is valid with:

openssl pkcs12 -in keyandcert.p12 -noout -info

Import CA cert:

• Start | Run | mmc
  
• Add snap-in Certificates
  
• Right click on Trusted Root Certificates | All tasks | Import | Choose the CA certificate: demoCA/cacert.pem | Choose Trusted Root CA store
  
• Find the certificate in the Trusted Root CA store | Right click | Properties | Enable only the following: untick everything except Server Authentication and Client Authentication
  

Import server cert:

• Right click on Personal Certificates | All tasks | Import | Choose the server certificate: keyandcert.p12 | Enter private key password, mark as exportable | Choose Personal store
  
• Find the certificate in Peronal store | Dbl click | Ensure the certificate dates are correct and the phrase "You have a private key that corresponds to this certificate" is present on the General tab.
  

Test with python (install the CA cert in a directory first):

import ldap
ldap.set_option(ldap.OPT_X_TLS_CACERTFILE,"/etc/ssl/cacert.pem")
ldap.set_option(ldap.OPT_DEBUG_LEVEL,255)
l = ldap.initialize("ldap://computername.mydom.com")
l.start_tls_s()

This should return successfully. If not, turn up the schannel logging level on the windows box by setting the eventlogging key:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL\EventLogging = 0x4

Then re-try the python code. You should see the following events from schannel in the event viewer:

• Creating an SSL server credential
  
• Server credential has following properties....
  
• An SSL server handshake completed successfully
  

The Microsoft HOWTO for doing this is pretty poor. It says all you need to do is install the certificates and reboot, then the DC will be listening on the ldaps port (tcp 686). In fact, a reboot is *not* required to get TLS working on the regular ldap port using start_tls as above, and even if you reboot the DC won't listen on port 686. I have trawled the net and can't find any other instructions for how to get ldaps listening.

I hate this windows black magic voodoo shit where debugging is practically impossible, error messages are completely uninformative, and rebooting is the cure for everything.

Update: If you need to update expired certificates, that does require a reboot. Deleting the old certs and restarting the certificate service doesn't cut it. To debug you can run a packet capture and look at the 'server hello' in wireshark; it parses the whole certificate so you can see what you are serving.

HOWTO configure ntp on a windows 2003 server

Wow, this is really sucky. I can't believe this is the blessed procedure. I have reproduced it here:

Open Registry Editor (regedit.exe) and configure the following registry entries:

    HKLM\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\Type

This registry entry determines which peers W32Time will accept synchronization from. Change this REG_SZ value from NT5DS to NTP so the PDC Emulator synchronizes from the list of reliable time servers specified in the NtpServer registry entry described below.

    HKLM\SYSTEM\CurrentControlSet\Services\W32Time\Config\AnnounceFlags

This registry entry controls whether the local computer is marked as a reliable time server (which is only possible if the previous registry entry is set to NTP as described above). Change this REG_DWORD value from 10 to 5 here.

    HKLM\SYSTEM\CurrentControlSet\Services\W32Time\Parameters\NtpServer

This registry entry specifies a space-delimited list of stratum 1 time servers from which the local computer can obtain reliable time stamps. The list may consist of one or more DNS names or IP addresses (if DNS names are used then you must append ,0x1 to the end of each DNS name). For example, to synchronize the PDC Emulator in your forest root domain with tock.usno.navy.mil, an open-access SNTP time server run by the United States Naval Observatory, change the value of the NtpServer registry entry from time.windows.com,0x1 to tock.usno.navy.mil,0x1 here. Alternatively, you can specify the IP address of this time server, which is 192.5.41.209 instead.

Now stop and restart the Windows Time service using the following commands:

    net stop w32time

    net start w32time

You should see some entries in the system event log stating that the box is receiving time from the server.

Nagios check_proc fooled by 15 character limit in /proc/pid/status

Interesting problem with the nagios check_proc command. It does not take into account that the status variable in /proc/pid is limited to 15 characters. Names of processes longer than this are truncated.

Using python ldap to authenticate a django app to a windows domain controller

I used a HOWTO and the auth backend from django ticket 2507 to get django working with a linux openldap. The next task was to get it working with windows. This will probably depend on your AD structure more than anything else. I used the following in settings.py:

    import ldap
    AUTHENTICATION_BACKENDS = (
    'myapp.ldapauth.LDAPBackend',
    )
    LDAP_DEBUG=True
    LDAP_SERVER_URI='ldap://mydomain.com'
    LDAP_SEARCHDN='ou=Staff,dc=mydomain,dc=com'
    LDAP_SEARCH_FILTER = 'sAMAccountName=%s'
    LDAP_PREBINDDN = 'bindacct@mydomain.com'
    LDAP_PREBINDPW = 'pass'
    LDAP_BIND_ATTRIBUTE = 'cn'
    LDAP_FIRST_NAME = 'givenName'
    LDAP_LAST_NAME = 'sn'

I used ipython to debug my ldap setup:

ipython
import ldap
ldap.set_option(ldap.OPT_DEBUG_LEVEL,255)
l = ldap.initialize('ldap://server:port')
l.simple_bind_s('domainuser@mydomain.com','pass')
l.search_s('ou=people,dc=mydomain,dc=com',ldap.SCOPE_SUBTREE,'sAMAccountName=domainuser')

The next step is to follow the Microsoft instructions for enabling SSL so the creds don't travel in cleartext.

Upgrading ubuntu

When upgrading ubuntu I usually just change my sources.list and do a dist-upgrade. There is now a utility that essentially does this for you: 'do-release-upgrade'. It replaces all occurrences of the old distro eg. 'intrepid' with the new distro 'jaunty' in your sources.list, downloads packages and tells you to reboot. Nice!

You can also upgrade with the GUI 'update manager', but I found it wasn't very good at reporting progress so I wasn't sure what was actually happening.

Backup and restore openldap 2.4

Backup

/etc/init.d/slapd stop
sudo slapcat -n 0 > backup/config.ldif
sudo slapcat -n 1 > backup/users.ldif
sudo cp /etc/ldap.secret backup/ldap.secret
/etc/init.d/slapd start

Restore

/etc/init.d/slapd stop
sudo slapadd -n 0 -l backup/config.ldif
sudo slapadd -n 0 -l backup/users.ldif
sudo cp backup/ldap.secret /etc/ldap.secret
/etc/init.d/slapd start

Openldap 2.4 and TLS

The HOWTOs I used were:

• TLS section of the admin guide
  
• The TLS FAQ entry - also includes a quick howto for creating your own CA
  

The most annoying thing about openldap is that pretty much every bit of advice and howto on the Internet is for the old version that uses slapd.conf. In the new version (2.4) everything is stored in the LDAP database in ldif itself. So where is the advice about how to add the TLS config directives? Nowhere! Not only that, but ldapadd and ldapmodify are really difficult to use, with poor error messages if you screw up your ldif syntax.

You need to write a file tls_ldap.ldif:

dn: cn=config
add: olcTLSCACertificateFile
olcTLSCACertificateFile: /etc/ldap/ssl/demoCA/cacert.pem
-
add: olcTLSCertificateFile
olcTLSCertificateFile: /etc/ldap/ssl/servercrt.pem
-
add: olcTLSCertificateKeyFile
olcTLSCertificateKeyFile: /etc/ldap/ssl/serverkey.pem

Then run:

sudo ldapmodify -f tls_config.ldif -D cn=admin,cn=config -x -y /etc/ldap.secret

This assumes that the admin password is stored in /etc/ldap.secret - this is how the debian package installs ldap. Most advice on the internet tells you to look in slapd.conf for rootpw - retarded. Interestingly, after I disabled regular ldap in favour of ldaps below, I couldn't use ldapmodify anymore, even when I specified ldaps:// with the -H parameter. Had to re-enable regular ldap, run the command then turn it off again.

Add the following line to /etc/default/slapd (if you only want SSL then just use ldaps):

SLAPD_SERVICES="ldap:/// ldaps:///"

Restart slapd.

On the client you need to copy over the cacert, and add these lines to /etc/ldap.conf:

uri ldaps://myserver.fqdn.com/
tls_cacertfile /etc/ssl/ldapcacert.pem
tls_checkpeer no

I had to turn off tls_checkpeer, even though this shouldn't be necessary. The server wasn't giving any error logs, until I ran it manually in super debug mode:

sudo slapd -d -1 -g openldap -u openldap -h ldaps:/// -F /etc/ldap/slapd.d/

When it gave "unable to get TLS client DN". I figured out what the problem was: I was just using "myserver" in the URI, instead of the FQDN in the certificate. So make sure you put the same domain in your ldaps uri as appears in your certificate (should be fully qualified like "myserver.fqdn.com").

Python distutils installer and user_options

I found the standard doco for distutils to be pretty appalling. It is really only useful for simple cases. The API Reference is slightly more useful, but still not great. I couldn't find a good explanation of how user_options worked to pass extra stuff into the setup script.

Here a few things I found out after a lot of mucking around:

• Tuples that go into the user_options array should look like:

  ('mysql-root-passwd=', "p", 'MySQL root password for local server [default: None]')

  The '=' on the first element tells distutils this options should have a value (i.e. it is not a boolean option)
  
• Options get stored as object variables in your class so the one above turns up as self.mysql_root_passwd - note the substitution for '-'.
  
• To subclass the install command inherit from

  from distutils.command.install import install

  and set

  'cmdclass': {'install': WhitetrashInstallData}

  in your call to setup.
  
• If the doco sucks, take a look at the code:

  /usr/lib/python2.5/distutils/command/install_data.py

  helped me.
  

Installing zenoss-core on Ubuntu Intrepid

Zenoss only has instructions for older versions of ubuntu. I had to install the following packages to get it to compile on intrepid:

sudo apt-get install \
libpango1.0-dev \
libcairo2-dev \
libxml-2 \
python-cairo-dev \
gettext \
mysql-dev \
python-setuptools bzip2 \
mysql-server mysql-client python-dev build-essential subversion snmpd autoconf snmp

Which got me to this error:

/usr/include/asm-generic/fcntl.h:117: error: redefinition of ‘struct flock’
/usr/include/asm-generic/fcntl.h:140: error: redefinition of ‘struct flock64’

but thankfully someone has a patch already:

--- Samba/source/ntvfs/sysdep/inotify.c.~1~   2008-07-30 15:44:55.000000000 -0400
+++ Samba/source/ntvfs/sysdep/inotify.c   2008-11-11 15:56:44.000000000 -0500
@@ -29,10 +29,10 @@
 #include "lib/util/dlinklist.h"
 #include "libcli/raw/smb.h"
 
-#include 
-#include 
+#include 
 
-#ifndef HAVE_INOTIFY_INIT
+#if 0
+#include 
 /*
   glibc doesn't define these functions yet (as of March 2006)
 */

apply with:

$ cd inst/build/wmi-*/
$ patch -p0 < patchfile 

Configuring ssh on a cisco switch or router

To get ssh working on cisco gear, you first need an image which actually supports it. Seriously? This is retarded. Upgrading is fairly easy via the web interface once you have navigated the cisco downloads maze. They have a HOWTO enable ssh, that boils down to:

aaa new-model
username someuser password 0 thisisabadpword
service password-encryption
line vty 0 4
transport input telnet

Test with telnet and the username/password you used above then:

ip domain-name mydomain.com
cry key generate rsa
ip ssh time-out 60
ip ssh authentication-retries 2

Disable everything except ssh:

line vty 0 4
transport input ssh

If there is another vty line, then do the same for that one.

Bridging a bonded network interface on ubuntu to a cisco switch using LACP 802.3ad

There are a lot of out of date howto's for doing bridging on linux. I found a good one - it is really easy (this is works on Ubuntu intrepid):

apt-get install ifenslave

Then change your /etc/network/interfaces - this setup bonds the two interfaces together so both are used to maximise throughput:

auto bond0
iface bond0 inet dhcp
  slaves all
  bond-mode 4
  bond-miimon 100

It is also possible to bridge a bonded interface, with a interfaces file like this (you'll also need bridge-utils):

auto bond0
iface bond0 inet manual
    slaves eth1 eth0
    bond-mode 4
    bond-miimon 100

auto br0
iface br0 inet static
 address 192.168.0.1
 netmask 255.255.255.0
 network 192.168.0.0
 broadcast 192.168.0.255
 gateway 192.168.0.254
        bridge_ports bond0
        bridge_fd 9
        bridge_hello 2
        bridge_maxage 12
        bridge_stp off
 dns-nameservers 192.168.0.1
 dns-search my.lan

You switch needs to support 802.3ad, which apparently most modern ones do. I have it working with cisco following their howto, which boils down to:

Router> enable 
Router# configure terminal
Router(config)# interface port-channel 1
Router(config-if)# interface g1/0/24
Router(config-if)# channel-group 1 mode active 
Router(config-if)# exit
Router(config)# interface g1/0/23
Router(config-if)# channel-group 1 mode active 
Router(config-if)# end
Router# copy run start

This should give you lines like this (show run):

interface GigabitEthernet1/0/23
 channel-group 1 mode active
!
interface GigabitEthernet1/0/24
 channel-group 1 mode active
!

Vmware server console 1.0.4 broken on 64bit intrepid

I was getting the following errors on intrepid amd64 when trying to install vmware server console 1.0.4:

/usr/lib/vmware-server-console/lib/wrapper-gtk24.sh: 316: /usr/lib/vmware-server-console/bin/vmware-server-console: not found
/usr/lib/vmware-server-console/lib/wrapper-gtk24.sh: 370: /usr/lib/vmware-server-console/bin/vmware-server-console: not found

The fix was to install the ia32-libs package.

Using ssh as a SOCKS proxy

SSH makes an awesome socks proxy for tunneling whatever, but particularly web. The nicest way to do it is to create a ~/.ssh/config file with:

host mysocks
   dynamicforward 9988
   hostname mysocksbox.domain.name
   user blah

You can put any options in that config file that are listed in the ssh_config man page. So now all you need to do is

ssh mysocks

and it will set up a listener on local port 9988, traffic forwarded over ssh to mysocksbox.domain.name. You can then use the firefox switchproxy add on or foxyproxy to point to localhost 9988 SOCKS5. You also want to set network.proxy.socks_remote_dns to true in about:config to get DNS to go through the proxy too. Simple!

HOWTO convert docx, docm, xlsx, pptx office 2007 "open" xml docs to open office

There is a commandline tool you can use to convert docx, docm, xlsx, pptx Office 2007 "open" xml docs to Open Office.

HP Proliant raid - ubuntu intrepid won't boot after install

I've had a problem with HP Proliant SAS raid drives and ubuntu intrepid. The OS installs fine, but won't boot after installation. HP says something about attempting to boot from C: drive. I love how HP assumes I have a C: drive. It seems plenty of people are having the same problem in this thread. The solution is to boot into rescue mode using the installation media and do:

grub-install /dev/cciss/c0d0

or just use the re-install grub rescue menu option with

/dev/cciss/c0d0

Network bridging broken on Dell Poweredge 850 BCM5721

I couldn't get network bridging to work on a Dell Poweredge 850 with the inbuilt nic (Broadcom BCM5721) on Ubuntu intrepid.

The bridge was created fine, brctl showed eth0 joined properly. No iptables on host or guest. Routing rules correct.

The behaviour was very similar to this bug. When using DHCP, the vm guest would make DHCP requests that reached the DHCP server, but never get any replies sent by the DHCP server. If a static IP was set the box continually arps without receiving any of the arp replies being sent by other boxes. If you look at the arp cache it shows all entries as incomplete.

I eventually gave up, copied to different hardware, and it worked perfectly - same vm, same xml definition, same version of the installed host OS.

Migrating a libvirt/kvm virtual machine between hosts

Migrating a libvirt/kvm virtual machine between hosts is possible, but not with the version of libvirt in ubuntu intrepid. You get something like:

libvir: error : this function is not supported by the hypervisor: virDomainMigrate

The help is also appalling, would it kill you to include an example?

migrate myimage qemu+ssh://mynewhost/system

Virt Manager

Older versions of virt manager have a bug that only allows you to ssh as root (in newer versions you can use the user@hostname syntax in the hostname box). A workaround for the old versions is: use gconf-editor to change the xml under Apps|virt-manager|connections.

HOWTO check an SSH host key

Check an SSH host key with:

ssh-keygen -lf /etc/ssh/ssh_host_rsa_key

Or, as a one-liner for both RSA and DSA:

find /etc/ssh/*{dsa,rsa}_key -exec ssh-keygen -lf {} \;

This also works with known_hosts, which is handy:

ssh-keygen -lf ~/.ssh/known_hosts

Creating a self-signed SSL certificate for apache

Now that the apache2-ssl-certificate utility has disappeared, it takes a few more steps. This gives you an unencrypted server key (so you don't have to enter a password on restart), make the permissions 400:

sudo openssl genrsa -out server.key 2048
sudo openssl req -new -x509 -key server.key -out server.crt -days 1095

If you did create an encrypted key, you can remove the password with:

openssl rsa -in server.key -out server.key.nopass

Then you want the following in your site config:

SSLCertificateFile /etc/apache2/ssl/server.crt
SSLCertificateKeyFile /etc/apache2/ssl/server.key
SSLEngine On

And tell it to listen on the right ports in /etc/apache2/ports.conf:

Listen 80
Listen 443

Recommended packages now installed by default

Having recommended packages installed by default in ubuntu intrepid took me by surprise. This meant when I installed darcs, I also got the full exim4 mta. Worse, when I uninstalled exim4, it didn't stop the server listener, remove the init script, or remove it from the runlevels. Poor effort!

Set up a darcs repository for write by a group of users

Create a group, add everyone to the group then set permissions and sticky on the group:

sudo find /code -type d -exec chmod 770 {} \;
sudo find /code -type f -exec chmod 660 {} \;
sudo find /code -type d -exec chmod g+s {} \;

Building JEOS Ubuntu machines for libvirt/KVM

The ubuntu server doco has a pretty good howto.

To keep a record of your config and replicate it across multiple VMs you'll want to use a config file. Then use this command:

sudo vmbuilder kvm ubuntu --domain=hostname \
--dest=/path/to/hostname-intrepid-amd64 \
-c thisconfig.cfg

Or if it's a one-off here's a version with most options on the commandline (you'll want to be changing that password soon):

sudo vmbuilder kvm ubuntu --dest=/data/kvm/temp-ubuntu-amd64 --bridge=br0 \
--mem=2048 -suite precise --flavour virtual --arch amd64 -o --libvirt qemu:///system \
--hostname temp-ubuntu-amd64 --part /usr/local/vms/templates/vmbuilder.partition \
--user user --name user --pass password

To edit a virtual machine's attributes you can just edit the xml under /etc/libvirt/ and then restart libvirt but the supported way is better because it tells you about any errors:

$ virsh dumpxml foo > /tmp/foo.xml
(edit /tmp/foo.xml as needed)
$ virsh define /tmp/foo.xml

The networking configuration is hereSomehow I ended up with VMs using the same port numbers for their display, giving this error:

bind() failed

It took me a while to track this down (BTW: virt-manager logs to ~/.virt-manager and the kvm server logs into /var/log/libvirt). Just had to change the port numbers with dump/define above in the machine config and all was well.

To connect to the console, use VNC to localhost, or you can install virt-viewer, which basically does the same:

sudo apt-get install virt-viewer
sudo virt-viewer my_vm_name

List hardware on a linux box

This is a nice little command to list all the hardware details of your linux box: lshw.

How to delete all waypoints from a garmin etrex

Deleting all waypoints from the garmin etrex legend is extremely un-intuitive. You need to get to the find waypoint screen then click the menu next to the X in the top right hand corner.

Why distributed source code management (git) is better than X

Great article detailing why git is better than a list of other SCMs (mercurial,bzr,svn,perforce). But what about darcs? I think possibly the only advantage over darcs is speed. This might be enough to make the switch now that the git user interface has been simplified. I remember it was pretty bad when I gave it a try a couple of years ago.

http://whygitisbetterthanx.com/