Copying a compressed disk image across the network using netcat and dd

Here is a handy command to copy a disk image (such as a VM) across the network compressed. We also do a SHA1 hash to make sure it copied correctly. The idea is to only read and write the data once, to make it as quick as possible.

On the box you are copying the disk from:

mkfifo /tmp/disk.dat; sha1sum /tmp/disk.dat & dd bs=256k if=mydisk.dd | tee /tmp/disk.dat | gzip -1 | nc -q 2 10.1.1.1 8181

On the box you are copying the disk to:

nc -l 8181 | gunzip | tee image.dd | sha1sum | tee image.dd.sha1

The quick and dirty version with no hash checking is below. Note if your source is OS X you want -w instead of -q in the netcat command. I've used this with two macs connected via thunderbolt/firewire, one in TDM, and one sending the image to a linux box:

dd bs=256k if=mydisk.dd | gzip -1 | nc -q 2 10.1.1.1 8181
nc -l 8181 | gunzip > image.dd

Django user profiles

While the Django doco is pretty good, it is a bit light-on for user profiles. User profiles are for when you want to extend the information stored per user on top of the Django defaults (firstname, lastname, email etc.). There is a good blog post that fills in the gaps, although be sure to read the comments, because there is a gotcha and a work around. Basically this is what you want:

from site.app.models import UserProfile
from django.contrib.auth.admin import UserAdmin as RealUserAdmin

class UserProfileInline(admin.StackedInline):
    model = UserProfile

class UserAdmin(RealUserAdmin):
    inlines = [ UserProfileInline ]

admin.site.unregister(User)
admin.site.register(User,UserAdmin)

Time offsets in python

I always forget how to do time offsets (i.e. current time - 20 seconds) in python. Here's how:

from datetime import datetime,timedelta
datetime.now() - timedelta(seconds=20)

Installing django and postgres on ubuntu

To install django with postgres on ubuntu:

sudo apt-get install python-django postgresql python-psycopg2

django-admin.py startproject mysite

Edit settings.py:

DATABASE_ENGINE = 'postgresql_psycopg2'
DATABASE_NAME = 'blahdb'
DATABASE_USER = 'blah'
DATABASE_PASSWORD = 'blah'
DATABASE_HOST = 'localhost'
DATABASE_PORT = ''

Use psql to create the user and database, granting all privs on the database. If you want to use django testing, your user also needs to be able to create a database. Use this syntax:

alter user django createdb;
\du django

Then syncdb and startapp.

dpkg basics - listing installed packages etc.

Here are some basic dpkg operations that come in handy.

List all installed packages:

dpkg -l

List the files that a package installs:

dpkg -L [packagename]

Find out which package a particular file was installed by:

dpkg --search [filename]

HOWTO allow multiple users write access to a directory without changing umask

I have often run into the problem of giving multiple users write access to a code repo. The main problem is what permissions are set on files which are added in new commits. The default umask is 022, so you get directories as 755 and files as 644, which obviously doesn't work.

The solution I have used in the past is to change the umask in /etc/profile and /etc/login.defs to 002. You have to do both, otherwise files added via ssh and other means don't get the right mask. The disadvantage is that now all files get created as 775,664, when you only really need it for one directory. There is a better way, enter filesystem acls.

First, change your /etc/fstab to include the 'acl' option for the mount point where your repo resides:

/dev/sda1    /    ext3    defaults,acl   0       0

Do some of the regular prep to make sure you files are owned right, and dirs have the sticky bit set.

chown -R user:group /code
chown -R g+w /code
find /code -type d -exec chmod g+s {} \;

Use setfacl to set the default acls for new files and directories:

setfacl -R -m d:u::rwx,d:g::rwx,d:o:r-x /code

And check the result with 'getfacl'. Also when you use 'ls', you should see a '+' at the end of the usual permissions string that indicates there are more acls:

drwxrwsr-x+

Possibly the stupidest IT security comment I have ever read

From SANS news bites:

TOP OF THE NEWS
--High Stakes in Covert Cyber War
(January 26, 2010)
Christian Science Monitor Editor John Yemma points out that the recently disclosed long term cyber attacks against US oil companies could result in "lost jobs and higher energy prices." The attackers infiltrated the companies' networks and remained inside, quietly stealing valuable bid data, which could allow them to make bids on potentially valuable oil and gas tracts without having to invest the considerable research funds spent by the targeted companies. Evidence suggests that the attacks originated in China.
http://www.csmonitor.com/Commentary/editors-blog/2010/0126/Why-the-China-virus-hack-at-US-energy-companies-is-worrisome
(Northcutt): One sensible approach is pretty simple. We make people stand in long lines to clear customs, let's do the same thing for packets. Now before you flame me for being an idiot, I am not suggesting all packets; let's start with SMTP. If a mail message comes from a known site or country that is a major source of malicious traffic, or has a link back to such a place, force it through a series of gateways. Who pays for this? The entity that wants to deal with the US. We can call it a packet visa. Counterpoint 1: "It will never work because there are a million pathways between here and there." Ah, very true, but there are a finite number of targets, US Government including DoD, the industrial defense contractors, Fortune 500 companies, critical infrastructure, and resource brokers such as oil companies. It is the old 80/20 rule. I am betting a guy like Tom Liston can write the code in an afternoon, though it will take some DHS contractor sixty people to maintain and improve it.]

Northcutt, wtf? Does having long lines at Customs actually make your border more secure, or just slower? Presumably the security is in the checking that happens when you get to the counter, or beforehand when you book the flight. How does having a line make you more secure?

So what you would like to do is purposefully implement a DOS on SMTP? If you are so sure the sources are malicious, why not just block them instead of delivering the mail slowly? If you aren't sure enough to block them you are probably DOSing legitimate email. And what difference does it make to the attacker if the email is delivered slowly? The attack is still delivered.

I could go on, but I think this definitely wins the prize for stupidest IT security comment. I'll let you know when I read something worse.