Blogs

Are sploggers getting smarter?

Some time ago I switched from using Akismet to Mollom. When I made the switch I was preparing to (finally) migrate to Drupal 6, but that project has stalled. I also though it would be a better idea to use something developed by acquia, rather than the compeition.

After switching to Mollom I noticed the number of comments I was getting went through the floor, even though I kept the module up to date.

After some complaints from users I disabled caching and allowed Mollom approved comments to be posted without moderation.

There has been a big problem with my recent changes - the sploggers are winning. It seems if a splogger quotes a paragraph or 2 from the item they are commenting on, they're getting through. I haven't been able to find a way of telling if it is humans entering the CAPTCHAs or if it is bots exploiting a weakness in Mollom.

Either way I think this is a problem that needs to be sorted. No I don't have any suggestions for solving it, I am busy solving other issues - so please don't label me an OSDB. Well given only the Mollon client is FOSS and I suspect the problem is in the (proprietary) backend, I can duck that accusation :)

Work, Business, Travel and Family

Financially the last 12 months has been pretty good for DHC. What financial crisis? This has come at quite a cost.

Once again I have ended up having an extended period of time away from home for business. Today is day 37, country number 5. I am sitting in a hotel room on the evening of my only day off this week wondering why am I doing this. I have a family, friends, clients and a life back in Australia.

Others have blogged about hitting the travel wall - for me it kicks in after about 2 weeks and I keep fighting it until I get home. This year I have been away for 103 of 311 days - that doesn't include the days spent in Melbourne or visiting family. I really understand what Dries meant in a recent tweet.

I am starting to realise humans aren't designed to work like this.

Sure there is a lot of stuff that can be done via email, irc, chat or on the phone. All of the technology in the world can't replace face to face meetings. Just as technology can't provide the opportunity to exchange a funny story or joke in the corridor, it certainly can't make up for just wanting to be at home.

I know a few developers like myself who have ideas kicking around in the back of their mind most of the time, then it clicks - bang solution! This background noise is a blessing and curse. It never goes down too well when I am suppose to be spending time with the family, but while pushing one of the kids on the swing the answer to a problem from yesterday comes to me, ring the client to discuss the solution and make sure they are happy with it - damn junior has fallen over while I was distracted.

The background noise also has another significant downside, it is very hard to shut off. I find alcohol works pretty well at giving me some "mental space", while also being legal and readily available. The downside is the harder I work the more I need that mental space and so drink more. Earlier today, I found a site which claims to guestimate how much alcohol you would have to consume in 3 hours before you were dead, I tweeted my results, I thought it was amusing. The other thing I came across was a blog post from Stephan Hermann (aka @ubuntuworker on identi.ca) on his fight with alcohol. Stephan's post was a timely reminder of how things can slide.

Most of us in IT want to see the system delivered, the product shipped, the customer happy, and most of us will put 120% into making that happen. When it is all done, we go for a beer or 10. Just about every IT shop I have worked for had beers on a Friday night, after a project was delivered or sometimes just when we came up with a lame excuse.

I don't think many people outside of the IT industry understand how stressful it can be and drinking, or other substance (ab)use, is considered an acceptable way of dealing with that stress.

Earlier this year I was in Beijing and some client data was lost due to a mix up (mostly caused by me not being able to read Chinese and someone else's lack of English). I felt awful. Around the same time a guy working for Foxconn a new iPhone prototype threw himself out of the office after being accused of stealing one of the units. This was an extreme case but I so felt for the guy, he was just doing his job, something stuffs up and he feels powerless.

There are many more stories I could link to and discuss, but I won't.

In Australia and other places there is a lot of talk about making work environments more family friendly. Sadly I don't see that happening in our industry. I see the opposite. The global financial crisis is used by some to make their staff work harder and longer for less money. As a contractor I get to set a price and run the meter, but I find myself saying yes to projects or demands that I should not be accepting.

When I return to Australia I plan to give my liver a break, spend more time with my family and develop strategies for "switching off" more often. Life is too short to be a slave to work. I can't continue to work like this claiming I am building a future for my children when I am undermining my own future, and so theirs. Time to deal with my issues, make a plan and be successful, like another Dave.

Sorry for the rant but I felt that this all needed to be said.

Building Debian (and Ubuntu) Meta Packages

Over the last few weeks I have been building a bunch of Debian packages (aka debs) for a new Ubuntu server roll out. Most of the packages are either updates to existing packages or meta packages. Building meta packages is pretty easy, once you know how.

I will outline how to build a simple package which pulls in a couple of useful packages.

First off we need to create the directory structures and files. I do all of my packaging work in /home/$USER/packaging, with my meta packages living in a directory called meta.

For your first package run the following command

$ mkdir -p /home/$USER/packaging/meta/my-meta/DEBIAN

The key to creating meta packages is the "control" file. I have a very generic package I use for all of my servers, called dhc-base . This pulls in what I consider to be the minimum dependencies needed for a basic server. My

~/packaging/meta/dhc-base/DEBIAN/control
file looks something like this:

Package: dhc-base
Version: 0.1
Section: main
Priority: standard
Architecture: all
Depends: dhc-archive-keyring, fail2ban, iptables, openssh-server, screen, shorewall, ubuntu-minimal
Maintainer: Dave Hall <EMAIL-ADDRESS>
Description: Base install meta package for boxes administered by Dave Hall Consulting

The fields should all be pretty self explanatory. The key one is "Depends" which lists all of the packages which you want your package to pull in. I try to keep the list alphabetical so it is easier to manage.

In my example I pull in some basic things which I use all the time as well as the the gpg signing key for my packages, which I have also packaged - I may blog how to do that one day too.

Now we are ready to build the package. simply run

$ dpkg-deb -b /home/$USER/packaging/meta/my-meta

and wait for dpkg-deb to work its magic. Now you should have a shiny new deb called my-meta.deb sitting in /home/$USER/packaging/meta

If you have a bunch of meta packages to build, it can become tedious to have run the command over an over again, and each time the packages will overwrite the previous version. To save me some effort I wrote a little shell script which build a package, and gives it a nice version number too.

#!/bin/bash # # build-meta - dpkg-deb wrapper script for building meta packages # # Developed by Dave Hall Consulting # # Copyright (c) 2009 Dave Hall Consulting - http://davehall.com.au # # You may freely use and distribute this script as long as the copyright # notice is preserved # function usage { SCRIPT=`basename $0` echo Usage: $SCRIPT package-path output-path } if [ $# != 2 ]; then usage $0 exit 1 fi DIR=$1 OUT=$2 DPKG_DEB=dpkg-deb PKGNAME=`basename $DIR` BUILDREV=`date +%Y%m%d$H%I%S` VERSION=`cat $DIR/DEBIAN/control | awk '$1~/^Version:/{print $2}'` echo "Building $PKGNAME" $DPKG_DEB -b $DIR $OUT/${PKGNAME}_$VERSION-${BUILDREV}_all.deb

The script it pretty simple. It takes to arguments, the path for the package and directory to put the final package in, it will even read the version number from the control file.

To process all of the meta packages at once, simply run:

$for pkg in `find /home/$USER/packaging/meta -maxdepth 1 -type d | egrep -v '(.bzr|.svn|.git)'`; do /path/to/build-meta $pkg /home/$USER/packaging/built; done

Now you should have a nice collection of meta packages to deploy.

If you want to setup your own debian repository for your meta packages, I would recommend reading Ian Lawrence's reprepro howto.

I have found meta packages really simplify the tedious task of setting up common package sets - especially for server roll outs.

Update: If you are storing your meta packages under version control, as you should be, there is a problem. If you build the debs direct from a subversion checkout the .svn directory is included - so make sure you svn export your meta packages. Same principle applies for other version control systems.

Newstead is Online - Freely

Since moving to Newstead, Victoria just over a year ago I have been involved in several tech projects in the town.

The first of these was helping to organise the inaugural StixCamp. StixCampNewstead was a great success - many described it as awesome (insert wikipedia citation/s needed here link). The event got some good media coverage too. Although the overwhelming majority of the 60 or so attendees were from Melbourne, several locals and people from across the goldfields turned up. Mount Alexander Shire Council kicked in some cash to cover the budget shortfall which ensured the event was free, and Telstra Central Victoria & Sunraysia gave us free internet via their Next G network. StixCamp showed that although the Newstead district has a population of under 600 people, we are a switched on community that can host successful events.

Just after moving into town I was roped into building the new community website. The site was launched on the 19 September. The new Newstead community website is built using Drupal 6, with a theme developed by a local designer and myself. Maldon & District Financial Services, operator of the Maldon branch and Newstead agency of the Bendigo Bank, provided some funding for this project, while DHC donated many hours of free labour and hosting. Unlike many community websites the Newstead site is maintained by the community for the community. Each community group has a person nominated to create and maintain their content. There is a local editorial team which takes care of the basic maintenance work.

The Newstead site is built around the existing community groups. The site lists around 40 groups based in the town. Each group can post news and upcoming events. This approach also allows us to showcase the spirit of involvement in the town. The business directory has over 40 business listed, which range from micro businesses to the large employers in town. Go check out the Newstead community website for yourself.

The other project launched at the same time was Network Newstead. Network Newstead is the local free community run wifi network. The network comprises of 3 access points which cover most of the activity points in the town. The Mount Alexander Shire's community grants program paid for the hardware. After some discussions with Internode, they came to the party with free backhaul. Without the support of both organisations this project would not have been possible.

Network Newstead allows locals and tourists a like to access the internet for 30 minute sessions. We only allow access to whitelisted ports, sorry unless you know how to run openVPN you can use p2p - but leave everything a geek should need open. OpenDNS provide simple DNS based filtering of inappropriate content - no I don't think I am Mr Conroy, but I do think we should be able control what is available on our free network. WifiDog is used for the captive portal.

Newstead's telecommunications infrastructure, like many other small communities, has been neglected by Telstra. The local telephone exchange only had a DSLAM installed in 2007, and Telstra could be only bothered to install some clearance sale ADSL1 unit. Many properties outside of central Newstead (read more than 2kms from the exchange) can't get ADSL due to pair gain and poor quality copper. In our case we ended up spending close to $1000 on equipment to get a decent signal from Optus. Unfortunately you can't buy good service from Optus and have switched to Telstra Business at $165 per month for 15Gb. For many in the community these costs are just excessive.

Some members of the local community don't own a laptop, but we have facilities for them too. When I moved into town the internet cafe at the community run Rural Transaction Centre (RTC), was referred to by some as "the biggest botnet in town". Over a period of 4 to 6 weeks DHC have replaced the Celeron machines running Windows XP, with newer Pentium 4 with 17" LCD screens from ComputerBank, running Ubuntu Linux. Dave Hall Consulting donated the PCs and services to the RTC to get the new network up and running, some equipment may have even come from our spare parts bin.

Although DHC has been the main driver and implementer of these projects, the Newstead community has been heavily involved in the planning and implementation. Structures are in place to ensure the community is also involved in the maintenance of the projects.

Being involved in tech projects in a small town has been an excellent learning experience. During these projects I was able to work with community leaders, senior members of council and other members of the community. I used to cringe at the thought of the local grave vine gossiping about what I bought from the supermarket, but now I think it makes you more accountable - if you do the wrong thing everyone knows about, but if you deliver everyone knows too. At the same time this network allows for rapid feedback on ideas. I am also conscious of the fact that if I argue with someone, I am likely to see them in the main street next week.

In a former life I worked in local government in metropolitan Melbourne, even as an employee, councilors and management seemed disconnected. When I'm in Newstead, I have a weekly meeting with the ward councilor to discuss where my projects are at. I even have the Shire CEO's mobile number along with direct numbers and email addresses for several other council employees. This level of accessibility makes it a lot easier to get things done.

If you are a geek who has moved (or is considering moving) to a small community, if you have the time, I would highly recommend offering some of your skills for free. Not only will you get that warm inner glow from doing something to help someone else, you build up some experience / street cred / brownie points / connections / local knowledge - all important things in a small community. Community projects can also be part of your portfolio / resume.

Expansion of the website and wireless projects are already on the drawing board. Watch this space!

Drupal Book Review

OK, I am not reviewing a book today, but I soon will be. Packt Publishing have asked me to review Matt Butcher's new book Drupal 6 JavaScript and jQuery. The book looks pretty interesting. Alhtough it isn't on the same scale, being asked to review a serious Drupal developer book, is a bit like Obama winning the noble peace prize - ok maybe I am exaggerating a little there.

I really like YUI, but Drupal has made me interested in jQuery. One of the things awesome things about Drupal is that you can use jQuery without ever having to touch jQuery. This has made me lazy about learning jQuery - especially in the context of Drupal. It look like I have run out of excuses.

The book should arrive in France by the end of the week, but I won't be back in France for a couple of weeks, I have heading off to China for 10 days or so to catch up with friends and discuss some ideas about doing cool things with Drupal. Watch this space.

Goodbye phpGroupWare

I am writing this post with a strong sense of sadness. At the same time I feel that I have no other option. I am walking away from phpGroupWare.

Before I walk out the door, lets go for a trip down memory lane. Insert tacky music wavy lines down the screen and bad hair styles from here on in.

I think the first time I heard Linux mentioned was in 1996/97. In 1999 I had my first experience with Linux, through an anarchist friend, I was very curious, but didn't have much of an opportunity to play with it.

In 2000, over a few months I started to play with Linux and GNOME dual booting my machine. I could never really make it do what I wanted, but I persisted. The first version of Linux I bought was Macmillan Linux 6.5 - I still have it in a cupboard somewhere. I then moved on to Red Hat 6.2.

In 2001 Julie's work was sick of waiting for their shared calendar solution. I was contracted to install it. They had an NT4 box running Lotus Notes, which no one wanted to touch - including me. I found phpGW by searching (probably "yahooing") for "open source groupwise". I would later discover I had embarked on an epic journey. I knew of open source, but I didn't know a lot about it. PHP seemed pretty cool, as did phpGroupWare. I recommned they get a new server - a good one, dual PIII-800s, RAID-1 and Linux, it cost over $3000. I learnt a lot on that box including basic Linux sysadmin, compiling stuff from source and what happens when you forget to update lilo. Today that server lies idle in a cupboard in my office.

Very quickly I became involved in the phpGroupWare commununity. By 2002 I considered myself active and later that year I was promoted to release manager. By 2003 I had landed my first overseas gig indirectly through phpGroupware. By now I felt very strongly that I was a free software developer. 2004 saw the EGroupware.Org fork, or as I still like to refer to it, the EGO fork. This did considerable damage to the project and sucked a lot of life out of it.

Since then the project has done some cool things, but struggled to get a release of trunk for over 5 years. There are some really smart people around the project, but none of us seem focused to shipping a product, and I think we all have different ideas of what that product should be. I have met many awesome people over the years, include the Paris conference in 2006.

It is now late 2009, I have 2 kids and a growing business. PHP 5.3 is out and we are still trying to port to PHP5. There are many solid application frameworks around - my personal favourites are Zend and Drupal. Both of these projects have large developer bases, active communities, clear development processes and documentation, phpGroupWare fails on each count.

The project has exposed me to so many great people and ideas. These things will stay with me for the rest of my life.

If anyone is to take over phpGroupWare, my first suggestion would be to start over. Clearly define a purpose for the project, define a target market, build a product which fits that purpose and appeals to that market. Such an approach will involve a significant investment of resrources. I just can't commit any resources to the project at this time. If things change I may come back one day, but now things feel too far gone.

As much as I hate to say this, I think phpGroupWare is a dead project. I feel that it is time for me to move on. I will make myself available to assist with any handover or shut down of the project.

Thanks to everyone who has contributed to making phpGroupWare such a great project over the years. It was fun while it lasted.

Brotherhood Books Launches - Giving Books a Second Chance

In late 2008 I worked on building an online second hand bookstore for the Brotherhood of St Laurence. Today Brotherhood Books was launched by comedian and writer Corinne Grant, at the Sacred Heart Primary School in Fitzroy. I'm hanging out for Corinne to publish her short story about the snot eating witch.

Brotherhood Books is a really exciting project, it allows people who can't access bricks and mortar second hand book stores to access quality second hand books at good prices. All of the profits from the service flow directly back to Brotherhood programs and services. Before this project I thought the Brotherhood ran a few big Opys and gave out some food vouchers. They do a lot more than that.

I have been to the East Brunswick warehouse a few times to meet with the workers to discuss how the platform is working and what can be done to improve it. Most of the Brotherhood Books workers are volunteers, who are trying to get some experience so they can enter or return to the workforce. They all seem to really love their books.

The identity, webdesign and initial marketing material was done by imageffect.com.au. The platform is built using Zend Framework, where possible we used off the shelf libraries, but there is a large amount of original work. The cataloguing functionality is pretty slick, almost all of the books are catalogued by volunteers scanning a barcode, entering a condition and adjusting the system recommended price and it is done. Yahoo's YUI Toolkit provides most of the AJAX yummy-ness for the volunteers, while most of the data is pulled from Bowker's web services.

Brotherhood Books has over 30,000 items for sale, and growing every day. So far over 45 tonnes of books have been saved from going to landfill!

If you have some spare books to donate to the Brotherhood, they have many ways of accepting donations.

At the launch today there was some media present, and someone from the BSL was interviewed by Red Symonds on 774 ABC Melbourne.

Many of computer books are dated, but they still make for fun reading, These days I have several titles from the Brotherhood on my bookshelf - including a mid 90s guide to cracking.

Go check it out and buy a book so you can claim you were there at the start.

Yes I am doing the hard sell on it, but I have been waiting since the soft launch earlier this year to be able to promote it widely.

Missing Software Freedom Conference Kosova

Today I should be in Prishtina Kosovo running Drupal workshops at the first Software Freedom Conference Kosova. Unfortunately due to work and family commitments I had to decline the invitation. I hope to make it there next year.

I will also be missing out on DrupalCon Paris next week too.

Sometimes it sucks to be in Australia, especially when Europe is so far away and so many cool things happening there too.

Back Blogging Again

Bless me internet for I haven't blogged, it has been 274 days since my last post.

I have wanted to blog, but I kept on finding excuses to avoid it - need to upgrade the software, need to finish x, y and z, need to focus on projects a, b and c etc. One of the main reasons is that I have been too lazy to put the effort in. I find it takes time to think of what to blog and then to bash it out, refine it and post it. When I have had the time to put that effort into my blog, I haven't had the inclination.

While things have been quiet here, I have microblogging using the open source laconica platform through identi.ca, which also posts to StixCampNewstead.

More recently I have been working with a client in France who has some serious collaboration requirements. At last count they have almost 2100 instances of drupal running. This has involved a lot of work, and some travel. I will blog about this project soon - it is pretty awesome (even if I say so myself).

We have built a small drupal powered site for a local assest management consulting business. They are very happy with the results. Now their customers just log in to download the software. Everything was off the shelf drupal - except for the theme and a 60 line custom permissions module.

We have built the Newstead community website using drupal. It still needs some polish before final launch. The community has been heavily involved in the development of the site. So far over 30 locals have been trained in maintaining their page/s on the site. There is no "webmaster", each local business and community group will maintain their own content

A couple of months ago I/we joined the drupal association. One day the buttons will be added to the site.

Where to next?

I plan to blog more about projects I am involved in. I also plan to switch this site to drupal 7 as close as possible to the release date - it looks like others will be switching too. Next week I will be commencing the build of the Newstead community wireless network.

Watch this space, lots happening - including more frequent updates from here on in.

Updated IMCE plugin for Drupal YUI Editor

My IMCE plugin for YUI Editor has been included in drupal CVS and the 6.x-2.33 release. Now I can claim to have code included in an official drupal release, ok it is a small plugin for a contrib module, we all have to start somewhere.

The version included in Drupal only supports YUI 2.5.x as the API has changed in 2.6. I have a new version which supports 2.6.x, but it has a layout bug, so I won't be submitting it until this bug is fixed. If you can tolerate the visual bug or want to help fix it, grab the lastest version of the IMCE plugin for Drupal's YUI Editor. Use the same installation instructions as last time.

Feedback welcome.