Member's Areas and wget

Earlier this evening I was discussing mirroring restricted areas of sites with wget on #ubuntu-au. The solution is pretty simple.

  1. Install the web developer extension for firefox
  2. Login to the target site
  3. On the webdev toolbar select Cookies > View Cookie Information
  4. For each of the cookie entries add the following to a file called wget-cookies.txt which should be saved in your home directory

This is what it all means

  • <> the domain of the site
  • TRUE the domain wide flag, if the domain starts with . this should be TRUE
  • / the path the cookie applies to
  • FALSE is the cookie secure (or available via HTTPS)
  • 1496836642 the expiry of the cookie (i am using 11:57:22 UTC on 7-Jun-2017)
  • <key> the name of the cookie
  • <value> the value of the cookie

If you just want to pull down a single page use the following command:

wget  --load-cookies ~/wget-cookies.txt <target-url>

Then you should have the target page

If you want to mirror the whole site as an authenticated user try something like:

wget --mirror -w 2 -p --convert-links  --load-cookies ~/wget-cookies.txt <target-url>

I tested this with a couple of my own sites and it seems to work well.

Before doing something like this, check the term of service and the license of the content to ensure that you are not in violation of either.

Yamakasi and Maccas

Some time ago I watched a documentary on Yamakasi, which was shown here on SBS TV. SBS's press release gives a pretty good summary of the doco. For those of you interested in seeing it, a quick search on Google, should turn up a link to it on Google Video - I would post a link to it, but I am unsure if it was posted/approved by the copyright holder.

The doco was pretty impressive. Some parts of it were shot in Évry (just outside of Paris), where Benoit lives and I stayed last year, so a couple of places looked a little familiar. The guys (I think they were all guys) in the video have some pretty amazing ways of getting around. They scale walls, jump between tall buildings and generally have a good time getting between point A and B. Even if I had the physical strength to do half the stuff I saw them doing, I am not sure I would have the mental strength to jump between the tops of 10 storey + buildings.

I first heard about this kind of stuff from a friend of mine, he was telling me about Le Parkour. It all sounded pretty cool, albeit a bit far fetched, so much so that by the end of it I thought he was talking crap. Although Parkour is similar to Yamakasi, in that they believe in not letting physical structures impeded movement through the (urban) environment, there are some differences - primarily Yamakasi more into making it look good. I am sure for people in Australia, the Australian Pakor Association could explain the differences in more detail. There seems to be some other politics involved too, but Wikipedia is hardly an authoritative source for that kind of info.

Tonight I was watching "mainstream" commercial TV and I saw an ad which from about 5 seconds in was pretty obviously for Mc Donald's breakfast. The basic plot is that a young guy wakes up at 10:25 and so he uses Yamakasi style moves to get to Maccas by 10:30 (when they stop serving breakfast at 10:30), this is interspersed with images of "breakfast" being cooked. This isn't the first time a large multi national (has used underground/fringe culture to sell their product, but I did find this one rather shocking. Sorry no links to the ad, this post already adds to word of mouth marketing I am sure they were hoping for.

Being vegetarian, the ad didn't encourage me to head down to Maccas for a egg and bacon roll tomorrow morning, while jumping a few buildings and scaling a wall or 2 to get there. It make we want to watch the doco again though.

Coding has begun and now I get mentoring

Johan has already started coding for his Google Summer of Code project for the GNU Project and phpGroupware. This morning I received confirmation from the FSF that his copyright assignment has been sorted.

I have done a quick review of some of Johan's work, but I hope to spend some serious time on getting things moving this weekend. The new job has really put a big whole in my time available for anything other than work, commuting or my family.

When I read Federico Mena-Quintero's Summer of Code Mentoring HOWTO, I was working pretty flexibly from home and I thought mentoring was checking over code and answering a few questions. My new job has taught me that good mentoring can make a big difference for someone coming into an organisation.

I hope over the next week or so I can establish a good routine which gives me enough time to help Johan find his way and get on with hacking on sync.

The Disruption Caused by Blackouts

In the last 6 months I have endured 2 blackouts at home (which is also my office). Yesterday's was about 13 hours long, easily beating the previous record of almost 8 hours. The power supplier has given me the information on how to make a claim for the food in our freezer which has to be thrown out.

Now that I am working away from home for large parts of my day, I really need a working internet connection in the evenings so I can run the rest of my business. I can use my data card and laptop battery to work at reduced efficiency during the short outages, but anymore than 2 hours and it starts cause major disruption. The problem is that I use my laptop as a MP3 player while catching up on mail and feeds on the train, so my battery is almost dead when I get home.

Although I live a bit over 35kms from the CBD of the second largest city in Australia, theree are times where I feel like I live out the back of nowhere. I have an electric train station 2 mins walk away, I have a "03 97XX XXXX" phone landline number and a 24hour supermarket less than 5mins drive away. On the downside, couriers considering us country, so "next day delivery" means it will arrive 2 or 3 days after it being shipped. The other big reminder is the reliability of electricity supply.

In the last 3 and half years shortish blackouts (1 to 2 hours) have occurred on average once every 3 months or so. When I was living in Footscray (inner Western Melbourne), in almost 5 years, we had 2 outages I remember, both of them less than 2 hours, and one was from someone wrapping their car around a pole just down the road.

The compensation regulations (PDF) (p21) which apply to power companies in Victoria are completely inadequate and encourage poor maintenance. In the past our electricity retailer has claimed that they are not liable for any compensation claims where the outage is caused by matters outside their control, such as possums shorting out lines, cars hitting poles or various other natural causes. I think inadequate staffing levels is something within their control.

Maybe the compensation claims from the local strip shopping centre traders will force the distributor to improve their response times, rather than being rude to customers who call for an ETA.

The following items have been added to the household shopping list:

  • Battery powered lantern
  • Battery powered radio

I just wish I could buy a more reliable power supply for a rental property (so solar panels on the roof is out).

Update Earlier this morning (31-May-2007), I had another 1 hour+ outage. This time the network didn't come back up cleanly. A switch needed to be power cycled and another server decided to swap the network card allocations.

Ubuntu on Dell PCs and XP License Refunds

Now that Dell is shipping Ubuntu loaded machines in the US, there has been some discussion on the Australian Ubuntu LoCo list about when Dell will be shipping them in Australia. The consensus seems to be, not any time soon. I tend to agree, there are many other larger markets Dell is likely to target if Linux/Ubuntu on Dells takes off.

Tom Schinckel mentioned that he is waiting for Dell to offer Window license refunds. I have already obtained one.

About 2 years ago I bought my Dell Latitude D810, which came preloaded with XP Pro. During the order process I asked twice if the OEM copy of XP would run under QEMU (after explaining what it was), I was told both times it would. I was pretty suspicious. After receiving the machine I set it all up, including loading ubuntu on it - I can't remember if I used Hoary or a Breezy Colony. Then I installed QEMU and tried installing XP from the OEM media. So far so good, until it came time to supply the license key. I kept on being told it was invalid and to contact Microsoft. I checked on #qemu on freenode and was told that the Dell OEM version of XP is BIOS locked to Dell kit. No problem, I called Microsoft who (after a 20min hold) referred me to Dell. Another long hold and I was told no go. I started asking for a refund and was transferred to various places. After a while I mentioned that Australian consumer law applied and the product was not functioning as advertised, so I wanted a refund on the faulty part. I was eventually offered a full refund if I returned the whole machine. I wasn't taking that option. Instead offered to contact the media and see if they could be of any assistance, I was then told I would receive a call back in a few days.

About a week later I received a call back from Dell. They made a final offer, I could keep the XP license and they would give me a "goodwill credit" of 150AUD. As I could get XP Pro OEM licenses for about 190AUD, wholesale, I pushed for that. I was told that Dell paid less than 150AUD per XP Pro OEM license, so I could take 150AUD or take nothing. I soon realised that this was the best I was going to get, so I took it. I wasn't required to return the license key as "Dell has no way of cancelling or reusing a license key".

For a bit over 1 hour of effort on the phone, I ended up with an XP Pro OEM license which could be used for dual booting(if I ever wanted to), and 150AUD refunded to my credit card. I am sure Dell doesn't think this sets a precedent, but the NSW Office of Fair Trading might beg to differ. As Dell Australia is based in NSW, NSW consumer law applies to all transactions.

Disclaimer: I am not a lawyer, this is not a legal opinion. I am not responsible if Dell (or anyone else) declines your request for a refund/credit based on your use of the information contained in this post. blah blah, you can't sue me :)


$ cat hello_world.c #include <stdio.h> main() { printf ("Hello World!\n"); }
$ gcc hello_world.c -o hello_world
$ chmod +x hello_world
$ ./ hello_world
Hello World!

I have started playing with C recently. I am still finding it annoying, yet enjoyable at the same time. The PHP developers have spoilt me (and other PHP coders). PHP gives you most of the power of C, without having to deal with annoying string handling, easy arrays, memory management and having to build to test. I am sure there is more.

Unlike OS X, I think I will persist with C.

Other than reminding myself of how to do hello world in C, why did I post this? I thought you would never ask, it is because my blog is now syndicated on Planet Linux Australia. I am pretty honoured by being added. Hopefully there are others who appreciate my rants :)

OS X and Macs - the Windows killer?

For the last week I have almost exclusively been using a PowerPC Mac - claimed by Apple to be a great platform just a few years a ago. Personally, I think that Mac OS X is an interesting platform. The mac hasn't grabbed me.

On the up side, OS X (and Darwin) is based on BSD, so it has some good security foundations, it also uses many tools common to Linux, such as bash and CUPS. The 3D desktop effects are kinda cool for the first day, but then just become part of the day to day experience. I am yet to see a real advantage to the OS X 3D desktop.

The Mighty Mouse is pretty slick. The scroll wheel feels very nice and is well positioned. The side buttons for expose are addictive on the first day. A let down is that you have to change your preferences to enable the right button.

I don't claim to understand the whole Mac software management system, but from what I do know, you drop a disk image (a dmx file) into the applications folder in finder and it is installed. Want to remove it? delete the folder. This is pretty neat, once you understand how it works. It reminds me of the klik package management system.

The file open dialog is a crazy hierarchical beast, that works. Jumping between levels in a tree really works. Pity more than 3 levels down it can involve some vertical scrolling and you need to select a file to get its full name if it is too long.

Now for the downsides of using a Mac running OS X.

The keyboard feels awful, this is one of the times I would recommend a Microsoft product, but as MS keyboard feels far better than an Apple Keyboard. The standard mac keyboard feels plasticy and the key travel doesn't feel right. I have used a range of keyboards over the years and the Max keyboard feels awful. Maye apple should rebrand Logitech's kit, like Microsoft does.

My next complaint is key bindings. For ever since I remember, [home] takes you to the start of current line and [end] takes you to the end of the current line. Many apps even ignore the [home]/[end] keys. Windows, GNOME and KDE all bind [alt] [F4] to close window - but not the mac. There are many other standard combinations ignored by Apple. Another annoyance is the apple key - for most things it functions like a [ctrl] on a PC, but not in a shell, then it functions like an apple key and [ctrl] functions like a [ctrl] key under *nix - I have lost track of how many windows i have closed when trying to delete a word in the console (bash fiends know what i mean).

Inconsistent use of key combinations. In the console and some other apps, [apple] [arrow] loops through the windows of the application, but not Apple Mail, it has decided that the combo expands/collapses message threads, very annoying when trying to compose a message while trying to copy and paste from another.

The maximise button doesn't actually maximise. I am not sure if it is up to the application or the window manager, but clicking maximise (the green circle) may increase or decrease the width or height of the window. When I click maximise, I expect the window to be maximised - or at the very least increased in dimensions.

The real deal breakers for me are the [home]/[end] keys, the inconsistent shortcuts and other crazy behaviour of OSX mean that I won't be switching to a Mac anytime soon.

Over the next week I plan to load more FLOSS on the mac, such as Mozilla Thunderbird for email, which will join Mozilla Firefox web browser and gvim - my referred text editor. I doubt this will be enough for me to stick with OS X.

The indigo iMac G3 I landed last week is likely to be running Copland (a PPC port of xubuntu) real soon now. I am still trying to work out what I do with Julie's Apple Powerbook G3, which currently runs Xubuntu 6.06.1 LTS, as ubuntu has dropped support for PowerPC in feisy. Maybe I can find other PowerPC machines to install Copland or Debian onto :)

I am yet to see how OS X is more user friendly and and easier crossgrade path for windows users than a Linux desktop.

Most homes don't really have broadband in Australia

I am bashing out this post on a unreliable 384/64kpbs UTMS (3G data) connection, which feels painfully slow at times compared to my 8000/384kpbs (which is more like 6500/300) ADSL connection at home. Even when I had 1536/256kpbs at home I felt like I was better off than a lot of other people. Most non geeks I know have 512/128kbps.

According Democrats in the US congress 2Mbit/s should be the minimum speed for "broadband". Although that is now available to many homes in Australia, the cost of anything faster than 512/128 isn't seen as value for money by many people. I know several people who would like a lower quota and higher speeds for the same price as they are paying now. Even for many businesses (less than 5 people), they see 512/128 as being good enough for email and web browsing. 2048/512 would allow business to use new services such as VoIP and cheap video conferencing, which could improve the level of service they offer to their customers while improving the bottom line.

The biggest barriers to affordable high speed connections in Australia is Sol Trujillo and the other untrustworthy Wiggles at Telstra. Hopefully if there is a change of government at the next Federal election, the "Group of 9 Fibre to the Node network might get built and be open to competition.

Who is Planet Summer?

Until tonight I didn't know who Planet Summer was. It turns out she is a young model, who doesn't wear a lot.

Many people wanting to find Planet Summer have been ending up on my site. A google search for planet summer lists my blog post about Planet Summer of Code.

Hint look higher up the search results if you are looking for the model's site.

Using Technology on the train

It has been many years since I have caught peak hour trains. Last time I did it was about 5 years ago when I was working at the City of Darebin. Back then I used to lug a heavy Compaq Armada (I forget the model number) home most nights, but I rarely used it on the train. MP3 players weren't worn as fashion accessories back then.

These days I have a Dell Latitude D810, which has a 15.4" widescreen screen, which runs well at 1920x1200, but it is a too big to use comfortably on the train. The UMTS/GRPS data card hanging out one side and the headphones lead out the other take up even more room. I have quite broad shoulders, so it very hard to type on a laptop which is as wide as my lap, with people sitting either side of me. I have also noticed that my laptop screen becomes communal reading material, just like a newspaper. Maybe I should offer to come to their work and read their emails and monitor their web surfing while they are at work.

Time for me to return to listening to psy trance and catching up on my feeds in liferea.