php

Keyword Bookmarking

When doing development work, from time to time it is handy to be able to look up documentation. Bookmarking manuals is handy, but often you still need to search for the function you're after. Firefox, and possibly other browsers (not Chrome or Chromium), allows you to setup a keyword bookmark linked to a search.

I've setup a few search bookmarks for development resources. This is how I've done it:

  1. Select Bookmarks > Organise Bookmarks... from the menu.
  2. Right click on the bookmarks menu (or any folder) on the left pane
  3. Select New Bookmark... from the context menu
  4. Drupal bookmark example
    Fill in the information for the bookmark, the import piece is the keyword, that will allow us to search.
  5. Click save and return to the browser

Now when we want to search the Drupal 7 API, we can just type "dapi Example Drupal API search in location bar

Now we should see the appropriate page from the Drupal API documentation.

Example Drupal API page

The same method can be used for other PHP web app developer resources, here are some which I'm using.

I could have implemented this using OpenSearch plugins, but that requires changing the search provider every time I want to look for something. By using keyword bookmarks I just type the keyword and the search term into the location bar.

Feel free to share your keyword bookmarks in the comments.

Making it Easier to Spawn php-cgi on Debian and Ubuntu

Apache is a great web server, but sometimes I need something a bit more lightweight. I already have a bunch of sites using lighttpd, but I'm planning on switching them to nginx. Both nginx and lighttpd use FastCGI for running php. Getting FastCGI up and running on Ubuntu (or Debian) involves a bit of manual work which can slow down deployment.

The normal process to get nginx and php-cgi up and running is to install the spawn-fcgi package, create a shell script such as /usr/local/bin/php-fastcgi to launch it, then a custom init script, after making both of these executable you need to run update-rc.d then finally should be right to go. Each of these manual steps increases the likelihood of mistakes being made.

Instead, I created a deb contains a configurable init script. It is pretty simple, the init script calls spawn-fcgi with the appropriate arguments. All of the configuration is handled in /etc/default/php-fastcgi. The main config options are:

  • ENABLED - Enable (or disable) the init script. default: 0 (disabled)
  • ADDRESS - The IP address to bind to. default: 127.0.0.1
  • PORT - The TCP Port to bind to. default: 9000
  • USER - The user the php scripts will be excuted as. default: www-data
  • GROUP - The group the php scripts will be executed as. default: www-data
  • PHP_CGI - The full path to the php-cgi binary. default: /usr/bin/php5-cgi

The last 3 variables are not in the defaults file as I didn't think many users would want/need to change them, feel free to add them in if you need them.

Once you have set ENABLED to 1, launch the init scipt by executing sudo /etc/init.d/php-fastcgi start. To check that it is running run sudo netstat -nplt | grep 9000 and you should see /usr/bin/php5-cgi listed. Now you can continue to configure your webserver.

The package depends on php5-cgi and spawn-fcgi, which is available in Debian testing/squeeze, unstable/sid, along with Ubuntu karmic and lucid. For earlier versions of ubuntu you can change the dependency in debian/control from spawn-fcgi to lighttpd and disable lighttpd once it is installed so you can get spawn-fcgi . I haven't tested this approach and wouldn't recommend it.

You can grab the http://davehall.com.au/sites/davehall.com.au/files/php-fastcgi_0.1-1_all.deb">binary package and install it using dpkg or build it yourself from the source tarball.

For more information on setting up nginx using php-cgi I recommend the linode howto - just skip the "Configure spawn-fcgi" step :)

Solr Replication, Load Balancing, haproxy and Drupal

I use Apache Solr for search on several projects, including a few using Drupal. Solr has built in support for replication and load balancing, unfortunately the load balancing is done on the client side and works best when using a persistent connection, which doesn't make a lot of sense for php based webapps. In the case of Drupal, there has been a long discussion on a patch in the issue queue to enable Solr's native load balancing, but things seem to have stalled.

In one instance I have Solr replicating from the master to a slave, with the plan to add additional slaves if the load justifies it. In order to get Drupal to write to the master and read from either node I needed a proxy or load balancer. In my case the best lightweight http load balancer that would easily run on the web heads was haproxy. I could have run varnish in front of solr and had it do the load balancing but that seemed like overkill at this stage.

Now when an update request hits haproxy it directs it to the master, but for reads it balances the requests between the 2 nodes. To get this setup running on ubuntu 9.10 with haproxy 1.3.18, I used the following /etc/haproxy/haproxy.cfg on each of the web heads:

global
    log 127.0.0.1   local0
    log 127.0.0.1   local1 notice
    maxconn 4096
    nbproc 4
    user haproxy
    group haproxy
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    retries 3
    maxconn 2000
    balance roundrobin
    stats enable
    stats uri /haproxy?stats

frontend solr_lb
    bind localhost:8080
    acl master_methods method POST DELETE PUT
    use_backend master_backend if master_methods
    default_backend read_backends

backend master_backend
    server solr-a 192.168.201.161:8080 weight 1 maxconn 512 check

backend slave_backend
    server solr-b 192.168.201.162:8080 weight 1 maxconn 512 check

backend read_backends
    server solr-a 192.168.201.161:8080 weight 1 maxconn 512 check
    server solr-b 192.168.201.162:8080 weight 1 maxconn 512 check

To ensure the configuration is working properly run

wget http://localhost:8080/solr -O -
on each of the web heads. If you get a connection refused message haproxy may not be running. If you get a 503 error make sure solr/jetty/tomcat is running on the solr nodes. If you get some html output which mentions Solr, then it should be working properly.

For Drupal's apachesolr module to use this configuration, simply set the hostname to localhost and the port to 8080 in the module configuration page. Rebuild your search index and you should be right to go.

If you had a lot of index updates then you could consider making the master write only and having 2 read only slaves, just change the IP addresses to point to the right hosts.

For more information on Solr replication refer to the Solr wiki, for more information on configuring haproxy refer to the manual. Thanks to Joe William and his blog post on load balancing couchdb using haproxy which helped me get the configuration I needed after I decided what I wanted.

Group Redent Plugin for Status.net / Identi.ca

For a bit over a year now I've been using the free software based microblogging service identi.ca. Unlike twitter, the identi.ca code base is released under the AGPLv3, through the status.net project. Anyone is free to setup their own instance of status.net and can even federate with other instances, using the Open Micro Blogging standard.

Another feature which identi.ca/status.net has over twitter is support for groups, so someone can send a message to a group of people, a bit like how a mailing list works. One down side of this feature is that some people redent (like retweeting, but on identi.ca) without removing the group "bang tag" (aka !). This leads to the same message being sent to the group over an over again which gets annoying. I left !linux and have considered leaving !ubuntu, because of how often this happens.

As status.net is freely available, I decided to write a plugin which handles this problem. If someone redents a message to a group without changing the group tag to a hash tag, the plugin does it for them. It is like the status.net twitter bridge when reposting dents to twitter. If someone really wants to redent something to a group, they simply use !! instead of ! and it will be sent to the group. This is useful, if for example, a dent from someone is relevant to a group, but wasn't posted there initially, it can be redented to the group

I disagree with Bela Hausmann (aka @and3k), I think it should be enabled by default on identi.ca. I hope others consdier it to be a useful feature and ask for it to be added to identi.ca.

If you run your own instance of status.net running, http://davehall.com.au/sites/davehall.com.au/files/GroupRedentCleanerPlugin.php.txt">download the plugin, drop it in

/path/to/statusnet/plugins/GroupRedentCleanerPlugin.php

and add the following to your config.php: addPlugin('GroupRedentCleaner', array()); Have a play!

Packaging Doctrine for Debian and Ubuntu

I have been indoctrinated into to the everything on production machines should be packaged school of thought. Rather than bang on about that, I intend to keep this post relatively short and announce that I have created Debian (and Ubuntu) packages for Doctrine, the ORM for PHP.

The packaging is rather basic, it gets installed just like any other Debianised PEAR package, that being the files go in /usr/share/php, the package.xml and any documentation goes into /usr/share/doc/<package>, and the tests are stored as examples in /usr/share/doc/<package>/examples. The generated package will be called php-doctrine_1.2.1-1_all.deb (or similar), to comply with the Debian convention of naming all PEAR packages php-<pear-package-name>_<version>_<architecture>.deb. I have only packaged 1.2.1, but the files can easily be adapted for other versions, some of the packaging is designed to be version agnostic anyway.

To create your own Doctrine deb, follow these steps:

  • Create a directory, such as ~/packaging/php-doctrine-1.2.1
  • Change into the new directory
  • Download my debian/ tarball and extract it in your php-doctrine-1.2.1 directory
  • Download the PEAR package tarball from the project website and extract it in your php-doctrine-1.2.1 directory
  • If you don't already have a standard Debian build environment setup, set one up by running sudo apt-get install build-essential
  • To build the package run dpkg-buildpackage -k<your-gpg-key-id> -rfakeroot . If you don't have a gpg key drop the "-k<your-gpg-key-id>" from the command

Now you should have a shiny new Doctrine deb. I think the best way to deploy it is using apt and private package repository.

Update: @micahg on identi.ca pointed me to a Doctrine ITP for Debian. Hopefully Federico's work will mean I no longer need to maintain my own packaging of Doctrine.

Packaging Drush and Dependencies for Debian

Lately I have been trying to avoid non packaged software being installed on production servers. The main reason for this is to make it easier to apply updates. It also makes it easier to deploy new servers with meta packages when everything is pre packaged.

One tool which I am using a lot on production servers is Drupal's command line tool - drush. Drush is awesome it makes managing drupal sites so much easier, especially when it comes to applying updates. Drush is packaged for Debian testing, unstable and lenny backports by Antoine Beaupré (aka anarcat) and will be available in universe for ubuntu lucid. Drush depends on PEAR's Console_Table module and includes some code which automagically installs the dependency from PEAR CVS. The Debianised package includes the PEAR class in the package, which is handy, but if you are building your own debs from CVS or the nightly tarballs, the dependency isn't included. The auto installer only works if it can write to /path/to/drush/includes, which in these cases means calling drush as root, otherwise it spews a few errors about not being able to write the file then dies.

A more packaging friendly approach would be to build a debian package for PEAR Console_Table and have that as a dependency of the drush package in Debian. The problem with this approach is that drush currently only looks in /path/to/drush/includes for the PEAR class. I have submitted a patch which first checks if Table_Console has been installed via the PEAR installer (or other package management tool). Combine this with the Debian source package I have created for Table_Console (see the file attached at the bottom of the post), you can have a modular and apt managed instance of drush, without having to duplicate code.

I have discussed this approach with anarcat, he is supportive and hopefully it will be the approach adopted for drush 3.0.

Update The drush patch has been committed and should be included in 3.0alpha2.

Upcoming Book Reviews

Packt Publishing seem to have liked my review of Drupal 6 Javascript and jQuery, so much so they have asked me to review another title. On my return from linux.conf.au and Drupal South in New Zealand, a copy of the second edition of AJAX and PHP was waiting for me at the post office. I'll be reading and reviewing the book during February.

I will cover LCA and Drupal South in other blog posts once I have some time to sit down and reflect on the events. For now I will just gloat about winning a spot prize at Drupal South. I walked away with Emma Jane Hogbin and Konstantin Käfer's book, Front End Drupal. I've wanted to buy this title for a while, but shipping from the US made it a bit too pricey even with the strong Australian Dollar. I hope to start reading it in a few weeks, with a review to follow shortly after.

Got a book for me to review? I only read books in dead tree format as I mostly read when I want to get away from the screen. Feel free to contact me to discuss it further.

Essential Tools for a PHP Developer

Tobias Schlitt has just posted some slides from his talk entitled "6 essential PHP development tools in 60 minutes". I flicked the 90 or so slides in PDF format, they pretty much mirror my development environment.

Tobias left out 2 must haves from my personal list. Vim, the only editor I can use for any prolonged period of hacking (go easy emacs fanbois). Although not really a PHP tool, Firebug, is an essential tool for any serious modern web application developer,

With this environment hacking on PHP based web apps should be a breeze.

As a side note I am starting to play with git after watching Linus' Google Tech Talk on it, and I am starting to like a it, so maybe soon it will be s/svn/git for me.

A Virtual Host per Project

Not long before my old laptop got to the end of it usable lifespan I started playing with the Zend Framework in my spare time. One of the cool things about ZF is that it wants to use friendly URLs, and a dispatcher to handle all the requests. The downside of this approach, and how ZF is organised, it works best if you use a Virtual Host per project. At first this seemed like a real pain to have to create a virtual host per project. One Saturday afternoon I worked through the apache docs and found a solution - then I found it fantastic. Rather than bore you with more of my views on Zend Framework, I will explain how to have a virtual host model that requires a little work up front and is very low maintenance.

It gets tedious copying and pasting virtual host config files each time you want to start a new project, so instead I let Apache do the work for me.

I added a new virtual host config file called projects to

/etc/apache2/sites-available
. The file contains

UseCanonicalName Off

LogFormat "%V %h %l %u %t \"%r\" %s %b" vcommon

<Directory /home/dave/Projects>
Options FollowSymLinks
AllowOverride All
</Directory>

NameVirtualHost 127.0.0.2
<VirtualHost 127.0.0.2>
	ServerName projects

	CustomLog /var/log/apache2/access_log.projects vcommon

	VirtualDocumentRoot /home/[username]/Projects/%1/application/www
	AccessFileName     .htaccess
</VirtualHost>

The important bit is the VirtualDocumentRoot directive which tells Apache to map a hostname to a path. I use an IP address from the 127.0.0.0/8 range for the virtual host, so they aren't accessible to the outside world and I don't have to worry about it changing every time I check locations.

All of my projects live under ~/Projects and each one gets a directory structure that looks something like this.

[projectname]
  |
  +- notes - coding notes, like grep output when refactoring etc
  |
  +- resources - any reference material or code snippets
  |
  +- application - the code for the project
     |
     +- www - document root for vhost

There are usually other paths here too, but they vary from project to project.

To make this work there are few more steps. First enable the new virtual host

$ sudo a2ensite projects

Don't reload apache yet.

Next you need to add the apache module

$ sudo a2enmod vhost_alias

Time to edit your

/etc/hosts
file so you can find the virtual hosts. Add a line similar to this
127.0.0.2 projects phpgw-trunk.project [...] phpgw-stable.project

Now you can restart apache.

$ sudo /etc/init.d/apache2 reload

This is handy for developing client sites - especially using drupal.

Now my

/var/www/index.html
is just an empty file.

I am getting a bit bored with adding entries to

/etc/hosts
all the time. If I get around to adding dnsmasq with wildcard hosts to the mix, I will post a follow up.

This setup is based on my current dev environment (Ubuntu Hardy), but it also works on older versions of Ubuntu. The steps should be similar for Debian and derivatives. For other distros, it should work, just how to make it work may be a little different. Feel free to post tips for others in the comments.

Day 2 at PHP Unconference Hamburg

I arrived back in Bergen late last night after spending another day the PHP Unconference in Hamburg. I even managed to get one speaker to do his talk in English, which made things a lot easier for me.

My brain started to adjust to German a bit more, which made things easier than on day 1. Overall I think I understood about 25% of what was being discussed, which sound like a waste of time, but that 25% was pretty good quality. Also the discussions in the corridors was great too. At the end of the day the language spoken isn't very important when compared to the ideas shared.

For me, the only attraction of web based social networks, is to provide a backup of my addressbook online. FOSS on the other hand is a global "social network" that is real. Events like linux.conf.au, the PHP Unconference in Hamburg, Bar Camp Melbourne and other similar events are a vital part of the networks - they provide the space for us to meet and discuss ideas.

I also used the trip as an opportunity to catch up with Christian Böttger, Release Coordinator for phpGroupWare. Not only did we discuss the project, but we caught up on how business and life in general was going. It is always good to catch up with Christian, I just wish I had more than a couple of hours to spare.

My next couple of events are locked in. Wednesday night is drinks with Johan Gunnarsson from phpGroupWare, at the airport in Copenhagen. Overnight Google emailed me a confirmation for the Google Developer Day 2008 in Sydney on June 18, there is some interesting stuff on there too - less FOSS centric but still seems pretty cool.