I am Peter Breuls. I write web applications in PHP, movie reviews and irregularly something on this weblog. Welcome!
Through my company Devize, I'm available as a developer or a consultant for websites or web applications.
I work as an Administrator at online community FOK! and as a Lead Developer at frontoffice supplier SIMgroep.

Suicidal mailserver

Linuxcomment

DoveCot - kill message

I reset the time on our mailserver, because NTP wasn't working (or at all running, I guess). Immediately my mail client started complaining about the connection. The above error message was the cause. It's both funny and explanatory. I like it.

Using Lighttpd for exceptional performance

Linuxcomment

To increase performance at the site I work for, we started separating static webfiles from dynamic pages using a separate hostname and web server for static files a while ago. For this, we placed almost all of the images, style sheets and Javascript files on a separate server and installed Cherokee for light and fast web serving on that server. And it worked. Loading of images became easier, and we could apply some of the server-related suggestions from Yahoo!'s Exceptional Performance.

However, Cherokee didn't always try its hardest to serve us well: sometimes it decided to pull the server's load to a performance-decreasing number, configuration of much needed features was "not yet implemented" or buggy and the software is overall not done yet. Also, the developers have, weirdly, decided to use exclamation marks as separators within the configuration, which is just plain annoying.

So we started looking towards other solutions. And we decided to go back to our other option before we chose Cherokee: Lighttpd. With Lighty, as it's pronounced, we can apply these performance rules:

» Add an Expires or a Cache-Control Header
» Gzip Components
» Configure ETags

I'd like to show you which configuration options we used. The following applies to a default Lighttpd (version 1.4.19) as shipped with Ubuntu 8.10. If you're using a different OS or if you've downloaded Lighttpd yourself, I assume you understand enough of it to translate the following to your own situation.

Add an Expires or a Cache-Control Header
For this, we need mod_expire. With that module, we can instruct Lighttpd to include both the "Expires" and the "Cache-Control" headers in the response. To enable the module, we uncomment this line:

# "mod_expire",

This line is mentioned in a list of modules at the top of lighttpd.conf. If it's not, you should look for the list somewhere else in the file and uncomment the line or simply add it, but without the hash sign.

Next, look up a line that says 'expire.url'. It should be there and commented by default. Uncomment it and configure it to do what you want it to do. For us, Lighttpd is entirely dedicated to serving static files, which all need to be cached by the client for a long time. Let say, for documentation's sake, we choose two weeks as the caching time. This would then be our configuration:

expire.url = ("/" => "access plus 14 days")

This leads to these two headers when a URL from the Lighttpd-server is requested:

Expires: Thu, 25 Dec 2008 13:30:31 GMT
Cache-Control: max-age=1209600

And that's it! Enable the 'expire' module, configure the expire time, and you're done.

Gzip Components
To apply gzip encoding of the repsonse body, we use mod_compress, which is enabled and confgured by default. However, not everything we want to compress is actually being compressed, so we change this:

compress.filetype = ("text/plain", "text/html", "application/x-javascript", "text/css")

into this:

compress.filetype = ("text/plain", "text/html", "application/x-javascript", "text/css" ,"image/jpeg","image/jpg", "image/gif","image/png")

Over time, we might add other mime-types we forgot to include, but the above covers most of the requests.

When requesting an image with any regular browser, it will be sent to us, gzip-encoded. If you're using FireFox with the Live HTTP Headers extension, you can find these in the response headers:

Vary: Accept-Encoding
Content-Encoding: gzip

This confirms that it's working.

Configure ETags
This is the easy one. In our default installation, a request to the server gave us this as one of the response headers:

ETag: "2832627283"
Last-Modified: Wed, 10 Dec 2008 13:16:32 GMT
Content-Length: 1395

Because we currently use one server for the static serving, this is all we want. So we're done.

And that concludes the confguration of Lighttpd as the optimal static-files-webserver.

Tools of the trade

Linuxcomment

It's always fun to compare tools. Who works with what and especially, why? Following the example of Flickr and some others, let me list my tools, see if you match:

Working:


  • Main machine: MacBook Pro. I have an Ubuntu PC, but that's just 'extra'. I do everything on my Mac, from working to living.

  • Editors: Zend Studio 6 for all the main development tasks, completed by TextMate (and the handy 'mate' cli-command) and vim for serveral minor things.

  • Transmit: used for access to (s)FTP code locations, and to manually check whether (s)FTP import applications do what they should

  • iTerm with usually about six tabs. I traverse folders, grep through them, use CVS/SVN commands and access MySQL from the commandline. And of course I connect to development and production servers using ssh, but that goes without saying.

  • MySQL Query Browser: I can usually do what I want by just using the commandline client, but every now and then I need a little more visual help.

  • Zend Core: used as an all-in-one package for Apache and PHP. I also use MAMP to run a good old PHP4 environment because at one of my employers we're still in the midst of upgrading to PHP5 (I know, shut up).

  • Xdebug: I use it for profiling and I love the way it adapts var_dump() to a more usable way of displaying variables

  • FireFox and FireBug: very important indeed. I can't image having to work without FireBug. I still remember trying to think really hard about my HTML/CSS and placing alerts in my JS as a way of doing some poor-mans-debugging. FireBug is a godsend.

  • YSlow: a man needs performance, and YSlow helps me determine what to do. Very nice!

  • CSSedit: editors for CSS don't do a lot more than text editors, but they help a little and a little is enough.

  • OPML Editor: I keep my notes, todo's and more in outlines. The best outline editor used to be the one from Radio UserLand, until Dave Winer took the tool and released it apart from the weblog editor.

  • VMWare Fusion: although I love working on my Mac, I'm still missing what I already mentioned before: the combination of Krusader and Kompare (and to a lesser degree, Cervisia) for development work. For that, I am trying out using an Ubuntu virtual machine which uses the three beforementioned apps and sshfs to mount the (development) servers I'm working on. Works like a charm!

Living and working:
I take my Mac everywhere. I work on it at work, even though it is a private machine. At home, I use VLC to watch video's and DVD's, NetNewsWire for the daily read, Celtx for screenwriting, Mail.app for.. well duh, RealPlayer to listen to BBC Radio 1 or iTunes for my music collection, Twitterific for Twitter and Unison to eh, browse newsgroups.

Well, that's most of it. What about you?

Wubi: the new way of dual-booting

Linuxcomment

When I work, I usually use two computers: an Ubuntu-powered PC and my MacBook Pro. Of those, the Mac is my main machine: I have all my development tools and environments running on it, I use it for mail and documents, my music is on it (because I can't work in silence), etcetera. It has this value to me because I can carry it to wherever I need to, which means I can use it at both of my two jobs, plus on location if I ever need to (and sometimes I do).

But there's one thing missing: although Mac OSX is a very nice OS, probably the best I've worked with, I still need Linux to complete my wishlist. The combination of Krusader and Kompare, for instance, is a golden one if you need to organize your development projects. Krusader is a two-pane file manager, with the option to use FTP and SFTP or just local files. Whenever I need to examine the difference between two versions of the same file (and I need to do that a lot: before commiting my changes to version control, I want to know exactly what I'm doing), I set up Krusader to have a folder with one version of the project on the left and one version on the right. I then select the files I want to compare and the magic happens: Kompare starts.

Kompare is a frontend to the diff tool and as such not that special: if I just want to quickly see the difference between foo.php and it's current state in CVS I can just use the built-in comparison tools from Zend Studio for Ecplise. But: Kompare is so much better! It has coloring to differentiate between additions, removals and changes, is more precise (due to the tried-and-tested diff being the backend) and has the ability to apply some of my changes to the other file, which enables me to be more exact in what I commit (you know, sometimes you have several changes in a file, but just a few of them belong to that relevant bugfix and the others are for that new feature that's still unfinished, so you want to do a partly commit).

So, I need Linux next to Mac, because those tools don't run on Mac OSX. When at home (for Job #1), that's no problem. My only PC is an Ubuntu machine. But at work (at Job #2), I have an old PC which is kinda slow which I use for this. And the slow part, that's annoying. Because whenever I need to use Kompare, I need five to ten minutes to boot the thing, and every action after that needs a lot of patience.

But, I share my desk with a colleague who sits there when I'm at Job #1. And he has his own PC, stalled under the desk next to mine. A new, fresh one, nice and fast, running Windows, and I already discussed making that one a dual-boot so we can share not just the desk, but the PC as well. But I don't have the time to go and install Ubuntu next to Windows, carefully selecting partitions, making sure I don't nuke his installation, and whatnot. So even months after "Hey, can I make that one a dual-boot?" - "Sure!" I'm still working on the slow machine.

Enter Wubi. Shipping with Ubuntu 8.04 next week, it's an Ubuntu installer for Windows. And it does exactly (exactly!) what I need:

When I rebooted my machine, an option to boot Ubuntu was added to my Windows boot list, and after selecting it, Ubuntu started loading just as it would if installed on a dedicated drive. I was even given the normal GRUB menu.
(Linux.com)

So now I can just boot my colleage's machine, put in the Ubuntu CD, click through the installer and enjoy Ubuntu. No need for me to run the Ubuntu installer, carefully selecting drives, doing all kinds of stuff that costs me time. Just click-click-install, the Windows way.

Life really does get better with every Ubuntu release.

Ubuntu Landscape

Linux1 comment

Canonical: "Landscape provides users with a hosted web interface on which all machines are registered. From this single interface, packages and security updates are deployed to the entire network of servers and/or desktops with a single click. Additionally a wealth of monitoring data is provided graphically to the administrator showing process and resource use as well as flagging any available security fixes for the system."

Wow. Cool. Seems like a real time saver; currently, when I have to apply changes to a group of machines, I find myself logging in and out of each of them and that's not very funny when you know your server farm increases in size regularly...

Software developers: please log everything

Linuxcomment

Okay, this annoys me. Since a few hours, the wireless connection on my MacBook Pro is gone. No reason given. Of course I'm trying to connect back, while I'm using Google to find solutions when it seems I can't solve this on my own.

MacBook ProThe problem is, my Mac recognises my wireless network. It can see my accesspoint, read its name and it asks me kindly if I would like to connect to it. Sure I would, so I press Yes. It then tries a couple things and comes up with the message that tells me connecting failed. No reason given.

Being both a Linux desktop user and Linux server admin, I have seen my share of failing pieces of software or equipment, and one of the reasons I like Unices to much is that there's a whole bunch of logiles being kept in the /var/log folder, providing me with the how and why of these errors. Mac runs on a Unix-type OS, so during my wireless-problem I'm listing the files in that folder, sorted by date and time, and I'm using tail to view the latest entries in what seems to be the only active log file: system.log. Indeed it shows me some errors, but there's nothing corresponding to the pop-up message that told me the connection to my accesspoint failed. Nothing! Not a single logfile seems to have updated with information I can use to solve this problem.

And I've seen that before: software that's dealing with its problems all on its own, without asking for help or enabling the user, me, to find out more using Google or my own knowlegde. That is a mistake. If you create software, make sure you have nice friendly error messages for the average user. That's important. But the "extra mile" in this is just as important: make sure you write every error into a log file and point me to it in case it's not system.log (or syslog or messages or anything that can be considered a default). Make sure enough information is in there to enable me to exactly understand what's going on. I am a technical user. I understand technical information if it's described in a human-readable way. I can think for myself and come up with solutions, or spend time pasting the log entry in search boxes to I can find other people's solutions.

To me, having enough information is very important. So, software developers, please log all those information somewhere.

Ubuntu is here

Linuxcomment

The new Ubuntu is out. It's version 6.06 LTS, in which LTS stands for Long Term Support. For desktops, this means that for 3 years, packages wil be kept up-to-date with security patches. For server users this is 5 years.

Ubuntu logo

I run a few Ubuntu machines myself. Some desktops, some servers. My desktop PC at home was the first one. It started with Ubuntu Hoary, now runs Breezy and in a few days or weeks, when I have time, it will be upgraded to Dapper, which is the code name for 6.06.

Ubuntu is easy to use, based upon Debian, which rocks, and alltogether a great platform for any desktop or server. Really. You should try it!

Ubuntu package updates

Linuxcomment

I'm looking for a page on the Ubuntu website that tells me which software packages have recently updated. Or something like that. I can't seem to find such a page. Surely there must be something like that, but where is it?

Passing the HELO check with Postfix

Linux1 comment

I run several webservers in a pool. Guided by a loadbalancer, they serve over 2 million pages a day, so they're quite busy altogether. One of the applications running as webserver-service sends mail to users. That's fine, because with PHP/Apache and Linux, that's easy to do.

However, my server logs keep on showing errors concerning these sent e-mails. They get bounced back by mail transfer agents who are very strict in their spam control. The messages all point to this page., saying my server did not pass the HELO check.

With the HELO check, an SMTP server can check whether the sending server is properly configured, and thus reliable. It does so by reading the HELO name the server sends, and checking if it's a valid hostname or IP.

While the explanation is quite clear, it didn't help me much. It gives some suggestions on how to fix my problem, but the configuration options for Postfix are missing. That's why I'm posting them here, for Google to find.

In your main.cf file, find the myhostname setting. By default, it will have something like localhost.localdomain as its value. (That's what caused my servers to not pass these helo checks, because localhost.localdomain is not a valid hostname.) Change the value into something that can be resolved to the server itself, like the actual hostname of the server. In my case, one of the servers has aphrodite.fok.nl as its hostname, so I changed the setting into:

myhostname = aphrodite.fok.nl

Save the file, give a postfix reload command, and you're good to go.

Yakuake!

Linux1 comment

This is cool: Yakuake, a KDE console application. Instead of opening just another application that lives in your task bar, it hides itself, to be shown when you press F12 (or a key you assign to it yourself). It drops down from the top when you press the key, let's you do the work, and rolls up again when you're done. Quite nifty!

KDE 3.5 Released

Linuxcomment

Seems like I've got something to do tonight at home: KDE 3.5 Released Currently, I'm using version 3.4.3 in the (K)ubuntu Breezy version. There's a separate KDE 3.5 repository for Ubuntu users:

deb http://kubuntu.org/packages/kde35 breezy main

Just add it to /etc/apt/sources.list, update and upgrade.

By the way, the KDE web site has a visual guide which explains some of the new features. It makes clear what's hot and new in KDE, in an easy way.

Mirdir can backup directories

Linuxcomment

Linux and Open Source Blog:"Mirdir provides a quick and easy way to make an ad hoc backup of important data. With it you can copy a file or directories to your keydisk, or save redundant copies of data you can’t afford to lose. It tries to do only one thing, and do it well: mirror a directory."

That's a nice one to remember.

Default crontab editor

Linux7 comments

I was working on some sysadmin tasks on a freshly installed server, and I found out the crontab editor was not what I expected it to be. I really like vi (or vim) for command line editing, so I wanted 'crontab -e' to use vi instead of something else. To make that happen, I put this into my bashrc file (location: ~/.bashrc):

export VISUAL=vi

I keep forgetting this, so I decided to blog it. Don't forget to reload the bashrc by doing this:
source ~/.bashrc

And vi will be at your service!

Editor's choice: Ubuntu

Linuxcomment

I like Ubuntu. It's my operating system of choice, both at work and at home. The people at ZDnet seem to agree: they tested some Linux distros, and Ubuntu Breezy came out as the winner.