Google Takes Another Bite

Quietly, and almost overnight, Google have moved into another market. At the time of writing there is not yet any announcement on the Official Google Blog but it looks as though Google have begun the process of unveiling maps based on their own data.

The Google Maps product that we have all grown so used to was driven by data from Tele Atlas, a Netherlands-based company, as witnessed by the map data copyright notice that appears in the bottom right hand corner of a Google Map.

But now Google Maps, in the US at least, have started to replace these with Google map data copyright notices. We should probably expect to see these changes rolled out elsewhere in due course too.

Although Google have not revealed the sources of their map data, the suggestion is that it is a by-product of the work they did in building Google Street View with additional data coming from other public domain sources, such as the rather poor quality TIGER data in the US. Interestingly, Google did blog about the introduction of the Street View Trike in the UK and how they are being used to reach those parts that they cannot easily reach by car – they even invite readers to suggest locations that are poorly mapped.

The other big map data provider is NAVTEQ who provide data for roughly 85% of the world’s in-car navigation systems as well as for portable GPS devices from Garmin and Magellan and for Bing, MultiMap and Yahoo! online maps. Last year NAVTEQ was acquired by Nokia and Tele Atlas was acquired by TomTom. I would imagine that both NAVTEQ and Tele Atlas are now feeling rather relieved.

Map Quality

One of the first indications of a change was that users had noticed a sudden degradation in the quality of Google’s maps. Roads that used to be mapped no longer appeared on the maps and roads appeared where buildings had recently been built. Anecdotal evidence suggests that the quality seems higher in those areas that have been visited by a Street View car and lower elsewhere. Google has started to aggregate data from a number of public sources and to combine them with their own map data, in particular, US land parcel data is now also visible on the maps. Some users have observed that buildings which had previously been unlabelled, for security reasons, are now clearly labelled.

Data Liberation

Google have introduced a link at the bottom of the map inviting users to report problems that they spot. They must be aware that the map quality is not as good as it used to be but they must also be confident that any failings can be rectified by their enormous user community. Perhaps this early lack of quality is the main reason why the whole process has been kept quiet. Rather hypocritically, for a company that started the Data Liberation Front whose mission statement says:

“Users should be able to control the data they store in any of Google’s products.
Our team’s goal is to make it easier for them to move data in and out.”,

Google don’t provide a mechanism for users to retrieve the map corrections that they have submitted to Google. I wonder if that will change.

Good News for Mobile Users

All of this is probably very good news for users of mobile map devices. The licence that Google had previously signed with Tele Atlas precluded the use of the map data for turn-by-turn applications. Such navigation applications are often expensive because of the extra cost of purchasing such a licence which has to be passed on to the end user. Google will now be able to move forward without being tethered by such restrictions.

An advantage to Google of owning their own data is that a mobile Google Maps application will now be able to pre-download map data to the device, enabling the maps to work where either there is no reliable mobile signal (such as in the Lake District in the UK) or in places where the cost of downloading the map data over the mobile network could be prohibitively expensive (such as when travelling abroad with a mobile data plan). Currently, on the iPhone at least, this kind of offline mapping is only available at no cost to applications that use the excellent OpenStreetMap data, such as the OffMaps application.

The value to Google of the flow back of data from mobile devices that are using Google Maps is enormous. Feedback from Android phones and other devices that allow background processing will almost certainly be used to enhance map data. As an example, consider a GPS enabled mobile device travelling in a vehicle along the road network. The Google Maps application running on the device will be able to feed back to Google not only information about the likely locations of new and unmapped roads, about one-way streets and permanent and temporary speed restrictions, but also information about the average speed at any given time of day on any road. This kind of information can be used to provide accurate and optimum routing. And once you know all of this information, it is not difficult to see how, by comparing it with new real-time data, it can be used to spot traffic incidents and hotspots as they occur. Nokia and TomTom already have agreements in place with mobile phone providers that allow such data to feed back but Google will be cutting out the middle man.

OpenStreetMap

If you haven’t already contributed towards the construction of the map of your own town or city you should really take a look at OpenStreetMap and contribute at least a little of your time to enhance the excellent free and open map of the world whose data belongs to everyone.

OpenStreetMap is the Wikipedia of the mapping world. As other Wikipedia-like sites prove there is really only room for one such successful system at a time (remember Google Knol, Citizendium, and the late Encarta). Both Knol and Citizendium still exist, but how many times a year do you refer to them? The ease with which ordinary people can contribute towards the construction of accurate maps of the world and the effects of such crowd sourcing is evident in the surprisingly high quality of the OpenStreetMap maps. It is still a work in progress and some parts of the world are better covered than others. But at the current rate of improvement it has been estimated that within a year or two, OpenStreetMap will also contain enough good quality routing data to start to compete with the commercial offerings from Tele Atlas and NAVTEQ.

OpenStreetMap is possibly already the biggest threat to Tele Atlas and NAVTEQ but now that Google has entered the mapping market other large companies may be persuaded to lend their support to OpenStreetMap. Yahoo has already allowed OpenStreetMap to use their aerial imagery for the purposes of tracing map features. What if Microsoft or Apple were to get involved? It is likely that within a 12 to 24 month time span Microsoft may well re-enter the mobile market with a competitive operating system. It seems unlikely, however, that they will be happy to display the Google logo on their maps.

With feedback from mobile applications and the input of ordinary users OpenStreetMap could well be the map data source of the future. But there is clearly going to be a lot of competition from all of the data providers to gather a good deal of accurate and useful data and to offer it at a reasonable price. In the meantime, for those of us who are mobile map users, I think we are in for a good time.

Satellite Above – I Watched it for a Little While

A moving light in the sky is guaranteed to catch my attention. If it is slow moving and flashing then I know that it is likely to be an aircraft. If it shoots across the sky in an instant then I know it to be a meteor or shooting star. If it is fixed then I conclude that it is a star or a planet. But what if it is none of these? What if it is bright orange and moves across the sky slowly over a period of thirty seconds or so?

I asked myself that very question recently as I watched in amazement as an object that looked to me like a distant ball of fire passed silently and slowly from north-west to south-east across the clear evening sky. My initial thoughts were that I had just seen my first fireball but I knew that to be very unlikely and, besides, I was sure that fireballs were associated with freak weather conditions and on this night everything was still.

The object, whatever it was, appeared to be some distance off and moved across the sky with a speed that I readily associated with that of orbiting satellites which I had seen many times before. But these had always been white in colour and this one was a distant flaming red.

I wondered if it would be easy to check whether any satellites had passed overhead and so turned to the Internet for a solution. I came across a number of great resources which provided more than enough information to solve the mystery.

The first was a free piece of software by Sebastian Stoff called the Orbitron Satellite Tracking System which gives graphical and tabular information about the position of satellites and their visibility at a given time and place. The Orbitron software suggested that what I may have seen was a satellite which goes by the name of Iridium 43, one of a family of about seventy such satellites that provide communication services and orbit the earth from pole to pole at a height of about 500 miles and at a speed of about 17,000 miles per hour.

The Iridium satellites made the news back in February of this year when one of them, Iridium 33, collided with a retired Russian satellite and with a combined impact speed of 26,000 miles per hours both were destroyed leaving thousands of pieces of space debris to fall back to earth over the following days. The Russian satellite had been uncontrolled since, at least, 1995 but the authorities had predicted that the two satellites should have missed each other by about half a kilometre – they were clearly wrong.

Iridium satellites are known to give rise to an interesting phenomenon – the Iridium Flare. The satellites are equipped with three highly reflective door sized antenna made of silver-coated Teflon on polished aluminium  and occasionally one of these will pick up light from the sun and reflect it down onto the earth’s surface generating an illuminated spot on the earth about 6 miles across. To an observer on the surface of the earth the satellite appears as if from nowhere as a faint object that slowly increases in brightness to a maximum and then just as quickly dims until it is no longer visible, with the whole show lasting no longer than, perhaps, thirty seconds. A simulation is shown here. The satellite that I saw appeared a rich flaming red in colour but I put that down to atmospheric conditions and its effect on the light as it was reflected from the satellite down to earth.

A really excellent web site that makes it easy to determine when and where to look out for satellites that are likely to be visible to the naked eye is Heavens Above. Start by declaring your location and follow links from the main page to get predictions for when Iridium flares, the International Space Station or other such objects will be visible in your area. The site also displays charts showing you where in the sky these objects will appear.

If you like to see a more earth-based and dynamic view of how any given satellite is orbiting then this real time satellite tracking web site has a mashup showing the live movement of selected satellites superimposed over the familiar Google Maps background. You can combine this view with an Iridium flare  prediction from the Heavens Above web site to get a Google Maps view of the expected track of a visible satellite too.

It is all a little geeky, but I find it reassuring to be able to get an explanation for such phenomena.

Lessons Learned from the Coast To Coast Walk

The Coast To Coast Walk starts on the west coast of England at St Bees in Cumbria and runs for approximately 200 miles to Robin Hood’s Bay in North Yorkshire on the east coast. In between it crosses three National Parks: the Lake District, the Yorkshire Dales and the North York Moors.  It usually takes something between 12 and 18 days to complete the walk with a crossing of, perhaps, 14 or 15 days being the most commonly attempted. The walk was originally devised by Alfred Wainwright as a west to east journey so that the prevailing `weather’ was always behind the walker and, indeed, this is how most people today choose to complete the walk. A smaller number of walkers opt to travel east to west using the argument that the most stunning scenery arises in the west and so an east-west path saves the best for the end.

I recently walked the first part of the walk from west to east and can highly recommend it as a complete tonic. I have done quite a lot of walking in the past, including some long distance walks, but never one that spans more than a single day. However, many of my preconceptions turned out to be completely wrong and so here I attempt to document some of my conclusions. These are clearly just my opinions and are not given in any particular order but are presented here in the hope that they may be useful to someone considering this walk as their first long distance trail.

Use a Baggage Carrier

There are a number of companies that will move your bags each day from place to place. The two most commonly used would appear to be Sherpa and Packhorse. We used Sherpa but the service provided by both would seem to be very similar: you leave your bags at the camp site, hotel, B&B or guest house in the morning and you find them at your new destination in the evening when you arrive. My initial plan had been to carry all of my kit end equipment, but a couple of long trial walks had persuaded me that I would enjoy the walk a lot more if I didn’t have to carry a heavy weight on my back and at just £7 a day I didn’t take a lot of persuading that employing a baggage carrier was a good idea. As well as carting baggage both companies also provide transport to St Bees and from Robin Hood’s Bay and secure car parking. We left our car with Sherpa at their base in Richmond for £2/day and paid the £20/person to be transported from Richmond to St Bees in time to start walking at about 9.30am on the first day of our walk. This worked out to be a lot cheaper than getting public transport to St Bees and paying to stay there overnight.

Accommodation or Camping

If you are considering saving yourself a little money by camping along the route you should take some time to consider what it really means to camp for the duration of the walk. At the start and end of each day you are going to have to pack-up and re-set you tent as you are unlikely to be staying in the same place for two consecutive nights. If you are planning to carry your camping equipment you are going to be either lugging a good number of extra kilos around with you each day or else skimping on luxury items that you might otherwise have had room to bring. If you are using a baggage carrier then you will be expected to have all of you kit packed up by, at the latest, 8:30 each morning and on certain parts of the walk carry the risk of arriving before your tent (this happened to me in Keld). When I complete the second half of the walk I intend to make full use of the excellent B&Bs and bunk barns along the way. Partly this is because I want to enjoy the walking without worrying about getting a good pitch or setting off in good time in the morning. Partly it is because I felt that in sections of the route the coast to coast walk has become an important part of the local economy and I was pleased to be contributing to that. But mostly it is because the advantage of being able to wash and dry tee-shirts and socks overnight, spend a few hours in the local pub eating a heart-warming meal, get a good night’s sleep, enjoy a full English breakfast and then depart in my own time is worth quite a lot to me. Having said all of that, the advantages of camping will appeal to many others. The cost savings made by camping may play a big part in your final decision, of course. I can vouch for the quality of the Pennine View camp site in Kirby Stephen and the Park House camp site in Keld, both of which were very clean and friendly places.

Sturdy Walking Boots

I plan to visit our local branch of Cotswold and thank the helpful assistant (I’m going to call him Mark) who persuaded me not to invest in a pair of lightweight walking boots. I was concerned that my boots were too sturdy or rather too heavy and that they would take their toll on my feet at the end of the day. Many years ago I had walked the 42 mile Lyke Wake Walk in 12½ hours wearing just a pair of old running shoes on my feet (to be fair, I was still a student and they were the only pair of shoes that I owned) and I was under the mistaken impression that a lightweight pair of shoes would be required to complete this walk. What Mark pointed out to me was that there are parts of the walk which remain wet and boggy for 12 months of the year and how right he was. In particular, the section that lies between Kirby Stephen and Keld is likely to be a challenge for anyone without a good pair of watertight boots. My recommendation is to wear good sturdy boots but to make sure that they are well worn in before you start the walk.

Waterproof Bags

I’ve always used plastic bags to organised the contents of my rucksack and had always assumed that in doing so I was not only simplifying the task of finding anything but was additionally guaranteeing an extra level dryness should I be caught in the hills in a particularly heavy downpour.

I learned a lesson on the second day of our walk when we were caught on the Ennerdale Bridge to Borrowdale section of the walk by the tail end of hurricane Bill or Brian (or some similarly harmless sounding storm). As we passed Black Sail hut and began our ascent of Loft Beck the winds were so strong that not only was it difficult to remain upright but our rucksack waterproof covers were catching the wind and being whipped free and flapping about wildly. The only solution was to temporarily remove them and store them inside the rucksack.

On arriving in Borrowdale I discovered just what a fool I had been. When they advise the use of plastic bags they don’t mean bags of the supermarket `bag for life’ variety because they contain tiny little air holes to prevent babies and small children from suffocating should they feel a sudden urge to wrap their heads up inside them. The correct kind of waterproof bag can be purchased from any good outdoor shop.

Shower Gel

A surprisingly useful discovery for me was that not only can shower gel be used for the more obvious purposes of washing and shampooing but it also proved to be useful for hand washing tee-shirts and socks and as a washing-up liquid when camping and cooking on the camp site. What a versatile little product.

Gaiters

I’ve never owned a pair of gaiters but I might just treat myself to a pair for next year. It was clear that those walkers sporting gaiters were much more comfortable negotiating the boggy ground, presumably because if their boots sunk in to below the top of the bog they still had several inches of reserve before their socks and the inside of the boots became wet. I also noticed that on a long day of walking in the wet even when wearing waterproof trousers water would somehow find its way down through long inner trousers and socks and into my walking boots but those fellow walkers who were gaitered-up ended the day with drier boots. My conclusion is that they also act to reduce the amount of water that can run into the inside of the boot. I suspect there is a more formal explanation of their advantages elsewhere.

Consider Doing the Walk in Stages

My initial plan had been to complete the entire walk in a two-week period but as we had left our plans to the very last minute there was not enough holiday remaining to complete the walk until the following season. When my wife suggested that we do just the first week I thought that I might feel somewhat unfulfilled or dissatisfied with such a compromise and I agreed but reserved the right in my mind to continue into the second week alone if necessary.

Once I had started to talk to others along the way I was surprised by just how many different approaches people were taking to the walk. Some were doing the entire crossing, others were doing so but taking several days off half way and going home for a long weekend or longer. Others, still, were just completing the Lake District section or the Dales. One walker (a retired schoolteacher from Durham) was picking off a section at a time “whenever the weather was good” and was camping and so didn’t have any accommodation booking complications. Another couple were starting the walk in Ennerdale Bridge and walking for 3 days to Kirby Stephen and then taking a Taxi back to St Bees to do day 1 of the walk to Ennerdale Bridge before returning to Kirby Stephen to continue, simply because they couldn’t find accommodation in the Lake District for the days that they would otherwise have needed to have been there.

When we finished our walk in Reeth we met a couple who were continuing for the second week and who wished that they had chosen to break their walk in two simply because they had really enjoyed their first week and felt that they were unlikely to double that enjoyment by in the second week but would rather have come back at a later date to pick up from where they left off.

So tear up all of your preconceptions about how the walk should be done because as people say “It’s a trail and not a trial”.

Go to Grasmere

The guidebooks will implore you to visit Grasmere. I chose not to because, living little more than a couple of hours drive away, I felt I knew that area well and wanted to spend more time on those sections of the walk that pass through less popular spots. I think that was a mistake and I would suggest that the walk is improved by lingering in the Lakes a little longer. The overnight in Grasmere is also the first point on the walk at which your newly found walking acquaintances  will divide into two groups (those that stop and those that don’t) and go their separate ways. This is a good thing because at a stroke it doubles the number of new faces.

You will reach Grasmere after your first three days of walking in which you will have covered 14 miles on each of the first two days, much of it over hilly Lakeland terrain. If you continue directly to Patterdale you will cover 19 miles on your third day. Breaking the third day in two by visiting Grasmere leaves you with a pleasant 9 mile and 10 mile walk on each of days three and four in some of the most delightful countryside England has to offer. You should stop in Grasmere.

Blisters

One of the most common physical impediments to completing the walk is the development of blisters. Even with well worn in boots, if you dramatically increase the lengths of your walks you raise the chances of finding some part of the boot that rubs enough to generate a sore, which in turn will become a blister unless treated. I came across several people who were suffering even after just two or three days of the walk and one man who was bailing out completely after five days because of the pain from foot blisters.

People seem to talk highly of a product called Compeed which I took with me but didn’t need. In the past, for treating the kind of sores you get on the top of a toe or the back of a heel,  I’ve used corn plasters (or pads) which are little circular or oval raised cutouts about 3 or 4mm high with a hole in the middle and adhesive on one side which stick to the foot around the sore. They work by surrounding and protecting the area of sensitive skin and preventing further rubbing. Another trick that has worked for me for treating those sores that develop on the base of the foot is simply to apply one of those “water-resistant” plasters whose smooth surface prevents the sock from gripping that part of the base of the foot and so relieves that lateral tug which can give rise to painful blisters if left unchecked.

However, by far the best mechanism for preventing any of this, for me at least, is to put in a couple of days of walking with a rucksack loaded to the expected weight about a fortnight before setting off on the coast to coast walk. If you aim to walk upwards of fifteen miles on each day, any parts of the skin that are going to rub will do so and any sores that develop will have time to harden up before your real walk starts. You will also get an indication of the kinds of weaknesses your feet may have and you will be able to prepare appropriately.

Water

We were lucky enough to have sun for all but one of our walking days and when the weather is warm it is surprising just how much water you will need to carry. Many of the walkers that I saw or spoke to had invested in the platypus-style water systems that allowed them to be drip fed by a rubber tube that finds its way out of a rucksack and to a point somewhere near your mouth which allows water to be taken at frequent intervals and in small quantities. I’m told this is considered to be the best way to take on water.

We carry our water in the small clear plastic bottles that spring water is sold in – we re-use them, of course. The advantage of this is that, if you are walking with a friend, you can distribute the water between you in whatever balance seems fair. I tend to carry more water than my wife because I carry a bigger rucksack and she is prone to back injuries and so the small plastic bottle system works best for us.

On a hot day I found that I needed to carry no less than five small (½ litre) water bottles. I’ve heard others quote figures that would suggest that this is not enough but it seemed to work for me.

Sun

I am fair-skinned and like to keep the sun off me as much as possible. A large bottle of factor 50 sunblock was in the back of my rucksack and was applied liberally a number of times throughout the day. In general, you are walking towards the sun in the morning and away from it in the afternoon (if you cross from west to east) so special attention has to be given to different parts of the anatomy throughout the day.

A good hat is essential for keeping the sun away from the face and neck. I wear one the the `foreign legion’-style mountain caps with long peak and an adjustable cotton flap at the back that is long enough to keep the back of my neck completely sun-free and if you don’t mind looking like a complete dork (and I don’t mind) the cap is almost perfect. This design has the advantage that it can be dipped into clear cool mountain streams, retaining water in the cotton flap, to enhance the cooling effect to the back of the head and neck.

Mobile Phones

Because of the mountainous and remote nature of a lot of the walk you can easily go for the entire day without a hint of a mobile phone signal. This is great because it guarantees no unexpected interruptions from work or the kids at home but also means that there is little opportunity to make any last minute alterations to your plans once you have left your base each morning. You won’t walk past any phone boxes either, so plan ahead.

Rainy Days

Be prepared for a few rainy days. If you can stay reasonably dry then most things will remain manageable and so, for this reason alone, a good set of waterproofs is essential.

One of the big advantages of staying in B&Bs is the opportunity they give for completely drying out both you and your belongings.

On a typical day you can expect to be walking for between six and nine hours, often with little chance to take shelter. If you are uncertain about whether your walking gear will cope with a long period of rain but don’t want to invest in anything expensive you could purchase a bottle of one of the readily available re-proofing treatments. They typically cost about £10 and come as either a liquid that you spray-on or one that you add to the washing machine and wash in to your waterproofs.

I found that a very wet day could have quite a demoralising effect on us. We were absolutely soaked to the skin with boots and rucksacks full of water. But after drying out overnight, and especially helped by beautiful weather the following day, things took on a new light again.

Take it Slowly

I noticed that I had a tendency to want to rush and that some effort was required, at least in the early stages of the walk, to resist the urge not to be left behind. Most people who have slept overnight in a guest house or a camp site will be up and on their way by about 8:30 to 8:45 in the morning. Although there is some variation in the routes given in the numerous guide books they do all pretty much tend to direct people the same way with the obvious consequence that if you walk at an average speed and you start at the same time as most people you will invariably find yourself walking amongst other coast-to-coasters. There are advantages to this if you enjoy that kind of walking. If you prefer to walk without the crowds, and you are not a particularly slow walker, then the best approach is probably to enjoy your breakfast and not to rush off.

I met one couple at one of our overnight stops who were both fit and fast walkers and who would linger for a couple of hours in the morning at each of their stops aiming to start walking each day at between 9:30 and 11:00 in the morning. They said that they found that by doing so they had a lot of the scenery to themselves.

Socks and other Underwear

My guidebook says “Three or four changes of underwear is fine. Any more is excessive, any less unhygienic”. I had to re-read that sentence a couple of times and, to this day, I’m still sure that this must be a misprint. Perhaps I’m old fashioned or out of touch but why would you want to economise on underwear? It’s not heavy or bulky or otherwise expensive and it is the bit of your clothing that comes into contact with your body and therefore does need changing. My advice is don’t skimp in this area.

Of course, if you are staying somewhere that gives you access to a washbasin and a good drying radiator then you can take the opportunity to wash and dry socks and underwear. I knew a couple that were travelling light who discarded their underwear as they walked the coast to coast (not outdoors, but properly in dustbins) and who re-stocked when they reached Richmond.

Whatever you do, my advice would be to take at least as much as you would normally wear at home unless you can confirm with one or more of your bookings that washing facilities will be available and then you can plan around that.

Accommodation

I felt that the quality of accommodation on the walk was excellent and was also aware of how important the supply of coast-to-coasters is to the local economy in many of the places that the walk passes through. I suspect that this quality is maintained partly as a result of the exchange of information on internet discussion forums which gives positive and negative feedback on many of the possible choices of accommodation. The result of this is that those places that get good reviews also getting good business and those that don’t provide a good service soon start to lose custom.

In general, everywhere but the camp sites will make you up a packed lunch for the following day if you remember to ask for one the night before. In my experience these are easily enough to keep you going through the whole day.

We stayed at the following places and would stay at any of them again. They are all to be recommended:

Low Cock How Farm – Ennerdale Bridge – Friendly bunk barn and small number of twin or double rooms. Wood burning fire in guest lounge with plenty of opportunity to dry boots and clothes. Wholesome farmhouse food in large quantities.

Gillercombe – Rosthwaite – Clean and friendly family run guesthouse.

Borrowdale Youth Hostel – Rosthwaite – Modern, clean, large well-organised hostel. Single-sex  dormitories of 6 people. Good evening meals with choice of beers, excellent self service multi-course breakfast and large hot drying room.

White Lion – Patterdale – Pub with good food and beer and helpful staff. Clean rooms with excellent views and very hot radiators – perfect for washing and drying.

Greyhound Hotel – Shap – Friendly hotel with comfortable bar. Excellent food – the locally sourced Cumbrian Lamb Henry was the best lamb I have ever tasted. Large comfortable en-suite rooms.

Pennine View Caravan & Camping Park – Kirby Stephen – Clean and tidy site. Good showers and washroom. Dishwashing facilities. Mostly caravans but plenty of space for tents too – level pitches. Helpful and friendly staff. A two minute walk from the Croglin Castle pub.

Park House Bunk Barn & Camp Site – Keld – Small friendly tidy family run campsite. Very comfortable bunk barn with space for 7 in two rooms. Simple evening meals available. Superb location next to the stream and waterfalls. Five minutes walk from the bar at Keld Lodge (the old Youth Hostel) but take a torch to find your way back after dark.

In Summary

Do the walk. Either on your own or with a friend or group of friends. Whether you walk for two weeks, a week or for just a few days you will get a tremendous reward from the experience and, as a result of the continuously varying landscape and outlook, a feeling of having had a holiday quite unlike any break you may have taken before. I’m looking forward to starting the second half.

The Right Tool for the Job: Comparing GPS Devices

I have just walked the first 100 miles of the Wainwright Coast to Coast walk; the section between St. Bees and Reeth in Swaledale. Before setting off I was keen to ensure that, as well as the obvious set of guides, maps and compass, I had a good GPS device to help with navigation when visibility got poor. I’ve been caught out in dense fog on a high fell in the Lake District and I know how difficult it can be to find the right route down in such conditions.

I already own two GPS devices: an iPhone and a TomTom GO 730 both of which are excellent devices and purchases that I am entirely happy with. I have seen a lot of discussion in online forums about the merits of various handheld GPS units and was aware of more and more software for the iPhone that made use of its built in GPS hardware. I wanted to know if either of my existing two devices could be usefully used on a trek or if I needed to invest in yet another gadget.

After much research I purchased a Garmin GPSMAP 60CSX and I was very very happy with it. I carry an iPhone all the time but I wouldn’t even bother switching it on if I had the Garmin with me. That’s not to say that it doesn’t have its uses. For example, RouteBuddy have released a series of Ordnance Survey 1:25000 scale maps of the UK National Parks which are stored on the iPhone and so can be viewed even when out of network contact. Their free iPhone application `Atlas’ can also be used to view the free OpenStreetMap map of the UK (and the rest of the world) and of course there are the excellent iPhone Google Maps and Google Earth applications. All of these are good for browsing in the pub or at home.

There are, however, a couple of big problems with the iPhone when used outdoors. Firstly, the battery life is appalling. With location services enabled to allow a GPS fix I estimate that you would be lucky to get more than two or three hours of continuous service from it, possibly less. A separate battery pack might extend that by a factor of two or three but you’d still be worrying about your chances of lasting for a good day’s walk and you would have the additional inconvenience of having to lug the weight of the battery pack. Secondly, and perhaps more seriously as far as safety is concerned, you will find that the iPhone is useless in the wet because when you are wet you won’t be able to operate the touchscreen. I discovered this inconvenience on a particularly wet day when I needed to call a Youth Hostel to book a bed for the night.

The map that comes with the Garmin is hopelessly basic, but if you download the free contour maps from the Scottish Mountaineering Society website and install (either manually or from the Internet) some routes and waypoints or POIs onto the Garmin it is an excellent hiking tool. Oh and it’s waterproof to 1 metre depth of water, its GPS is considerably more accurate than an iPhone and having it on continuously for 100 miles of walking it only got through 4 AA batteries. I’d rather have the combination of a paper map in a waterproof case (or one of those waterproof laminated ones) and a handheld GPS than risk having my map in the GPS. At least then if the device fails you still have a map (and, it goes without saying, a compass). The Garmin will give you a very accurate grid reference to read off on the map.

My advice is stick to the right tool for the job: iPhone for indoors, Garmin GPS60CSX for outdoor on foot and a TomTom (or similar) for the car. I don’t think any of the devices work well in the wrong environment.

A Solution to the "iTunes has stopped working" Problem on Vista

About a week ago iTunes began crashing each time I synced my iPhone. Initially it happened right at the end of the sync and so, although irritating,  it was possible to ignore it while simultaneously searching for a solution online. The problem started immediately after I had installed the new version of Google Earth and so my initial thoughts were that it was in some way related to a change that had been made as part of that install. But uninstalling Google Earth had no effect.

The nature of the problem was that the more I ignored it the further forward in the sync process it got until eventually iTunes would crash immediately on recognising the iPhone and display the "iTunes has stopped working" popup.

The Apple support forums were of little help. They seemed to contain little in the way of advice from Apple and a lot in the way of posts from similar and confused iTunes users who were experiencing similar problems to me.

One apparently useful suggestion from the forums was that the issue might be resolved by uninstalling QuickTime and then re-installing iTunes+QuickTime together. This had no effect either.

Following another line of hope I performed a restore on the iPhone which reloads the firmware and attempts to recover the iPhone settings from the most recent backup. Again, no solution.

After a number of frustrating evenings of tinkering I stumbled on the solution that worked for me: From the iTunes store menu choose "Deauthorize Computer…" followed by "Authorize Computer…". I have no idea why that solved the problem but I can now sync again and if you have stumbled upon this post because you have the same problem I really hope it works for you.

I can’t understand how Apple can produce such a wonderful user experience on the iPhone and Apple TV and such an awful one on iTunes and the Apple support web site.

Twitter Ye Not (there’s #uksnow time to lose)

I feel like I’ve just seen a meteor shower – an unexpected but surprisingly satisfying event took place last night. The micro-blogging site Twitter receives mixed reviews. Many people love it, are perhaps even a little addicted to it, others fail to see what all the fuss is about. Whatever your views there is no doubt that it has spawned a number of interesting discussions about the future directions of blogging and social networking.

Several tools have formed around twitter which aim to apply some form of structuring to the naturally unstructured twitter message (a tweet). One such tool is #hashtags which filters all tweets that contain a hashtags pattern (a word that begins with a hash character) and presents them as a chronological list of tweets collected under the heading of the given hashtag thus enabling communities of twitterers to follow a stream of conversations about a common topic. There is no organization to all of this, but rather a community that feeds off a number of sites that make use of this data such as twitterfall.com, search.twitter.com, twist.flaptor.com and hashtags.org itself.

Last night a tag formed to discuss the amount and location of snow that was falling in the UK. Although the conversation was informative it was often difficult to make use of the information because tweeters were inconsistent in declaring their location and the amount of snow they were experiencing. At some point in the evening one twitter user decided to build a mashup that took the #uksnow stream of tweets and parsed their content for the sequence

#uksnow postcode score/10

Where postcode is replaced by the first portion of the tweeters UK postcode and score is a rating (out of ten) used to indicate the amount of snow that is currently falling in that postcode region. The mashup combined this information with google maps, plotting in white regions with the heaviest amounts of snow. It was observed by one twitter user just how consistent this map was with the MET Office radar but with the advantage of being updated in real time.

Clearly there are obvious weaknesses, such as the subjective nature of a  zero to ten snow scale (although the, once popular,  Beaufort scale for measuring wind speed springs to mind) and the influence of the natural differences in population density which would bias the influence of the tweets in favour of heavily (twitter) populated areas of the UK, but I can’t help feeling that I witnessed something that was both spontaneous and surprisingly useful.

Broadband Network Usage Monitoring and Measurement Tools

If you have found that your ISP has been restricting your broadband bandwidth the obvious question you will ask your ISP is why? If you ask that question the answer you are likely to get is that you have been using the “broadband service inappropriately”. You might also be told that you have exceeded your usage allowance but if you, like me, are on an unlimited contract you are unlikely to be told what the upper usage limit is. The reason you won’t be told the actual value of the upper limit is that it is likely to be more complicated than a simple figure and the reason that this is the case is because the ISPs are principally interested in avoiding router congestion at peak times. At off-peak times, such as in the early hours of the morning and during the working day, it makes little difference to an ISP if the capacity of their network is 25%, 50% or 75% used, they still have the same equipment costs and other overheads. But once the network reaches capacity, and routers are forced to drop packets, then customers start to notice and the ISP begins to get a bad name. For this reason most ISPs have, quietly, begun to apply traffic shaping at peak times.

Traffic shaping involves restricting the bandwidth of `heavy usage’ customers in such a way as to prevent them from interfering with the network experience enjoyed by `lighter usage’ customers. Unfortunately it looks like we are in for a lot more of this as network demand grows. The popularity of the Internet as a medium for watching media has rocketed in recent months and looks to continue to grow as more and more people switch their viewing habits from the more traditional broadcast medium to Internet based technologies. In the UK, the BBC iPlayer alone has been responsible for tremendous changes in network usage.

I recently found myself in the position of having determined from my ISP that I was being traffic shaped. Unfortunately, my router provided me with little help in identifying the volumes of data that were passing through my broadband connection each month but my ISP furnished me with figures which seemed to be considerably higher than I would have expected. Finding myself in a very weak position I decided to rearrange my home network to allow me to gain a better understanding of my broadband usage.

My broadband is supplied by BT on an unlimited tariff and I use the BT supplied broadband router (2-Wire 1800) which hosts two wired Ethernet connections, one to a PC and one to a network attached storage device, and a number of wireless connections to PCs or laptops. There are nine devices in total but typically a maximum of 4 might be actively using the broadband line at any one time, and by active I usually mean browsing web pages. Living reasonably close to our exchange we can manage to achieve a download speed of about 6.3 Mb/sec and even with the bandwidth restriction we still reach this speed outside of peak times.

WallWatcher

The problem with having a wired network is that the only device that can really determine how much traffic is flowing is the broadband router as everything else talks directly to it and it funnels data into and out of the ADSL line. I attempted to get my existing router to log traffic information to a PC so that I could take a close look at what was travelling up and down the external line.

There are many different logging analysers available but the one I chose to use was WallWatcher a free tool with support for a large number of routers.

Tomato

Unfortunately I found that I couldn’t get WallWatcher to correctly recognise the format of the packet logs coming from my model of 2-Wire router. In my case the solution was to make use of a spare wireless router that I had which was not being usefully employed, a Linksys WRT54GL. This variant of the WRT54 family of router runs Linux and can be easily upgraded to run an alternative piece of firmware. I wanted to concentrate my network devices on the Linksys router and then run a single connection from the Linksys router to an Ethernet port on the back of the 2-Wire broadband router. I also wanted to make use of the fine bandwidth reporting available from the Tomato firmware which this router can be upgraded to run. The process of upgrading the router took about 10 minutes as it can be done from a menu option within the native Linksys firmware. Once Tomato was up and running on the Linksys it was easy to configure it to provide a good quality wireless network that replaced the old 2-Wire network and as an added benefit was also faster.

I configured the Linksys to store its bandwidth logs on a network Windows share and to forward its packet logs to WallWatcher as before and the results were immediately interesting. Tomato hosts a number of web pages that show bandwidth over varying periods of time. I could see straight away that the download bandwidth on the WAN port was considerably higher than I would have expected and at quite a sustained value (see the image on the right). The graph shows consistently high volume of download overnight and then a period of very low activity in the morning when all of the PCs were switched off followed by high usage again from about 2pm when they were restarted. Tomato also hosts a real time bandwidth display. Using these displays combined with WallWatcher it was easy to identify the PC responsible for the heavy usage and by examining the addresses of the remote end-points shown by WallWatcher it was also easy to determine the offending program.

TV Tonic

In my case, the program generating the traffic was the TV Tonic RSS service, a program responsible for downloading video podcasts from the Internet. I hadn’t realised that the program was still active as I had not made use of the client for a number of weeks. Incidentally, the TV Tonic client runs as an add-on to Windows Media Center (under Vista) and is quite a nice addition. Had I looked a little more closely at its configuration I would have noticed that not only did it have an option to limit the download bandwidth but also it had a download scheduler to control the time of day that it be allowed to download at all (both of these options would be nice to see adopted by the BBC iPlayer). I’m not quite sure why TV Tonic was downloading such large amounts of data but, not wishing to experience another month of bandwidth restriction, I immediately disabled the TV Tonic service and the Tomato real time monitor showed the corresponding reduction in bandwidth usage.

BBC iPlayer

Even after disabling the TV Tonic RSS service there still seemed to be a lot of network activity from my PC although I wasn’t running any obvious client program. A closer look at the WallWatcher log display showed a large number of incoming and outgoing UDP packets being sent to external machines. WallWatcher comes with a charting tool called WallReviewer which gives a useful interactive graphical picture of incoming and outgoing traffic information over a given period of time. The WallReviewer chart of “Outbound Leaks by Remote Names” showed a large number of packets being sent to the machines iplaykdms82.telhc.bbc.co.uk and iplaykdms6.telhc.bbc.co.uk. The names of these remote sites suggested the BBC iPlayer might be responsible but the application wasn’t running and the option “allow programmes to be shared when you exit download manager” was not ticked in the iPlayer configuration dialogue so I had assumed that there ought to be no networking activity from the iPlayer Kontiki-based software. I found that if I disabled the Windows service named “KService” (which runs the BBC iPlayer program “C:\Program Files\Kontiki\KService.exe”) then all of this network activity stopped immediately. From the WallWatcher display it was clear to see that these packets were being sent about every 2 to 4 seconds but WallWatcher is not able to give any indication of the size of the packets.

Wireshark

To get a better indication of packet sizes a protocol analyzer is required. The “old faithful” in this area used to be called Ethereal but development on Ethereal has now been moved to Wireshark. Wireshark is simple to install and can be used on many platforms. It is also free and licensed under the GNU General Public License. There is a lot more to Wireshark that the casual user is ever likely to need and a basic knowledge of networking protocols and terminology helps but there is plenty of documentation.

Running Wireshark on my PC confirmed that data packets were being sent to the BBC domain every two to four seconds but also showed that the packet sizes were small, 16 bytes of payload which by the time they have been wrapped in the UDP and IP packets amount to a 58 byte Ethernet frame. I find that having disabled the KService service I am unable to start up the BBC iPlayer but as soon as I re-enable the service the iPlayer functions as normal.

Conclusion

Having made these networking changes I am now in a much better position to know exactly how much traffic is being downloaded (or uploaded) over my broadband line and also able to detect this traffic early on to avoid triggering any ISP penalties. The tools required to monitor bandwidth are not expensive (in fact they are free) and are easily configured. I think that my ISP should have been able to give me the information that I needed to monitor and control my bandwidth – it feels a little like having been sold a car which has no fuel gauge.

One lesson that can be learnt from all of this is that it is becoming more and more important for anyone with a reasonable grasp of networking to take matters into their own hands to monitor their own network usage. I don’t see the ISPs relaxing their grip on our usage patterns in the short term, at least not until their own issues of congestion have been addressed. So by tightening up on wasted bandwidth we should be left with more to do the things that we really want to use it for.

Fair Use for BT Unlimited Broadband Traffic Shaping

Have you ever noticed large variations in your broadband performance? If so, there are a number of factors you might want to check before putting it down to bad luck. It may be that you, like me, are having your download bandwidth silently restricted by your ISP.

My broadband is supplied by BT and known as BT Business Broadband Share, I’ve been a BT customer for a number of years now and I think the equivalent (and, I notice, somewhat cheaper) current package is known as BT Business Total Broadband – Option 3. Both packages are advertised with the term ‘unlimited usage’ and both refer you, in the small print, to BT’s fair use policy.

I wouldn’t describe myself as a heavy user of broadband. I think I may have purchased and downloaded a dozen music files from the Internet, in my life, and I have used the BBC iPlayer to download a similar number of TV programmes, although I was careful to tick the little box that prevented the iPlayer from re-sharing those files after I had finished watching them. I download, perhaps, 6 large DVD size images from Microsoft’s MSDN subscriber site a year as part of my developer network subscription. I connect to the office with a VPN connection to collect email and occasionally use remote desktop or VNC to connect to one of several remote servers to perform evening or weekend maintenance. Other than that, I browse the web and listen again to a repeat of a Radio 4 audio programme on average about once a month. None of these things are particularly expensive in terms of bandwidth.

We are only a short distance from our local exchange and so usually comfortably achieve download bandwidth figures of over 6M bits/sec, but recently I had noticed much longer delays in displaying web pages from all sources and interrupted video streaming. We have a number of other PCs in the house on our wireless LAN and they were all experiencing similar problems so I checked the router (a BT supplied 2-Wire 1800) and noted that its broadband monitor showed low download and upload demand. This made me suspicious that there was a problem with the wireless network itself and so I checked the various settings, rebooted PCs and restarted the router and all of this made no difference at all to the download performance. I even chose to accept the firmware upgrade that the router was offering me in the hope that it might fix the problem but, rather ironically, the only visible difference I could detect was that the router’s nice bandwidth monitoring page has now been removed which means that I no longer have any indication of the upload or download bandwidth in use at any given point in time.

Noting that the download bandwidth was low, and knowing that our line normally performs well, I assumed then that the problem might be congestion at the exchange. There are good congestion checking tools at nildram and plusnet and plenty of information about broadband exchanges at samknows.com. These resources all suggested that my local exchange had no congestion issue.

Puzzled, I thought I’d monitor the bandwidth and see if I could determine a pattern. I initially suspected some form of interference from, say, a poor electrical connection or a fluorescent light, both of which can have this effect on broadband speeds, or so I had read.

You can check you broadband bandwidth with the excellent speed checker at speedtest.net which allows you to maintain a nice record of the measurements you have taken over a period of time or the less impressive BT offering at speedtester.bt.com which I found had to be run with administrative privileges on my Vista system but which does additionally provide you with what BT call your IP profile. There is an excellent description of this IP Profile at kitz.co.uk and a wealth of background information about ADSL too.

According to speedtester.bt.com my IP profile was 6.5Mbits/sec which was what I had expected:

IP profile for your line is - 6500 kbps
DSL connection rate: 448 kbps(UP-STREAM)  7616 kbps(DOWN-STREAM)
Actual IP throughput achieved during the test was - 2569 kbps

But depending on the time of day that I ran the test the IP throughput would vary from the expected 6340 kbps to as low as 379 kbps in the evening, rising through 3788 kbps after 11pm back to full speed after midnight and during the daytime. I saw this pattern repeat itself over a number of days.

Armed with this information I began to become suspicious that my line was being traffic shaped in some way. I called BT Business Broadband support and the nice lady there confirmed that my number was being restricted but she seemed surprised that I had not received an email alerting me to this. Broadband support said they were unable to give me any more information other than to say that it was due to excessive usage and gave me a phone number to ring to investigate further. That number turned out to be for reporting security breaches but they, in turn, gave me the email address: liteusage@btbroadbandoffice.com to which issues regarding traffic shaping and bandwidth restriction can be addressed.

I understood from the phone conversation that it was BT’s policy to review these restrictions at the billing points and that the restrictions could be gradually reduced if not completely removed when the excessive usage ceased. This meant that I should expect restrictions to be in place for several more weeks until BT saw fit to restore my service.

I emailed a request asking if they could help me to:

1. Understand why the restriction has been applied .
2. Help me get it released as soon as possible as it is interfering
   with my business use of this line.
3. Give me an idea of the volume (or nature) of downloads that must have
   been present to have this restriction applied so that I can ensure
   it doesn’t happen again.

and the reply I got back said:

From the description you have given it appears that you are being
traffic shaped under the terms of our Fair Usage Policy, this is
why your connection is slow between 5pm and midnight.

and then followed this up with the surprising statement:

I am sorry but we cannot lift this restriction for you as our suppliers implement these measures.

Neither of these was particularly helpful or acceptable so I thought I’d better have a closer look at the fair use policy to see what I had done wrong and that is where the confusion really begins. The fair use policy doesn’t explain what BT consider fair use to be. I’d be only to happy to make a considered judgement about whether my £50/month unlimited broadband package was worth that amount if I could see what I was allowed to use it for. The policy explains in its three major sections:

Why do we have a fair use policy?

BT explain that their “… Fair Use Policy manages inappropriate use and makes sure the service can be used fairly by everyone”. And they define inappropriate use:

“A very small number of our customers use their broadband service inappropriately, for example when sending or downloading very large files, or using ‘peer to peer’ and file sharing software (which may be sending and receiving video and other large files constantly)”

So their concern appears to be related to the transmission of very large files and “peer to peer” or file sharing software. There is no mention of an acceptable download (or upload) usage figure, especially for their unlimited services.

How does the fair use Policy work?

Quite simply:

“If you regularly use the service inappropriately during peak hours, and we believe this is unfairly affecting other customers’ use of the service, we’ll manage your bandwidth during peak times (which could result in reduced service speeds).”

The policy clearly states that BT will control bandwidth for what they deem inappropriate use (earlier rather feebly defined as something involving large files and, possibly “peer to peer” and file sharing software) if this is done regularly (again undefined). There is no indication of how long they will continue to manage bandwidth for or an advanced warning that it might be about to happen. They do say that:

“If you continue to use your service inappropriately we reserve the right to end your agreement with us and will give you notice before doing so.”

But, of course, if you weren’t aware that you were using the service in an inappropriate way to begin with how are you supposed to know that you have continued to use it inappropriately. This seems to me to be an opportunity for BT to silently manage bandwidth to whatever extent suits them whilst continuing to charge the full rate for the service.

How do I know if the Fair Use Policy affects me?

BT say: “Our Fair Use Policy applies to all our customers but it’ll only actually affect you if you’re one of the very few customers who make inappropriate use of our service”. So another recursive and incomplete piece of information.

But don’t worry because:

“If you don’t use peer to peer, file sharing or other inappropriate software and you’re not, for example, constantly downloading or uploading: videos or very large files, you’re unlikely to be affected by our Fair Use Policy.”

So quite simply, don’t trouble your pretty little heads over our fair usage policy as it is unlikely to affect you. But hang on a minute,

• I use the BBC iPlayer and that is peer-to-peer file sharing software,
• I pay several hundred pounds a year to Microsoft to allow me to download operating system DVD images or sometimes videos from their web site and those are large files.

So, on both those counts BT can legitimately claim that I have fallen foul of their fair use policy and without warning restrict my broadband connection to whatever extent they wish and for however long they wish and still continue to bill me the same amount of money even though they neither specify the acceptable usage limits or provide me with any mechanism by which I can determine my current usage?

That doesn’t seem like fair use to me.

ORDB – Gone But Not Forgotten

ORDB has posed another Web traceability problem for me. They recently announced that they were closing down their Open Relay database that has been used by mail servers to help identify incoming mail likely to be considered spam based on knowledge of the mail senders IP address; the idea being that if enough people reported spam to a centralised organisation then a database could be maintained that could track IP addresses from which spam appeared to originate and then that organisation could provide a lookup service to mail servers which would allow them to rapidly check all incoming mail to determine if it originated from one of these blacklisted IP addresses.

The problem is that although I recall reading about their impending closure, there is now nowhere authoritative that I can find that confirms the current status; their website, at http://ordb.org/news/?id=38, where the original news item was posted (I believe) is currently offline and the word on numerous online sites is that the ORDB service is now reporting all IP addresses as being blacklisted in an attempt to force mail server administrators to remove the ORDB database from their list of blacklist services, presumably to reduce the load on their servers if they decide to re-open in some form. Looking through online forums and blog posts it is clear that plenty of people are experiencing the results of this change.

There is really no technical issue here for me – the correct solution is simply to remove ORDB from my mail servers, which is something I did a while ago anyway. The real issue is that when I googled for “ordb” it was apparent that most of the articles were based on the same sources at www.nabble.com and in turn Slashdot which reported the nabble.com message. The one site I did find that reported a conversation with a former ORDB operator was on a German site at www.heise.de which reported:

Andreas Plesner Jacobsen, a former ORDB operator, explained to heise online sister publication iX that this measure has been introduced because the zone is still swamped with queries. The intention is to get mail server operators to stop using ORDB. Simply deleting the domain was not a viable alternative, since the load would then merely be directed to the .org name server.

Ironically, it seems that the only way ORDB could get people to act to remove them from their mail servers was by breaking their service in a way which forced administrators to investigate – simply making an announcement on their web site was clearly not sufficient.

My 10 Favourite (free) Windows Tools of all Time.

Reading Ed Bott’s postings about his and his readers’ favourite Windows programs of all time I was surprised to note just how many of the programs on the list had an associated price tag rather than being free (as in beer). In particular, what attracted my attention was that had I been asked to guess which were free and which were not I would probably have failed miserably. For example, a text editor for $33, a note taking tool for $60 and a screen capture utility for $40, but a complete news aggregator for free.

I work, mostly, in a Microsoft environment and so the majority of my main software development tools for that platform are either purchased or licensed through an (expensive) subscription but, like most readers, I like to adorn that environment with utilities that make for a more agreeable working experience. Sometimes those utilities relate directly to work tasks and sometimes less so, but what I notice is that most often those utilities are free (or effectively so – more on this later).

I constructed a list of the utilities that I use on a regular basis at work and at home and very quickly the list grew well beyond 10 in size. As it doesn’t seem sensible to attempt to order them in any way (because such an ordering would make an assumption about your motives for having them in the first place) I leave them unordered. Likewise, as I don’t feel comfortable choosing my top ten, I describe more than that number here but the real list is much longer and growing.

TrueCrypt

TrueCrypt is disk encryption software which allows either an entire disk partition to be encrypted or else a virtual encrypted disk to be created from a file and then mounted as a Windows drive. The software is Open Source, well documented and thoroughly well thought out. I haven’t had the courage to get to encrypt a real partition yet but do use it to maintain a number of well protected virtual drives that I can mount when I need access to the documents that I store securely inside them. A drive can be mounted once the required password (or password and key file, or correct encryption keys) are provided and once mounted it can be used just like any other Windows drive. The contents of a TrueCrypt drive are never stored in their decrypted state on disk they are only ever held temporarily in RAM. TrueCrypt drives are a great place to store that collection of documents that you know should really be kept secure.

Cygwin

For software developers, like me, who were brought up in a Unix environment the lack of a real command line in the Windows environment can be stifling. Now I know (the awfully named PowerShell) is now available, but what made the Unix environment so complete was the rich set of commands that could be glued together with whichever variant of the Bourne shell was in vogue. Cygwin provides that same environment but hosted under Windows. The choice of programs is truly massive: editors, shells, compilers, interpreters, text and document processors, libraries, windowing systems. Most things GNUish can be found there courtesy of the GNU C compiler and friends too. Integration with Windows through the filesystem means that all of these tools can be used to process files and media residing on any Windows drive.

TimeSnapper

TimeSnapper quietly records your activity by taking snapshots of your computer screen at regular intervals through the day. The interval between snapshots can be configured to a given number of seconds and the recording is achieved without any noticeable pause or flicker. This is really handy on those days where you have moved from one task to another and have not been as meticulous about recording your exact timings as you should have been as it allows you to replay the day a snapshot at a time or to jump quickly to a particular time of day and see what you were working on at that point in time. You provide TimeSnapper with a folder it can use to store the snapshot images and chose the format (.png, .jpg, .gif, .wmf, .tiff, .bmp, .emf) and the resolution of the stored images as a percentage of the full screen resolution. TimeSnapper will also manage the archiving of the snapshots if you provide it with an age beyond which you wish it to delete old images or an upper limit to the amount of space you would like it to allocate to storage. Multiple displayed are handled too. This is a tool you can forget about until you need it, and then its a lifesaver.

Copernic Desktop Search

Copernic Desktop Search is one the many similar search products but what I really like about this program is its intuitive interface. Of course, it is packed with all of the features you would expect from any such search tool and, of course, it indexes a myriad of document and media file formats inspecting meta data inside the files for rapid lookup. It also understands, and so can index, email and contact information from Outlook, Outlook Express, Eudora and Mozilla Thunderbird.

The using interface, rather than relying on a web browser as some search tools do, reacts dynamically as you type, homing-in on the information being sought. Indexing happens on-the-fly and only when the machine is not heavily loaded (and this is configurable). Copernic confirm on their web site that you can “Rest assured that the data indexed by CDS stays on your PC and on no account will it be transferred to us or any of our partners”. The licence only allows for non-commercial use. A separate licence exists for commercial application. That said, I know people for whom this has revolutionised the way they use their PC and I recommend this as a productivity tool.

Virtual CloneDrive

As a software developer I find that I am often presented with application software in ISO format. It is always a pain to have to burn a DVD just so that it can be mounted in a Windows drive and then discarded and probably never used again (probably never even labelled) once the installation has been completed. Most of my MSDN software arrives this way. SlySoft’s Virtual CloneDrive allows these images to be mounted directly from the ISO file on the file system. Several other formats are supported in addition to ISO.

MusicBrainz Picard

If you have ripped your CD collection to MP3 or other digital format you will almost certainly have found errors in the track and album metadata that the music files contain or inconsistencies in the naming conventions used by each of the different people who have provided this information. MusicBrainz Picard comes to the rescue by applying the accumulated knowledge from the very well moderated MusicBrainz database. MusicBrainz is a community music metadatabase that attempts to create a comprehensive music information site and you can use the Picard tagger to automatically identify digital music and then tag it and to clean up the existing metadata tags in your digital music collection. I used Picard to correct the Windows Media Player created tags in my own music library when I ripped my entire CD collection to mp3 format and use it regularly each time I purchase music.

Pidgin

Pidgin is a multi-protocol messaging client that handles a large number of instant messaging protocols: AIM, Bonjour, Gadu-Gadu, Google Talk, Groupwise, ICQ, IRC, MSN, MySpaceIM, QQ, SILC, SIMPLE, Sametime, XMPP, Yahoo!, Zephy. I can really only claim to have used the MSN and IRC protocols but the reason for turning to Pidgin was to allow me to communicate with my family members on MSN without having to endure advertisement hell. Pidgin supports away messages, typing indications and file transfer between clients.

Firebug

If you are anything more than the most casual of Firefox users or if you create any kind of HTML content or even if you are simply interested in the structure of the HTML page that you are viewing in Firefox you should be interested in the Firebug extension to Firefox. Firebug integrates with Firefox to enable rich examination of a web page structure including:

1. an interactive and graphical identification of the effect of individual sections of HTML on the resulting display going from both HTML to display and from display to HTML.
2. an indication of the CSS rules, and the order in which they have been applied, that determine the final appearance of a screen element.
3. the ability to change elements of the CSS or HTML source and immediately see the resulting effect on the display.

Firebug was written by one of the original Firefox developers and the slickness of the integration is evident. If I could only keep one Firefox extension it would be Firebug.

7Zip

Its difficult to get to excited about a file archive tool, especially one that performs well is unobtrusive and just gets the job done. 7Zip is just that kind of tool, integrating well with the Windows explorer shell context menus but providing more functionality and better performance than the native Windows archiver (Compressed folders). When writing an archive, 7z, ZIP, GZIP, BZIP2 and TAR target formats are available and when reading an archive any of RAR, CAB, ISO, ARJ, LZH, CHM, MSI, WIM, Z, CPIO, RPM, DEB and NSIS formats are available. 7Zip can optionally apply AES-256 encryption when creating 7z and ZIP format archives.

Gimp

Gimp, the GNU Image Manipulation Program, is as close as you will get to a tool like Photoshop or Paint Shop Pro without spending a lot of money. For most of the image related tasks that I need to perform it is overkill (by a long shot too) but if you are prepared to put some time into learning the basic techniques some impressive results can be obtained. There are quite a lot of helpful web sites within reach of Google that contain hints, tips and tutorials for those who make the effort. Also take a look at paint.net, a relative newcomer but receiving a lot of praise.

JungleDisk

JungleDisk is a tool that puts a user-friendly front-end on top of Amazon’s S3 Storage Service. S3 enables inexpensive off-site storage of files up to 5GB in size to an unlimited capacity. Storage costs are of the order of $0.18 / month per GB with data transfer rates of between $0.10 and $0.20 per GB. JungleDisk itself is not free (in spite of my claim in the title of this post), it costs $20, but can be used on as many PCs as you like with the same Amazon S3 account. I include it here because compared to the cost and worry of on-site storage the combined cost of JungleDisk and Amazon S3 is effectively free, at least as far as I am concerned. JungleDisk can perform on-the-fly encryption of data as it travels from the PC to S3 and decryption on its return journey, it can make the S3 storage appear as a mapped local drive and it can perform scheduled backups from the PC to S3.

Launchy

Launchy is a smart search program which tries to guess which program you are looking for and will launch it with the minimum number of keypresses required to satisfactorily identify the desired program. It is designed to help you forget about your start menu, the icons on your desktop, and even your file manager.

This is a utility that I didn’t expect to survive my move from Windows XP to Windows Vista because at first glance it appears to provide much the same functionality that is now found with the search facility that is built into Vista’s Start Menu search box. Indeed, to begin with, I survived without it for a couple of months but then I began to miss the fact that Launchy is started with only a hot-key combination and requires no mouse movement or clicks. Launchy lurks in the background and responds to the Alt-spacebar key sequence by opening a small input field to accept keyboard input. On typing, Launchy searches its indexed list of known programs for the closest match, the search being refined with each additional keypress. When the desired program is identified a hit of the enter key is all that is required to launch the program.

Launchy can be customised to search specific locations for commands and to recognise additional files type, or to provide additional arguments or accept user supplied parameters to commands and it can also perform online searches with google, msn, yahoo, live, weather, amazon, wikipedia, dictionary, thesaurus, imdb, netflix, and msdn.

SharpReader

SharpReader is an RSS feed aggregator created by Luke Hutteman and is the only RSS reader that I have ever been completely comfortable with. The application is infrequently updated but (possibly as a result) runs without faulting and simply does the job well. In addition to allowing a collection of feeds to be browsed it also presents a stacked list of alerts up the right hand edge of the display whenever new items arrive. The lifetime of these alerts can be adjusted to allow just enough time to quickly scan them without them becoming too much of a distraction to the job in hand. At the time that I started using SharpReader the only other utility that I felt came anywhere close to it was FeedDemon. FeedDemon is now also a free product and I have been dual running it alongside SharpReader – the jury is out, but SharpReader still has the edge.

gVim

Vi was one of the first Unix visual text editors, taking its name from the two character command that switched its predecessor ex into a, so called, visual mode. Ex, in turn, is a descendent of ed which was written by Ken Thomson back in the 1960s as part of the Multics environment and contained one of the first implementations of regular expressions. Vim was created in 1991 for the Amiga computer as an extended version of the vi editor and gVim is the graphical variant of vim. The expressiveness of regular expressions combined with the rather terse but necessary and sufficient approach command driven editing that this family of editors supported went on to fuel many of the ideas in other important Unix commands, notably grep, sed and later awk (which you could argue was responsible for the creation of Perl). The lineage continues with Rob Pike’s sam and acme for the Plan 9 and Inferno operating systems.

You could argue that Vim is part of Cygwin which I have described elsewhere but I think it deserves to be singled out here if for no other reason than for the fact that on a Windows system it allows you to replace the hopeless notepad with something that at least allows you to perform some useful tasks and, if you are prepared to make the effort to learn its command syntax, become more productive too.