Having experimented more with the filter, I've noticed a bit more of what I was hoping for. When the sun is behind me, and slightly to one side, it seems to have the most effect.
Also, a cool tip for anyone else new to using a polariser – try framing your shot through the viewfinder, and turning the filter round and round in front of the lens (my one is designed to turn like this, it doesn't unscrew itself). You can see the effect 'turn on' in front of your eyes before you take the shot. Very cool!