I think the £5 fee is refundable if you decide that you're never ever going to use the Oyster card again, e.g. you take a one way trip to Planet Zorg, or you enter a coma caused by drinking too much...