Tankless water heaters probably do not save much energy, either. And may actually even waste it [update: they almost certainly do, see next article] compared to conventional tanked heaters. In short, they are generally overhyped.
After having coped with one for a little over six months, I think I am qualified to make those statements.
When I first took a shower in my current residence (which has such a heater), I was astounded at how much water I had to first run down the drain before it got the least bit warm. For a moment, I even thought the water heater had just broke, then I realized that not only does a tankless heater have to warm the pipe between the fixture and the heater (the reason you have to let the water run with any type of water heater), it also has to first warm up the water heater itself.
Then I learned to my shock that one can’t start showering and incrementally adjust the temperature as the pipe warms up further. Tankless heaters operate in a very delicate and easily-disturbed equilibrium. Adjust the flow of either the hot or the cold water, and you disturb that equilibrium, and it takes almost as long to regain it as it did for the water to get warm in the first place. So you have to let more water run down the drain — heated water, this time — until the hot water running down the drain has reached its maximum piping-hot temperature.
Then you start adjusting the temperature. This takes significantly longer than doing so with a tanked heater, because each time you touch a faucet, you disturb that delicate equilibrium and have to wait for the temperature to stabilize again (typically it will oscillate after you make a change). During that time, you send even more heated water down the drain. If you don’t, you will find yourself being scalded or frozen by a temperature fluctuation.
And forget about taking a “navy shower” to save water. If you shut the water off to lather up, you have just destroyed the delicate equilibrium you spent so much time and water creating. You’re better off just letting the water run and pushing the shower head to the side so more heated water gets sent down the drain while you soap up, because that will waste less water and energy than starting the process from scratch again.
And it’s not just during showers. Want to quickly wash a few dishes? You’ll send several times more cold water down the drain than with a tanked heater. Going to wash another dish after that? Better just leave the water running, because if you turn it off you’ll destroy the equilibrium, the water will get cold in the middle of that second dish, and you’ll waste at least as much water letting it run and warm up again as you would if you didn’t turn the faucet off.
Somehow, I have the sneaking suspicion that the calculations which show tankless heaters saving energy don’t take into account how the different usage patterns they force cause so much water (mostly heated water, at that) to get sent down the drain, unused.
If I were building a “green” house, I’d focus on other solutions, like super-insulating the water tank and putting it on a timer, to minimize energy usage. I bet it wouldn’t be hard to beat a tankless heater with such strategies, simply because I’d be sending far less hot water down the drain.