They did something similar when replacing the kitchen taps some years ago, so if I take the tap head off, it doesn't produce a fountain.

But it looks like the kitchen was built around the water heater, so I reckon it's going to be major work to replace it, anyway.