Our fears of several days without running water turned out, thankfully, not to be realized. As it turns out, we managed to get hold of our landlady last night, and she managed to get hold of her emergency plumber. About an hour and a half later, she and the plumber were here, and about an hour or so after that, we had a new relief valve on the water heater, plus new valves -- ball valves this time, not stopcocks -- on both the inlet and outlet to the water heater, so that should it malf again, we can totally isolate it from the house water system and shouldn't have to shut off all the water to the house again. Turns out, unsurprisingly, the reason the relief valve didn't close is it was clogged almost solid with calcium deposits.
Next week, he'll be coming back to install an expansion tank to prevent it from blowing off the relief again, and then we can turn the water temperature back up so as to make best use of a rather small heater. (It's basically a one-shower-capacity water heater.)