The problem is that this requires a program called git-receive-pack on the server, which I have installed with the rest of the git tools in $HOME/bin. However, apparently when connecting via ssh to a non-interactive session, $HOME/bin isn't in $PATH, because I get an error message from the server end indicating that git-receive-pack couldn't be found.
I tried putting
PATH=$PATH:/home/<myusername>/bin
in ~/.ssh/environment, but that didn't help.
Any hints are appreciated. (I can get it to work by using one of git-push's options that allows me to specify the path to git-receive-pack on the server, but I'd rather not have to keep doing that.)