I found the problem, or better, the solution to the problem. If the LANG variable is not set, you get this behaviour. As soon as as you set LANG to something (need not match the keyboard language), it works.
I remembered that OpenOffice once (or still?) showed the same behaviour. Probably has to do with some library used by both these products.
Probably something for the FAQ!