You're right, it's a bit weird, but I just figured that the prompt was standard for plink or something. Like it went <plink prompt><PS1>.
So, playing around some more, I still haven't fixed the plink problem, however, I notice that if I enter a command as a plink argument, it actually goes though. So, I did do that to see what LANG and PS1 were set to, and they were both set correctly.
The problem now is, some of my aliases don't work. If I use the alias command through plink, I see it has all the correct aliases configured. However, when I use my alias to cd to the correct source dir, it says bash: <alias I'm using>: command not found. It seems like others work fine, though I haven't tried them all yet.
Any idea about this?