Running ddd on the target is not the same thing. If you ran ddd on the same machine as SE is installed then you'd be using your cross-gdb and that would be the same as the SE setup (but without SE). The problem in undoubtedly to do with remote debugging, and my uneducated guess would be that it's a (cross)gdb problem, but since you're posting here there's a small possibility that SE isn't helping at all. Running a gui debugger (ddd or insight) on your dev machine would replicate both gdb and the method of driving it to ensure that it isn't a SE problem (or show that it is, of course).
Is it only a stepping problem? That is, if you step to a lib call then step over it (instead of into it) does it all still work? If so, it might be something like the running lib (on your target) is stripped whereas the original (on your dev system) is not, leading to breakpoints being put at the wrong location.