Instead of Hex and LineHex display modes, how about a Hex tool window?
The display in the hex tool would be much like a debugger Memory display window.
It would track the cursor position from the current editor window.
Allowing edits may be tricky, I don't know, but I don't usually care either - I mostly use Hex/LineHex for viewing.