You don’t seem to be binding any command to backspace. That’ll cause various kinds of flakiness. Adding the default or standard keymap seems to solve this problem.
However, adding the keymap solved the probrem but it seems to have caused another problem.
It seems that deleting the next character of the atomic widget also deletes the atomic widget itself.
For instance:
foo<widget>
|bar
then pressing backspace deletes line break and <widget> too.