Hi. 

When I go to google maps. I can't pan left or right (up and down work fine). C-h b showed that the C-f
and C-b keybindings are declared twice, once for forward and backward of one character and the other for
panning maybe this is the source of the problem?

Unable to reproduce problem.

  "... keybindings are declared twice, once for forward and backward of
   one character and the other for panning maybe this is the source of the
   problem?"

No, google_maps_keymap is at the top of the keymap stack, and therefore
any bindings in it shadow bindings in lower keymaps.