Look in System Preferences>Keyboard & Mouse>Keyboard Shortcuts. The system wide full keyboard access shortcuts for switching focus to various on screen elements such as windows and palettes are all listed there.
Once you bring up the colour panel with command-shift-C, control-F6 will give it focus and either command-`/~ or control-F4 will put the focus back to the document window.
Unfortunately, the OmniGraffle inspectors do not appear to be linked into this system and I can't work out how to give them focus using FKA shortcuts.