I experienced same symptoms -- swirling rainbow, 2 second delay, on every return to the application. Unchecking quicklook made the problem go away. Painful since no one buys for quicklook preview, but everyone buys to draw a diagram and everyone (on Lion at least) is effected.
On another note...the forum signup process is pretty harsh. Made me think/work pretty hard just to add this 'me too' note.
My document has only one canvas, and around 100 simple shapes (rectangles). I do have a lot of fills w/ opacity around 50%, and some nested shape groupings.