Aha! I was finally able to reproduce. I've gotten a few error messages, some about an invalid index into an NSArray but also the NSUndoManager ones. They just started appearing again.
One of the NSUndoManager alerts popped up and I stopped what I was doing to see what I could do to make it happen again. If I do the 2 finger scroll gesture, another alert pops up. Basically, I can make as many pop up as I want simply by making that gesture once one fires. If I get rid of them all and then immediately make that gesture again, it will pop up. If instead I click around and scroll using the scroll bars, they don't pop up anymore. I'm not sure what initially triggers them, but hopefully these extra details will help.
One of the NSUndoManager alerts popped up and I stopped what I was doing to see what I could do to make it happen again. If I do the 2 finger scroll gesture, another alert pops up. Basically, I can make as many pop up as I want simply by making that gesture once one fires. If I get rid of them all and then immediately make that gesture again, it will pop up. If instead I click around and scroll using the scroll bars, they don't pop up anymore. I'm not sure what initially triggers them, but hopefully these extra details will help.