Hmmm... I guess I'm not understanding what you're seeing then. When I start with a document like this:
(You can't see it, but the text insertion cursor is just after "typed".)
When I uncheck column 2 and hit Return, I get:
It definitely seems like the split behavior is activating, though I don't know why. If you quit OmniOutliner and set your preferences file aside - move it to the desktop, for example - does the problem occur when you restart the app?