You don't really "fix" the problem, you avoid it by only adding pages in pairs. Alternatively you can override the master items again and delete them, but that's a real paiin when they are behind other things (and master objeccts are always behind non-master objects on any given layer).
And you aren't really seeing two columns. Waht you have is two text frames on top of each other, one with left aligned text and one with right aligned text.