Those look to me like Anchored Objects with a custom position. That's another attribute that can be added to an object style. That's how the images and captions were handled here:
This book was done in CS4 before you could drag and drop to anchor, which makes things even easier. The only complication I see here is text wrap, which for custom anchored objects only affects text in lines following the line with the anchor. Note that the anchor point, marked by the Yen symbol, is in front of the word Notes. In your case it might be necessary to add a blank paragraph with leading set to 0 before each paragraph that requires a heading to give it a place to anchor.