First off, credit goes to the OpenX forums. I didn't bookmark the page, so I can't point you to the exact URL. But this trick is really cool and it will help you comply with Google Adsense TOS.
All you have to do in OpenX to make sure that no more than three Adsense ads are served is this:
- Create three separate "Advertisers" in OpenX for serving Adsense; call them "Adsense 1", "Adsense 2", and "Adsense 3"
- For each of those "Advertisers", make sure you go to the settings for each one of them and check the checkbox that says, "Display only one banner from this advertiser on a web page"
That's all you have to do to make sure that OpenX won't serve more than three Adsense ads per page.
To make matters simpler, if you already have an Advertiser in OpenX set up to serve Adsense, you can duplicate it to make the three "Advertisers" needed to perform this trick.