You can't do that because you'd lose the automatic bids to tournaments (i.e. coaches). I'm sure the BCS conferences felt the same way about some of their weaker members at one time or another or added an alternate choice when they expanded.
When a block of some of the original WAC members bolted and formed the Mountain West they surrendered bowl and broadcast revenue for several years. We can't wait out something like that out like BYU, Utah, Air Force, San Diego State, Wyoming, UNLV, and New Mexico did.
The only thing you can control is what you're doing. I'd like to do what you're suggesting but I think your only options are to try to get into another conference or work to build up the conference you're in.