There's a known bug with Bioware's code and CHA-based spellcasters. Loss of CHA buffs causes a progressive loss of available spell slots, as if you had cast the spells. I'm not aware of any fix to this, sadly...though if there is, I'd love to see it.
You can simulate this by repeatedly equipping and unequipping any one piece of CHA stat jewelry you have. Pretty soon, you'll be left with no spells available.
For this reason, I never use CHA buffs, either spells or Ioun stones, for my sorcerer, opting instead for "static" buffs from jewelry and other items that remain equipped all the time.
