I mostly ignored Illusion in this playthrough, though I did dabble with every spell type. It's Destruction/Conjuration/Restoration for me, and a bit of Alteration for the armors (though I didn't touch the perks there). Good mix of damage/crowd-control/defense.
Also another thing about the Destruction tree, and why damage not scaling is hardly a big deal - the perk that staggers enemies with dual casts.
That thing is just lol. As far as I can tell, every enemy seems to stagger from a dual cast shot with that perk, even with a lowly Firebolt. You can pretty much catch an enemy in a continous chain of stagger animations as they can't do anything. And people want greater damage on top of that?
I can only speak for myself, but from where I stand it isn't a matter of the game becoming difficult or anything, it's the problem of having a 35 hour progression curve for a game with 150 hours of content. Obviously you hit plateaus with all skills but it happens with destruction extremely rapidly. There's no next sword, or better enchantment or even really anything new to do with it once you get to expert level spells (the master spells are basically worthless, kind of a huge troll after the cool quest to unlock them). Once I got the ability to spam expert destruction spells for zero cost, the feeling of growth was gone. Maybe worse is the fact that the expert level spells are the most boring--shoot damage at single enemy, woooo. Yeah there are other support spells at that level, but I sure would like some more aoe options like the adept spells or something. Also, the other issue with the magicka reduction enchantment options is that they made my 25 levels spent on magicka boosts, as well as the perks I dumped into lowering costs in destruction an almost complete waste. So yeah, destruction in particular is a pretty disappointing skill tree, regardless of it's viability late game.
I made another character to play anyway, but guess what; I can spend almost all of my playthrough focusing on bow/melee up until I max enchanting, spend 3 perk points buffing out the damage on a destruction element of choice, and with a little time spamming spells to boost my skill level, be exactly as effective a destruction mage as I am with a character that spent 50 hours, lots of perks, and lots of stat boosts in the effort. To me, that's lame.
Sorry that got a little long, let me get the point: HELL YEAH I want more damage on top of that!