Meant to weigh in on this earlier. I have been to both. Four times to W-B, more to Aspen. I think that you should aspire to see both at some point. As others have mentioned, while different, you will certainly whichever you pick.
Right now, with current exchange rates, W-B is a relative bargain. Weather can be iffy, but snow is normally great. It's huge. We don't really love the "village", but it has everything. Yep, people party hard. Very diverse.
Aspen is iconic. It's got tradition, charm, history. It was a town long before skiing. But, as an example, this is the 80th anniversary of the ski club. Yes, a lot of the rich and famous head there. But it's not all like that. We love the skiing {we're serious skiers}. Pretty minimal lifelines, if any. And weather? It either snows, or it's bluebird for the most part. It is just beautiful. Elevation has never bothered us.
If Canada was not a bargain, I would say Aspen. I could probably make a strong hands down case. But I think I could be just as persuasive about W-B!
No bad choice. Two great choices. Visit one, then think about when you can visit the other. I doubt if we'll go back to W-B. We'll visit Aspen quite a bit. Part of it is that travel for us is much longer to W-B. That does not mean that we didn't like W-B.
You kind of find your favorites, in our case, over decades. Aspen is one.
Have fun!