Trekkin Girl, In general holidays always cause a bit of extra expense. And hassle! Memories of these things are good for kids, though. I am not really sure what you mean by buying baskets, because maybe you buy filled ones? But I wanted to be on record for admitting that for all of my 14 years of mothering, I have simply gone into my shelves, emptied out some baskets I use for storage (bought at yard sales) and used them for Easter baskets. My kids use them to collect their eggs (plastic ones purchased years ago and washed and reused every Easter). The eggs are filled with Annies bunny graham crackers, Annies bunny cheddar crackers (on sale this week!), a single box of trader joes jelly beans. A couple of Snickers in the egg shape and some homemade hot cross buns (with grated orange and lemon peel instead of citron) pretty much round out the morning. At the end of the day (or week, or whenever), the eggs get rewashed and stored again, the baskets just go back to their usual spot. And my kids have not eaten too much junk.
I determined a long time ago NOT to cave into the crazy holiday thing of massive buying and junk food eating but still make things special. It is possible to move away from the expense a bit. Not too far because if you go too far, it just isn't special. But you can definitely strike a happy medium. A friend of mine always made a bunny cake for Easter -- it was simply a matter of cutting a regular rectagular cake into pieces for a square head and triangular ears and frosting it. The kids love that! Especially because they don't regularly get sweets.
But I feel for you: sometimes it is so hard to try to do the good mama thing and still stay on a very tight budget.