I think in terms of categories...fine motor, active play/ large motor, imaginative play, art supplies, music and media, group games, "educational stuff", construction equipment, books.
Imaginative play...play kitchen, play wooden or plastic food, bowls, spoons/ forks, play silks (I really hesitated, but they love, love, love these), dress up clothes - hats and such, some cars and trucks, a little Thomas train set that they build the track for, Fisher Price Little People farm, set of plastic dinosaurs, dolls and stuffed animals, baby blankets and washcloths (regular ones, and hand towels, they use them with the dolls all the time), a large toy horse, a toy shopping cart (Little Tykes), doctor kit, some baskets and pails. I plan to get some play stands when I can afford them.
Active play: balls of different sizes, bean bags to toss, balloons (I hung one up from a doorway and they hit it like a pinata), a hoop, push bike/ tricycle, rocking horse, a slide my husband built that has wide steps and a railing, can play underneath it too (they play on it all the freaking time and it's the center for visiting children from young toddler up till about age 10. I can even slide on it and climb underneath it. It's a slide, a bridge for trolls, a barn for animals, a clothes line, a boat, etc....) A swingset or swing would be fabulous, scooter board, a little balance beam (we made one for about $6), a hop ball, rope ladder, trapeze or twizzler.
Art supplies: easel, paints and paint brushes (tempra, watercolor, fingerpaint), crayons, colored pencils, chalk, scissors and glue, tape, play dough, liquid starch and cornstarch for making goop, tissue paper, construction and drawing paper, butcher paper, magic markers (the short ones especially), a magna doodle, stamps and stamp pad
Construction toys: blocks of all kinds (unit blocks, large cardboard blocks, tree blocks, magnetic blocks, legos/ duplos, zoom tools)
Fine motor: I use a lot of Montessori toys (Michael Olaf, Montessori N Such catalogues are great). I took a shoe box and cut a slit and my 15 month old will spend a long time putting poker chips in through the slit. My 2.5 yr old likes putting coins in piggy banks. Puzzles, esp with pegs. Fishing poles and fish with magnets on them. sorting activities, lacing cards, peg board, hammering board, creamer pitchers for practice pouring, baster and eyedropper, bin with rice and measuring spoons and bowls (sand, beans, salt, flour, etc work well too), scrub brush, broom and dustpan, spray bottle, tweezers, cloths, dish pans....Awakening Your Toddler's Love of Learning is a great resource book!
Games: Goodnight Moon, candyland, rhyming and opposite puzzles, art memory, color/ shape bingo
rhythm instruments, cds, xylophone
Educational: at this age, I view the fine and gross motor and imaginative play/ art stuff as the primary work. I do a lot of Montessori practical life activities and music. We do have some educational dvds (Signing Time, Preschool Power, Leap Frog) and a Fridge Phonics. For your 3.5 yr old, I'd look at things like abc puzzles, sandpaper numbers, number balance scales, geoboards, unifix cubes/ cuissinarie rods, rhyming and sequencing and number puzzles