My experience as a parent: Because I work full-time, my son's activities have always been tied to my work schedule. We felt that his preschool offered a great variety of activities and he didn't have time for any other classes. The summer before kindergarten, my partner was under-employed and what work he was doing was from home, so our son got a summer vacation between schools and took half-day kids' classes at the museum. He really likes the museum classes (art and natural history, different ones offered each week) and they are full-day for age 6+, so each summer since then he has gone to the museum most weeks and spent a few weeks at home with Dad, but that's been his only extracurricular until this spring: He is joining the school garden club and will be staying after school once a week to work in the organic garden. There's no charge for that. His school has many more clubs for the older grades, so I expect he'll get involved in some. He is not into sports. His dad has taught him a bit of piano, and he likes it but has not expressed interest in taking lessons. He's very interested in acting and stagecraft, but for young kids in our area the only available theater groups require a serious Saturday commitment, which we're not willing to do--so he'll have to wait until he's old enough for school plays.
My experience as a child: I took scads of low-cost classes offered through the YWCA and gifted&talented organization. These were usually just a few weeks long, and my parents were pretty tolerant about driving me there or arranging carpools. I was interested in dance, but my parents would not commit to lessons until I could get myself to and from them, in 3rd grade, by walking or riding my bike to the dance studio a few blocks away; then I took dance twice a week through high school; my parents made it clear that they would pay for it only if I attended diligently, so I did. I joined Girl Scouts in 2nd grade; it was low-cost and the meetings were at my school, so I could walk home. In 5th grade I wanted to play in the school orchestra because my best friend did, and this is the activity my parents probably should have said no to--I'm not musically talented, the viola was expensive, and then my mom's friend who taught private lessons talked her into having me take Suzuki method lessons, for which my mom was supposed to be present the whole time and then coach me at home, and we both hated it yet kept it up for 2 years. Also, that meant I had activities 4 days a week every week all school year, which was a bit much.
I think in your situation, I would investigate the low-cost activities that are convenient for you to get to, and have your daughter choose ONE to start with. See how it goes: Does she like it, is she dedicated to going every week, does it seem to be improving her skills? I think it's fine to tell your son this is something you can do when you're 5--let him build anticipation!
If it's feasible, I would go with swimming lessons before any other sport because being able to swim can save your life. My son gets swimming lessons in school, and I'm very grateful--it's unusual for an elementary school to have a pool!