Yep, I am very similar to you OP. I used to feel a lot of guilt about it, but fortunately for me my kids are (and always have been) very interested in doing their own thing, and then showing/explaining to me rather than having me accidentally mess it up.

Now we are enjoying doing more things together, since they are older (6, 6, and 7) and I can find common spheres of interest. I try not to structure too many things at home, I feel they get so much of that at school and outside activities that I want home to be down time. (also, I am not really one of those organized craft-a-day type of people by personality, so when I try to do that I get burnt out and frankly it's fake.)
Have you asked the olders if there's a project they would like to do together with you? Or are there projects you're interested in that they could help with? I'm getting ready to remodel the playroom in august and sept. But I'm going to enlist the kids' help to paint shelves, do some stencilling, ect. To practice using stencils I took them with me to a craft store and we got some cheap ones plus a wood box/shape that each got to pick out, and then I showed them how to use them and then let them experiment. That's turned into an enjoyable thing for them so probably once a week or so they're painting a new project or adding to their boxes, and I showed them where the craft books are in the library (in the grown up section as well as the kids).
I'm not into building with legos, but I like seeing what others have built, so we've been to a mini exhibit at our library showcasing what older kids have built as well as them incorporating robotics into their lego creations. We happened to run in to one of the "coaches" of those teams, and got to see some of the kids in action. So that infused their lego play with that angle--I do research online and show the kids.
They did an art-wire project at a camp, so I've bought craft wire for them and we've learned a few basic things together.
I love Japanese cuisine and bento boxes, my daughter in particular loves it too so we've learned how to make cute things together. My sushi rolling is sloppy, so we're going to have a friend teach us how to do it right.
I think it helped me to stop looking at things like "OMG I need to do kids stuff!" Some people are really good at that. But for me personally, I love to introduce MY stuff to kids. Even when I was a preschool teacher and nanny, I tended to involve the kids in real-world stuff, with real tools, ect. I was pretty lucky in that I always had kids who ate that stuff up.
So I think you don't need to feel guilty. Just relax, maybe try to think about things you enjoy and how you can incorporate your kids into them just as much as you think about the reverse.