Three-year-olds are definitely like that. My three-year-old's parents (me and my husband!) are together, and some days he wants no one but Mommy, other days only Papa can make him smile. I think your best bet is to be available to him and to offer to do things together, but to continue letting him take the lead. Don't take it personally if he turns you down-- he might just be slow to warm up and need to consider your offer for a while before he takes you up on it.
Reading books together is a great bonding activity. You can definitely find books about trains, cars, trucks, etc. You could bring a couple with you when you come over, let him know they are there for him and offer to read them to him if he wants. He might prefer to look at them on his own, but eventually might be interested in having you read them to him. Even if he wants Dad to read at first, you could probably join him. Maybe at bedtime he can choose one book for dad to read and one book for you to read.
Taking walks or pushing him on the swings at the playground are also great for slow-to-warm-up children. Being outside, doing parallel activities without having to interact. On a walk you can point out interesting things to him, let him decide which direction to go, etc. Swinging can be soothing to kids who are anxious or unsure.
Play dough, coloring or painting, driving cars down a ramp (you can make one out of anything... a board, a book, a shelf, etc), washing dishes (i.e. playing in the sink) together, building the train tracks for him to drive his trains on, making a "road" to drive cars on (masking tape on the rug or chalk on the driveway/sidewalk work well)... anything you can do side-by-side and that can be done silently in parallel would allow him to warm up to you without forcing an interaction those times he doesn't want one.
Some three-year-olds love to help with grown-up jobs, so asking for his help watering plants, making lunch, sweeping up, etc, might catch his interest.
You might also keep your eye out for a special activity the two of you could eventually do together-- storytime at the library or bookstore, an art class, a playgroup. My step-daughter was a toddler when I started parenting her, and we built a strong bond at a parent-child gymnastics class. Despite its name, there were plenty of other non-parents there (like nannies and grandparents) and someone else was leading the activities so I didn't have to try to come up with things to do. Without my boyfriend there to mediate our interactions, it nudged us to find our own relationship and form our own bond. My step-daughter and I (9 years later) still have a very special relationship all our own.
I hope there were some helpful ideas in here! Above all, just try to be a consistent, patient presence. Offer yourself, but don't take it personally if he isn't ready. He's likely just trying to build trust in the relationship, taking his time and maybe trying to figure you out before he commits to a relationship with you. It sounds like you are doing just what he needs you to do. Remember that the relationship you are building is between you and this little guy, and the only people who know what that should look and feel like is you and this little boy-- not his dad, not his mom, not the neighbor or your best friend or the babysitter. Take it as slow as the two of you need to, and don't feel pressured by anyone else's idea if what it out to be.