Of my children, I would easily take my 6 year old and up (although my 15 year old would not want to go).
With a life jacket, I would not worry about them falling out, and they have no fear, or so it seems.
My 6 yo has been riding the "baby roller coasters" at King's Island since he was 4, and olders went on the Diamond Back rollercoaster (as well as ALL the others there) at age 8, 11, & 13. They LOVE adventure, fast rides, and action. They would LOVE to white water raft. The only one to opt out would be my 15 yo, as she does not like fast things or bumpy rides. And in a year or two, I am sure my almost two year old would also love it.
Perhaps it is just something I have passed down to them, because I love it myself.
As for us, we are an unschooling family, and I was a homeschooled kid myself. But, I don't think that really has anything do do with it other than opening up the times we could go.
And as for Dad, I have had to push him into more "risky" type stuff. It is not his style.