It really helps to toss down a small towel to wipe your feet on as you go. Then when you're done sweeping, you haven't tracked crumbly bits behind you. (E.g. 1:43 in this video
) A hand towel fits in any load of laundry.
Also, when you're clearing the floor in preparation for other cleaning, only do the middles of the floor. Don't move furniture, don't squiggle the broom into the corners, just get the big open areas that are easy to reach.
And only pick up big things and gross things before sweeping. It doesn't hurt blocks to be swept up with the dust they were sitting in in the first place and it's waaaaay less work to sweep everything into a pile rather than dashing around after fiddly bits.