First, I agree with pp who noted that teens aren't interested in Justin Bieber or the Jonas Brothers - at least, not any teens that I know. They cringe at the thought.
Second, there's a big difference between a 4 y.o and a 10 y.o. I'll guess that most of the Bieber merchandise is owned by 10 y.o's and other "tweens".
Third, my kids have had their own money (allowance, birthday cash etc.) to spend on whatever they wanted from a fairly young age - 5 or so. If they chose to buy celebrity stuff, it's not illegal or immoral, so although I might have suggested (with a lot of persuasion and enthusiasm) alternatives for spending, I don't think I would have outright forbidden it. My kids are older now, but I don't recall them being interested in pop celebrity culture at age 4 or 5. They did like to buy Pokemon or Disney or other pop culture stuff. I didn't object. I didn't encourage it, either. Mostly because I dislike buy junky clutter in the first place.