It's not really about caring, as a child. It's self-defence, and I don't think Sherlock would necessarily be immune; when you're a kid basically everyone has more power than you. Peter stopped talking, essentially, because he knew he'd be whipped if he said the wrong thing, and he was forever saying the wrong thing. The way Sherlock flaunts authority now, I can imagine he came into sharp clashes with it as a child and didn't usually come out the victor.