I was a late November baby, in a place with a December 31 cutoff. I wasn't held back - instead, I was registered in a French immersion program where the teacher made us ask to go to the bathroom in French in the first week! I don't regret it. I was always ahead of my class academically, even if I was smaller until puberty and a bit awkward socially (though I think this had less to do with age and more with intellectual mismatch with my peers). In high school, I skipped a grade in math and accelerated my science and English classes. Being in my "age appropriate" classroom was bad enough. Being held back would have been miserable.
Basically, it's all very individual. I know I had classmates (mostly boys) who probably benefitted from starting school a little later. I think it should be about how ready the kid is to learn in that environment, not about the desire to make them "competitive" or to not be the youngest in their class. Kids who are the youngest in their class can be the best readers, or the first to master the multiplication tables, just as much as the oldest kid can.