Thinking about holding back my 2nd grader

My son is very young for his grade. He was born in mid August and his school has a September cutoff. Some of the kids in his class are a whole year older than him. His first report card from 2nd grade was not good at all.... When we met with his teacher she said he can't sit still in the classroom, and if he doesn't get something he doesn't ask for help, he just doesn't do it.

Although he's younger age shows in the classroom, physically he's as fast and tall (or even taller) than his classmates and loves playing baseball with friends from class.

My husband and I have been thinking about holding him back in 2nd grade, but we're afraid it might affect his self-esteem or create social challenges. We don’t want him to feel like he's falling behind, but we also want to give him the best chance to catch up and thrive. We're trying to figure out if holding him back could help him focus more in school or if it would just make things harder for him socially. It’s a tough call. What do you guys think?