"Blasphemy! Ignorance!" I should groan you in my nerdy indignation!
The Harry Potter series has some of the best character development, story-telling, and wit of any contemporary literature. Sure, it's starts a bit "kiddish" and light-hearted - but as the series progresses, the plot gets deeper and the subject matter matures. You're doing yourself a disservice. Steven King called Dolores Umbridge (a character who first really arrives in book 5) "the best written villain since Hannibal Lecter." Read them, for sure. Watch them if you just want to hear the characters' names pronounced. The movies look cool, but take too many liberties to make them screen feasible.