Am I making a mistake by preferring IISER Kolkata over IIT Kharagpur MSc Integrated Physics?

No, it is not a mistake to prefer IISER Kolkata over IIT KGP for an MSc integrated course in Physics. You are going to get a pure science degree so it makes more sense to get the degree from an institute that focuses on that degree. IIT KGP is popular for its engineering degrees. So, you will be mostly surrounded by people who are pursuing engineering degrees. You won’t get to meet many like-minded students. The course structure of integrated MSc at IITs is also outdated. 

At IISER Kolkata, you will get a decent course structure and better faculties for an integrated MSc.