direct PIP installation try

>>>pip install -i https://pypi.tuna.tsinghua.edu.cn/simple box2d-py
...
 error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/

  ----------------------------------------
  Failed building wheel for box2d-py
  ...

we have to go to the wheel
https://www.lfd.uci.edu/~gohlke/pythonlibs/#pybox2d

>>> pip install Box2D-2.3.2-cp36-cp36m-win_amd64.whl

tests the following code:

import gym
env = gym.make('CarRacing-v0')
env.reset()
for _ in range(1000):
    env.render()
    env.step(env.action_space.sample())

the atari wheel here
https://github.com/Kojoley/atari-py/releases