First make a virtual environment with Python 3.8, we recommend Python 3.8 because that's the version supported by carla Python SDK 0.9.12 but you can self-compile a supported carla sdk up to Python 3.10.
Before installating ROAR_PY_RL, Make sure that ROAR_PY control suite(https://github.com/augcog/ROAR_PY) is installed and updated to the newest version. You need to install both the interface and the Carla Implementation backend for ROAR_PY.