Fixed cv2/numpy bug + Better Setup & Usage Instructions + Fixed pointnav_policy_path for Reality #50
Conversation
Fixed bug pertaining to cv2 requiring int8 input
Updated README.md with fixed setup instructions & instructions for running on spot
Fixed pointnav policy for spot
Instruction to save video from simulator
| git clone [email protected]:WongKinYiu/yolov7.git # if using YOLOv7 | ||
| git clone https://github.com/IDEA-Research/GroundingDINO.git | ||
| git clone https://github.com/WongKinYiu/yolov7.git # if using YOLOv7 | ||
| git clone https://github.com/facebookresearch/habitat-lab |
There was a problem hiding this comment.
This is already under pyproject.toml, installable under pip install -e .[habitat]
There was a problem hiding this comment.
Unfortunately for some reason in my Windows and Linux environment even though pip install -e .[habitat] ran without any errors it didn't git clone the repo's into the directory and files from the yolov7 and groundingdino were required to run the models from what I remember
| The following pip install's will also be required: | ||
|
|
||
| ```bash | ||
| pip install hydra-core --upgrade |
There was a problem hiding this comment.
It would be good to move these to pyproject.toml or the Dockerfile as much as possible.
There was a problem hiding this comment.
True! If I get the chance will move it there nonetheless thought it was useful information incase the requirements in pyproject.toml wasn't enough to get your environment ready
There was a problem hiding this comment.
Both habitat-lab and bd_spot_wrapper have been in the toml file already
| @@ -32,7 +32,8 @@ def process_request() -> Dict[str, Any]: | |||
| payload = request.json | |||
| return jsonify(model.process_payload(payload)) | |||
|
|
|||
| app.run(host="localhost", port=port) | |||
| app.run(host="0.0.0.0", port=port) # app.run(host="localhost", port=port) | |||
There was a problem hiding this comment.
this is for external access?
There was a problem hiding this comment.
This opens up the flask server to the local network. Still allows access to localhost so doesn't need any changes to any other code if models are still hosted on the device connected to spot.
But also allows for the option to host models elsewhere on the network allowing for bigger and better models while having smaller edge compute devices on the spot instead of needing all the compute to be on or near spot. Opens up the option to easily add more models of all sizes if interested
There was a problem hiding this comment.
The incorrect pointnav weights were provided in the reality config file assuming the reality config file correlated with the config to be used on the spot
There was a problem hiding this comment.
cv2 error which was fixed by specifying the input int type (uint8)
Error reproducible in habitat and reality when using original environment setup instructions as of October 1, 2024 since no specific opencv version is specified and any numpy version after "1.22.4" is accepted
|
Installation instructions have been updated to fix the cv2 error by simply downgrading the cv2 version. |
Fixed CV2/Numpy Bug:
Bug Source:
Bug:
Bug has been detected in multiple versions of OpenCV(4.5 to 4.8) and has been fixed in detection.py
Environment setup
Environment setup is missing certain libraries and some details which has been added + fixed
Issues: #34 ,
Usage Instructions
Better instructions for usage on simulator and on SPOT.
Issues: #29 , #26
pointnav_policy_path for reality
pointnav_policy_path for reality on spot was incorrect and has now been fixed.