A virtual environment is recommended for use. A CUDA installation is needed to run the LLM and STT models.
Install dependencies with:
pip install -r requirements.txt
The directory for the trait files, host IP address, and port are hard-coded onto the LLM-BioTalk_plots.py file. You should add the correct paths to run it locally. A HuggingFace API key is also required.
python ./VR-BioTalk-LLM/LLM-BioTalk_plots.py
The infoExtractor.py script depends on the CloudComPy wrapper and should be installed before running the script.
Additional dependencies can be installed with:
pip install open3d tqdm numpy scipy shapely
The file path of the .ply plot files should be hard-coded. You can generate the merged .ply files for plots with the mergePC.py script.
python ./VR-BioTalk-LLM/TraitExtraction/infoExtractor.py
The VR application should be run while the server is already running. The IP address and port number of the host must be set on the Voice2Command gameobject. The application was tested on a Meta Quest 3. For uttering a command, you should press the index trigger and say the commmand while holding the trigger, and stop pressing when you are finished talking.
The headset and the server should be connected on the same network. If this is not possible, a Virtual Local Area Network (VLAN) is recommended, and services like ZeroTier are recommended for this.