Feel free to reach me at: π« n4jidx@gmail.com
- π§© PR #147 β Contributed to
runpod-workers/worker-vllm
Added integration and environment support for thebitsandbytesquantization option based on vLLM docs:- Updated
requirements.txtand resolved compatibility issues - Modified
README.mdandworker-config.jsonto reflect the new option - Fixed compatibility issue with
typing-extensions - Added automatic
quantizationfallback logic inengine_args.py - Deployed and tested the feature successfully in a RunPod serverless environment
- Updated
|
|
|
|


