Tree: Refactor code organization

Move common functions into their own folder and refactor the backends
to use their own folder as well.

Also cleanup imports and alphabetize import statments themselves.

Finally, move colab and docker into their own folders as well.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-01-18 00:42:52 -05:00 committed by Brian Dashore
parent ee99349a78
commit 78f920eeda
22 changed files with 41 additions and 42 deletions

30
docker/Dockerfile Normal file
View file

@ -0,0 +1,30 @@
# Use an official CUDA runtime with Ubuntu as a parent image
FROM nvidia/cuda:12.2.0-devel-ubuntu22.04
# Install Python and pip
RUN apt-get update && apt-get install -y \
python3.11 \
python3-pip \
&& rm -rf /var/lib/apt/lists/*
# Set the working directory in the container
WORKDIR /usr/src/app
# Get requirements
COPY requirements.txt requirements.txt
# Install torch with CUDA support and exllamav2
RUN pip install torch --extra-index-url https://download.pytorch.org/whl/cu121
RUN pip install exllamav2
# Install any other needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
# Copy the current directory contents into the container at /usr/src/app
COPY . .
# Make port 5000 available to the world outside this container
EXPOSE 5000
# Run main.py when the container launches
CMD ["python3", "main.py"]