Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds compatibility with NVIDIA AI Workbench #6

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 56 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -11,4 +11,59 @@ get_helm.sh
.ruff_cache
data/minio
eval.txt
.DS_Store
.DS_Store

# Ignore generated or temporary files managed by the Workbench
.project/*
!.project/spec.yaml
!.project/configpacks

# General ignores

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# Temp directories, notebooks created by jupyterlab
.ipynb_checkpoints
.Trash-*/
.jupyter/
.local/

# Python distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Workbench Project Layout
10 changes: 10 additions & 0 deletions .project/configpacks
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
*defaults.ContainerUser
*bash.PreBuild
*defaults.EnvVars
*defaults.Readme
*defaults.Entrypoint
*apt.PackageManager
*bash.PreLanguage
*python.PipPackageManager
*bash.PostBuild
*jupyterlab.JupyterLab
133 changes: 133 additions & 0 deletions .project/spec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
specVersion: v2
specMinorVersion: 2
meta:
name: pdf-to-podcast
image: project-pdf-to-podcast
description: ""
labels: []
createdOn: "2024-12-06T19:19:11Z"
defaultBranch: main
layout:
- path: docs/
type: code
storage: git
- path: samples/
type: code
storage: git
- path: services/
type: code
storage: git
- path: shared/
type: code
storage: git
- path: tests/
type: code
storage: git
- path: launchable/
type: code
storage: git
- path: frontend/
type: code
storage: git
environment:
base:
registry: nvcr.io
image: nvidia/ai-workbench/python-basic:1.0.2
build_timestamp: "20241001182612"
name: Python Basic
supported_architectures: []
cuda_version: ""
description: A Python Base with Jupyterlab
entrypoint_script: ""
labels:
- ubuntu
- python3
- jupyterlab
apps:
- name: jupyterlab
type: jupyterlab
class: webapp
start_command: jupyter lab --allow-root --port 8888 --ip 0.0.0.0 --no-browser
--NotebookApp.base_url=\$PROXY_PREFIX --NotebookApp.default_url=/lab --NotebookApp.allow_origin='*'
health_check_command: '[ \$(echo url=\$(jupyter lab list | head -n 2 | tail
-n 1 | cut -f1 -d'' '' | grep -v ''Currently'' | sed "s@/?@/lab?@g") | curl
-o /dev/null -s -w ''%{http_code}'' --config -) == ''200'' ]'
stop_command: jupyter lab stop 8888
user_msg: ""
logfile_path: ""
timeout_seconds: 60
icon_url: ""
webapp_options:
autolaunch: true
port: "8888"
proxy:
trim_prefix: false
url_command: jupyter lab list | head -n 2 | tail -n 1 | cut -f1 -d' ' | grep
-v 'Currently'
programming_languages:
- python3
icon_url: https://workbench.download.nvidia.com/static/img/ai-workbench-icon-rectangle.jpg
image_version: 1.0.5
os: linux
os_distro: ubuntu
os_distro_release: "22.04"
schema_version: v2
user_info:
uid: ""
gid: ""
username: ""
package_managers:
- name: apt
binary_path: /usr/bin/apt
installed_packages:
- curl
- git
- git-lfs
- python3
- gcc
- python3-dev
- python3-pip
- vim
- name: pip
binary_path: /usr/bin/pip
installed_packages:
- jupyterlab==4.2.4
package_manager_environment:
name: ""
target: ""
execution:
apps:
- name: frontend
type: custom
class: webapp
start_command: PROXY_PREFIX=$PROXY_PREFIX python3 -m frontend
health_check_command: curl -f "http://localhost:7860/"
stop_command: pkill -f "^python3 -m frontend"
user_msg: ""
logfile_path: ""
timeout_seconds: 30
icon_url: ""
webapp_options:
autolaunch: true
port: "7860"
proxy:
trim_prefix: false
url: http://localhost:7860/
resources:
gpu:
requested: 0
sharedMemoryMB: 0
secrets:
- variable: ELEVENLABS_API_KEY
description: ""
- variable: NVIDIA_API_KEY
description: ""
mounts:
- type: project
target: /project/
description: Project directory
options: rw
- type: volume
target: /nvwb-shared-volume/
description: ""
options: volumeName=nvwb-shared-volume
40 changes: 40 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -10,6 +10,10 @@ The blueprint accepts a Target PDF and optionally multiple Context PDFs. The Tar

For more information about the PDF, Agent and TTS service flows, please refer to the mermaid [diagram](docs/README.md)

| :exclamation: Important |
| :-----------------------|
| Users running this blueprint with [NVIDIA AI Workbench](https://www.nvidia.com/en-us/deep-learning-ai/solutions/data-science/workbench/) should skip to the quickstart section [here](#nvidia-ai-workbench-quickstart-)! |

## Software Components
- NVIDIA NIM microservices
- Response generation (Inference)
@@ -195,3 +199,39 @@ We use GitHub Actions for CI/CD. We run the following actions:
- Implement proper certificate management
- Configure appropriate security headers
- Follow other web security best practices

## NVIDIA AI Workbench Quickstart [![Open In AI Workbench](https://img.shields.io/badge/Open_In-AI_Workbench-76B900)](https://ngc.nvidia.com/open-ai-workbench/aHR0cHM6Ly9naXRodWIuY29tL05WSURJQS1BSS1CbHVlcHJpbnRzL3BkZi10by1wb2RjYXN0)

If you do not NVIDIA AI Workbench installed, first complete the installation for AI Workbench [here](https://www.nvidia.com/en-us/deep-learning-ai/solutions/data-science/workbench/).

Let's get started!

1. Fork this Project to your own GitHub namespace and copy the link.

```
https://github.com/[your_namespace]/<project_name>
```

2. Open the NVIDIA AI Workbench Desktop App. Select a location to work in.

3. Clone this Project onto your desired machine by selecting **Clone Project** and providing the GitHub link.

4. Wait for the project to build. You can expand the bottom **Building** indicator to view real-time build logs.

5. When the build completes, set the following configurations.

* `Environment` &rarr; `Secrets` &rarr; `Configure`. Specify the ``NVIDIA_API_KEY`` and ``ELEVENLABS_API_KEY`` Key.
* (Optional) Add a ``SENDER_EMAIL`` variable and a ``SENDER_EMAIL_PASSWORD`` secret to the project to use the email functionality on the frontend application. Gmail sender accounts are currently supported; you can create an App Password for your account [here](https://support.google.com/mail/answer/185833).

6. Navigate to `Environment` &rarr; `Compose` and **Start** the Docker compose services. You can view progress under **Output** on the bottom left and selecting **Compose** logs from the dropdown. It may take a few minutes to pull and build the services.

* To run the blueprint with a _locally-running_ Llama 3.1 8B Instruct NVIDIA NIM, be sure to specify the ``local`` profile from the profile dropdown before selecting **Start**.

7. (Option 1) Run the **Jupyter Notebook**. On the top right of the AI Workbench window, select **Open Jupyterlab**. Navigate to ``launchable/PDFtoPodcast.ipynb``, skip the setup sections, and get started immediately with the provided sample PDFs.

8. (Option 2) Run the **Frontend application**. On the top right of the AI Workbench window, select **Open Frontend**.

* Upload your own locally-stored, custom PDFs
* View and download your generated podcast locally
* Specify your agent parameters (local vs Build endpoints),
* (optional) Email your generated podcast to a recipient
27 changes: 26 additions & 1 deletion docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,28 @@
services:
local-nim:
image: nvcr.io/nim/meta/llama-3.1-8b-instruct:latest
runtime: nvidia
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [ gpu ]
ports:
- "8000:8000"
volumes:
- type: bind
source: ~/.cache/nim
target: /opt/nim/.cache/
environment:
- NIM_MODEL_PROFILE=193649a2eb95e821309d6023a2cabb31489d3b690a9973c7ab5d1ff58b0aa7eb
- NGC_API_KEY=${NVIDIA_API_KEY:?Error NVIDIA_API_KEY not set}
networks:
- app-network
profiles:
- local

redis:
image: redis:latest
ports:
@@ -16,7 +40,7 @@ services:
- MINIO_ROOT_USER=minioadmin
- MINIO_ROOT_PASSWORD=minioadmin
volumes:
- ./data/minio:/data
- minio_data:/data
command: minio server /data --console-address ":9001"
networks:
- app-network
@@ -150,6 +174,7 @@ services:
volumes:
redis_data:
pdf_temp:
minio_data:

networks:
app-network:
Empty file added docs/.gitkeep
Empty file.
Empty file added frontend/.gitkeep
Empty file.
14 changes: 14 additions & 0 deletions frontend/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Loading
Loading