Skip to content

Commit

Permalink
nits
Browse files Browse the repository at this point in the history
  • Loading branch information
dglogo committed Jan 6, 2025
1 parent 65a3227 commit 256722d
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 7 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ For more information about the PDF, Agent and TTS service flows, please refer to
- Redis - [Redis](https://redis.io/)
- Storage - [MinIO](https://minio.io/)

> **Note:** Since NVIDIA blueprints are adaptable to your specific business use case and/or infrastructure, the above software components are configurable. For example, to decrease the amount of GPU memory required, you can leverage a smaller LLama 3.1-8B NIM and disable GPU usage for Docling in docker-compose.yaml.
> **Note:** Since NVIDIA blueprints are adaptable to your specific business use case and/or infrastructure, the above software components are configurable. For example, to decrease the amount of GPU memory required, you can leverage a smaller Llama 3.1-8B NIM and disable GPU usage for Docling in docker-compose.yaml.
Docker Compose scripts are provided which spin up the microservices on a single node. The Blueprint contains sample use-case PDFs but Developers can build upon this blueprint, by using their own PDFs based upon their specific use case.

Expand Down Expand Up @@ -150,11 +150,11 @@ The make `model-dev` target will let you spin up only the docling service.

2. **Use Self-hosted NIM**

By default this blueprint uses an ensemble of 3 LLMs to generate podcasts. The example uses the LLama 3.1-8B, LLama 3.1-70B, & LLama 3.1-405B NIMs for balanced performance and accuracy. To use a different model, update the models.json file with the desired model. The default models.json calls an NVIDIA-hosted API Catalog endpoints. This is the default configuration and is recommended for most users getting started with the blueprint but once you want to adapt the blueprint, locally hosted NIM endpoints are required.
By default this blueprint uses an ensemble of 3 LLMs to generate podcasts. The example uses the Llama 3.1-8B, Llama 3.1-70B, & Llama 3.1-405B NIMs for balanced performance and accuracy. To use a different model, update the models.json file with the desired model. The default models.json calls an NVIDIA-hosted API Catalog endpoints. This is the default configuration and is recommended for most users getting started with the blueprint but once you want to adapt the blueprint, locally hosted NIM endpoints are required.

3. **Change the Default Models and GPU Assignments**

It is easy to swap out different pieces of the stack to optimize GPU usage for available hardware. For example, minimize GPU usage by swapping in the smaller LLama 3.1-8B NIM and disabling GPU usage for docling in docker-compose.yaml.
It is easy to swap out different pieces of the stack to optimize GPU usage for available hardware. For example, minimize GPU usage by swapping in the smaller Llama 3.1-8B NIM and disabling GPU usage for docling in docker-compose.yaml.

4. **Enable Tracing**

Expand Down
10 changes: 6 additions & 4 deletions launchable/PDFtoPodcast.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"\n",
"- Transforms dense PDF documents into natural, engaging conversations\n",
"- Creates AI-generated podcasts with either single-speaker or two-person formats\n",
"- Uses cutting-edge language models (powered by LLama 3.1-70B NIM, LLama 3.1-8B NIM, and LLama 3.1-405B NIM) to ensure high-quality content\n",
"- Uses cutting-edge language models (powered by Llama 3.1-70B NIM, Llama 3.1-8B NIM, and Llama 3.1-405B NIM) to ensure high-quality content\n",
"- Leverages ElevenLabs' voice synthesis for natural-sounding audio"
]
},
Expand Down Expand Up @@ -82,9 +82,11 @@
"id": "5cec102b-21d3-441c-a1a8-e72dbfc6c6fd",
"metadata": {},
"source": [
"### Generate an API Key for NIM Endpoints\n",
"### Get an NVIDIA NIM Trial API Key\n",
"\n",
"This NVIDIA API Catalog key will be used to access cloud hosted models in API Catalog.\n",
"Prior to getting started, you will need to create API Keys to access NVIDIA NIM trial hosted endpoints.\n",
"\n",
"If you don’t have an NVIDIA account, you will be asked to sign-up. Each user gets a 1000 API trial credits upon signup to try NVIDIA NIM models.\n",
"\n",
"You can use different model API endpoints with the same API key."
]
Expand All @@ -102,7 +104,7 @@
"id": "1b707189-4848-4ebc-ab4e-d55d7084bff5",
"metadata": {},
"source": [
"2. Select a model, such as llama3-8b-instruct.\n",
"2. Select a model, such as Llama3-8b-instruct.\n",
" "
]
},
Expand Down

0 comments on commit 256722d

Please sign in to comment.