Skip to content

Commit

Permalink
Update tutorial links
Browse files Browse the repository at this point in the history
  • Loading branch information
okhat committed Dec 25, 2024
1 parent b7337d7 commit ed6b4e3
Show file tree
Hide file tree
Showing 4 changed files with 16 additions and 10 deletions.
3 changes: 3 additions & 0 deletions docs/docs/tutorials/classification/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Please refer to [this tutorial from Drew Breunig](https://www.dbreunig.com/2024/12/12/pipelines-prompt-optimization-with-dspy.html) using DSPy.

This tutorial demonstrates a few aspects of using DSPy in a highly-accessible, concrete context for categorizing historic events with a tiny LM.
6 changes: 4 additions & 2 deletions docs/docs/tutorials/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@

* [Entity Extraction](/tutorials/entity_extraction/)

* [Multi-Hop Search](tutorials/multihop_search/)
* [Classification](/tutorials/classification/)

* [Multi-Hop Search](/tutorials/multihop_search/)

* [Privacy-Conscious Delegation](/tutorials/papillon/)

* [Saving and Loading](tutorials/saving/index.md)
* [Saving and Loading](/tutorials/saving/index.md)

* [Deployment](/tutorials/deployment/)

Expand Down
16 changes: 8 additions & 8 deletions docs/docs/tutorials/rag/index.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"\n",
"You can always prompt the LM directly via `lm(prompt=\"prompt\")` or `lm(messages=[...])`. However, DSPy gives you `Modules` as a better way to define your LM functions.\n",
"\n",
"The simplest module is `dspy.Predict`. It takes a [DSPy Signature](/building-blocks/2-signatures), i.e. a structured input/output schema, and gives you back a callable function for the behavior you specified. Let's use the \"in-line\" notation for signatures to declare a module that takes a `question` (of type `str`) as input and produces a `response` as an output."
"The simplest module is `dspy.Predict`. It takes a [DSPy Signature](/learn/programming/signatures), i.e. a structured input/output schema, and gives you back a callable function for the behavior you specified. Let's use the \"in-line\" notation for signatures to declare a module that takes a `question` (of type `str`) as input and produces a `response` as an output."
]
},
{
Expand Down Expand Up @@ -76,7 +76,7 @@
"source": [
"Notice how the variable names we specified in the signature defined our input and output argument names and their role.\n",
"\n",
"Now, what did DSPy do to build this `qa` module? Nothing fancy in this example, yet. The module passed your signature, LM, and inputs to an [Adapter](/building-blocks/1-language_models#structured-lm-output-with-adapters), which is a layer that handles structuring the inputs and parsing structured outputs to fit your signature.\n",
"Now, what did DSPy do to build this `qa` module? Nothing fancy in this example, yet. The module passed your signature, LM, and inputs to an Adapter, which is a layer that handles structuring the inputs and parsing structured outputs to fit your signature.\n",
"\n",
"Let's see it directly. You can inspect the `n` last prompts sent by DSPy easily.\n"
]
Expand Down Expand Up @@ -204,13 +204,13 @@
"\n",
"You already know a lot about DSPy at this point. If all you want is quick scripting, this much of DSPy already enables a lot. Sprinkling DSPy signatures and modules into your Python control flow is a pretty ergonomic way to just get stuff done with LMs.\n",
"\n",
"That said, you're likely here because you want to build a high-quality system and improve it over time. The way to do that in DSPy is to iterate fast by evaluating the quality of your system and using DSPy's powerful tools, e.g. [Optimizers](/building-blocks/6-optimizers). You can learn about the [appropriate development cycle in DSPy here](/building-blocks/solving_your_task).\n",
"That said, you're likely here because you want to build a high-quality system and improve it over time. The way to do that in DSPy is to iterate fast by evaluating the quality of your system and using DSPy's powerful tools, e.g. Optimizers.\n",
"\n",
"## Manipulating Examples in DSPy.\n",
"\n",
"To measure the quality of your DSPy system, you need (1) a bunch of input values, like `question`s for example, and (2) a `metric` that can score the quality of an output from your system. Metrics vary widely. Some metrics need ground-truth labels of ideal outputs, e.g. for classification or question answering. Other metrics are self-supervised, e.g. checking faithfulness or lack of hallucination, perhaps using a DSPy program as a judge of these qualities.\n",
"\n",
"Let's load a dataset of questions and their (pretty long) gold answers. Since we started this notebook with the goal of building **a system for answering Tech questions**, we obtained a bunch of StackExchange-based questions and their correct answers from the [RAG-QA Arena](https://arxiv.org/abs/2407.13998) dataset. (Learn more about the [development cycle](/building-blocks/solving_your_task) if you don't have data for your task.)\n",
"Let's load a dataset of questions and their (pretty long) gold answers. Since we started this notebook with the goal of building **a system for answering Tech questions**, we obtained a bunch of StackExchange-based questions and their correct answers from the [RAG-QA Arena](https://arxiv.org/abs/2407.13998) dataset.\n",
"\n"
]
},
Expand Down Expand Up @@ -1041,7 +1041,7 @@
"\n",
"If there are many sub-modules in your program, all of them will be optimized together. In this case, there's only one: `self.respond = dspy.ChainOfThought('context, question -> response')`\n",
"\n",
"Let's set up and use DSPy's [MIPRO (v2) optimizer](/deep-dive/optimizers/miprov2). The run below has a cost around $1.5 (for the `medium` auto setting) and may take some 20-30 minutes depending on your number of threads."
"Let's set up and use DSPy's MIPRO (v2) optimizer. The run below has a cost around $1.5 (for the `medium` auto setting) and may take some 20-30 minutes depending on your number of threads."
]
},
{
Expand Down Expand Up @@ -1334,9 +1334,9 @@
"In general, you have the following tools:\n",
"\n",
"1. Explore better system architectures for your program, e.g. what if we ask the LM to generate search queries for the retriever? See, e.g., the [STORM pipeline](https://arxiv.org/abs/2402.14207) built in DSPy.\n",
"2. Explore different [prompt optimizers](https://arxiv.org/abs/2406.11695) or [weight optimizers](https://arxiv.org/abs/2407.10930). See the **[Optimizers Docs](/building-blocks/6-optimizers)**.\n",
"3. Scale inference time compute using DSPy Optimizers, e.g. this [notebook](https://github.com/stanfordnlp/dspy/blob/main/examples/agents/multi_agent.ipynb).\n",
"4. Cut cost by distilling to a smaller LM, via prompt or weight optimization, e.g. [this notebook](https://github.com/stanfordnlp/dspy/blob/main/examples/nli/scone/scone.ipynb).\n",
"2. Explore different [prompt optimizers](https://arxiv.org/abs/2406.11695) or [weight optimizers](https://arxiv.org/abs/2407.10930). See the Optimizers Docs.\n",
"3. Scale inference time compute using DSPy Optimizers, e.g. via ensembling multiple post-optimization programs.\n",
"4. Cut cost by distilling to a smaller LM, via prompt or weight optimization.\n",
"\n",
"How do you decide which ones to proceed with first?\n",
"\n",
Expand Down
1 change: 1 addition & 0 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ nav:
- Agents: tutorials/agents/index.ipynb
- Reasoning: tutorials/math/index.ipynb
- Entity Extraction: tutorials/entity_extraction/index.ipynb
- Classification: tutorials/classification/index.md
- Privacy-Conscious Delegation: tutorials/papillon/index.md
- Multi-Hop Search: tutorials/multihop_search/index.ipynb
- Saving and Loading: tutorials/saving/index.md
Expand Down

0 comments on commit ed6b4e3

Please sign in to comment.