diff --git a/docs/docs/tutorials/classification/index.md b/docs/docs/tutorials/classification/index.md new file mode 100644 index 000000000..bd1037440 --- /dev/null +++ b/docs/docs/tutorials/classification/index.md @@ -0,0 +1,3 @@ +Please refer to [this tutorial from Drew Breunig](https://www.dbreunig.com/2024/12/12/pipelines-prompt-optimization-with-dspy.html) using DSPy. + +This tutorial demonstrates a few aspects of using DSPy in a highly-accessible, concrete context for categorizing historic events with a tiny LM. \ No newline at end of file diff --git a/docs/docs/tutorials/index.md b/docs/docs/tutorials/index.md index 367bb172c..604da0859 100644 --- a/docs/docs/tutorials/index.md +++ b/docs/docs/tutorials/index.md @@ -6,11 +6,13 @@ * [Entity Extraction](/tutorials/entity_extraction/) -* [Multi-Hop Search](tutorials/multihop_search/) +* [Classification](/tutorials/classification/) + +* [Multi-Hop Search](/tutorials/multihop_search/) * [Privacy-Conscious Delegation](/tutorials/papillon/) -* [Saving and Loading](tutorials/saving/index.md) +* [Saving and Loading](/tutorials/saving/index.md) * [Deployment](/tutorials/deployment/) diff --git a/docs/docs/tutorials/rag/index.ipynb b/docs/docs/tutorials/rag/index.ipynb index 45ce46b7c..d8b6cd927 100644 --- a/docs/docs/tutorials/rag/index.ipynb +++ b/docs/docs/tutorials/rag/index.ipynb @@ -41,7 +41,7 @@ "\n", "You can always prompt the LM directly via `lm(prompt=\"prompt\")` or `lm(messages=[...])`. However, DSPy gives you `Modules` as a better way to define your LM functions.\n", "\n", - "The simplest module is `dspy.Predict`. It takes a [DSPy Signature](/building-blocks/2-signatures), i.e. a structured input/output schema, and gives you back a callable function for the behavior you specified. Let's use the \"in-line\" notation for signatures to declare a module that takes a `question` (of type `str`) as input and produces a `response` as an output." + "The simplest module is `dspy.Predict`. It takes a [DSPy Signature](/learn/programming/signatures), i.e. a structured input/output schema, and gives you back a callable function for the behavior you specified. Let's use the \"in-line\" notation for signatures to declare a module that takes a `question` (of type `str`) as input and produces a `response` as an output." ] }, { @@ -76,7 +76,7 @@ "source": [ "Notice how the variable names we specified in the signature defined our input and output argument names and their role.\n", "\n", - "Now, what did DSPy do to build this `qa` module? Nothing fancy in this example, yet. The module passed your signature, LM, and inputs to an [Adapter](/building-blocks/1-language_models#structured-lm-output-with-adapters), which is a layer that handles structuring the inputs and parsing structured outputs to fit your signature.\n", + "Now, what did DSPy do to build this `qa` module? Nothing fancy in this example, yet. The module passed your signature, LM, and inputs to an Adapter, which is a layer that handles structuring the inputs and parsing structured outputs to fit your signature.\n", "\n", "Let's see it directly. You can inspect the `n` last prompts sent by DSPy easily.\n" ] @@ -204,13 +204,13 @@ "\n", "You already know a lot about DSPy at this point. If all you want is quick scripting, this much of DSPy already enables a lot. Sprinkling DSPy signatures and modules into your Python control flow is a pretty ergonomic way to just get stuff done with LMs.\n", "\n", - "That said, you're likely here because you want to build a high-quality system and improve it over time. The way to do that in DSPy is to iterate fast by evaluating the quality of your system and using DSPy's powerful tools, e.g. [Optimizers](/building-blocks/6-optimizers). You can learn about the [appropriate development cycle in DSPy here](/building-blocks/solving_your_task).\n", + "That said, you're likely here because you want to build a high-quality system and improve it over time. The way to do that in DSPy is to iterate fast by evaluating the quality of your system and using DSPy's powerful tools, e.g. Optimizers.\n", "\n", "## Manipulating Examples in DSPy.\n", "\n", "To measure the quality of your DSPy system, you need (1) a bunch of input values, like `question`s for example, and (2) a `metric` that can score the quality of an output from your system. Metrics vary widely. Some metrics need ground-truth labels of ideal outputs, e.g. for classification or question answering. Other metrics are self-supervised, e.g. checking faithfulness or lack of hallucination, perhaps using a DSPy program as a judge of these qualities.\n", "\n", - "Let's load a dataset of questions and their (pretty long) gold answers. Since we started this notebook with the goal of building **a system for answering Tech questions**, we obtained a bunch of StackExchange-based questions and their correct answers from the [RAG-QA Arena](https://arxiv.org/abs/2407.13998) dataset. (Learn more about the [development cycle](/building-blocks/solving_your_task) if you don't have data for your task.)\n", + "Let's load a dataset of questions and their (pretty long) gold answers. Since we started this notebook with the goal of building **a system for answering Tech questions**, we obtained a bunch of StackExchange-based questions and their correct answers from the [RAG-QA Arena](https://arxiv.org/abs/2407.13998) dataset.\n", "\n" ] }, @@ -1041,7 +1041,7 @@ "\n", "If there are many sub-modules in your program, all of them will be optimized together. In this case, there's only one: `self.respond = dspy.ChainOfThought('context, question -> response')`\n", "\n", - "Let's set up and use DSPy's [MIPRO (v2) optimizer](/deep-dive/optimizers/miprov2). The run below has a cost around $1.5 (for the `medium` auto setting) and may take some 20-30 minutes depending on your number of threads." + "Let's set up and use DSPy's MIPRO (v2) optimizer. The run below has a cost around $1.5 (for the `medium` auto setting) and may take some 20-30 minutes depending on your number of threads." ] }, { @@ -1334,9 +1334,9 @@ "In general, you have the following tools:\n", "\n", "1. Explore better system architectures for your program, e.g. what if we ask the LM to generate search queries for the retriever? See, e.g., the [STORM pipeline](https://arxiv.org/abs/2402.14207) built in DSPy.\n", - "2. Explore different [prompt optimizers](https://arxiv.org/abs/2406.11695) or [weight optimizers](https://arxiv.org/abs/2407.10930). See the **[Optimizers Docs](/building-blocks/6-optimizers)**.\n", - "3. Scale inference time compute using DSPy Optimizers, e.g. this [notebook](https://github.com/stanfordnlp/dspy/blob/main/examples/agents/multi_agent.ipynb).\n", - "4. Cut cost by distilling to a smaller LM, via prompt or weight optimization, e.g. [this notebook](https://github.com/stanfordnlp/dspy/blob/main/examples/nli/scone/scone.ipynb).\n", + "2. Explore different [prompt optimizers](https://arxiv.org/abs/2406.11695) or [weight optimizers](https://arxiv.org/abs/2407.10930). See the Optimizers Docs.\n", + "3. Scale inference time compute using DSPy Optimizers, e.g. via ensembling multiple post-optimization programs.\n", + "4. Cut cost by distilling to a smaller LM, via prompt or weight optimization.\n", "\n", "How do you decide which ones to proceed with first?\n", "\n", diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index f4d85312c..9577774e4 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -49,6 +49,7 @@ nav: - Agents: tutorials/agents/index.ipynb - Reasoning: tutorials/math/index.ipynb - Entity Extraction: tutorials/entity_extraction/index.ipynb + - Classification: tutorials/classification/index.md - Privacy-Conscious Delegation: tutorials/papillon/index.md - Multi-Hop Search: tutorials/multihop_search/index.ipynb - Saving and Loading: tutorials/saving/index.md