contract_review.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jaideep11061982/GenAINotebooks/blob/main/contract_review.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a65d3850-3ddb-4db3-87fe-fa7d92e6c55b",
      "metadata": {
        "id": "a65d3850-3ddb-4db3-87fe-fa7d92e6c55b"
      },
      "source": [
        "# Contract Review Workflow\n",
        "\n",
        "<a href=\"https://colab.research.google.com/github/run-llama/llamacloud-demo/blob/main/examples/document_workflows/contract_review/contract_review.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "\n",
        "![](https://github.com/run-llama/llamacloud-demo/blob/main/examples/document_workflows/contract_review/contract_review.png?raw=1)\n",
        "\n",
        "This tutorial shows you how to create an agentic workflow that can review a contract for compliance with certain regulations. We will parse the contract into a set of key clauses, match it with relevant clauses from a guideline repository (here, we specifically do GDPR), and then produce a compliance summary."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "87c5f0d7-c884-475a-88ff-4b7057bb91a6",
      "metadata": {
        "id": "87c5f0d7-c884-475a-88ff-4b7057bb91a6"
      },
      "outputs": [],
      "source": [
        "!pip install llama-index llama-index-indices-managed-llama-cloud llama-cloud llama-parse"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8fe98ed0-cfcb-4c37-ac5a-7140d79fefd0",
      "metadata": {
        "id": "8fe98ed0-cfcb-4c37-ac5a-7140d79fefd0"
      },
      "outputs": [],
      "source": [
        "import nest_asyncio\n",
        "\n",
        "nest_asyncio.apply()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d76541e7-65d3-4c70-afcd-49658bc00954",
      "metadata": {
        "id": "d76541e7-65d3-4c70-afcd-49658bc00954"
      },
      "source": [
        "## Setup\n",
        "\n",
        "We setup an index for guidelines. In this case it's just the GDPR document.\n",
        "\n",
        "We also setup our parser."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "dec74e08-72fc-4ed8-9e13-e1ed609ed890",
      "metadata": {
        "id": "dec74e08-72fc-4ed8-9e13-e1ed609ed890"
      },
      "outputs": [],
      "source": [
        "!mkdir -p data\n",
        "!wget \"https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679\" -O data/gdpr.pdf"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e01c7a8f-25f9-4f3b-b040-ee15d34826f6",
      "metadata": {
        "id": "e01c7a8f-25f9-4f3b-b040-ee15d34826f6"
      },
      "source": [
        "### Setup Index\n",
        "Here we use LlamaCloud: https://cloud.llamaindex.ai/. If you don't have access yet, you're always welcome to use our open-source VectorStoreIndex."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "07ccec03-9aaf-42c7-99e1-f7187328548b",
      "metadata": {
        "id": "07ccec03-9aaf-42c7-99e1-f7187328548b"
      },
      "outputs": [],
      "source": [
        "# option 1\n",
        "from llama_index.indices.managed.llama_cloud import LlamaCloudIndex\n",
        "\n",
        "index = LlamaCloudIndex(\n",
        "  name=\"gdpr\",\n",
        "  project_name=\"llamacloud_demo\",\n",
        "  organization_id=\"cdcb3478-1348-492e-8aa0-25f47d1a3902\",\n",
        "  # api_key=\"llx-...\"\n",
        ")\n",
        "\n",
        "retriever = index.as_retriever(similarity_top_k=2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1e5d1dfc-5d1e-4acd-80c9-616bd9aff3cb",
      "metadata": {
        "id": "1e5d1dfc-5d1e-4acd-80c9-616bd9aff3cb"
      },
      "outputs": [],
      "source": [
        "!pwd"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2bb66d70-58e2-46cf-8dc7-9188a70891f3",
      "metadata": {
        "id": "2bb66d70-58e2-46cf-8dc7-9188a70891f3"
      },
      "source": [
        "### Setup Parser\n",
        "\n",
        "Here we use LlamaParse to parse the vendor agremeent."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8af32038-7ce0-49f0-9d67-bf7260a3f27f",
      "metadata": {
        "id": "8af32038-7ce0-49f0-9d67-bf7260a3f27f"
      },
      "outputs": [],
      "source": [
        "from llama_parse import LlamaParse\n",
        "\n",
        "# use our multimodal models for extractions\n",
        "parser = LlamaParse(result_type=\"markdown\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e634cc0d-fc3f-4864-ae64-7d681647cc10",
      "metadata": {
        "id": "e634cc0d-fc3f-4864-ae64-7d681647cc10"
      },
      "source": [
        "### Define Contract Output Schema\n",
        "\n",
        "We want to extract relevant clauses from the agreement in order to match it against relevant clauses in the GDPR. This schema defines a way to structuring the set of extracted clauses."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8d3aceab-26cb-424d-9b24-624e11550901",
      "metadata": {
        "id": "8d3aceab-26cb-424d-9b24-624e11550901"
      },
      "outputs": [],
      "source": [
        "from typing import List, Optional\n",
        "from pydantic import BaseModel, Field\n",
        "\n",
        "class ContractClause(BaseModel):\n",
        "    clause_text: str = Field(..., description=\"The exact text of the clause.\")\n",
        "    mentions_data_processing: bool = Field(False, description=\"True if the clause involves personal data collection or usage.\")\n",
        "    mentions_data_transfer: bool = Field(False, description=\"True if the clause involves transferring personal data, especially to third parties or across borders.\")\n",
        "    requires_consent: bool = Field(False, description=\"True if the clause explicitly states that user consent is needed for data activities.\")\n",
        "    specifies_purpose: bool = Field(False, description=\"True if the clause specifies a clear purpose for data handling or transfer.\")\n",
        "    mentions_safeguards: bool = Field(False, description=\"True if the clause mentions security measures or other safeguards for data.\")\n",
        "\n",
        "class ContractExtraction(BaseModel):\n",
        "    vendor_name: Optional[str] = Field(None, description=\"The vendor's name if identifiable.\")\n",
        "    effective_date: Optional[str] = Field(None, description=\"The effective date of the agreement, if available.\")\n",
        "    governing_law: Optional[str] = Field(None, description=\"The governing law of the contract, if stated.\")\n",
        "    clauses: List[ContractClause] = Field(..., description=\"List of extracted clauses and their relevant indicators.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9a4a1d37-c8c8-48ab-b394-51bb2412f3b5",
      "metadata": {
        "id": "9a4a1d37-c8c8-48ab-b394-51bb2412f3b5"
      },
      "source": [
        "### Define Compliance Check Schema\n",
        "\n",
        "Define a schema that matches clauses with relevant guidelines in GDPR."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7d0e0f4f-5f82-46b0-92a2-3e47fc061de3",
      "metadata": {
        "id": "7d0e0f4f-5f82-46b0-92a2-3e47fc061de3"
      },
      "outputs": [],
      "source": [
        "from typing import Optional\n",
        "from pydantic import BaseModel, Field\n",
        "\n",
        "class GuidelineMatch(BaseModel):\n",
        "    guideline_text: str = Field(..., description=\"The single most relevant guideline excerpt related to this clause.\")\n",
        "    similarity_score: float = Field(..., description=\"Similarity score indicating how closely the guideline matches the clause, e.g., between 0 and 1.\")\n",
        "    relevance_explanation: Optional[str] = Field(None, description=\"Brief explanation of why this guideline is relevant.\")\n",
        "\n",
        "class ClauseComplianceCheck(BaseModel):\n",
        "    clause_text: str = Field(..., description=\"The exact text of the clause from the contract.\")\n",
        "    matched_guideline: Optional[GuidelineMatch] = Field(None, description=\"The most relevant guideline extracted via vector retrieval.\")\n",
        "    compliant: bool = Field(..., description=\"Indicates whether the clause is considered compliant with the referenced guideline.\")\n",
        "    notes: Optional[str] = Field(None, description=\"Additional commentary or recommendations.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "15e1805c-a9ad-433c-8f22-a005de3fe0ab",
      "metadata": {
        "id": "15e1805c-a9ad-433c-8f22-a005de3fe0ab"
      },
      "source": [
        "### Define Final Output Schema\n",
        "\n",
        "This is the schema for the final compliance report. It contains the vendor name, if it's overall compliant, and also the summary notes.\n",
        "\n",
        "It will be inferred from the individual checks for every clause."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b9f275f4-66cc-4950-93f7-47da98d14e96",
      "metadata": {
        "id": "b9f275f4-66cc-4950-93f7-47da98d14e96"
      },
      "outputs": [],
      "source": [
        "from typing import Optional, List\n",
        "from pydantic import BaseModel, Field\n",
        "\n",
        "class ComplianceReport(BaseModel):\n",
        "    vendor_name: Optional[str] = Field(None, description=\"The vendor's name if identified from the contract.\")\n",
        "    overall_compliant: bool = Field(..., description=\"Indicates if the contract is considered overall compliant.\")\n",
        "    summary_notes: Optional[str] = Field(None, description=\"General summary or recommendations for achieving full compliance.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0ff8eebc-c00e-4499-ad5d-2d7843786eba",
      "metadata": {
        "id": "0ff8eebc-c00e-4499-ad5d-2d7843786eba"
      },
      "source": [
        "## Setup Contract Review Workflow\n",
        "\n",
        "Let's define the following contract review workflow:\n",
        "1. Extract out structured data from the vendor agreement.\n",
        "2. For each clause, do retrieval against GDPR to see if it's compliant with guidelines.\n",
        "3. Generate a final summary."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8b803856-630c-497c-9142-bd5aeb9efa5d",
      "metadata": {
        "id": "8b803856-630c-497c-9142-bd5aeb9efa5d"
      },
      "outputs": [],
      "source": [
        "from llama_index.core.workflow import (\n",
        "    Event,\n",
        "    StartEvent,\n",
        "    StopEvent,\n",
        "    Context,\n",
        "    Workflow,\n",
        "    step,\n",
        ")\n",
        "from llama_index.core.llms import LLM\n",
        "from typing import Optional\n",
        "from pydantic import BaseModel\n",
        "from llama_index.core import SimpleDirectoryReader\n",
        "from llama_index.core.schema import Document\n",
        "from llama_index.core.agent import FunctionCallingAgentWorker\n",
        "from llama_index.core.prompts import ChatPromptTemplate\n",
        "from llama_index.core.llms import ChatMessage, MessageRole\n",
        "from llama_index.core.retrievers import BaseRetriever\n",
        "from pathlib import Path\n",
        "import logging\n",
        "import json\n",
        "import os\n",
        "\n",
        "_logger = logging.getLogger(__name__)\n",
        "_logger.setLevel(logging.INFO)\n",
        "\n",
        "\n",
        "CONTRACT_EXTRACT_PROMPT = \"\"\"\\\n",
        "You are given contract data below. \\\n",
        "Please extract out relevant information from the contract into the defined schema - the schema is defined as a function call.\\\n",
        "\n",
        "{contract_data}\n",
        "\"\"\"\n",
        "\n",
        "CONTRACT_MATCH_PROMPT = \"\"\"\\\n",
        "Given the following contract clause and the corresponding relevant guideline text, evaluate the compliance \\\n",
        "and provide a JSON object that matches the ClauseComplianceCheck schema.\n",
        "\n",
        "**Contract Clause:**\n",
        "{clause_text}\n",
        "\n",
        "**Matched Guideline Text(s):**\n",
        "{guideline_text}\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "COMPLIANCE_REPORT_SYSTEM_PROMPT = \"\"\"\\\n",
        "You are a compliance reporting assistant. Your task is to generate a final compliance report \\\n",
        "based on the results of clause compliance checks against \\\n",
        "a given set of guidelines.\n",
        "\n",
        "Analyze the provided compliance results and produce a structured report according to the specified schema.\n",
        "Ensure that if there are no noncompliant clauses, the report clearly indicates full compliance.\n",
        "\"\"\"\n",
        "\n",
        "COMPLIANCE_REPORT_USER_PROMPT = \"\"\"\\\n",
        "A set of clauses within a contract were checked against GDPR compliance guidelines for the following vendor: {vendor_name}.\n",
        "The set of noncompliant clauses are given below.\n",
        "\n",
        "Each section includes:\n",
        "- **Clause:** The exact text of the contract clause.\n",
        "- **Guideline:** The relevant GDPR guideline text.\n",
        "- **Compliance Status:** Should be `False` for noncompliant clauses.\n",
        "- **Notes:** Additional information or explanations.\n",
        "\n",
        "{compliance_results}\n",
        "\n",
        "Based on the above compliance results, generate a final compliance report following the `ComplianceReport` schema below.\n",
        "If there are no noncompliant clauses, the report should indicate that the contract is fully compliant.\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "class ContractExtractionEvent(Event):\n",
        "    contract_extraction: ContractExtraction\n",
        "\n",
        "\n",
        "class MatchGuidelineEvent(Event):\n",
        "    clause: ContractClause\n",
        "\n",
        "\n",
        "class MatchGuidelineResultEvent(Event):\n",
        "    result: ClauseComplianceCheck\n",
        "\n",
        "\n",
        "class GenerateReportEvent(Event):\n",
        "    match_results: List[ClauseComplianceCheck]\n",
        "\n",
        "\n",
        "class LogEvent(Event):\n",
        "    msg: str\n",
        "    delta: bool = False\n",
        "\n",
        "\n",
        "class ContractReviewWorkflow(Workflow):\n",
        "    \"\"\"Contract review workflow.\"\"\"\n",
        "\n",
        "    def __init__(\n",
        "        self,\n",
        "        parser: LlamaParse,\n",
        "        guideline_retriever: BaseRetriever,\n",
        "        llm: LLM | None = None,\n",
        "        similarity_top_k: int = 20,\n",
        "        output_dir: str = \"data_out\",\n",
        "        **kwargs,\n",
        "    ) -> None:\n",
        "        \"\"\"Init params.\"\"\"\n",
        "        super().__init__(**kwargs)\n",
        "\n",
        "        self.parser = parser\n",
        "        self.guideline_retriever = guideline_retriever\n",
        "\n",
        "        self.llm = llm or OpenAI(model=\"gpt-4o-mini\")\n",
        "        self.similarity_top_k = similarity_top_k\n",
        "\n",
        "        # if not exists, create\n",
        "        out_path = Path(output_dir) / \"workflow_output\"\n",
        "        if not out_path.exists():\n",
        "            out_path.mkdir(parents=True, exist_ok=True)\n",
        "            os.chmod(str(out_path), 0o0777)\n",
        "        self.output_dir = out_path\n",
        "\n",
        "    @step\n",
        "    async def parse_contract(\n",
        "        self, ctx: Context, ev: StartEvent\n",
        "    ) -> ContractExtractionEvent:\n",
        "        # load output template file\n",
        "        contract_extraction_path = Path(\n",
        "            f\"{self.output_dir}/contract_extraction.json\"\n",
        "        )\n",
        "        if contract_extraction_path.exists():\n",
        "            if self._verbose:\n",
        "                ctx.write_event_to_stream(LogEvent(msg=\">> Loading contract from cache\"))\n",
        "            contract_extraction_dict = json.load(open(str(contract_extraction_path), \"r\"))\n",
        "            contract_extraction = ContractExtraction.model_validate(contract_extraction_dict)\n",
        "        else:\n",
        "            if self._verbose:\n",
        "                ctx.write_event_to_stream(LogEvent(msg=\">> Reading contract\"))\n",
        "\n",
        "            # no need to parse contract, it's already in markdown\n",
        "            # you can use LlamaParse to parse more complex PDFs + other docs\n",
        "\n",
        "            docs = SimpleDirectoryReader(input_files=[ev.contract_path]).load_data()\n",
        "\n",
        "            # extract from contract\n",
        "            prompt = ChatPromptTemplate.from_messages([\n",
        "                (\"user\", CONTRACT_EXTRACT_PROMPT)\n",
        "            ])\n",
        "            contract_extraction = await llm.astructured_predict(\n",
        "                ContractExtraction,\n",
        "                prompt,\n",
        "                contract_data=\"\\n\".join([d.get_content(metadata_mode=\"all\") for d in docs])\n",
        "            )\n",
        "            if not isinstance(contract_extraction, ContractExtraction):\n",
        "                raise ValueError(f\"Invalid extraction from contract: {contract_extraction}\")\n",
        "            # save output template to file\n",
        "            with open(contract_extraction_path, \"w\") as fp:\n",
        "                fp.write(contract_extraction.model_dump_json())\n",
        "        if self._verbose:\n",
        "            ctx.write_event_to_stream(LogEvent(msg=f\">> Contract data: {contract_extraction.dict()}\"))\n",
        "\n",
        "        return ContractExtractionEvent(contract_extraction=contract_extraction)\n",
        "\n",
        "    @step\n",
        "    async def dispatch_guideline_match(\n",
        "        self, ctx: Context, ev: ContractExtractionEvent\n",
        "    ) -> MatchGuidelineEvent:\n",
        "        \"\"\"For each clause in the contract, find relevant guidelines.\n",
        "\n",
        "        Use a map-reduce pattern.\n",
        "\n",
        "        \"\"\"\n",
        "        await ctx.set(\"num_clauses\", len(ev.contract_extraction.clauses))\n",
        "        await ctx.set(\"vendor_name\", ev.contract_extraction.vendor_name)\n",
        "\n",
        "        for clause in ev.contract_extraction.clauses:\n",
        "            ctx.send_event(MatchGuidelineEvent(clause=clause, vendor_name=ev.contract_extraction.vendor_name))\n",
        "\n",
        "    @step\n",
        "    async def handle_guideline_match(\n",
        "        self, ctx: Context, ev: MatchGuidelineEvent\n",
        "    ) -> MatchGuidelineResultEvent:\n",
        "        \"\"\"Handle matching clause against guideline.\"\"\"\n",
        "\n",
        "        # retrieve matching guideline\n",
        "        query = f\"\"\"\\\n",
        "Please find the relevant guideline from {ev.vendor_name} that aligns with the following contract clause:\n",
        "\n",
        "{ev.clause.clause_text}\n",
        "\"\"\"\n",
        "        guideline_docs = self.guideline_retriever.retrieve(query)\n",
        "        guideline_text=\"\\n\\n\".join([g.get_content() for g in guideline_docs])\n",
        "        if self._verbose:\n",
        "            ctx.write_event_to_stream(\n",
        "                LogEvent(msg=f\">> Found guidelines: {guideline_text[:200]}...\")\n",
        "            )\n",
        "\n",
        "        # extract from contract\n",
        "        prompt = ChatPromptTemplate.from_messages([\n",
        "            (\"user\", CONTRACT_MATCH_PROMPT)\n",
        "        ])\n",
        "        compliance_output = await llm.astructured_predict(\n",
        "            ClauseComplianceCheck,\n",
        "            prompt,\n",
        "            clause_text=ev.clause.model_dump_json(),\n",
        "            guideline_text=guideline_text\n",
        "\n",
        "        )\n",
        "\n",
        "        if not isinstance(compliance_output, ClauseComplianceCheck):\n",
        "            raise ValueError(f\"Invalid compliance check: {compliance_output}\")\n",
        "\n",
        "        return MatchGuidelineResultEvent(result=compliance_output)\n",
        "\n",
        "    @step\n",
        "    async def gather_guideline_match(\n",
        "        self, ctx: Context, ev: MatchGuidelineResultEvent\n",
        "    ) -> GenerateReportEvent:\n",
        "        \"\"\"Handle matching clause against guideline.\"\"\"\n",
        "        num_clauses = await ctx.get(\"num_clauses\")\n",
        "        events = ctx.collect_events(ev, [MatchGuidelineResultEvent] * num_clauses)\n",
        "        if events is None:\n",
        "            return\n",
        "\n",
        "        match_results = [e.result for e in events]\n",
        "        # save match results\n",
        "        match_results_path = Path(\n",
        "            f\"{self.output_dir}/match_results.jsonl\"\n",
        "        )\n",
        "        with open(match_results_path, \"w\") as fp:\n",
        "            for mr in match_results:\n",
        "                fp.write(mr.model_dump_json() + \"\\n\")\n",
        "\n",
        "\n",
        "        return GenerateReportEvent(match_results=[e.result for e in events])\n",
        "\n",
        "    @step\n",
        "    async def generate_output(\n",
        "        self, ctx: Context, ev: GenerateReportEvent\n",
        "    ) -> StopEvent:\n",
        "        if self._verbose:\n",
        "            ctx.write_event_to_stream(LogEvent(msg=\">> Generating Compliance Report\"))\n",
        "\n",
        "        # if all clauses are compliant, return a compliant result\n",
        "        non_compliant_results = [r for r in ev.match_results if not r.compliant]\n",
        "\n",
        "        # generate compliance results string\n",
        "        result_tmpl = \"\"\"\n",
        "1. **Clause**: {clause}\n",
        "2. **Guideline:** {guideline}\n",
        "3. **Compliance Status:** {compliance_status}\n",
        "4. **Notes:** {notes}\n",
        "\"\"\"\n",
        "        non_compliant_strings = []\n",
        "        for nr in non_compliant_results:\n",
        "            non_compliant_strings.append(\n",
        "                result_tmpl.format(\n",
        "                    clause=nr.clause_text,\n",
        "                    guideline=nr.matched_guideline.guideline_text,\n",
        "                    compliance_status=nr.compliant,\n",
        "                    notes=nr.notes\n",
        "                )\n",
        "            )\n",
        "        non_compliant_str = \"\\n\\n\".join(non_compliant_strings)\n",
        "\n",
        "        prompt = ChatPromptTemplate.from_messages([\n",
        "            (\"system\", COMPLIANCE_REPORT_SYSTEM_PROMPT),\n",
        "            (\"user\", COMPLIANCE_REPORT_USER_PROMPT)\n",
        "        ])\n",
        "        compliance_report = await llm.astructured_predict(\n",
        "            ComplianceReport,\n",
        "            prompt,\n",
        "            compliance_results=non_compliant_str,\n",
        "            vendor_name=await ctx.get(\"vendor_name\")\n",
        "        )\n",
        "\n",
        "        return StopEvent(result={\"report\": compliance_report, \"non_compliant_results\": non_compliant_results})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8557f4a7-ce61-4757-8f35-6785574c57cb",
      "metadata": {
        "id": "8557f4a7-ce61-4757-8f35-6785574c57cb"
      },
      "outputs": [],
      "source": [
        "from llama_index.llms.openai import OpenAI\n",
        "\n",
        "llm = OpenAI(model=\"gpt-4o\")\n",
        "workflow = ContractReviewWorkflow(\n",
        "    parser=parser,\n",
        "    guideline_retriever=retriever,\n",
        "    llm=llm,\n",
        "    verbose=True,\n",
        "    timeout=None,  # don't worry about timeout to make sure it completes\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "66d79ab2-1455-4f8f-bde3-8bbd51b7da51",
      "metadata": {
        "id": "66d79ab2-1455-4f8f-bde3-8bbd51b7da51"
      },
      "source": [
        "#### Visualize the workflow"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "470f298b-c3a3-485e-a440-b0576e45134d",
      "metadata": {
        "id": "470f298b-c3a3-485e-a440-b0576e45134d"
      },
      "outputs": [],
      "source": [
        "from llama_index.utils.workflow import draw_all_possible_flows\n",
        "\n",
        "draw_all_possible_flows(ContractReviewWorkflow, filename=\"contract_workflow.html\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0f11484c-52b8-424f-9a64-7456068ff1b1",
      "metadata": {
        "id": "0f11484c-52b8-424f-9a64-7456068ff1b1"
      },
      "source": [
        "## Run the Workflow\n",
        "\n",
        "Let's run the full workflow and generate the output!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b9a55b1b-8263-4365-b187-6204150ec4cb",
      "metadata": {
        "scrolled": true,
        "id": "b9a55b1b-8263-4365-b187-6204150ec4cb"
      },
      "outputs": [],
      "source": [
        "from IPython.display import clear_output\n",
        "\n",
        "handler = workflow.run(contract_path=\"data/vendor_agreement.md\")\n",
        "async for event in handler.stream_events():\n",
        "    if isinstance(event, LogEvent):\n",
        "        if event.delta:\n",
        "            print(event.msg, end=\"\")\n",
        "        else:\n",
        "            print(event.msg)\n",
        "\n",
        "response_dict = await handler\n",
        "print(str(response_dict[\"report\"]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bb0f5f52-ced9-49c2-82b9-131470fd291e",
      "metadata": {
        "id": "bb0f5f52-ced9-49c2-82b9-131470fd291e",
        "outputId": "8769c540-9805-4080-9148-f5b033fb2cd4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "vendor_name='ACME Office Supply, Inc.' overall_compliant=False summary_notes=\"The contract contains noncompliant clauses regarding subprocessors and data transfer. It allows engaging subprocessors without prior client approval and lacks the client's right to object. Additionally, it does not mention additional safeguards or compliance with standard contractual clauses for data transfer, which are recommended to protect data subjects' rights. To achieve full compliance, these clauses should be revised to align with GDPR guidelines.\"\n"
          ]
        }
      ],
      "source": [
        "print(str(response_dict[\"report\"]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "dda1efaa-f7cf-4a05-a2f7-13bcb15d26ef",
      "metadata": {
        "id": "dda1efaa-f7cf-4a05-a2f7-13bcb15d26ef",
        "outputId": "0ffd0bc9-f8c0-4873-f5ba-bdbe97d7f9ec"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[ClauseComplianceCheck(clause_text='- Vendor may engage subprocessors without prior Client approval - Subprocessors may be located in any jurisdiction globally - Notice of new subprocessors provided within 30 days of engagement - Client has no right to object to new subprocessors', matched_guideline=GuidelineMatch(guideline_text='The processor shall not engage another processor without prior specific or general written authorisation of the controller. In the case of general written authorisation, the processor shall inform the controller of any intended changes concerning the addition or replacement of other processors, thereby giving the controller the opportunity to object to such changes.', similarity_score=0.9, relevance_explanation='The guideline specifies that the processor must obtain prior authorization from the controller before engaging subprocessors, and must inform the controller of changes, allowing them to object. The contract clause does not comply with these requirements.'), compliant=False, notes='The contract clause does not comply with the guideline as it allows the vendor to engage subprocessors without prior client approval and does not provide the client the right to object to new subprocessors.'),\n",
              " ClauseComplianceCheck(clause_text='- Vendor maintains primary data centers in the United States - Vendor may transfer data to any country where it maintains operations - No prior notification required for new data storage locations - Vendor will rely on its standard data transfer mechanisms - Data may be processed by staff operating outside the EEA', matched_guideline=GuidelineMatch(guideline_text='Standard data-protection clauses in a wider contract, such as a contract between the processor and another processor, nor from adding other clauses or additional safeguards provided that they do not contradict, directly or indirectly, the standard contractual clauses adopted by the Commission or by a supervisory authority or prejudice the fundamental rights or freedoms of the data subjects. Controllers and processors should be encouraged to provide additional safeguards via contractual commitments that supplement standard protection clauses.', similarity_score=0.85, relevance_explanation=\"The guideline emphasizes the importance of standard data-protection clauses and additional safeguards, which is relevant to the clause's mention of standard data transfer mechanisms.\"), compliant=False, notes=\"The clause lacks mention of additional safeguards or compliance with standard contractual clauses, which are recommended by the guideline to ensure protection of data subjects' rights.\")]"
            ]
          },
          "execution_count": 124,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response_dict[\"non_compliant_results\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "710ae172-b7e7-4d95-960f-f9cd90b40e3f",
      "metadata": {
        "id": "710ae172-b7e7-4d95-960f-f9cd90b40e3f"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "llamacloud-demo",
      "language": "python",
      "name": "llamacloud-demo"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.8"
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}