Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Assistant code to RC branch #322

Merged
merged 57 commits into from
Jan 22, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
45c026b
Add latest code gen for 2025-01 RC
aulorbe Dec 4, 2024
cd48708
Incorporate newest code from stable
aulorbe Dec 9, 2024
a792a02
Update RC branch to include control plane for Assistants
aulorbe Dec 5, 2024
84b04c3
Save work
aulorbe Dec 6, 2024
e75dcb7
Save work
aulorbe Dec 7, 2024
37952ad
Got chat working!
aulorbe Dec 9, 2024
4a31806
Add newly-generated code for Assistant data, control, and eval planes…
aulorbe Dec 9, 2024
8c7dc4d
Match w/RC branch minus Assistant content
aulorbe Dec 9, 2024
ea135f7
update server paths
aulorbe Dec 9, 2024
20dcf53
Got list to work
aulorbe Dec 10, 2024
3d3e2c3
listFiles working
aulorbe Dec 10, 2024
13976b9
Keeping uploadFile as blob for user + got deleteFile working
aulorbe Dec 10, 2024
07902a2
Got Eval working
aulorbe Dec 12, 2024
887fd91
Upload file workaround
aulorbe Dec 13, 2024
813e5af
Got uploadFile with metadata to work!
aulorbe Dec 13, 2024
8430b47
Add chatCompletions and contextAssistant
aulorbe Dec 16, 2024
a509fb5
Reset hash of submodule
aulorbe Dec 28, 2024
eed47de
Add newest gen. code
aulorbe Dec 28, 2024
4a3bf40
Add Assistant singleton to deal with data plane regions
aulorbe Dec 29, 2024
6cd624d
Rename file
aulorbe Dec 29, 2024
690f279
Change from private to readonly to connote immutable intent of property
aulorbe Dec 29, 2024
e0e84ca
Start adding tests
aulorbe Dec 29, 2024
a5173b9
Finish unit tests
aulorbe Dec 30, 2024
9a91af8
Add interface exports to top-level index.ts file
aulorbe Dec 30, 2024
d0089e3
Add createAssistant int. tests
aulorbe Jan 3, 2025
e3c4ac7
Add updateAssistant int. tests + new interface for better UX
aulorbe Jan 3, 2025
4e9603a
Add delete int. tests + revise update int. tests
aulorbe Jan 3, 2025
67a642e
Clean up + add listAssistants int. tests
aulorbe Jan 3, 2025
0d578d4
Add getAssistant int. tests
aulorbe Jan 3, 2025
896ff6e
Save work: centralization of assistant creation for int. tests + chat…
aulorbe Jan 6, 2025
b8fe94e
Remove text files
aulorbe Jan 6, 2025
2dceee9
Add TEST_FILE env var to workflow files
aulorbe Jan 6, 2025
462d559
Chat int. tests
aulorbe Jan 6, 2025
187cb63
Add context and upload int. tests
aulorbe Jan 6, 2025
a5bcc7b
Add describeFile int. test beginnings
aulorbe Jan 7, 2025
4327128
Add eval reorg + tests
aulorbe Jan 8, 2025
1c5615a
Document control plane ops
aulorbe Jan 8, 2025
c6a1554
Cleanup
aulorbe Jan 8, 2025
769a5dc
Add tests
aulorbe Jan 9, 2025
f989ce1
Documentation finished
aulorbe Jan 9, 2025
2e1e63b
Up sleep to avoid flakiness
aulorbe Jan 9, 2025
a297ad4
Remove unnecessarily old version of TS + add newer versions
aulorbe Jan 9, 2025
bcb7a33
Update severe vulnerability in package
aulorbe Jan 9, 2025
60af784
Clean up
aulorbe Jan 9, 2025
9e7b454
Increase sleep, remove unnecessary export
aulorbe Jan 9, 2025
0465545
Update docstrings, write README
aulorbe Jan 9, 2025
187c4d3
Add more sleep so that file is available for describing
aulorbe Jan 9, 2025
435229a
Merge branch 'release-candidate/2025-01' into add-assistant
aulorbe Jan 9, 2025
2462a5f
refactor to move assistant control plane stuff into the Pinecone clas…
austin-denoble Jan 19, 2025
2acac44
fix AssistantHostSingleton unit tests and issue accessing field
austin-denoble Jan 19, 2025
4c343de
clean up docstrings, README, etc
austin-denoble Jan 19, 2025
e0367fb
fix Assistant import
austin-denoble Jan 19, 2025
2bb6384
fix other import
austin-denoble Jan 19, 2025
ff40551
rework uploadFile to use basic node rest() along with generated inter…
austin-denoble Jan 22, 2025
fcaa739
clean up older uploadFile function, clean up action naming for assist…
austin-denoble Jan 22, 2025
ad21dee
add a step to make sure the next app is up and running before testing…
austin-denoble Jan 22, 2025
b0e0dd3
rework testing script a bit
austin-denoble Jan 22, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 20 additions & 5 deletions .github/workflows/testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ jobs:
runs-on: ubuntu-latest
outputs:
serverlessIndexName: ${{ steps.step3.outputs.SERVERLESS_INDEX_NAME }}
assistantName: ${{ steps.step3.outputs.ASSISTANT_NAME }}
testFile: ${{ steps.step3.outputs.TEST_FILE }}
Comment on lines +12 to +13
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reviewers: You can ignore these diffs if you like. It's just me allowing CI to spin up a global Assistant and a file to play with.

steps:
- name: Checkout code
id: step1
Expand All @@ -23,8 +25,13 @@ jobs:
env:
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
run: |
SERVERLESS_INDEX_NAME=$(npx ts-node ./src/integration/setup.ts | grep "SERVERLESS_INDEX_NAME=" | cut -d'=' -f2)
SETUP_OUTPUT=$(npx ts-node ./src/integration/setup.ts)
SERVERLESS_INDEX_NAME=$(echo "$SETUP_OUTPUT" | grep "SERVERLESS_INDEX_NAME=" | cut -d'=' -f2)
ASSISTANT_NAME=$(echo "$SETUP_OUTPUT" | grep "ASSISTANT_NAME=" | cut -d'=' -f2)
TEST_FILE=$(echo "$SETUP_OUTPUT" | grep "TEST_FILE=" | cut -d'=' -f2)
echo "SERVERLESS_INDEX_NAME=$SERVERLESS_INDEX_NAME" >> $GITHUB_OUTPUT
echo "ASSISTANT_NAME=$ASSISTANT_NAME" >> $GITHUB_OUTPUT
echo "TEST_FILE=$TEST_FILE" >> $GITHUB_OUTPUT

unit-tests:
needs: setup
Expand All @@ -48,6 +55,8 @@ jobs:
runs-on: ubuntu-latest
outputs:
serverlessIndexName: ${{ steps.runTests1.outputs.SERVERLESS_INDEX_NAME }}
assistantName: ${{ steps.runTests1.outputs.ASSISTANT_NAME }}
testFile: ${{ steps.runTests1.outputs.TEST_FILE }}
strategy:
fail-fast: false
max-parallel: 2
Expand Down Expand Up @@ -87,9 +96,13 @@ jobs:
CI: true
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
SERVERLESS_INDEX_NAME: ${{ needs.setup.outputs.serverlessIndexName}}
ASSISTANT_NAME: ${{ needs.setup.outputs.assistantName}}
TEST_FILE: ${{ needs.setup.outputs.testFile}}
run: |
${{ matrix.config.runner }} run test:integration:${{ matrix.config.jest_env }}
echo "SERVERLESS_INDEX_NAME=${{ needs.setup.outputs.serverlessIndexName}}" >> $GITHUB_OUTPUT
echo "ASSISTANT_NAME=${{ needs.setup.outputs.assistantName}}" >> $GITHUB_OUTPUT
echo "TEST_FILE=${{ needs.setup.outputs.testFile}}" >> $GITHUB_OUTPUT

- name: Run integration tests (Staging)
if: matrix.pinecone_env == 'staging'
Expand All @@ -115,6 +128,8 @@ jobs:
env:
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
SERVERLESS_INDEX_NAME: ${{ needs.integration-tests.outputs.serverlessIndexName}}
ASSISTANT_NAME: ${{ needs.integration-tests.outputs.assistantName}}
TEST_FILE: ${{ needs.integration-tests.outputs.testFile}}
run: |
npx ts-node ./src/integration/teardown.ts

Expand All @@ -126,10 +141,6 @@ jobs:
matrix:
tsVersion:
[
'~4.1.0',
'~4.2.0',
'~4.3.0',
'~4.4.0',
'~4.5.0',
'~4.6.0',
'~4.7.0',
Expand All @@ -138,6 +149,10 @@ jobs:
'~5.0.0',
'~5.1.0',
'~5.2.0',
'~5.3.0',
'~5.4.0',
'~5.5.0',
'~5.6.0',
Comment on lines +152 to +155
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See this Slack thread for more color on this change

'latest',
]
steps:
Expand Down
286 changes: 286 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1245,6 +1245,292 @@ console.log(response);
//}
```

## Pinecone Assistant

The [Pinecone Assistant API](https://docs.pinecone.io/guides/assistant/understanding-assistant) enables you to create and manage AI assistants powered by Pinecone's vector database
capabilities. These Assistants can be customized with specific instructions and metadata, and can interact with
files and engage in chat conversations.

### Create an Assistant

[Creates a new Assistant](https://docs.pinecone.io/guides/assistant/create-assistant) with specified configurations. You can define the Assistant's name, provide instructions
that guide its behavior, and attach metadata for organization and tracking purposes.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();

const assistant = await pc.createAssistant({
name: 'product-assistant',
instructions: 'You are a helpful product recommendation assistant.',
metadata: {
team: 'product',
version: '1.0',
},
});
```

### Delete an Assistant

[Deletes an Assistant](https://docs.pinecone.io/guides/assistant/manage-assistants#delete-an-assistant) by name.

**Note:** Deleting an Assistant also deletes all associated files.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.deleteAssistant('test1');
```

### Get information about an Assistant

[Retrieves information](https://docs.pinecone.io/guides/assistant/manage-assistants#get-the-status-of-an-assistant) about an Assistant by name.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const test = await pc.describeAssistant('test1');
console.log(test);
// {
// name: 'test10',
// instructions: undefined,
// metadata: undefined,
// status: 'Ready',
// host: 'https://prod-1-data.ke.pinecone.io',
// createdAt: 2025-01-08T22:24:50.525Z,
// updatedAt: 2025-01-08T22:24:52.303Z
// }
```

### Update an Assistant

[Updates an Assistant](https://docs.pinecone.io/guides/assistant/manage-assistants#add-instructions-to-an-assistant) by name. You can update the Assistant's name, instructions, and/or metadata.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.updateAssistant({
name: 'test1',
instructions: 'some new instructions!',
});
// {
// assistantName: test1,
// instructions: 'some new instructions!',
// metadata: undefined
// }
```

### List Assistants

Retrieves a [list of all Assistants](https://docs.pinecone.io/guides/assistant/manage-assistants) in your account. This method returns details about each Assistant including their
names, instructions, metadata, status, and host.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();

const assistants = await pc.listAssistants();
console.log(assistants);
// {
// assistants: [{
// name: 'product-assistant',
// instructions: 'You are a helpful product recommendation assistant.',
// metadata: { team: 'product', version: '1.0' },
// status: 'Ready',
// host: 'product-assistant-abc123.svc.pinecone.io'
// }]
// }
```

### Chat with an Assistant

You can [chat with Assistants](https://docs.pinecone.io/guides/assistant/chat-with-assistant) using either the `chat` method or the `chatCompletion` method. The latter's output is
compatible with [OpenAI's Chat Completion](https://platform.openai.com/docs/api-reference/chat) format.

**Note:** Your Assistant must contain files in order for chat to work.

The following example shows how to chat with an Assistant using the `chat` method:

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const assistantName = 'test1';
const assistant = pc.Assistant(assistantName);
const chatResp = await assistant.chat({
messages: [{ role: 'user', content: 'What is the capital of France?' }],
});
console.log(chatResp);
// {
// id: '000000000000000023e7fb015be9d0ad',
// finishReason: 'stop',
// message: {
// role: 'assistant',
// content: 'The capital of France is Paris.'
// },
// model: 'gpt-4o-2024-05-13',
// citations: [ { position: 209, references: [Array] } ],
// usage: { promptTokens: 493, completionTokens: 38, totalTokens: 531 }
// }
```

### Inspect context snippets associated with a chat

Returns [context snippets associated with a given query and an Assistant's response](https://docs.pinecone.io/guides/assistant/understanding-context-snippets). This is useful for understanding
how the Assistant arrived at its answer(s).

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const assistantName = 'test1';
const assistant = pc.Assistant(assistantName);
const response = await assistant.context({
query: 'What is the capital of France?',
});
console.log(response);
// {
// snippets: [
// {
// type: 'text',
// content: 'The capital of France is Paris.',
// score: 0.9978925,
// reference: [Object]
// },
// ],
// usage: { promptTokens: 527, completionTokens: 0, totalTokens: 527 }
// }
```

### Add files to an Assistant

You can [add files to an Assistant](https://docs.pinecone.io/guides/assistant/manage-files#upload-a-local-file) to enable it to interact with files during chat conversations. The following
example shows how to upload a local `test-file.txt` file to an Assistant.

**Note:** You must upload at least 1 file in order to chat with an Assistant.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const assistantName = 'test1';
const assistant = pc.Assistant(assistantName);
await assistant.uploadFile({
path: 'test-file.txt',
metadata: { 'test-key': 'test-value' },
});
// {
// name: 'test-file.txt',
// id: '921ad74c-2421-413a-8c86-fca81ceabc5c',
// metadata: { 'test-key': 'test-value' },
// createdOn: 2025-01-06T19:14:21.969Z,
// updatedOn: 2025-01-06T19:14:21.969Z,
// status: 'Processing',
// percentDone: null,
// signedUrl: null,
// errorMessage: null
// }
```

### List all files in an Assistant

[Lists all files](https://docs.pinecone.io/guides/assistant/manage-files#list-files-in-an-assistant) that have been uploaded to an Assistant. Optionally, you can pass a filter to list only files that
meet certain criteria.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const assistantName = 'test1';
const assistant = pc.Assistant(assistantName);
const files = await assistant.listFiles({
filter: { metadata: { key: 'value' } },
});
console.log(files);
// {
// files: [
// {
// name: 'test-file.txt',
// id: '1a56ddd0-c6d8-4295-80c0-9bfd6f5cb87b',
// metadata: [Object],
// createdOn: 2025-01-06T19:14:21.969Z,
// updatedOn: 2025-01-06T19:14:36.925Z,
// status: 'Available',
// percentDone: 1,
// signedUrl: undefined,
// errorMessage: undefined
// }
// ]
// }
```

### Get the status of a file in an Assistant

[Retrieves information about a file](https://docs.pinecone.io/guides/assistant/manage-files#get-the-status-of-a-file) in an Assistant by ID.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const assistantName = 'test1';
const assistant = pc.Assistant(assistantName);
const files = await assistant.listFiles();
let fileId: string;
if (files.files) {
fileId = files.files[0].id;
} else {
fileId = '';
}
const resp = await assistant.describeFile({ fileId: fileId });
console.log(resp);
// {
// name: 'test-file.txt',
// id: '1a56ddd0-c6d8-4295-80c0-9bfd6f5cb87b',
// metadata: undefined,
// createdOn: 2025-01-06T19:14:21.969Z,
// updatedOn: 2025-01-06T19:14:36.925Z,
// status: 'Available',
// percentDone: 1,
// signedUrl: undefined,
// errorMessage: undefined
// }
```

### Delete a file from an Assistant

[Deletes a file(s)](https://docs.pinecone.io/guides/assistant/manage-files#delete-a-file) from an Assistant by ID.

**Note:** Deleting files is a PERMANENT operation. Deleted files _cannot_ be recovered.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const assistantName = 'test1';
const assistant = pc.Assistant(assistantName);
const files = await assistant.listFiles();
let fileId: string;
if (files.files) {
fileId = files.files[0].id;
await assistant.deleteFile({ fileId: fileId });
}
```

### Evaluate answers

You can also use the Assistant API to [evaluate the accuracy of a question-answer pair, given a known-true answer](https://docs.pinecone.io/guides/assistant/evaluate-answers).
The API will return a set of metrics (`correctness`, `completeness`, and `alignment`) that indicate how well the
Assistant performed.

```typescript
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.evaluate({
question: 'What is the capital of France?',
answer: "Lyon is France's capital city",
groundTruth: 'Paris is the capital city of France',
});
// {
// metrics: { correctness: 0, completeness: 0, alignment: 0 }, // 0s across the board indicates incorrect
// reasoning: { evaluatedFacts: [ [Object] ] },
// usage: { promptTokens: 1134, completionTokens: 21, totalTokens: 1155 }
// }
```

## Testing

All testing takes place automatically in CI and is configured using Github actions
Expand Down
2 changes: 1 addition & 1 deletion codegen/apis
Submodule apis updated from 3e5739 to 934bd7
2 changes: 1 addition & 1 deletion codegen/build-oas.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
set -eux -o pipefail

version=$1 # e.g. 2024-07
modules=("db_control" "db_data" "inference")
modules=("db_control" "db_data" "inference" "assistant_control" "assistant_data" "assistant_evaluation")

destination="src/pinecone-generated-ts-fetch"
build_dir="build"
Expand Down
5 changes: 3 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading