Skip to content

Commit

Permalink
Merge pull request #972 from milvus-io/update-docs
Browse files Browse the repository at this point in the history
Update docs
  • Loading branch information
AnthonyTsu1984 authored Nov 28, 2024
2 parents 13da608 + 1210e8a commit 70b17b6
Showing 165 changed files with 520 additions and 25 deletions.
70 changes: 70 additions & 0 deletions API_Reference/milvus-sdk-java/v2.5.x/About.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# About

Milvus Java SDK is an open-source project and its source code is hosted on [GitHub](https://github.com/milvus-io/milvus-sdk-java).

## **Compatibility**

<table>
<tr>
<th>Milvus version</th>
<th>Recommended SDK version</th>
</tr>
<tr>
<td>2.0.2</td>
<td>2.0.4</td>
</tr>
<tr>
<td>2.1</td>
<td>2.1.0-beta4</td>
</tr>
<tr>
<td>2.2.0 ~ 2.2.8</td>
<td>2.2.5</td>
</tr>
<tr>
<td>2.2.9 ~ 2.2.14</td>
<td>2.2.13</td>
</tr>
<tr>
<td>2.3.x</td>
<td>2.3.9</td>
</tr>
<tr>
<td>2.4.x</td>
<td>2.4.8</td>
</tr>
<tr>
<td>2.5.x</td>
<td>2.5.0</td>
</tr>
</table>

## **Installation**

You can use **[Apache Maven](https://maven.apache.org/install.html)** or **[Gradle](https://gradle.org/install/)** to download the SDK.

- Apache Maven

```xml
<dependency>
<groupId>io.milvus</groupId>
<artifactId>milvus-sdk-java</artifactId>
<version>2.5.0</version>
</dependency>
```

- Gradle/Groovy

```plaintext
implementation 'io.milvus:milvus-sdk-java:2.5.0'
```

- Gradle/Kotlin

```sql
implementation("io.milvus:milvus-sdk-java:2.5.0")
```

## **Contributing**

We are committed to building a collaborative, exuberant open-source community for Milvus. Therefore, contributions to Milvus Java SDK are welcome from everyone. Refer to [Contributing Guideline](https://github.com/milvus-io/milvus-sdk-java/blob/master/CONTRIBUTING.md) before making contributions to this project. You can [file an issue](https://github.com/milvus-io/milvus-sdk-java/issues/new) if you need any assistance or want to propose your ideas
257 changes: 257 additions & 0 deletions API_Reference/milvus-sdk-node/v2.5.x/About.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,257 @@
# About Milvus-sdk-node

Milvus-sdk-node is a node.js SDK for Milvus, an open-source vector database. Its source code is available on [GitHub](https://github.com/milvus-io/milvus-sdk-node).

## Compatibility

The following collection shows Milvus versions and recommended @zilliz/milvus2-sdk-node versions:

| Milvus version | Node sdk version | Installation |
| :------------: | :--------------: | :---------------------------------- |
| v2.2.0+ | **latest** | `yarn add @zilliz/milvus2-sdk-node` |
| v2.3.0+ | **latest** | `yarn add @zilliz/milvus2-sdk-node` |
| v2.4.0+ | **latest** | `yarn add @zilliz/milvus2-sdk-node` |
| v2.5.0+ | **latest** | `yarn add @zilliz/milvus2-sdk-node` |

## Dependencies

- [Milvus](https://milvus.io/)
- [Zilliz Cloud](https://cloud.zilliz.com/signup)
- Node: v14+

## Installation

The recommended way to get started using the Milvus node.js client is by using npm (Node package manager) to install the dependency in your project.

```javascript
npm install @zilliz/milvus2-sdk-node
# or ...
yarn add @zilliz/milvus2-sdk-node
```

This will download the Milvus node.js client and add a dependency entry in your package.json file.

## Quick Start

This guide will show you how to set up a simple application using Node.js and Milvus. Its scope is only how to set up the node.js client and perform the simple CRUD operations. For more in-depth coverage, see the [Milvus official website](https://milvus.io/).

### Create the package.json file

First, create a directory where your application will live.

```
mkdir myProject
cd myProject
```

Enter the following command and answer the questions to create the initial structure for your new project:

```shell
npm init -y
```

Next, install this client as a dependency.

```shell
npm install @zilliz/milvus2-sdk-node
```

### Start a milvus server

```shell
# Download the milvus standalone yaml file
$ wget https://github.com/milvus-io/milvus/releases/download/v2.2.8/milvus-standalone-docker-compose.yml -O docker-compose.yml

# start the milvus server
sudo docker-compose up -d
```

### Connect to Milvus

Create a new app.js file and add the following code to try out some basic vector operations using the Milvus node.js client.

```javascript
import { MilvusClient, DataType } from "@zilliz/milvus2-sdk-node";

const address = "your-milvus-ip";
const username = "your-milvus-username"; // optional username
const password = "your-milvus-password"; // optional password
const ssl = false; // secure or not

// connect to milvus
const client = new MilvusClient({ address, ssl, username, password });
```

| Parameters | Description | Type | Example |
| --------------- | ------------------------------------------------------------------------------------------------------------------------ | ------- | ------------------- |
| address | The Milvus IP address | String | '192.168.0.1:19530' |
| ssl? | SSL connection. It is false by default. | Boolean | false |
| username? | The username used to connect to Milvus | String | milvus |
| password? | The password used to connect to Milvus | String | milvus |
| maxRetries? | The number of retries for the grpc method, by default: 3 | Number | 3 |
| retryDelay? | The delay between attempts at retrying a failed grpc method in ms, by default: 30 | Number | 30 |
| channelOptions? | an optional configuration object that can be passed to a gRPC client when creating a channel to connect to a gRPC server | Number | 30 |

### define schema for collection

The code shows an example of how to define a schema for a collection in Milvus using the Milvus Node SDK. A schema defines the properties of a collection, such as the names and data types of the fields that make up the vectors.

```javascript
// define schema
const collection_name = `book`;
const dim = 128;
const schema = [
{
name: `book_id`,
description: `customized primary id`,
data_type: DataType.Int64,
is_primary_key: true,
autoID: false,
},
{
name: `word_count`,
description: `word count`,
data_type: DataType.Int64,
},
{
name: `book_intro`,
description: `word count`,
data_type: DataType.FloatVector,
dim: dim,
},
];
```

In the code, the schema variable is defined as an array of objects, where each object represents a field in the collection. The fields are defined using the following properties:

`name`: The name of the field, which must be unique within the collection.

`description`: A description of the field, which can be used to provide additional information about the field.

`data_type`: The data type of the field, which can be one of several predefined data types such as Int64 or FloatVector.

`is_primary_key`: A boolean flag that indicates whether the field is a primary key. Primary keys are unique identifiers for each vector in the collection, and can be used to efficiently retrieve specific vectors.

`autoID`: A boolean flag that indicates whether the primary key is automatically generated by Milvus.

`dim`: The dimensionality of the vector field. For fields of type FloatVector, this specifies the number of dimensions in each vector.

### create collection

```javascript
await client.createCollection({
collection_name,
description: `my first collection`,
fields: schema,
});
```

### prepare data

When using the Milvus Node SDK to insert data into a collection, it's important to ensure that the data format of the input matches the schema defined for the collection. The data format used by the Milvus Node SDK consists of an array of objects, where each object represents an entity. Typically, each object contains a unique identifier for the entity, which can be an integer or string, and a vector field that stores the feature values as an array of floating-point numbers.

```javascript
// generate mock data
const fields_data = [];

// generate mock data
for (let i = 0; i < 1000; i++) {
// create a new object with random values for each field
const r = {
book_id: Math.floor(Math.random() * 100000), // generate a random book ID
word_count: Math.floor(Math.random() * 1000), // generate a random word count
book_intro: [...Array(dim)].map(() => Math.random()), // generate a random vector for book_intro
};
// add the new object to the fields_data array
fields_data.push(r);
}
```

### insert data into collection

Once we have the data, you can insert data into the collection.

```javascript
await client.insert({
collection_name,
fields_data,
});
```

### create index

By creating an index and loading the collection into memory, you can improve the performance of search and retrieval operations in Milvus, making it faster and more efficient to work with large-scale datasets.

```javascript
// create index
await client.createIndex({
collection_name,
field_name: "book_intro",
index_name: "myindex",
index_type: "HNSW",
params: { efConstruction: 10, M: 4 },
metric_type: "L2",
});
```

Milvus supports [several different types of indexes](https://milvus.io/docs/index.md), each of which is optimized for different use cases and data distributions. Some of the most commonly used index types in Milvus include IVF_FLAT, IVF_SQ8, IVF_PQ, and HNSW. When creating an index in Milvus, you must choose an appropriate index type based on your specific use case and data distribution.

`collection_name`: The name of the collection to create the index for.

`field_name`: The name of the field to create the index on.

`index_name`: The name of the index to create.

`index_type`: The type of index to create.

`param`: The index build parameters, please refer to the [index docs](https://milvus.io/docs/index.md).

`metric_type`: The distance metric to use when computing the similarity between vectors. In this case, the metric type is set to L2, which is a commonly used metric for computing the Euclidean distance between vectors.

### load collection

When you create a collection in Milvus, the collection data is initially stored on disk, and it is not immediately available for search and retrieval. In order to search or retrieve data from the collection, you must first load the collection into memory using the loadCollectionSync method.

```javascript
// load collection
await client.loadCollectionSync({
collection_name,
});
```

### vector search

Now you can perform vector search on your collection.

```javascript
// Generate a random search vector
const searchVector = [...Array(dim)].map(() => Math.random());

// Perform a vector search on the collection
const res = await client.search({
collection_name, // required, the collection name
vector: [0.1, 0.2, 0.3, 0.4], // required, vector used to compare other vectors in milvus
filter: "word_count > 0", // optional, filter
params: { nprobe: 64 }, // optional, specify the search parameters
limit: 1, // specify the number of nearest neighbors to return
metric_type: "L2", // optional, metric to calculate similarity of two vectors
output_fields: ["book_id", "word_count"], // optional, specify the fields to return in the search results
});
```

The code snippet performs a vector search on the collection.

First, a random search vector is generated. Then, the `client.search()` method is called with several parameters:
`collection_name`: Required. Specifies the name of the Milvus collection to search.

`vectors`: Required. Specifies the vector used to compare other vectors in Milvus. The example code passes an array of floating-point numbers as the vector.

`filter` or `expr`: Optional. Specifies a filter to apply to the search query. The example code passes a filter that only includes vectors where the word_count field is greater than 0.

`params`: Optional. Specifies additional search parameters. The example code sets nprobe to 64, which controls the number of clusters to search during the query. By default, nothing will be set.

`limit` or `topk`: Optional. Specifies the number of nearest neighbors to return. The example code sets this to 1. By default, it will be `100`

`metric_type`: Optional. Specifies the metric used to calculate the similarity of two vectors. The example code sets this to `"L2"`. By default, it will be `"L2"`.

`output_fields`: Optional. Specifies which fields to include in the search results. The example code specifies the `book_id` and `word_count` fields, by default the node sdk will output all fields.
Loading

0 comments on commit 70b17b6

Please sign in to comment.