Skip to content

Commit

Permalink
Update documentation for kvstore titles.
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 580968592
Change-Id: Icdc507cf1061d325d6584697534b4df2937bbf59
  • Loading branch information
laramiel authored and copybara-github committed Nov 9, 2023
1 parent 5ecc6e2 commit 1c1963a
Show file tree
Hide file tree
Showing 12 changed files with 36 additions and 25 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ storage and manipulation of large multi-dimensional arrays that:

* Natively supports multiple storage systems, such as local and network
filesystems, [Google Cloud Storage](https://cloud.google.com/storage),
HTTP servers, and in-memory storage.
Amazon S3-compatible object stores, HTTP servers, and in-memory storage.

* Offers an asynchronous API to enable high-throughput access even to
high-latency remote storage.
Expand Down
9 changes: 4 additions & 5 deletions tensorstore/kvstore/file/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/file
title: Read/write access to the local filesystem.
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand All @@ -12,16 +14,13 @@ allOf:
file_io_concurrency:
$ref: ContextResource
description: |-
Specifies or references a previously defined
`Context.file_io_concurrency`.
Specifies or references a previously defined `Context.file_io_concurrency`.
file_io_sync:
$ref: ContextResource
description: |-
Specifies or references a previously defined
`Context.file_io_sync`.
Specifies or references a previously defined `Context.file_io_sync`.
required:
- path
title: JSON specification of file-backed key-value store.
definitions:
url:
$id: KvStoreUrl/file
Expand Down
2 changes: 2 additions & 0 deletions tensorstore/kvstore/gcs/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/gcs
title: Read/write access to Google Cloud Storage (GCS).
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand Down
2 changes: 2 additions & 0 deletions tensorstore/kvstore/grpc/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/grpc
title: Read/write key-value store using the tensorstore-specific gRPC protocol.
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand Down
2 changes: 2 additions & 0 deletions tensorstore/kvstore/http/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/http
title: Read-only access to arbitrary HTTP servers.
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand Down
2 changes: 2 additions & 0 deletions tensorstore/kvstore/memory/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/memory
title: Read/write, non-persistent in-memory storage.
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand Down
11 changes: 5 additions & 6 deletions tensorstore/kvstore/neuroglancer_uint64_sharded/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/neuroglancer_uint64_sharded
title: Read/write adapter for the :ref:`Neuroglancer Precomputed<neuroglancer-precomputed-driver>` sharded format.
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand All @@ -15,7 +17,7 @@ allOf:
$ref: kvstore/neuroglancer_uint64_sharded/ShardingSpec
cache_pool:
$ref: ContextResource
description: |-
description: |
Specifies or references a previously defined `Context.cache_pool`. It
is normally more convenient to specify a default `~Context.cache_pool`
in the `.context`.
Expand All @@ -26,20 +28,17 @@ allOf:
`~Context.cache_pool.total_bytes_limit` value. Otherwise, every read
operation will require 2 additional reads, to read the shard index
and the minishard index.
default: cache_pool
data_copy_concurrency:
$ref: ContextResource
description: |-
Specifies or references a previously defined
`Context.data_copy_concurrency`. It is normally more
convenient to specify a default `~Context.data_copy_concurrency` in
Specifies or references a previously defined `Context.data_copy_concurrency`.
It is normally more convenient to specify a default `~Context.data_copy_concurrency` in
the `.context`.
default: data_copy_concurrency
required:
- base
- metadata
title: JSON specification of neuroglancer_uint64_sharded key-value store.
definitions:
sharding-spec:
$id: 'kvstore/neuroglancer_uint64_sharded/ShardingSpec'
Expand Down
7 changes: 4 additions & 3 deletions tensorstore/kvstore/ocdbt/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/ocdbt
title: Read/write adapter for the OCDBT format.
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand Down Expand Up @@ -79,20 +81,19 @@ allOf:
$ref: ContextResource
description: |-
Specifies or references a previously defined `Context.cache_pool`. It
is normally more convenient to specify a default `~Context.cache_pool`
is typically more convenient to specify a default `~Context.cache_pool`
in the `.context`.
default: cache_pool
data_copy_concurrency:
$ref: ContextResource
description: |-
Specifies or references a previously defined
`Context.data_copy_concurrency`. It is normally more
`Context.data_copy_concurrency`. It is typically more
convenient to specify a default `~Context.data_copy_concurrency` in
the `.context`.
default: data_copy_concurrency
required:
- base
title: JSON specification of OCDBT key-value store.
definitions:
zstd_compression:
$id: kvstore/ocdbt/Compression/zstd
Expand Down
3 changes: 2 additions & 1 deletion tensorstore/kvstore/s3/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
``s3`` Key-Value Store driver
===============================

The ``s3`` driver provides access to S3. Keys directly correspond to HTTP paths.
The ``s3`` driver provides access to Amazon S3 and S3-compatible object stores.
Keys directly correspond to paths within an S3 bucket.

.. json:schema:: kvstore/s3
Expand Down
2 changes: 2 additions & 0 deletions tensorstore/kvstore/s3/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/s3
title: Read/write access to Amazon S3-compatible object stores.
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand Down
10 changes: 5 additions & 5 deletions tensorstore/kvstore/zarr3_sharding_indexed/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/zarr_sharding_indexed
title: Read/write adapter for the :ref:`zarr v3<zarr3-driver>` sharded_indexed format.
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand All @@ -21,7 +23,7 @@ allOf:
$ref: driver/zarr3/CodecChain
cache_pool:
$ref: ContextResource
description: |-
description: |
Specifies or references a previously defined `Context.cache_pool`. It
is normally more convenient to specify a default `~Context.cache_pool`
in the `.context`.
Expand All @@ -35,13 +37,11 @@ allOf:
data_copy_concurrency:
$ref: ContextResource
description: |-
Specifies or references a previously defined
`Context.data_copy_concurrency`. It is normally more
convenient to specify a default `~Context.data_copy_concurrency` in
Specifies or references a previously defined `Context.data_copy_concurrency`.
It is normally more convenient to specify a default `~Context.data_copy_concurrency` in
the `.context`.
default: data_copy_concurrency
required:
- base
- grid_shape
- index_codecs
title: JSON specification of zarr_sharding_indexed key-value store.
9 changes: 5 additions & 4 deletions tensorstore/kvstore/zip/schema.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
$schema: http://json-schema.org/draft-07/schema#
$id: kvstore/zip
title: Read-only adapter for the ZIP archive format.
description: JSON specification of the key-value store.
allOf:
- $ref: KvStore
- type: object
Expand All @@ -8,22 +10,21 @@ allOf:
const: zip
base:
$ref: KvStore
title: Underlying key-value store with path to shard.
title: Underlying key-value store with path to a ZIP file.
cache_pool:
$ref: ContextResource
description: |-
Specifies or references a previously defined `Context.cache_pool`. It
is normally more convenient to specify a default `~Context.cache_pool`
is typically more convenient to specify a default `~Context.cache_pool`
in the `.context`.
default: cache_pool
data_copy_concurrency:
$ref: ContextResource
description: |-
Specifies or references a previously defined
`Context.data_copy_concurrency`. It is normally more
`Context.data_copy_concurrency`. It is typically more
convenient to specify a default `~Context.data_copy_concurrency` in
the `.context`.
default: data_copy_concurrency
required:
- base
title: JSON specification of zip key-value store.

0 comments on commit 1c1963a

Please sign in to comment.