Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 50 additions & 7 deletions _includes/code/howto/configure-sq/sq-compression-v3.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
// not yet supported in client 3.0.8

import assert from 'assert';
import weaviate from 'weaviate-client';
// START-ANY
Expand Down Expand Up @@ -27,7 +25,7 @@ const collection = await client.collections.create({

let collectionConfig = await collection.config.get();

assert.equal(collectionConfig.vectorizers.default.indexConfig.quantizer.type, "SQ")
assert.equal(collectionConfig.vectorizers.default.indexConfig.quantizer.type, "sq")

// Clean-up
await client.collections.delete(collectionName);
Expand All @@ -49,20 +47,65 @@ const collection = await client.collections.create({
vectorizers: weaviate.configure.vectors.selfProvided({
vectorIndexConfig: weaviate.configure.vectorIndex.hnsw({
quantizer: weaviate.configure.vectorIndex.quantizer.sq({
cache: true, // Enable caching
rescoreLimit: 200, // The minimum number of candidates to fetch before rescoring
rescoreLimit: 200, // The minimum number of candidates to fetch before rescoring
trainingLimit: 50000, // The size of the training set used to determine the bucket boundaries
}),
vectorCacheMaxObjects: 10000 // Cache size (used if `cache` enabled)
vectorCacheMaxObjects: 100000 // Maximum number of objects in the vector cache
Comment thread
g-despot marked this conversation as resolved.
})
})
})
// END SQWithOptions

let collectionConfig = await collection.config.get();

assert.equal(collectionConfig.vectorizers.default.indexConfig.quantizer.type, "SQ")
assert.equal(collectionConfig.vectorizers.default.indexConfig.quantizer.type, "sq")

// Clean-up
await client.collections.delete(collectionName);

client.close();


// UPDATE SCHEMA
{
const client = await weaviate.connectToLocal();

const collectionName = 'MyCollection';

// Prep
await client.collections.delete(collectionName);
await client.collections.create({
name: collectionName,
vectorizers: weaviate.configure.vectors.selfProvided({
vectorIndexConfig: weaviate.configure.vectorIndex.hnsw({
quantizer: weaviate.configure.vectorIndex.quantizer.none(),
})
})
})

// START UpdateSchema
const collection = client.collections.use('MyCollection');

await collection.config.update({
vectorizers: [
weaviate.reconfigure.vectors.update({
name: 'default',
vectorIndexConfig: weaviate.reconfigure.vectorIndex.hnsw({
quantizer: weaviate.reconfigure.vectorIndex.quantizer.sq({
rescoreLimit: 20,
}),
}),
}),
],
})
// END UpdateSchema

let collectionConfig = await collection.config.get();

assert.equal(collectionConfig.vectorizers.default.indexConfig.quantizer.type, "sq")

// Clean-up
await client.collections.delete(collectionName);

client.close();
}
14 changes: 7 additions & 7 deletions _includes/code/howto/configure-sq/sq-compression.options-v3.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,24 +11,24 @@ const collectionName = 'MyCollection';
// Prep
await client.collections.delete(collectionName);

// START BQWithOptions
// START SQWithOptions
const collection = await client.collections.create({
name: 'MyCollection',
vectorizers: weaviate.configure.vectors.selfProvided({
vectorIndexConfig: weaviate.configure.vectorIndex.hnsw({
quantizer: weaviate.configure.vectorIndex.quantizer.bq({
cache: true, // Enable caching
rescoreLimit: 200, // The minimum number of candidates to fetch before rescoring
quantizer: weaviate.configure.vectorIndex.quantizer.sq({
rescoreLimit: 200, // The minimum number of candidates to fetch before rescoring
trainingLimit: 50000, // The size of the training set used to determine the bucket boundaries
}),
vectorCacheMaxObjects: 10000 // Cache size (used if `cache` enabled)
vectorCacheMaxObjects: 100000 // Maximum number of objects in the vector cache
})
})
})
// END BQWithOptions
// END SQWithOptions

let collectionConfig = await collection.config.get();

assert.equal(collectionConfig.vectorizers.default.indexConfig.quantizer.type, "bq")
assert.equal(collectionConfig.vectorizers.default.indexConfig.quantizer.type, "sq")

// Clean-up
await client.collections.delete(collectionName);
Expand Down
16 changes: 7 additions & 9 deletions docs/cloud/embeddings/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -80,14 +80,14 @@ import PromptStarter from "/src/components/PromptStarter";

To use Weaviate Embeddings, you will need:

<!-- TODO[g-despot]: Update prerequisites with correct client versions -->
- A Weaviate Cloud free cluster
- A Weaviate client library that supports Weaviate Embeddings

- A Weaviate Cloud free cluster running at least Weaviate `1.28.5`
- A Weaviate client library that supports Weaviate Embeddings:
- **Python** client version `4.9.5` or higher
- **JavaScript/TypeScript** client version `3.2.5` or higher
- **Java** or **C#** clients
- **Go** client is not yet officially supported; you must pass the `X-Weaviate-Api-Key` and `X-Weaviate-Cluster-Url` headers manually upon instantiation as shown below.
import CodeClientInstall from "/\_includes/code/quickstart/clients.install.new.mdx";

<CodeClientInstall />

The Go client does not support Weaviate Embeddings directly. Pass the `X-Weaviate-Api-Key` and `X-Weaviate-Cluster-Url` headers manually when you instantiate the client.

## Step 1: Set up Weaviate

Expand All @@ -103,8 +103,6 @@ import LatestWeaviateVersion from "/_includes/latest-weaviate-version.mdx";

We recommend using a [client library](/weaviate/client-libraries) to work with Weaviate. Follow the instructions below to install one of the official client libraries, available in [Python](/weaviate/client-libraries/python), [JavaScript/TypeScript](/weaviate/client-libraries/typescript), [Go](/weaviate/client-libraries/go), and [Java](/weaviate/client-libraries/java).

import CodeClientInstall from "/_includes/code/quickstart/clients.install.mdx";

<CodeClientInstall />

### 1.3 Connect to Weaviate Cloud
Expand Down
4 changes: 4 additions & 0 deletions docs/cloud/manage-collections/enable-compression.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ Replace `YOUR-WEAVIATE-CLOUD-URL` with your cluster URL (e.g., `https://your-clu

The update syntax depends on your collection's **vector index type** (HNSW, flat, or dynamic) and whether it uses **named vectors**.

:::note HFresh collections are already compressed
This procedure does not apply to collections that use the [HFresh index](/weaviate/concepts/vector-index#hfresh-index), which is the index behind the **Cost Optimized** optimization profile and therefore the only index available on [free clusters](../manage-clusters/create.mdx#optimization-profile). HFresh has [rotational quantization (RQ)](../../weaviate/configuration/compression/rq-compression.md) built in and always on: the in-memory centroid index uses 8-bit RQ and the on-disk posting lists use 1-bit RQ. Weaviate rejects a request that tries to add PQ, SQ or BQ to an HFresh index, or that tries to disable its RQ, so there is nothing to enable. HFresh collections are also not listed by the categorization example below.
:::

#### HNSW index (default)

Most collections use the HNSW index. To enable compression:
Expand Down
10 changes: 7 additions & 3 deletions docs/deploy/configuration/env-vars/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ import APITable from '@site/src/components/APITable';
| --- | --- | --- | --- |
| `ASYNC_INDEXING` | If set, Weaviate creates vector indexes asynchronously to the object creation process. This can be useful for importing large amounts of data. (default: `false`) | `boolean` | `false` |
| `AUTOSCHEMA_ENABLED` | Whether to infer the schema where necessary with the autoschema (default: `true`) | `boolean` | `true` |
| `CORS_ALLOW_HEADERS` | Value of the `Access-Control-Allow-Headers` response header on the REST API, which controls the request headers a browser may send cross-origin. The default is the long list of headers Weaviate itself reads, including `Content-Type`, `Authorization` and the per-provider API-key headers. Default: the built-in header list | `string - comma separated names` | `Content-Type, Authorization` |
| `CORS_ALLOW_METHODS` | Value of the `Access-Control-Allow-Methods` response header on the REST API, which controls the HTTP methods a browser may use cross-origin. Default: `*` | `string - comma separated names` | `GET, POST, OPTIONS` |
| `CORS_ALLOW_ORIGIN` | Value of the `Access-Control-Allow-Origin` response header on the REST API, which controls the origins a browser may call Weaviate from. Set this to reach Weaviate directly from browser code on a specific site. Default: `*` | `string` | `https://example.com` |
| `DEFAULT_QUANTIZATION` | Default quantization technique - can be overridden by the quantization method specified in the collection definition. Available values: `rq-8`, `rq-1`, `pq`, `bq`, `sq` and `none`. Default: `none`.<br/><br/>Note: If the selected quantization method isn't supported for the index type of a collection (for example PQ & SQ aren't supported for the flat index), the quantization won't be applied to that collection.<br/><br/>Added in `v1.33` | `string` | `rq-8` |
| `DEFAULT_SHARDING_COUNT` | Default `desiredCount` for new single-tenant collections, used when the collection definition does not specify one. An explicit `desiredCount` in the class creation request still takes precedence. A value of `0` (default) uses the cluster node count. Multi-tenant collections are unaffected. Must be `<= 512`. Runtime-configurable. Default: `0`<br/>Added in `v1.37` | `string - number` | `12` |
| `DEFAULT_VECTOR_INDEX` | Default vector index type for new collections (and named vectors), used when the collection definition does not specify one. An explicit `vectorIndexType` in the collection definition still takes precedence. Available values: `hnsw`, `flat`, `dynamic`, and `hfresh`. Runtime-configurable. Default: `hnsw`<br/>Added in `v1.37.3` | `string` | `flat` |
Expand All @@ -57,7 +60,7 @@ import APITable from '@site/src/components/APITable';
| `INVERTED_SORTER_DISABLED` | Forces the "objects bucket" strategy and doesn't consider inverted sorting. Most users should never set this flag; it exists for benchmarking and as a safety net. Default: `false` | `boolean` | `false` |
| `GO_PROFILING_DISABLE` | If `true`, disables Go profiling. Default: `false`. | `boolean` | `false` |
| `GO_PROFILING_PORT` | Sets the port for the Go profiler. Default: `6060` | `integer` | `6060` |
| `DEBUG_ENDPOINTS_ENABLED` | Gate for the debug HTTP listener (the profiling port set by `GO_PROFILING_PORT`, default `6060`), which serves Weaviate's **unauthenticated** internal debug and profiling endpoints `/debug/config`, Go profiling (`/debug/pprof/*`, `/debug/fgprof`), and various maintenance and diagnostic routes. [Runtime-configurable](/deploy/configuration/env-vars/runtime-config.md) via the `debug_endpoints_enabled` override. Default: `false`. `GO_PROFILING_DISABLE` still controls whether the listener binds at all. <br/>Added in `v1.37.9` | `boolean` | `true` |
| `DEBUG_ENDPOINTS_ENABLED` | Gate for the debug HTTP listener (the profiling port set by `GO_PROFILING_PORT`, default `6060`), which serves Weaviate's **unauthenticated** internal debug and profiling endpoints: `/debug/config`, Go profiling (`/debug/pprof/*`, `/debug/fgprof`), and various maintenance and diagnostic routes. [Runtime-configurable](/deploy/configuration/env-vars/runtime-config.md) via the `debug_endpoints_enabled` override. Default: `false`. `GO_PROFILING_DISABLE` still controls whether the listener binds at all. <br/>Added in `v1.37.9` | `boolean` | `true` |
| `GRPC_MAX_MESSAGE_SIZE` | Maximum gRPC message size in bytes. Requests larger than this limit (e.g. a large `insert_many` call) are rejected. Default: `104858000` (approximately 100 MB) | `string - number` | `2000000000` |
| `GRPC_PORT` | The port on which Weaviate's gRPC server listens for incoming requests. Default: `50051` | `string - number` | `50052` |
| `HNSW_GEO_INDEX_EF` | Balance geo index search speed and recall. This value controls the search depth for geo-based queries. Default: `800`<br/>Added in `v1.31.22` | `string - number` | `1000` |
Expand Down Expand Up @@ -96,11 +99,12 @@ import APITable from '@site/src/components/APITable';
| `QUERY_BOOST_DEFAULT_DEPTH` | Default candidate-pool size used when a [Boost](/weaviate/search/boost.md) query does not set its own `depth`. The primary search retrieves this many candidates before the boost rescorer runs. Must be a positive integer and is hard-capped by `QUERY_MAXIMUM_RESULTS`. Default: `100`<br/>Added in `v1.38` | `string - number` | `200` |
| `QUERY_CROSS_REFERENCE_DEPTH_LIMIT` | Sets the maximum depth of cross-references to be resolved in a query. Defaults to 5. | `string - number` | `3` |
| `QUERY_DEFAULTS_LIMIT` | Sets the default number of objects to be returned in a query. | `string - number` | `25` <br/> Defaults to `10`|
| `QUERY_HYBRID_MAXIMUM_RESULTS` | Minimum size of the candidate pool that each leg of a [hybrid search](/weaviate/search/hybrid.md) retrieves before fusion. Each of the keyword and vector sub-searches fetches at least `offset` plus this many candidates, so that paging deeper into a hybrid result set stays consistent. Raise it if hybrid results shift between pages; higher values cost more work per query. Default: `100`<br/>Added in `v1.32`, and backported to `v1.30.12` and `v1.31.6` | `string - number` | `200` |
| `QUERY_MAXIMUM_RESULTS` | Sets the maximum total number of objects that can be retrieved. | `string - number` | `10000` |
| `QUERY_SLOW_LOG_ENABLED` | Log slow queries for debugging. Requires a restart to update. | `boolean` | `False` |
| `QUERY_SLOW_LOG_THRESHOLD` | Set a threshold time for slow query logging. Requires a restart to update. | `string` | `2s` <br/> Values are times: `3h`, `2s`, `100ms` |
| `REINDEX_SET_TO_ROARINGSET_AT_STARTUP` | Allow Weaviate to perform a one-off re-indexing to use Roaring Bitmaps. | `boolean` | `true` |
| `REVECTORIZE_CHECK_DISABLED` | Disables the optimization that reuses an object's existing vector when it is updated and none of its vectorized properties changed. By default (`false`) this check runs and skips unnecessary re-vectorization; set to `true` to re-vectorize on every update. Disabling removes a read-before-write — which can raise write throughput, or force always-fresh vectors — at the cost of an extra embedding call (and API cost, for remote vectorizers) per update. Default: `false` | `boolean` | `false` |
| `REVECTORIZE_CHECK_DISABLED` | Disables the optimization that reuses an object's existing vector when it is updated and none of its vectorized properties changed. By default (`false`) this check runs and skips unnecessary re-vectorization; set to `true` to re-vectorize on every update. Disabling removes a read-before-write. That can raise write throughput, and it guarantees a fresh vector on every update, at the cost of one extra embedding call per update (and the associated API cost, for remote vectorizers). Default: `false` | `boolean` | `false` |
| `TENANT_ACTIVITY_READ_LOG_LEVEL` | Sets the log level for tenant read activity. Useful for analysis or debugging purposes. Default: `debug` | `string` | `info` |
| `TENANT_ACTIVITY_WRITE_LOG_LEVEL` | Sets the log level for tenant write activity. Useful for analysis or debugging purposes. Default: `debug` | `string` | `info` |
| `TOKENIZER_CONCURRENCY_COUNT` | Limit the combined number of GSE and Kagome tokenizers running at the same time. Default: `GOMAXPROCS` | `string - number` | `NUMBER_OF_CPU_CORES` |
Expand Down Expand Up @@ -224,7 +228,7 @@ For more information on authentication and authorization, see the [Authenticatio
| `CLUSTER_GOSSIP_BIND_PORT` | Port for exchanging network state information. | `string - number` | `7102` |
| `CLUSTER_HOSTNAME` | Hostname of a node. Always set this value if the default OS hostname might change over time. | `string` | `node1` |
| `CLUSTER_JOIN` | The service name of the "founding" member node in a cluster setup | `string` | `weaviate-node-1:7100` |
| `HNSW_STARTUP_WAIT_FOR_VECTOR_CACHE` | If `true`, vector cache prefill is synchronous when a node starts. The node reports ready to serve when the cache is hot. Default changed to `true` in `v1.36.6`. For collections where [dynamic lazy shard loading](/weaviate/concepts/storage#dynamic-lazy-shard-loading) is active, this is always overridden to `false` regardless of the configured value. The configured value only applies to eagerly-loaded collections. | `boolean` | `true` |
| `HNSW_STARTUP_WAIT_FOR_VECTOR_CACHE` | If `true`, vector cache prefill is synchronous when a node starts. The node reports ready to serve when the cache is hot. Default changed to `true` in `v1.36.6`. **Deprecated in `v1.36.6`.** Setting it still overrides auto-detection, but Weaviate logs a deprecation warning at startup and the variable will be removed in a future version. When it is unset, prefill behavior is governed by [dynamic lazy shard loading](/weaviate/concepts/storage#dynamic-lazy-shard-loading). For collections where dynamic lazy shard loading is active, this is always overridden to `false` regardless of the configured value. The configured value only applies to eagerly-loaded collections. | `boolean` | `true` |
| `COLLECTION_RETRIEVAL_STRATEGY`| Set collection definition retrieval behavior for a data request. <br/><br/> <ul><li>`LeaderOnly` (default): Always requests the definition from the leader node. </li><li>`LocalOnly`: Always use the local definition</li><li>`LeaderOnMismatch`: Requests the definition if outdated.</li></ul> ([Read more](/weaviate/concepts/replication-architecture/consistency.md#collection-definition-requests-in-queries)) | `string` | `LeaderOnly` |
| `RAFT_BOOTSTRAP_EXPECT` | The number of voter notes at bootstrapping time | `string - number` | `1` |
| `RAFT_BOOTSTRAP_TIMEOUT` | The time in seconds to wait for the cluster to bootstrap | `string - number` | `90` |
Expand Down
4 changes: 2 additions & 2 deletions docs/deploy/configuration/env-vars/runtime-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,14 +120,14 @@ The following overrides are currently supported:

### MCP

Added in `v1.38`. Toggling these at runtime does not require a cluster restartthe HTTP handlers stay registered and per-request checks pick up the new value. See [MCP server Toggle without restart](/weaviate/configuration/mcp-server.mdx#toggle-without-restart) for behavior details.
Added in `v1.38`. Toggling these at runtime does not require a cluster restart: the HTTP handlers stay registered and per-request checks pick up the new value. See [MCP server: Toggle without restart](/weaviate/configuration/mcp-server.mdx#toggle-without-restart) for behavior details.

| Runtime override name | Environment variable name |
| :-------------------------------- | :---------------------------------- |
| `mcp_server_enabled` | `MCP_SERVER_ENABLED` |
| `mcp_server_write_access_enabled` | `MCP_SERVER_WRITE_ACCESS_ENABLED` |

`MCP_SERVER_CONFIG_PATH` is intentionally **not** runtime-configurable tool descriptions are baked into the tool schemas at registration.
`MCP_SERVER_CONFIG_PATH` is intentionally **not** runtime-configurable, because tool descriptions are baked into the tool schemas at registration.

Refer to the [Environment variables](./index.md) page for descriptions on each configuration option

Expand Down
4 changes: 2 additions & 2 deletions docs/deploy/configuration/export.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import FilteredTextBlock from '@site/src/components/Documentation/FilteredTextBl
import PyCode from '!!raw-loader!/\_includes/code/howto/configure.export.py';
import CsCode from '!!raw-loader!/\_includes/code/csharp/ManageDataExportTest.cs';

:::caution Preview added in `v1.37`
:::caution Preview (added in `v1.37`)
This is a preview feature. The API may change in future releases.
:::

Expand Down Expand Up @@ -206,7 +206,7 @@ Files are named `{collection}_{shard}_{rangeIndex}.parquet`. Collection and tena
| COLD | Exported directly from disk without loading into memory (remains COLD). |
| OFFLOADED | Skipped. The skip reason is recorded in the shard status. |

The tenant list is snapshotted when the export is created — tenants created during the export are not included.
The tenant list is snapshotted when the export is created. Tenants created during the export are not included.

## Permissions

Expand Down
Loading
Loading