Data operations

Beyond upsert and search, Quiver can change metadata without resending vectors, count points and the values of a field, and delete points by id or by filter. The examples use a small collection:

curl -X POST localhost:6900/collections -H 'content-type: application/json' \
  -d '{"name": "movies", "dimension": 2}' > /dev/null
curl -X POST localhost:6900/collections/movies/points -H 'content-type: application/json' -d '{
  "points": [
    {"id": "alien",   "vector": [0.9, 0.1], "metadata": {"genre": "sci-fi",  "year": 1979, "stats": {"views": 10, "likes": 4}}},
    {"id": "arrival", "vector": [0.8, 0.3], "metadata": {"genre": "sci-fi",  "year": 2016}},
    {"id": "amelie",  "vector": [0.1, 0.9], "metadata": {"genre": "romance", "year": 2001}}
  ]
}'

{"ids":["alien","arrival","amelie"],"upserted":3}

Updating metadata

POST /collections/{name}/points/metadata changes the metadata of the selected points and leaves their vectors and the index alone. Select points with exactly one of ids and filter; ids that do not exist are skipped. The answer is how many points changed.

Merge

The default mode is merge, an RFC 7386 JSON Merge Patch: keys in the patch are set, nested objects merge recursively, and null removes a key.

curl -X POST localhost:6900/collections/movies/points/metadata -H 'content-type: application/json' \
  -d '{"ids": ["alien", "nope"], "metadata": {"seen": true, "stats": {"views": 11, "likes": null}}}'
curl localhost:6900/collections/movies/points/alien?with_vectors=false

{"updated":1}
{"id":"alien","metadata":{"genre":"sci-fi","seen":true,"stats":{"views":11},"year":1979}}

alien kept genre and year, gained seen, and its stats object was merged: views changed and likes was removed. nope does not exist and was skipped. A point with no metadata, or with metadata that is not an object, starts from {}. A patch that is not an object replaces the value outright, as RFC 7386 says.

Replace

"mode": "replace" swaps the whole value; null removes the metadata. Here for every point a filter matches:

curl -X POST localhost:6900/collections/movies/points/metadata -H 'content-type: application/json' \
  -d '{"filter": {"genre": "sci-fi"}, "metadata": {"genre": "classic"}, "mode": "replace"}'
curl localhost:6900/collections/movies/points/arrival?with_vectors=false

{"updated":2}
{"id":"arrival","metadata":{"genre":"classic"}}

The new values are checked (the 64-level depth limit) before anything changes, so an update applies to every selected point or to none. Over gRPC the call is UpdateMetadata, with the metadata as a JSON string.

Counting

POST /collections/{name}/points/count counts every point, or with a filter, the matching ones:

curl -X POST localhost:6900/collections/movies/points/count -H 'content-type: application/json' -d '{}'
curl -X POST localhost:6900/collections/movies/points/count -H 'content-type: application/json' \
  -d '{"filter": {"genre": "classic"}}'

{"count":3}
{"count":2}

POST /collections/{name}/points/facet counts how many points hold each value of a metadata field, most common first. On the films collection from filtering:

curl -X POST localhost:6900/collections/films/points/facet -H 'content-type: application/json' \
  -d '{"field": "tags"}'
curl -X POST localhost:6900/collections/films/points/facet -H 'content-type: application/json' \
  -d '{"field": "genre", "filter": {"year": {"$gt": 1990}}, "limit": 2}'

{"values":[{"count":2,"value":"classic"},{"count":1,"value":"drama"},{"count":1,"value":"horror"}]}
{"values":[{"count":1,"value":"crime"},{"count":1,"value":"romance"}]}

Field	Default	Meaning
`field`		the metadata field, a dotted path
`filter`	none	only count points matching this
`limit`	10	how many values, 1 to 1,000

Values are strings, numbers and booleans; 1 and 1.0 count as one value. A field holding an array counts each distinct element once per point, and null, objects and missing fields are not counted. Ties are ordered by value: booleans, then numbers, then strings. Facets read every matching point, like a count.

films = Client("http://localhost:6900").collection("films")
print([(f.value, f.count) for f in films.facet("tags")])
print([(f.value, f.count) for f in films.facet("genre", filter={"year": {"$gt": 1990}}, limit=2)])

[('classic', 2), ('drama', 1), ('horror', 1)]
[('crime', 1), ('romance', 1)]

Deleting

POST /collections/{name}/points/delete deletes by ids (at most 10,000) or by filter, and answers how many points existed:

curl -X POST localhost:6900/collections/movies/points/delete -H 'content-type: application/json' \
  -d '{"ids": ["amelie", "nope"]}'
curl -X POST localhost:6900/collections/movies/points/delete -H 'content-type: application/json' \
  -d '{"filter": {"genre": "classic"}}'

{"deleted":1}
{"deleted":2}

nope did not exist, so the first delete counts one point; the second removes the two points the replace above turned into classic.

An empty filter would match every point, so a delete refuses it with 400 invalid_argument. To delete everything, drop the collection with DELETE /collections/{name}.

curl -X POST localhost:6900/collections/movies/points/delete -H 'content-type: application/json' \
  -d '{"filter": {}}'

{"error":{"code":"invalid_argument","message":"an empty filter would delete every point; drop the collection to delete everything"}}

A single point can also go with DELETE /collections/{name}/points/{id}, which answers 204, or 404 point_not_found. Deleted slots are reclaimed by compaction.

From Python

from quiver import Client
 
movies = Client("http://localhost:6900").collection("movies")
movies.upsert([("alien", [0.9, 0.1], {"genre": "sci-fi", "year": 1979})])
 
print(movies.update_metadata({"seen": True}, ids=["alien"]))
print(movies.update_metadata({"genre": "classic"}, filter={"year": {"$lt": 1990}}, mode="replace"))
print(movies.count(), movies.count({"genre": "classic"}))
print(movies.delete_where({"genre": "classic"}), movies.delete(["alien"]), movies.count())

update_metadata needs exactly one of ids= and filter= (a ValueError otherwise). In Rust, Client::update_metadata takes a Selector::Ids or Selector::Filter, and the embedded Collection::update_metadata a PointSelector.

Errors

Status	Code	When
400	`invalid_argument`	both or neither of `ids` and `filter`, more than 10,000 ids, an empty delete filter, metadata nested too deep, a facet `limit` outside 1–1,000 or an invalid facet `field`
400	`invalid_filter`	the filter does not parse
400	`invalid_body`	malformed JSON, an unknown field, a missing `metadata`, an unknown `mode`
404	`not_found`	no such collection