Vector DB extension

Chromia’s Vector DB extension allows for the efficient storage and querying of complex multi-dimensional data in a decentralized approach. It integrates relational database principles to support high-performance indexing and similarity searches, making it ideal for AI-driven applications such as recommendation systems, natural language processing, and image recognition.

You can easily use Chromia’s Rell language and the Chromia CLI to define vector-based schemas, execute operations and queries, and perform similarity searches. The integration with Chromia’s decentralized network ensures high availability, scalability, and tamper-proof data management. This combination of blockchain technology, relational indexing, and vector search opens up new possibilities for decentralized AI, gaming, and large-scale data analytics.

Leasing a container

If you are unfamiliar with the process of leasing a container, follow these steps.

info

Ensure that the Vector DB extension is selected when leasing the container.

Additionally, verify that the configuration contains the correct values for the blockchain you are trying to connect to.

.chromia/config

key.id=<key_id>
brid=97E2F.....9C530 
api.url=https://node0.testnet.chromia.com:7740 # for testnet

Configuring blockchain in the dapp

When leasing your container, ensure that you select the Vector DB extension. Configure your blockchain to use operations and queries to start utilizing this extension. Below is a sample configuration:

chromia.yml

blockchains:
  my_chain:
    module: my_chain_module
    config:
      gtx:
        modules:
          - "net.postchain.gtx.extensions.vectordb.VectorDbGTXModule"
      vector_db_extension:
        dimensions: 300 # Set number of dimensions to use

Integrating with Rell

Integrating the Vector DB Extension library into your Rell project is optional but recommended. To do this, add it to your configuration as follows:

chromia.yml

libs:
  vector_db:
    registry: https://gitlab.com/chromaway/core/vector-db-extension.git
    path: rell/src/lib/
    tagOrBranch: 0.3.0
    rid: x"1989D7328CC6B06BD07CB9E76904592F1E19667FBD300ACF5BB61D168B90E795" # Update to match version
    insecure: false

tagOrBranch can be found there.

Run chr install.

Once installed, you can add and remove vectors by invoking the store_vector or delete_vector functions.

Inserting vectors

Here is a simple dapp to store and remove vectors:

import lib.vector_db.*;

operation add_vector(context: integer, vector: text, id: integer) {
    store_vector(context, vector, id);
}

operation delete_vector(context: integer, id: integer) {
    delete_vector(context, id);
}

Operation examples:

chr tx -brid $vector_brid add_message hej "[1.0, 2.0, 3.0]"
chr tx -brid $vector_brid add_message hello "[1.0, 2.5, 3.0]"
chr tx -brid $vector_brid add_message hei "[1.0, 2.0, 3.1]"
chr tx -brid $vector_brid add_message "guten tag" "[1.0, 1.5, 3.5]"

Querying vectors

The extension introduces a query function named query_closest_objects, which allows you to search for vectors. It supports the following parameters:

It can be any number, allowing a dapp to use multiple contexts to separate vectors.

Name	Type	Required	Default	Description
context	integer	true		Context used by the dapp. It can be any number, allowing a dapp to use multiple contexts to separate vectors.
q_vector	vector as text	true		The vector to search for, formatted as text in the form of [1, 2, 3].
max_distance	decimal	true		The maximum allowed distance from q_vector to the stored vectors.
max_vectors	integer	false	10	The maximum number of vectors to return.
query_template	text	false	Not set	Provide a Rell query function for transforming the results (see below).

Query examples

# Plain query with no query_template:

chr query -brid $vector_brid query_closest_objects context=0 q_vector="[1.0, 2.0, 3.0]" max_distance=1.0 max_vectors=2
[
  [
    "distance": "0",
    "id": 1
  ],
  [
    "distance": "0.0001212999220387978",
    "id": 3
  ]
]

# Basic query_template provided to return the text messages:

chr query -brid $vector_brid query_closest_objects context=0 q_vector="[1.0, 2.5, 3.0]" max_distance=1.0 max_vectors=2 'query_template=["type":"get_messages"]'
[
  "hello",
  "hej"
]

# Another query_template which returns text and distance:

chr query -brid $vector_brid query_closest_objects context=0 q_vector="[1.0, 2.5, 3.0]" max_distance=1.0 max_vectors=2 'query_template=["type":"get_messages_with_distance"]'
[
  [
    "distance": "0",
    "text": "hello"
  ],
  [
    "distance": "0.005509683802306209",
    "text": "hej"
  ]
]

# Additional arguments passed to the query_template function:

chr query -brid $vector_brid query_closest_objects context=0 q_vector="[1.0, 2.5, 3.0]" max_distance=1.0 max_vectors=2 'query_template=["type":"get_messages_with_filter", "args":["text_filter": "j"]]'
[
  "hej",
]

You can find the source code and additional details about the Vector DB extension in the official repository.