Core concepts

Language overview

Rell is a language for relational blockchain programming. It combines the following features:

1. Relational data modeling and queries similar to SQL. People familiar with SQL should feel at home once they learn the new syntax.
2. Normal programming constructs: variables, loops, functions, collections, etc.
3. Constructs which specifically target app backends and, in particular, blockchain-style programming including request routing, authorization, etc.

Rell aims to make programming as ergonomic as possible. It minimizes boilerplate and repetition. At the same time, as a static type system it can detect and prevent many kinds of defects.

Blockchain programming

There are many different styles of blockchain programming. In the context of Rell, we see blockchain as a method for secure synchronization of databases on nodes of the system. Thus Rell is very database-centric.

Programming in Rell is pretty much identical to programming app backends: you need to handle requests to modify the data in the database and other requests which retrieve data from a database. Handling these two types of requests is basically all that a backend does. But, of course, before you implement request handlers, you need to describe your data model first.

Entity definitions

In SQL, usually you define your data model using CREATE TABLE syntax. In Java, you can define data objects using class definition. In Rell, we define them as entity.

Rell uses persistent objects, thus an entity definition automatically creates the storage (a table) necessary to persist objects of a entity. As you might expect, Rell's entity definition includes a list of attributes:

entity user {
    pubkey: pubkey;
    name: text;
}

It's very common that the name of the attribute is the same as its type. For example, it makes sense to call user's pubkey "pubkey." Rell allows you to shorten pubkey: pubkey; to just pubkey;. Rell also has a number of convenient semantic types, so there is a type called name as well. Thus you can rewrite the definition as just:

entity user { pubkey; name; }

Typically a system shouldn't allow different users to have the same name. That's, names should be unique. If name is unique, it can identify a user. In Rell, you can do this by defining a key, key name;. Note that it's not necessary to define both key and attribute. Rell is smart enough to figure out that if you use an attribute in a key, that attribute should exist in a entity.

It also might be useful to find a user by his pubkey. Should it also be unique? Not necessarily. A user might have several different identities. When you want to enable fast retrieval, but don't need uniqueness, you can use index definition:

entity user {
   key name;
   index pubkey;
}

However, if you want pubkey to be unique for a user, you can add a second key:

entity user {
   key name;
   key pubkey;
}

Typically, when you define a class in a programming language, it creates a type which you can use to refer to instances of that class. This is exactly how it works in Rell. The definition of entity user creates a type user which is a type of references to objects stored in a database. You can use references as attributes. For example, you might want to define something owned by a user, say, a channel. You can describe it like this:

entity channel {
   index owner: user;
   key name;
}

index makes it possible to efficiently find all channels owned by a user. key makes sure that channel names are unique within the system.

You might analyze channel entity definition from a point of view of a traditional relational database terminology. A single user can get associated with multiple channel objects, but a single channel is always related to a single user. Thus this represents one-to-many relationship. owner attribute of a channel refers to user object and thus constitutes a foreign key.

If channel names should be unique only in context of a single user (e.g. alice/news and bob/news are different channels), then you nee to use a composite key:

entity channel {
   key owner: user, name;
}

This basically means that a pair of (owner, name) should be unique.

Finally, one might ask: what changes if we change index owner: user to key owner: user? This makes a user reference unique per channel table, thus there can be at most one channel per user in that case. (If owner is the key, the the relationship between users and channels becomes a one-to-one relationship.)

Operations

Now that we defined the data model, we can finally get to handling requests. As previously mentioned, Rell works with two types of requests:

1. Data-modifying requests. We call them operations which get applied to the database state.
2. Data-retrieving requests. We call them queries.

But for both types of requests you are going to need to refer to things in the database, so you must consider relational operators first.

Creating objects

First, have a look at how we create objects:

create user (pubkey=x"0373599a61cc6b3bc02a78c34313e1737ae9cfd56b9bb24360b437d469efdf3b15",
             name="Alice");

This is essentially the same as INSERT operation in SQL, but the syntax is a bit different. Rell is smart enough to identify the connection between arguments and attributes based on their type. x"..." notation is a hexadecimal byte_array literal which is compatible with pubkey type. On the other hand, name gets provided via text literal. Thus we can write:

create user("Alice", x"0373599a61cc6b3bc02a78c34313e1737ae9cfd56b9bb24360b437d469efdf3b15");

The order of arguments doesn't matter here, they're matched with attributes based on types.

Finding objects

How do we find that object now?

val alice = user @ {.name=="Alice"};

• The @ operator retrieves a single record (or an object in this case) satisfying the search criteria you provided. If there is no such record, or more than one exists, it raises an error. It's recommended to use this construct when an operation needs a single record to operate on. If this requirement gets violated, then the operation gets aborted and all its effects get rolled back. Thus it's a succinct and effective way to deal with requirements.
• val defines a read-only variable that you can later use in an expression. A variable defined using var can get reassigned later.

If you want to retrieve a list of users, you can use the @* operator. For example:

val all_users = user @* {};

This returns a list of all users (since no filter expression was available, all users match it). Value declarations can include a type, for example, we can specify that all_users is of type list<user> like this:

val all_users: list<user> = user @* {};

Since the Rell compiler knows a type of every expression it doesn't really need a type declaration, however, if one is available, it checks against it. Type declarations are mostly useful as documentation for programmers reading the code and they msut not get used in cases where there is no ambiguity.

Both @ and @* correspond to SELECT in SQL.

Operation example

Make an operation which allows a user to create a new channel:

operation register_channel (user_pubkey: pubkey, channel_name: name) {
    require( op_context.is_signer(user_pubkey) );
    create channel (
        owner = user@{.pubkey == user_pubkey},
        name = channel_name
    );
}

You can go through this line by line. First we declare the operation name and a list of parameters:

operation register_channel (user_pubkey: pubkey, channel_name: name) {

This is very similar to a function definitions in other languages. In fact, an operation is a function of a special kind: you can invoke it by using a blockchain transaction by its name. When invoking register_channel, the caller must provide two arguments of specified types, otherwise it fails.

require( op_context.is_signer(user_pubkey) );

We don't want Alice to be able to pull a prank on Bob by registering a channel with a silly name on his behalf. Thus we need to enusre that the transaction gets signed with a key corresponding to the public key specified in the first parameter. (In other words, if Bob's public key is user_pubkey, the transaction must also gets signed by Bob, that's, Bob is a signer of this transaction.) This is a common pattern in Rell -- typically you specify an actor in a parameter of an operation and in the body of the operation you verify that the actor was actually the signer. require fails the operation if the specified condition isn't met.

create channel, obviously, creates a persistent object channel. You don't need to explicitly store it, as all created objects persist if the operation succeeds.

user@{.pubkey=user_pubkey} -- now we retrieve a user object by its pubkey, which should be unique. If no such user exists the operation fails. We don't need to test for that explicitly as @ operator does this job.

Rell can automatically find attribute names corresponding to arguments using types. As user and name are different types, you can write create channel as follows:

create channel (user@{.pubkey=user_pubkey}, channel_name);

Function

Sometimes multiple operations (or queries) need the same piece capability, for example some kind of a validation code, or code which retrieves objects in a particular way. In order to not repeat yourself you can use function. Functions work similarly to operations: they get some input and can perform validations and work with data. Additionally, they also have a return type which you can specify after the list of parameters. For example, if you want to allow the user of a channel to change the name of the channel itself:

// We added mutable specifier to channel's attribute "name" to make name editable.
// Note that in case both an attribute and a key need to be declared.

entity channel {
    mutable name;
    key name;
    index owner: user;
}

function get_channel_owned_by_user(user_pub: pubkey, channel_name: name): channel {
    val user = user@{.pubkey == user_pub};
    return channel@{channel_name, .owner == user};
}

operation change_channel_name(signer: pubkey, old_channel_name: name, new_channel_name: name) {
    require(op_context.is_signer(signer));
    val channel_to_change = get_channel_owned_by_user(signer, old_channel_name);
    update channel@{channel == channel_to_change}(.name = new_channel_name);
}

In the function get_channel_owned_by_user the code first retrieves a user with given public key and returns a channel owned by the retrieved user with the given channel name. Operator @ expects to find exactly one object (see Cardinality for more information), thus you can be sure that in case there is no user or channel with such a pubkey or name the function fails and so does the operation that's calling it. Finally, the function returns the pre-validated channel instance, thus saving the developer the hassle to verify owner every time a channel gets retrieved.

Please note that you must mark the attribute name with the keyword mutable. This is because you can only change the fields that are mutable by using the update statement.

Query

Storing data without the ability to access it again would be useless. Here's a simple example - retrieving channel names for a user with a certain name:

query get_channel_names (user_name: name) {
    return channel @* {
        .owner == user@{.name==user_name}
    } (.name);
}

Here you see a selection operator you're already familiar with --@*. We select all the channels with a given owner (which we first find by name).

Then we extract name attribute from retrieved objects using the (.name) construct.

Note that since we only need name from channel, is also possible to write

query get_channel_names (user_name: name) {
    return channel @* {
        .owner == user@{.name==user_name}
    }.name;
}

Relational expressions

In general, a relational expression consists of five parts, some of which can get omitted:

| FROM OPERATOR { WHERE } (WHAT) LIMIT

1. FROM describes the data provider. It can be a single entity, such as just user. Or, it can be combination of multiple entities, (user, channel). In the latter case, conceptually you're dealing with a Cartesian product, which is a set of all possible combinations. But, typically WHERE part then provides a condition which defines a correspondence between objects of different entities. For example, one can select such (user, channel) combinations where user is an owner of the channel. This works same way as JOIN in SQL, in fact, the optimizer typically translates it to JOINs.
2. OPERATOR -- there are different operators depending on required cardinality. They're:
   • @ -- exactly one, returns a value
   • @* -- any number, returns a list of values
   • @+ -- at least one, returns a list of values
   • @? -- one or zero, returns a nullable value
3. WHERE describes how to filter the FROM set. So, you would use your search criteria as well as JOINs.
4. WHAT describes how to process the set, for doing a projection, aggregation or sorting. If it's omitted then members of the set are returned as they're.
5. LIMIT for operators which return a list, limits the number of elements returned.

In SQL, the logical processing order doesn't match the order in which you write the clauses. For example, FROM is logically processed before SELECT even though SELECT comes first. (SQL logical processing order is in SQL Server documentation).

The order of components of a relational expression in Rell matches the logical processing order. So, first you define a set, then it's filtered, and then it's post-processed. Of course, the query planner can perform operations in a different order, but that shouldn't affect the results. A relational expression is kind of a pipeline.

Here are some examples of relational expressions. Suppose in addition to user and channel entities we provided before, we also have:

entity message {
    index channel;
    index timestamp;
    text;
}

We can retrieve all messages of a given user:

(channel, message) @* {
    channel.owner == given_user, message.channel == channel
}(message.text);

So, basically, we join channel with message. We can shorten the expression using entity aliases:

(c: channel, m: message) @* { c.owner == given_user, m.channel == c } (m.text, m.timestamp)

We can easily read this expression left to right:

• consider all pairs (c, m) where c is channel and m is message
• find those where c.owner equals given_user and m.channel equals c
• extract text and timestamp from m

The result of this expression is a list of tuples with text and timestamp attributes.

The preceding expression can be easily modified to retrieve the latest 25 messages:

(c: channel, m: message) @* {
    c.owner == given_user, m.channel == c
} (m.text, @sort_desc m.timestamp) limit 25

Here we sorted results by timestamp in a descending order using @sort_desc and limited the number of returned rows.

Composite indices

We can also select recent messages by adding, for example, m.timestamp >= given_timestamp condition to WHERE part. But a database can't filter messages efficiently (that's, without considering every message) using two criteria at once unless we create a composite index, changing the message entity definition in the following way:

entity message {
    index channel, timestamp;
    text;
}

Instead two separate indexes we got one composite index. The idea here is that we want to retrieve not the latest messages overall, but the latest messages for a given channel. Thus, we need to order messages by channels first. You can do paged retrieval by using the following query:

query get_next_messages (user_name: name, upto_timestamp: timestamp) {
   val given_user = user@{user_name};
   return (c: channel, m: message) @* {
       c.owner == given_user, m.channel == c, m.timestamp < upto_timestamp
   } (m.text, -sort m.timestamp) limit 25;
}

You can use this in an app like Twitter. A visitor might first retrieve the latest 25 messages, then go further - in which case the client sends a query with a timestamp of the oldest message retrieved.

To understand why this can work efficiently, consider that the index stores an ordered collection of pairs. For example:

1. (channel_1, 1000000) -> m1
2. (channel_1, 1000050) -> m3
3. (channel_1, 1000100) -> m5
4. (channel_2, 1000025) -> m2
5. (channel_2, 1000075) -> m4

A database can efficiently find a place which corresponds to a given timestamp in a given channel and traverse the index through it.

note

It's worth noting that all SQL databases work this way, this feature is not unique to Rell. But in a decentralized system resources are typically precious, thus it's important for Rell programmers to understand the query behavior and use indices efficiently.