Core concepts
Language overview
Rell is a language for relational blockchain programming. It combines the following features:
- Relational data modeling and queries similar to SQL. People familiar with SQL should feel at home once they learn the new syntax.
- Normal programming constructs: variables, loops, functions, collections, etc.
- Constructs which specifically target app backends and, in particular, blockchain-style programming including request routing, authorization, etc.
Rell aims to make programming as ergonomic as possible. It minimizes boilerplate and repetition. At the same time, as a static type system it can detect and prevent many kinds of defects.
Blockchain programming
There are many different styles of blockchain programming. In the context of Rell, we see blockchain as a method for secure synchronization of databases on nodes of the system. Thus Rell is very database-centric.
Programming in Rell is pretty much identical to programming app backends: you need to handle requests to modify the data in the database and other requests which retrieve data from a database. Handling these two types of requests is basically all that a backend does. But, of course, before you implement request handlers, you need to describe your data model first.
Entity definitions
In SQL, usually you define your data model using CREATE TABLE
syntax.
In Java, you can define data objects using class
definition. In Rell,
we define them as entity
.
Rell uses persistent objects, thus an entity definition automatically creates the storage (a table) necessary to persist objects of a entity. As you might expect, Rell's entity definition includes a list of attributes:
entity user {
pubkey: pubkey;
name: text;
}
It's very common that the name of the attribute is the same as its
type. For example, it makes sense to call user's pubkey "pubkey."
Rell allows you to shorten pubkey: pubkey;
to just pubkey;
. Rell
also has a number of convenient semantic types, so there is a type
called name
as well. Thus you can rewrite the definition as
just:
entity user { pubkey; name; }
Typically a system shouldn't allow different users to have the same
name. That's, names should be unique. If name is unique, it can identify a user. In Rell, you can do this by defining a key,
key name;
. Note that it's not necessary to define both key and
attribute. Rell is smart enough to figure out that if you use an
attribute in a key, that attribute should exist in a entity.
It also might be useful to find a user by his pubkey. Should it also be
unique? Not necessarily. A user might have several different identities.
When you want to enable fast retrieval, but don't need uniqueness, you
can use index
definition:
entity user {
key name;
index pubkey;
}
However, if you want pubkey to be unique for a user, you can add a second key:
entity user {
key name;
key pubkey;
}
Typically, when you define a class in a programming language, it creates
a type which you can use to refer to instances of that class. This is
exactly how it works in Rell. The definition of entity user
creates a
type user
which is a type of references to objects stored in a
database. You can use references as attributes. For example,
you might want to define something owned by a user, say, a channel. You
can describe it like this:
entity channel {
index owner: user;
key name;
}
index
makes it possible to efficiently find all channels owned by a
user. key
makes sure that channel names are unique within the system.
You might analyze channel
entity definition from a point of view of a
traditional relational database terminology. A single user can get
associated with multiple channel
objects, but a single channel
is
always related to a single user
. Thus this represents one-to-many
relationship. owner
attribute of a channel refers to user
object and
thus constitutes a foreign key.
If channel names should be unique only in context of a single user (e.g.
alice/news
and bob/news
are different channels), then you nee to use a composite
key:
entity channel {
key owner: user, name;
}
This basically means that a pair of (owner, name)
should be unique.
Finally, one might ask: what changes if we change index owner: user
to
key owner: user
? This makes a user reference unique per channel
table, thus there can be at most one channel per user in that case.
(If owner
is the key, the the relationship between users and
channels becomes a one-to-one relationship.)
Operations
Now that we defined the data model, we can finally get to handling requests. As previously mentioned, Rell works with two types of requests:
- Data-modifying requests. We call them
operations
which get applied to the database state. - Data-retrieving requests. We call them
queries
.
But for both types of requests you are going to need to refer to things in the database, so you must consider relational operators first.
Creating objects
First, have a look at how we create objects:
create user (pubkey=x"0373599a61cc6b3bc02a78c34313e1737ae9cfd56b9bb24360b437d469efdf3b15",
name="Alice");
This is essentially the same as INSERT
operation in SQL, but the
syntax is a bit different. Rell is smart enough to identify the
connection between arguments and attributes based on their type.
x"..."
notation is a hexadecimal byte_array
literal which is
compatible with pubkey
type. On the other hand, name gets provided via
text
literal. Thus we can write:
create user("Alice", x"0373599a61cc6b3bc02a78c34313e1737ae9cfd56b9bb24360b437d469efdf3b15");
The order of arguments doesn't matter here, they're matched with attributes based on types.
Finding objects
How do we find that object now?
val alice = user @ {.name=="Alice"};
- The
@
operator retrieves a single record (or an object in this case) satisfying the search criteria you provided. If there is no such record, or more than one exists, it raises an error. It's recommended to use this construct when an operation needs a single record to operate on. If this requirement gets violated, then the operation gets aborted and all its effects get rolled back. Thus it's a succinct and effective way to deal with requirements. val
defines a read-only variable that you can later use in an expression. A variable defined usingvar
can get reassigned later.
If you want to retrieve a list of users, you can use the @*
operator.
For example:
val all_users = user @* {};
This returns a list of all users (since no filter expression was
available, all users match it). Value declarations can include a type,
for example, we can specify that all_users
is of type list<user>
like this:
val all_users: list<user> = user @* {};
Since the Rell compiler knows a type of every expression it doesn't really need a type declaration, however, if one is available, it checks against it. Type declarations are mostly useful as documentation for programmers reading the code and they msut not get used in cases where there is no ambiguity.
Both @
and @*
correspond to SELECT
in SQL.
Operation example
Make an operation which allows a user to create a new channel:
operation register_channel (user_pubkey: pubkey, channel_name: name) {
require( op_context.is_signer(user_pubkey) );
create channel (
owner = user@{.pubkey == user_pubkey},
name = channel_name
);
}
You can go through this line by line. First we declare the operation name and a list of parameters:
operation register_channel (user_pubkey: pubkey, channel_name: name) {
This is very similar to a function definitions in other languages. In
fact, an operation is a function of a special kind: you can invoke it by
using a blockchain transaction by its name. When invoking
register_channel
, the caller must provide two arguments of specified
types, otherwise it fails.
require( op_context.is_signer(user_pubkey) );
We don't want Alice to be able to pull a prank on Bob by registering a
channel with a silly name on his behalf. Thus we need to enusre that
the transaction gets signed with a key corresponding to the public key
specified in the first parameter. (In other words, if Bob's public key
is user_pubkey
, the transaction must also gets signed by Bob,
that's, Bob is a signer of this transaction.) This is a common pattern
in Rell -- typically you specify an actor in a parameter of an
operation and in the body of the operation you verify that the actor was
actually the signer. require
fails the operation if the specified
condition isn't met.
create channel
, obviously, creates a persistent object channel
. You
don't need to explicitly store it, as all created objects persist
if the operation succeeds.
user@{.pubkey=user_pubkey}
-- now we retrieve a user object by its
pubkey, which should be unique. If no such user exists the operation
fails. We don't need to test for that explicitly as @
operator
does this job.
Rell can automatically find attribute names corresponding to arguments
using types. As user
and name
are different types, you can write create channel
as follows:
create channel (user@{.pubkey=user_pubkey}, channel_name);
Function
Sometimes multiple operations (or queries) need the same piece
capability, for example some kind of a validation code, or code which
retrieves objects in a particular way. In order to not repeat yourself
you can use function
. Functions work similarly to operations: they get
some input and can perform validations and work with data. Additionally,
they also have a return type which you can specify after the list of
parameters. For example, if you want to allow the user of a channel to
change the name of the channel itself:
// We added mutable specifier to channel's attribute "name" to make name editable.
// Note that in case both an attribute and a key need to be declared.
entity channel {
mutable name;
key name;
index owner: user;
}
function get_channel_owned_by_user(user_pub: pubkey, channel_name: name): channel {
val user = user@{.pubkey == user_pub};
return channel@{channel_name, .owner == user};
}
operation change_channel_name(signer: pubkey, old_channel_name: name, new_channel_name: name) {
require(op_context.is_signer(signer));
val channel_to_change = get_channel_owned_by_user(signer, old_channel_name);
update channel@{channel == channel_to_change}(.name = new_channel_name);
}
In the function get_channel_owned_by_user
the code first retrieves a
user with given public key and returns a channel owned by the retrieved
user with the given channel name. Operator @
expects to find exactly one
object (see
Cardinality
for more information), thus you can be sure that in case
there is no user or channel with such a pubkey or name the function fails and so does the operation that's calling it. Finally, the function
returns the pre-validated channel instance, thus saving the developer
the hassle to verify owner every time a channel gets retrieved.
Please note that you must mark the attribute name
with the keyword
mutable
. This is because you can only change the fields that are mutable by using the update statement.
Query
Storing data without the ability to access it again would be useless. Here's a simple example - retrieving channel names for a user with a certain name:
query get_channel_names (user_name: name) {
return channel @* {
.owner == user@{.name==user_name}
} (.name);
}
Here you see a selection operator you're already familiar with --@*
.
We select all the channels with a given owner (which we first find by
name).
Then we extract name attribute from retrieved objects using the
(.name)
construct.
Note that since we only need name
from channel, is also possible to
write
query get_channel_names (user_name: name) {
return channel @* {
.owner == user@{.name==user_name}
}.name;
}
Relational expressions
In general, a relational expression consists of five parts, some of which can get omitted:
| FROM OPERATOR { WHERE } (WHAT) LIMIT
- FROM describes the data provider. It can be a single
entity, such as just
user
. Or, it can be combination of multiple entities,(user, channel)
. In the latter case, conceptually you're dealing with a Cartesian product, which is a set of all possible combinations. But, typically WHERE part then provides a condition which defines a correspondence between objects of different entities. For example, one can select such(user, channel)
combinations whereuser
is an owner of thechannel
. This works same way asJOIN
in SQL, in fact, the optimizer typically translates it to JOINs. - OPERATOR -- there are different operators depending on required
cardinality. They're:
@
-- exactly one, returns a value@*
-- any number, returns a list of values@+
-- at least one, returns a list of values@?
-- one or zero, returns a nullable value
- WHERE describes how to filter the FROM set. So, you would use your search criteria as well as JOINs.
- WHAT describes how to process the set, for doing a projection, aggregation or sorting. If it's omitted then members of the set are returned as they're.
- LIMIT for operators which return a list, limits the number of elements returned.
In SQL, the logical processing order doesn't match the order in which you write the clauses. For example, FROM
is logically processed before
SELECT
even though SELECT
comes first. (SQL logical processing order
is in SQL Server
documentation).
The order of components of a relational expression in Rell matches the logical processing order. So, first you define a set, then it's filtered, and then it's post-processed. Of course, the query planner can perform operations in a different order, but that shouldn't affect the results. A relational expression is kind of a pipeline.
Here are some examples of relational expressions. Suppose in addition
to user
and channel
entities we provided before, we also have:
entity message {
index channel;
index timestamp;
text;
}
We can retrieve all messages of a given user:
(channel, message) @* {
channel.owner == given_user, message.channel == channel
}(message.text);
So, basically, we join channel
with message
. We can shorten the
expression using entity aliases:
(c: channel, m: message) @* { c.owner == given_user, m.channel == c } (m.text, m.timestamp)
We can easily read this expression left to right:
- consider all pairs
(c, m)
wherec
ischannel
andm
ismessage
- find those where
c.owner
equalsgiven_user
andm.channel
equalsc
- extract
text
andtimestamp
fromm
The result of this expression is a list of tuples with text
and
timestamp
attributes.
The preceding expression can be easily modified to retrieve the latest 25 messages:
(c: channel, m: message) @* {
c.owner == given_user, m.channel == c
} (m.text, @sort_desc m.timestamp) limit 25
Here we sorted results by timestamp in a descending order using
@sort_desc
and limited the number of returned rows.
Composite indices
We can also select recent messages by adding, for example,
m.timestamp >= given_timestamp
condition to WHERE part. But a
database can't filter messages efficiently (that's, without
considering every message) using two criteria at once unless we create a
composite index, changing the message
entity definition in the
following way:
entity message {
index channel, timestamp;
text;
}
Instead two separate indexes we got one composite index. The idea here is that we want to retrieve not the latest messages overall, but the latest messages for a given channel. Thus, we need to order messages by channels first. You can do paged retrieval by using the following query:
query get_next_messages (user_name: name, upto_timestamp: timestamp) {
val given_user = user@{user_name};
return (c: channel, m: message) @* {
c.owner == given_user, m.channel == c, m.timestamp < upto_timestamp
} (m.text, -sort m.timestamp) limit 25;
}
You can use this in an app like Twitter. A visitor might first retrieve the latest 25 messages, then go further - in which case the client sends a query with a timestamp of the oldest message retrieved.
To understand why this can work efficiently, consider that the index stores an ordered collection of pairs. For example:
1. (channel_1, 1000000) -> m1
2. (channel_1, 1000050) -> m3
3. (channel_1, 1000100) -> m5
4. (channel_2, 1000025) -> m2
5. (channel_2, 1000075) -> m4
A database can efficiently find a place which corresponds to a given timestamp in a given channel and traverse the index through it.
It's worth noting that all SQL databases work this way, this feature is not unique to Rell. But in a decentralized system resources are typically precious, thus it's important for Rell programmers to understand the query behavior and use indices efficiently.