CSL Application-Level Messaging

In this document we explore messaging in Cardano SL. The goal of this document is to explain how all the pieces, such as Time-Warp, Network-Transport, and Kademlia DHT, click together and make implementing a full CSL node possible.

Message Typeclass and Message Types

When reading the source code, you often encounter things like

-- | 'GetHeaders' message (see protocol specification).
data MsgGetHeaders = MsgGetHeaders
    { -- not guaranteed to be in any particular order
      mghFrom :: ![HeaderHash]
    , mghTo   :: !(Maybe HeaderHash)
    } deriving (Generic, Show, Eq)

instance Message MsgGetHeaders where
    messageName _ = varIntMName 4
    formatMessage _ = "GetHeaders"

How do you read this? First, let’s examine the instance part. This particular snippet says that the data structure defined by type MsgGetHeaders is used as a message payload. The name of such message is "GetHeaders".

In this particular case, the data structure has two fields: mghFrom and mghTo. Prefixes like mgh are used because Haskell puts symbols for record fields in the global namespace, so it’s programmer’s duty to avoid clashes.

It should be noted that sometimes you see messages that are parametrized with a ssc type variable. That is done for the code to be polymorphic with respect to the way we carry out shared seed computation. Here is an example of a message that sends newest headers first, minding ssc.

The way messages are serialized can be seen in Pos.Binary.Communication module.

Every message type should have an instance of the Message typeclass. Please see Time-Warp-NT guide.

Inv/Req/Data and MessagePart

Most of messages in Cardano SL are generalized with Inv/Req/Data standard (see Pos.Communication.Relay module). Within this framework we define three data types:


-- | Inventory message. Can be used to announce the fact that you have
-- some data.
data InvMsg key tag = InvMsg
    { imTag  :: !tag
    , imKeys :: !(NonEmpty key)
    }

-- | Request message. Can be used to request data (ideally data which
-- was previously announced by inventory message).
data ReqMsg key tag = ReqMsg
    { rmTag  :: !tag
    , rmKeys :: !(NonEmpty key)
    }

-- | Data message. Can be used to send actual data.
data DataMsg contents = DataMsg
    { dmContents :: !contents
    } deriving (Show, Eq)

Here:

  • key is a type representing the node identifier.

  • tag is a type used to describe announced/requested data. It should contain enough information for other nodes to check if they need these data and request them.

  • contents is a type representing actual message payload.

To introduce a new message using Inv/Req/Data one should create two types: tag type and contents type for this message, and then implement MessagePart typeclass for both of them.

class MessagePart a where
    pMessageName :: Proxy a -> MessageName

Here, pMessageName is an identifier for a particular message type (e.g. the type should be the same for tag and contents, but it should differ between messages).

Message typeclass for InvMsg key tag, ReqMsg key tag and DataMsg contents is automatically derived from the MessagePart typeclass for particular tag and contents.

Please see Pos.Communication.Message module for the examples of messages that are using Inv/Req/Data.

Block Exchange Messages

This table explains Pos.Block.Network.Types module.

Message type Payload Commentaries
GetHeaders Header hash checkpoints; Optional newest hash we’re interested in Expect newest header first
GetBlocks Oldest header hash; Newest hash Both hashes have to be present
BlockHeaders Non-empty collection of block headers, newest first Polymorphic in ssc
Block A single block Polymorphic in ssc

For more details see binary protocols.

Message names

All messages are given custom names, since using full type names would be excessive. Each name is concatenation of one or two encoded UnsignedVarInts.

This table contains names for all used messages / message parts; they could also be found in Pos.Communication.Message module. To distinguish from integers addition, concatenation is denoted here as (++).

Message type Name
NOP 0
SendProxySK 2
ConfirmProxySK 3
MsgGetHeaders 4
MsgHeaders 5
MsgGetBlocks 6
MsgBlock 7
InvMsg 8 ++ pMessageName tag
ReqMsg 9 ++ pMessageName tag
DataMsg 10 ++ pMessageName contents
SysStartRequest 1001
SysStartResponse 1002
Message part type Name
TxMsgTag 0
TxMsgContents 0
ProposalMsgTag 1
(UpdateProposal, [UpdateVote]) 1
VoteMsgTag 2
UpdateVote 2
GtTag 3
GtMsgContents 3

Communication Messages

This table is about data structures that are included in communication maintenance messages. It covers the following modules:

Data Type* or Message type Payload Commentaries
SysStartRequest Ø Polls peer’s timestamps. To be answered with SysStartResponse
SysStartResponse Peer’s timestamp Answer to SysStartRequest

Delegation Messages

Delegation is a feature that allows one stakeholder, called issuer, to let another stakeholder, called delegate, generate blocks on her behalf.

To do this, issuer should create proxy signing key that allows delegate to sign blocks instead of issuer. Any stakeholder can verify that a proxy signing key was actually issued by a specific stakeholder to a specific delegate and that this key is valid at time.

Delegation can be of two types: per-epoch delegation and delegation with revocable long-lived certificates. Per-epoch delegation is called “lightweight”, and the long-lived delegation is called “heavyweight”.

Lightweight Delegation

Lightweight delegation allows delegate to sign blocks instead of issuer for some epochs (range [low, high] of epochs is specified for a signing key created).

To do this, issuer should send message containing time range, issuer public key, delegate public key and certificate over network. Every node from network receives this message and can check later if the one who generated the block had right for it. Lightweight delegation data is stored in memory and gets deleted after some time (500 seconds).

This delegation type can be used to delegate blocks generating right to some trusted node when issuer knows she will be absent in some time range.

Heavyweight Delegation

Heavyweight delegation serves two purposes:

  1. Delegate block generation right, like lightweight delegation.
  2. Share stake with some delegate, thus allowing delegate to take part in Follow-The-Satoshi. No real money is transferred; stake of issuer is added to stake of delegate when calculating stakeholders for Follow-The-Satoshi.

Every particular stakeholder can share stake with one and only one delegate. To revoke certificate a node should create a new certificate and put itself as both issuer and delegate.

Messages table

This table describes delegation-related messages, found in Pos.Delegation.Types module. The format of delegation messages is described in Binary protocols section.

Message type Commentaries
SendProxySK Message with proxy delegation certificate
ConfirmProxySK Used to confirm proxy signature delivery

Update System Messages

You can see how system messages are implemented under WorkMode here.

Message type Commentaries
UpdateProposal Serialized update proposal, sent to a DHT peers in Pos.Communication.Methods
VoteMsgTag A tag for vote message. Works with UpdateVote to tag the payload
UpdateVote Message, payload of which contains the actual vote

Workers, Listeners and Handlers in CSL

You can think about it as «operating personnel» for messages.

Workers initiate messages exchange, so a worker is an active communication part of Cardano SL. Listeners accept messages from the workers and may send some messages as answers, so a listener is a passive communication part of Cardano SL. After a message was received, a listener uses the function called handler to actually perform the corresponding job. A particular handler is used based on the type of received message (as it has been said above, messages have different types).

To be able to perform necessary actions, all workers and handlers work in the WorkMode’s constraints (see below).

Block Processing

Block exchange messages are described above.

Block Processing Workers

Block acquisition is handled in Pos.Block.Network.Retrieval module.

Function retrievalWorker is very important: it’s a server that operates on block retrieval queue validating headers, and these blocks form a proper chain. Thus, at this point it sends a message of type MsgGetBlocks to the listener, and at this point it receives an answer from this listener, a message of MsgBlock type.

Here’s another example — the requestHeaders function. This function handles expected block headers, tracking them locally. So at this point it sends a message of type MsgGetHeaders to the listener, and at this point it receives an answer from that listener, a message of MsgHeaders type.

Additional worker for the block processing is defined in Pos.Block.Worker module. We reuse retrievalWorker described above and define a well-documented blkOnNewSlot worker. It represents an action which should be done when a new slot starts. This action includes the following steps:

  1. Generating a genesis block, if necessary;
  2. Getting leaders for the current epoch;
  3. Initiation block generation, if we’re the slot leader or we’re delegated to do so (optional).

Logic

The way in which blocks are processed is specified in the Pos.Block.Logic module. Please read about blocks in Cardano SL for more info.

Block Processing Listeners

Listeners for the block processing are defined in Pos.Block.Network.Listeners module.

Handler handleGetHeaders sends out the block headers: at this point it receives a message of type MsgGetHeaders from the worker, get the headers and then, at this point, it sends a response message of type MsgHeaders to that worker.

A handler handleGetBlocks sends out blocks. This handler corresponds to retrieveBlocks from main retrievalWorker. Thus, it receives a message of type MsgGetBlocks from the worker here, gets corresponding headers, and then it sends response message of type MsgBlock to that worker here.

A handler handleBlockHeaders sends out block headers for unsolicited use case in a similar way: it receives a message of MsgHeaders type from the worker and handles it.

All these handlers work in the ListenerActionConversation mode because they send replies to corresponding workers (so we have a conversation between workers and listeners).

Delegation

Another example is working with delegation messages described above.

Workers

Workers for delegation messages are defined in Pos.Delegation.Methods module. There are 3 workers: sendProxySKEpoch, sendProxySKSimple and sendProxyConfirmSK.

All these workers do not send messages to one particular node. They send messages to all neighbors. Thus, sendProxyConfirmSK worker sends a message of type ConfirmProxySK to all neighbors.

Listeners

Listeners for delegation messages are defined in Pos.Delegation.Listeners module.

Handler handleSendProxySK handles SendProxySK*-messages. This handler works in another mode, called ListenerActionOneMsg, which means there’s no conversation with the worker — we receive a single incoming message. In this case handler receives a message of type SendProxySK from the worker, but doesn’t reply to it. Instead, it sends a message of SendProxySKSimple type to its neighbors.

Handler handleConfirmProxySK works in the same way. It receives a message of type ConfirmProxySK from the worker and sends a message of ConfirmProxySK type to its neighbors.

Handler handleCheckProxySKConfirmed works in ListenerActionOneMsg mode too, but after it receives a message of CheckProxySKConfirmed type, it sends a message of CheckProxySKConfirmedRes type as a reply to the worker.

Security

Workers for security operations are defined in Pos.Security.Workers module. Let’s have a look at checkForReceivedBlocksWorker: in this case, again, we send a message to all neighbors using converseToNode function. This function tries to establish connection with the listeners to start a conversation with them.

Update System

Below is the list of workers and listeners related to update system.

Workers

Workers for update system are defined here. The only thing that the update system does is checking for a new approved update on each slot.

Listeners

Listeners for update system are defined here.

UpdateProposal handlers:

  • Req — local node answers to a request about update proposal with the set of votes for/against this proposal.
  • Inv — checks if we need the offered proposal, and records the data if this inventory message is relevant.
  • Data — carries the proposal information along with votes, which is verified and recorded.

UpdateVote listeners:

  • Req — sends our vote to whoever requests it.
  • Inv — checks if we need the offered vote, and records it if relevant.
  • Data — carries a single vote, which is verified and recorded.

WorkMode and WorkModeMin

A special type called WorkMode represents a bunch of constraints to perform work for the real world distributed system. You can think about constraint as a compile-time guarantee that particular actions can be performed in the particular context. For example, if we define type of some function f in the terms of logging constraint, we definitely know that we can log different info inside of this function f.

All workers and handlers described above work in the WorkMode’s constraints. These constraints guarantee the following abilities:

There’s a minimum version of WorkMode called MinWorkMode, for more specific functions like this one. MinWorkMode includes these absolutely necessary constraints only: