Observing Squad

The N+1 setup for connecting to the Elrond Network

In order to integrate with the Elrond Network and be able to broadcast transactions and query blockchain data in an optimized approach, one needs to setup an on-premises Observing Squad.

An Observing Squad is defined as a set of N Observer Nodes (one for each Shard, including the Metachain) plus an Elrond Proxy instance which will connect to these Observers and provide an HTTP API (by delegating requests to the Observers).

Currently the Elrond Mainnet has 3 Shards, plus the Metachain. Therefore, the Observing Squad is composed of 4 Observers and one Proxy instance.

By setting up an Observing Squad and querying the blockchain data through the Proxy, the particularities of Elrond's sharded architecture are abstracted away. This means that the client interacting with the Proxy does not have to be concerned about sharding at all.

System requirements

The Observing Squad can be installed on multiple machines or on a single, but more powerful machine.

In case of a single machine, our recommendation is as follows:

  • 8 x CPU

  • 32 GB RAM

  • HDD that can grow up to 5TB

  • 100 Mbit/s always-on Internet connection

  • Linux OS (Ubuntu 18.04 recommended)

Setup via the Mainnet scripts

Installation and Configuration

The Observing Squad can be setup using the installation scripts. Within the installation process, the LookupDatabaseExtension feature (required by the Hyperblock API) will be enabled by default.

Documentation within this section is preliminary, and subject to change.

Clone the installer repository:

git clone https://github.com/ElrondNetwork/elrond-go-scripts-mainnet.git

Checkout to the special branch exchanges-integration:

cd elrond-go-scripts-mainnet
git checkout exchanges-integration

Edit config/variables.cfg accordingly. For example:

CUSTOM_HOME="/home/ubuntu/ObservingSquad"
CUSTOM_USER="ubuntu"

Run the installation script:

./script.sh

And choose the option observers.

After installation, 5 new systemd units will be available (and enabled).

Start the nodes and the Proxy:

./script.sh

And choose option start.

Upgrading the Observing Squad

The Observing Squad can be updated using the installation scripts.

First, make sure you are on branch exchanges-integration of the elrond-go-scripts-mainnet.

cd ~/elrond-go-scripts-mainnet
git checkout exchanges-integration

Fetch the latest version of the installation scripts

./script.sh

Then choose option github_pull.

Stop the instances (Observers and Proxy)

./script.sh

Then choose option stop.

Upgrade the Observers

./script.sh

Then choose option upgrade.

Upgrade the Proxy

./script.sh

Then choose option upgrade_proxy.

Restart the instances (Observers and Proxy)

./script.sh

Then choose option start.

Monitoring and trivial checks

One can monitor the running Observers using the termui utility (installed during the setup process itself), as follows:

~/elrond-utils/termui --address localhost:8080 # Shard 0
~/elrond-utils/termui --address localhost:8081 # Shard 1
~/elrond-utils/termui --address localhost:8082 # Shard 2
~/elrond-utils/termui --address localhost:8083 # Metachain

Alternatively, one can query the status of the Observers by performing GET requests using curl:

curl http://localhost:8080/node/status | jq # Shard 0
curl http://localhost:8081/node/status | jq # Shard 1
curl http://localhost:8082/node/status | jq # Shard 2
curl http://localhost:8083/node/status | jq # Metachain

The Proxy does not offer a termui monitor, but its activity can be inspected using journalctl:

journalctl -f -u elrond-proxy.service

Optionally, one can perform the following smoke test in order to fetch the latest synchronized hyperblock:

export NONCE=$(curl http://localhost:8079/network/status/4294967295 | jq '.data["status"]["erd_highest_final_nonce"]')
curl http://localhost:8079/hyperblock/by-nonce/$NONCE | jq

Setup via Docker

Documentation in this section is preliminary and subject to update.