December 9, 2021

Evmoscan and ETL

Given our interest in alternative assets, Alt has started experimenting with blockchain technologies. We've started our experiments on Evmos, an EVM compatible-chain built on the Cosmos SDK. We'll be releasing a series of blog posts on the tooling and lessons we learn from this journey.

At Alt, we decided to build a tool to see recent gas usage across previous Evmos blocks. For people unfamiliar with blockchain transactions, gas fees are the fees used to pay miners/validators on a network to have your transaction included in a block. As transactions and congestion increase on a network, this will increase the gas fees, so we thought it would be useful to build a tool to see the previous history.

When we look at other networks like Ethereum, Etherscan shows you the recent gas history so we can estimate how much gas we're paying relative to the average.

Difference from traditional CRUD development

What makes building this app different from traditional apps is that the data is online on the blockchain. This means that most of our endpoints are just wrappers around RPC calls to a node.

Sample endpoint for getting block data.


@app.route("/block/:block_number>")
def block(block_number):
    block_ = block_loader(json.loads(get_block_info_rpc(block_number)))
    return render_template("block.html", block=block_)

@backoff.on_exception(backoff.constant, Exception, interval=0, max_tries=2)
def get_block_info_rpc(block_number: int, rpc: str = RPC_ENDPOINT, cur=None) -> str:
    res = requests.post(rpc, data=f'{{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":[{block_number}, false],"id":1}}', headers={'Content-Type': 'application/json'})
    res.raise_for_status()
    return res.text
    


Validator Data

While building this tool, we wanted to get validator data.

However, that is not available in the gRPC endpoints. This data can be fetched from the API endpoint though (docs). We used a community API node to get the data, but if you wanted your node to return this data. In your .evmosd/app.toml, under the API Configuration section, set enable to true.


###############################################################################
###                           API Configuration                             ###
###############################################################################

[api]

# Enable defines if the API server should be enabled.
enable = true
    


ETL

Given that the data is available online via RPC calls, this makes traditional data analysis hard. The data is however structured so it makes a good candidate to store in a SQL database. For this, we wrote an ETL scraper to store it in a database layer so that people can run their traditional analysis on top of it.

The scraper allows a user to specify a block range or a catch-up mode in order to catch-up to the latest block. In addition, if a -1 is passed in for block_to, the scraper will fetch the latest block and set that as the target.

Credits

Finally, we would like to thank the Evmos community for helping us with a ton of questions regarding Emvos. Join their Discord and check out their project.