EventStoreDB Cluster with AWS
While EventstoreDB is just gaining its popularity, some have already called it one of the most useful tools for implementing event-driven architecture. However, there is just a pinch of information online about EventstoreDB.
And, considering that I don’t want to keep you in the dark, I’ve prepared a guide on setting up an EventStoreDB cluster with AWS.
Written by Oleg Koval
The EventStoreDB cluster functions without shared disks. How does it work then? It uses the shared-nothing philosophy according to its principle you have three different physical drivers (aka nodes). The main benefit of the shared-nothing approach is that, when one node crashes, the data remains safe because each of the three nodes retains identical pieces of data.
Now that you know the basics, we can set off our installation of the EC2 instance group.
This article uses Amazon free tier, so don’t hesitate to try all the steps on your own.
The First thing you want to do — is to open your AWS management console. Then go to the EC2 section and then click on Instances. After that, navigate to a beautiful orange Launch instances button.
Step 1
Proceed by choosing “Ubuntu Server 20.04 LTS” as your chosen AMI.
Step 2
Good job! Now select the free tier instance type.
(Instance with type t2.micro fits good for testing purpose and anyone will be able to follow along with our steps without spending money)
Step 3
We need to change the number of instances to 3 because EventStoreDB uses a quorum-based replication model. So, by changing the number of instances to 3 we’ll allow the model to tolerate 1 node failure.
Within the model, the majority of nodes in the cluster must acknowledge that they commit a write to the client after they write to disk.
Step 4
Make no changes on the next step to continue using a free plan.
Step 5
Skip this step as well. We won’t need tags in this guide. But I strongly recommend adding tags that are very useful and informative, especially for huge infrastructure.
Step 6
Create a new group titled “eventstore-example-sg”. It’ll allow us to access the instances through ssh (22), default EventStoreDB port/http (2113) and enable communication between nodes via port/http 1112.
I’ll leave the instances open for all IPs just to simplify the example. AWS and I personally recommend that you set your security group rules to allow access from known IP addresses only.
Step 7
Excited? You’d better be because we’ve approached the moment when we have to press the Launch button.
After a new window pops up, we create a new key pair and download the “.pem” file. Without this file, you won’t be able to connect to your EC2 instances via ssh.
Press Launch instances button to create your instances. Once they’re up and running, we can proceed with the configuration of the cluster.
Now you should prepare a list of private IPs. We’ll reference them in our config files as private-ip-node-1, private-ip-node-2, private-ip-node-3.
Click on the instance and open the Details tab to take a look at the instance’s private IP. To do so select one particular instance and check the private IPv4 addresses field.
Moving on! I hope you’re still with me, aren’t you? It’ll be over soon, I promise.
And now we should prepare a config file for each instance.
To make life easier for you, I’m adding a template for the config file (please, pay attention to spaces when copying, since it uses yaml format).
This template should be used for the 1st file, so keep in mind that it should be linked to 2nd and 3rd nodes via GossipSeed. The Same logic applies to the rest of the nodes.
This is a guide with simplified security configuration. To get more information about security configuration, address EventStoreDB documentation.
---
# Paths
Db: /var/lib/eventstore
Index: /var/lib/eventstore/index
Log: /var/log/eventstore
# Run in insecure mode
Insecure: true
# Network configuration
IntIp: 0.0.0.0
ExtIp: 0.0.0.0
HttpPort: 2113
IntTcpPort: 1112
EnableExternalTcp: false
EnableAtomPubOverHTTP: true
# Cluster gossip
ClusterSize: 3
DiscoverViaDns: false
GossipSeed: private-ip-node-2:2113,private-ip-node-3:2113
# Projections configuration
RunProjections: All# Timeouts and intervals
CommitTimeoutMs: 5000
GossipIntervalMs: 2000
GossipTimeoutMs: 3000
And if you don’t like the template, try EventStoreDB developers offer you to prepare configs using configuration wizard.
Congratulations, your configs are prepared and you can move on with configuration.
The Next thing you want to do is to simply connect to the instance and run the following commands (*repeat the action for each node):
Switch to the Connect page and follow all the necessary steps to establish ssh connection.
Once you are connected to the node, run the next commands:
- Install EventStoreDB package (currently the latest version is v21.2.0)
$ curl -s https://packagecloud.io/install/repositories/EventStore/EventStore-OSS/script.deb.sh | sudo bash
$ sudo apt-get install eventstore-oss=21.2.0 - Put eventstore config for particular instance in this file
$ sudo nano /etc/eventstore/eventstore.conf - Start instance
$ sudo systemctl start eventstore
Do you remember our instances are accessible?
You can check whether your cluster is ready for using http://public-ip-node-1:2113/web/index.html#/clusterstatus. (The public IP is indicated right near the private IP). As you can see all the nodes are alive, with one Leader node and two Follower nodes.
Conclusion
And just like that, you’ve integrated EventStoreDB Cluster with AWS.
Mind, however, that this isn’t a production-ready cluster, just a working example to start with.
Thanks for reading! Hope you enjoyed learning about EventStoreDB Cluster with AWS. Stay tuned for what’s new, follow us!
