How Clouden uses AWS Cloud Directory

We spend a lot of effort in choosing the right technologies for our cloud services at Clouden. Over the years we have found Amazon Web Services to be a reliable provider of cloud platform services that scale in both performance and cost. Our strategy is to build web applications that have very low fixed costs and grow as the user base grows – sometimes faster and sometimes slower.

In the past most cloud services were based on hourly usage prices. To keep the service running, you had to pay for each hour it was turned on, regardless of actual usage. Luckily, in recent years AWS has been developing more and more cloud services that charge you for the actual amount of resources used. Examples of such services are Amazon S3, AWS IoT, Lambda, API Gateway, Cognito User Pools and Cloud Directory.

In this case study we take a look at Clouden Ping, one of Clouden’s newest web services. Clouden Ping is a website monitoring service that passes on Amazon’s pay-as-you-go pricing model to the end-customer. Instead of paying for a monthly subscription, you get charged for the number of pings you actually make to your websites. The payment model is based on prepaid credits.

Technologies chosen for Clouden Ping The diagram below illustrates the main technologies that Clouden Ping is based on.

Cognito User Pool manages the user logins and registrations and handles everything else related to them, such as resetting forgotten passwords. Web applications can talk directly to Amazon’s Cognito API, which means there is very little custom code needed.

Cloud Directory provides the database for Clouden accounts. Users can have multiple accounts and accounts can have multiple users, which allows sharing of ping hosts, results and notifications.

Aurora (SQL Server) maintains credit account balances for accounts. Credits are deposited when making Stripe payments and consumed when hosts are pinged.

DynamoDB provides the database for storing ping hosts and results of ping operations. Some older results are also archived into S3 although it’s not shown in the picture.

Lambda performs scheduled ping operations, stores the results in DynamoDB and applies charges to the Aurora credit balances.

How we use Cloud Directory AWS Cloud Directory is based on a fairly simple graph database model and an API that resembles other AWS native APIs such as DynamoDB and S3. You can use the AWS Console to perform some Cloud Directory tasks (like creating new directories) in the web browser, but you need to use AWS SDK or the AWS CLI for mosts operations.

The diagram below shows how we have structured our Users and Accounts in Cloud Directory.

We use four types of Cloud Directory objects, described below.

Containers are used to group other objects under known top-level paths. There is one container at /users which contains User objects, and another container at /accounts which contains Account objects. This makes it possible to locate any user in the directory by their userId at /users/{userId} and any account by their accountId at /accounts/{accountId}, and also enables easy enumeration of all users and accounts.

Users are objects that correspond to Cognito User Pool users. We basically only need to store their userId attributes, which are UUIDs generated by Cognito. Users are added to Cloud Directory when their default account is created by a Cognito event trigger.

Accounts are objects that can be created and deleted by users. Each account has an accountId and a name. Every user has at least one default account.

Memberships are typed links that are used to attach users to accounts. Each membership link has a role attribute, which allows different kind of memberhips. The default role is “owner” which is granted to the user that creates a new account. The other allowed role is “member”. These roles are string values defined by the application logic and more roles can easily be added without modifying the Cloud Directory schema.

Some Cloud Directory takeaways for developers AWS Cloud Directory is a relatively new product and might not yet be well known to all developers. Here are a couple of usage hints based on our experiences.

Use batch operations to perform consistent transactions. Cloud Directory has many use cases where you need to apply several changes to the database in a consistent manner. Wrapping them into a BatchWrite operation will ensure that they all either succeed or fail together. For instance, when you remove an object, you should detach it and delete it within the same batch operation. Avoid unnecessary path traversals when querying objects in Cloud Directory, because traversal steps are counted as read operations and charged separately. When possible, you should instead refer to objects directly using their ObjectIds. For instance, we could optimize user lookups by storing ObjectIds in the Cognito User Pool instead of looking up users by their /users/{userId} paths. You can add new attributes to applied schemas using in-place schema upgrades. This works as long as the changes you make are backwards compatible with the previous minor version of the schema. At the time of writing you can’t see the minor versions in AWS Console, but you can manage them using AWS CLI. If you became interested in what Clouden can offer, check out our home page to see all our services.