Full microservice

In this lab we will create a full microservice, by using a database (Amazon DynamoDB), a computing service (AWS Lambda), and an API service to expose the service to users and other microservices.

Prerequisites

You need:

Context

In this lab we are going to create a simple microservice that computes statistics for the top 10 players of a game.

The architecture is shown in the diagram below

Task: Creating the DynamoTB table


Step 1. Create the table

  1. Visit the DynamoDB console
  2. Visit the Tables menu option, and then click on Create table
  3. At the Create DynamoDB table:
    1. For Table name*, input <prefix>GameTopX
    2. For Primary key*, input SessionId
  4. Leave everything else as it is
  5. Click on Create

Wait until the table is created. We will use this task to insert some data into this table.


Step 2. Inserting the data

  1. Be sure that you have the table GameTopX selected on the DynamoDB console
  2. Click on the Items tab
  3. Click on the button Create item. DynamoDB will open a window so you can input data on it
  4. On the top left of the window, click on the drop-down list that is indicating Tree, and select Text
  5. Copy-and-paste the content below, replacing what is in that window:
    {
      "SessionId": "TheTestSession",
      "TopX": [
        {
          "Level": 5,
          "Lives": 3,
          "Nickname": "alanis",
          "Score": 1500,
          "Shots": 372,
          "Timestamp": "2019-06-21T00:07:01.328Z"
        },
        {
          "Level": 2,
          "Lives": 0,
          "Nickname": "blanka",
          "Score": 300,
          "Shots": 89,
          "Timestamp": "2019-06-26T08:28:52.264Z"
        },
        {
          "Level": 1,
          "Lives": 0,
          "Nickname": "bugsbunny",
          "Score": 200,
          "Shots": 88,
          "Timestamp": "2019-06-26T08:49:19.049Z"
        },
        {
          "Level": 1,
          "Lives": 0,
          "Nickname": "billythekid",
          "Score": 195,
          "Shots": 46,
          "Timestamp": "2019-06-25T22:43:32.050Z"
        },
        {
          "Level": 1,
          "Lives": 0,
          "Nickname": "nobodyknows",
          "Score": 65,
          "Shots": 17,
          "Timestamp": "2019-06-26T08:33:47.951Z"
        },
        {
          "Level": 1,
          "Lives": 0,
          "Nickname": "mordorlord",
          "Score": 5,
          "Shots": 1,
          "Timestamp": "2019-06-26T08:29:29.639Z"
        },
        {
          "Level": 1,
          "Lives": 3,
          "Nickname": "naruto",
          "Score": 0,
          "Shots": 0,
          "Timestamp": "2019-06-26T14:24:17.748Z"
        },
        {
          "Level": 1,
          "Lives": 3,
          "Nickname": "ramon",
          "Score": 0,
          "Shots": 0,
          "Timestamp": "2019-06-24T17:36:38.646Z"
        },
        {
          "Level": 1,
          "Lives": 3,
          "Nickname": "bruceb",
          "Score": 0,
          "Shots": 0,
          "Timestamp": "2019-06-24T12:24:12.238Z"
        },
        {
          "Level": 1,
          "Lives": 3,
          "Nickname": "hackz",
          "Score": 1,
          "Shots": 0,
          "Timestamp": "2019-06-24T17:36:38.646Z"
        }
      ]
    }
  6. Click on the Save button. Check that the record was properly created
  7. Click on the Overview tab. Scroll down on that page, then find and copy the Amazon Resource Name (ARN) for the table to an auxiliary text file

Task: Creating the Lambda function


Step 1. Creating the Lambda function and inputting the code

  1. Visit your Lambda page on the AWS Console
  2. Click on the button Create a function. If the button is not visible, check on the console for a menu on the left, with the label Functions, and then hit on the Create function button
  3. Select Author from scratch.
  4. Under the section Basic information:
    1. For Function Name, input <prefix>GameTopXStats
    2. For Runtime, select the latest supported version for Node.js
    3. For Permissions, we need to give permissions to our Lambda function to get data from the DynamoDB table that we have just created. Expand the section, and select Create a new role with basic Lambda permissions.
    4. Click on the Create function button. You are going to get transported to the Designer page for the function.
    5. Scroll down to the section Function code
    6. Open this link and copy the code inside the function code window:
    7. Click on Save, just to guarantee that the code was saved properly.

Step 2. Adjusting the permissions for your Lambda function

  1. Scroll down on the page of your lambda function, and find the section Execution Role
  2. Check that there is a link for the role that you have created before (View the <prefix>GameTopXStats-role). Click on it. It will open another tab on your browser, showing to you the role configuration. See that AWS Lambda has automatically added permissions so your function can provide log records to CloudWatch
  3. Click on Add inline policy.
  4. On the Create policy page, on the tab Visual editor
    1. For the Service section, select DynamoDB
    2. For the Actions section, on the Filter actions input box, write GetItem, and mark the method GetItem.
    3. For the Resources section, select Specific, and then click on the label Add ARN. On the top of the window, for the field Specify ARN for table, copy the DynamoDB table ARN that you have previously saved.
      If you are not sure on how to to get the ARN, just go to DynamoDB, select the <prefix>SessionTopX table, and on the tab Overview you are going to find its ARN at the end of the page.
  5. On the bottom right, click on the button Review Policy. You are going to be taken to the Review policy page
  6. On that page, for the field Name*, input DynamoDBPermissions
  7. On the bottom of the page, click on Create policy
  8. You can close this tab and get back to the Lambda Management Console page


Step 3. Configuring the Environment variables section

Visit the code of your Lambda function, and check that the function readTopxDataFromDatabase (it is at the top) specifies the table to query by reading the variable process.env.TABLENAME. Let's configure it with the name of the table that we have created:
  1. Scroll down to the section Environment variables. You will see two fields, one for Key, and another one for Value
  2. For Key, input TABLENAME
  3. For Value, input the name that you used to define your DynamoDB table (<prefix>GameTopX)
  4. On the top right, click on the button Save


Task: Configuring events and testing your Lambda function

Before moving forward, let's be sure that our lambda function is working properly. As we are intending to integrate the Lambda Function with an (to be created) API Gateway component, let's create some events that will help us to check if our code is correctly implemented.

  1. On the top of the page, click on Select a test event, and then click on Configure test events. A new window will be shown to you, so you can configure events
  2. Be sure that, near to the top of the new window, the Create new test event is selected
  3. On the field Event name, input and appropriate name (for the first one, the suggestion is DirectIntegration).
  4. Copy-and-paste the following JSON to the body of the event, replacing what is in there
    {
      "params": {
        "path": {},
        "querystring": {
          "sessionId": "TheTestSession"
        }
      }
    }
  5. Scroll down and click on the button Create, on the lower right of the window
  6. Click on the Test button. Scroll up to check the results on the screen
  7. Check the results on CloudWatch logs:

Repeat the steps 1 to 7 above, for the following pairs of Event name and body of the event

Event name body of the event
DirectMissingSession
{
  "params": {
    "path": {},
    "querystring": {}
  }
}
DirectWrongSession
{
  "params": {
    "path": {},
    "querystring": {
      "sessionId": "WRONG"
    }
  }
}
ProxyIntegration
{
  "queryStringParameters": {
    "sessionId": "TheTestSession"
  }
}
ProxyWrongSession
{
  "queryStringParameters": {
    "sessionId": "WRONG"
  }
}
ProxyMissingSession
{
  "queryStringParameters": {}
}

If at this point everything went well, then we have our lambda function working properly. Now, let's expose it via an API.


Task: Creating your API Gateway to consume the lambda function

Step 1 - Creating the API

You can execute this task by creating a new API, or by adding a resource to an existing API. If the latter is the case, skip to the step 2 of this task.

  1. Visit the API Gateway console page.
  2. Depending on the state of your account, you can find a Create API or a Get Started button. Click on the one that has appeared to you. You are going to be taken to a Create API page.
    1. Click on Get started. You are going to be taken to a Create API page.
    2. For the section Choose the protocol, select REST
    3. For the Create new API section, select New API
    4. For the section Settings, input the following:
You are going to be taken to a page where you can configure the resources of your API. See that this page has a drop-down button labeled Actions.

Step 2 - Creating the resource for your API

  1. On the Resources page, click on the drop-down button Actions, and then click on Create Resource
    A section entitled New Child Resource will be shown
  2. On the section New Child Resource
    1. For Resource Name*, input topxstatistics. The field Resource Path will be automatically updated with this same value. Let it as it is
    2. Mark the box in front of the field Enable API Gateway CORS
    3. Click on Create Resource. You will see that the Resources section of the page will be updated with this new resource
  3. Select the resource topxstatistics, then click on the button Actions, and then on Create Method. A drop-down list will appear
  4. On the drop-down list, select GET
  5. Click on the check button at the right of the GET method, to confirm its creation. A section entitled  /topxstatistics - GET - Setup will appear, for you to configure the method
  6. On the section /topxstatistics - GET - Setup:
    1. For Integration type, make sure that Lambda function is selected
    2. On the field Lambda Function, begin typing the name of the Lambda function that you have created (<prefix>GameTopXStats). A selection pop-up will appear, so you can select the name of the function. Select it
    3. Leave everything else as is, and click on Save
      A pop-up window will appear, asking you to give permissions to your API Gateway to access invoke the lambda function. Click on the OK button
    4. A section like the one below is going to be shown to you

Step 3 - Configuring the integration

  1. Click on Method Request
    1. Find the section URL Query String Parameters and expand it
    2. On the top, click on the label <- Method Execution to get back to the configuration page
  2. Click on Integration Request
    1. Find the section URL Query String Parameters, and expand it
      1. Click on Add query string
      2. For the field Name, input sessionId
      3. For the field Mapped From, input method.request.querystring.sessionId
      4. On the right hand side of that line, click on the check button. This will confirm the configuration for the query string parameter
    2. Scroll down to the section Mapping Templates, and expand it
      1. For Request body passthrough, select When there are no templates defined (recommended)
      2. Click on Add mapping template. A field will be make available under the section Content-Type
      3. On that field, input application/json (you need to write it)
      4. Click on the check button to confirm the definition of the content-type
        Scroll down a bit. A window will be made available for you to input with the mapping templates. Mapping templates are a powerful resource for you to configure integration in API Gateway without needing to write code.
      5. On the drop-down list Generate template, select Method Request passthrough.
        API Gateway will add, automatically, a mapping template that forwards the request to the Lambda function in a structured way
      6. Click on the button Save
    3. Scroll to the top, and click on the label <- Method Execution to get back to the configuration page


Step 4 - Testing your API

4.1. Testing without any session

  1. Click on TEST (you must click exactly on the label). You are going to be taken to the testing page for that resource/method
  2. You will see a page where you can configure the HTTP content for your request
  3. Check that there is no information on the field {topxstatistics} under the Query Strings section
  4. Click on the Test button with the lightning symbol, down on the page. As we have provided no sessionId yet, your Lambda function must respond accordingly, with a 400 statusCode. On the right of the screen you will be able to check the response body, response headers, and the logs, in a screen similar to the one below
  5. Visit the CloudWatch logs for your lambda function. Compare it with the results of your previous tests.


4.2 Testing with a invalid sessionId

  1. Still at the testing page, on the Query Strings section, input the following value to the field {topxstatistics}
    sessionId=WRONG
  2. Click on Test.
  3. Now you should receive a little bit different response
    {
      "isBase64Encoded": false,
      "statusCode": 404,
      "body": "Inexistent session",
    "headers": { "Content-Type": "text/plain" } }


4.3. Testing with a valid session

  1. Still at the testing page, on the Query Strings section, input the following value to the field {topxstatistics}
    sessionId=TheTestSession

    This is the same value that we have used to insert data on DynamoDB. Check on the previous Step.

  2. Click on Test.
  3. Now you should receive a response similar to the following one

Check that the status code is 200, and that the body contains the expected result, but not in the expected format. We would like to have it as a JSON. Let's fix this.

On the top of the page, click on <- Method Execution to get back to the configuration page


Step 5 - Fixing the Integration Response

  1. Being on the configuration page for the method execution, click on Integration Response. The page will show a line with a configuration for the Method response status 200
  2. Click to expand the line corresponding to the Method response status 200
  3. Scroll down and expand the section Mapping Templates, and expand it
    We are going to configure a Mapping Template that converts the response from String to a JSON
  4. If there is a record labeled application/json under the section Content-Type, click on it
    If there is NOT that record, add it:
  5. On the right, a field will be opened so you can input the mapping template. Input the following content:
    #set($inputRoot = $input.path('$'))
    ## The next line changes the HTTP response code with the one provided by lambda
    #set($context.responseOverride.status = $inputRoot.statusCode)
    ## Decoding base64 (this could have been left to the application)
    #if( $inputRoot.isBase64Encoded == true )
    $util.base64Decode($inputRoot.body)
    #else
    $inputRoot.body
    #end
  6. Click on the Save button below the mapping template field (see that are two Save buttons on that page. Be sure of clicking on the right one)
  7. Scroll to the top, and click on the label <- Method Execution to get back to the configuration page
  8. Test the API (run again the tests on Step 4)


Step 6 - Deploy the API

Your need to deploy the API so you can have access to it, externally.

  1. Visit the home page for API Gateway
  2. Be sure that you have your <prefix>GameAPI selected (just for the case of you to have other APIs on your account)
  3. Click on the drop-down Actions, and then click on Deploy API. The Deploy API pop-up window will be shown to you.
  4. On the Deploy API window:
    1. For Deployment Stage, select [New Stage]
    2. For Stage name*, input the value prod
    3. For Stage description, input Production environment for the API
    4. For Deployment description, input first deployment
    5. Click on the button Deploy
      You are going to be forwarded to the Stage page of the console, where the your deployment URL is at the top. It will be something similar to the following:

      Invoke URL: https://<API-id>.execute-api.<region>.amazonaws.com/prod
  5. At the left, you will see the section Stages, and prod under it. Click on prod to expand the section. You will see the topxstatistics resource, with GET and OPTIONS under it


Step 7 - Test the API from outside

  1. Click on the GET method under the topxstatistics resource. Copy the Invoke URL for the resource
  2. Open a new tab or window on your browser
  3. Using the invoke URL for topxstatistics that you have just copied (let's say it is https://xyz.execute-api.region1.amazonaws.com/prod/topxstatistics), run each one of these requests on the browser tab or window. IMPORTANT: Depending if you are creating a new API or just creating a new resource in an existing API, you may need to fix the path to the resource. Visit you API deployment on the API Gateway console, and check the path to the resources topxstatistics.

Check the results of the Network log of your browser. Check that the HTTP return codes area appropriate.


Step 8 - Adding security to your API

One problem remains to your API: it is publicly exposed. There are many options to implement security for your API. In this lab, we are going to implement access control via API KEY. This kind of access is recommended for Business-to-Business scenarios, when you have a key shared exclusively with the business who wants to consume your API.

8.1. Enabling the security via API Key

  1. On the menu on the left of the API page, under the name of your API, click the option Resources
  2. Click on the GET method for /topxstatistics
  3. Click on Method Request
  4. Make API Key Required = true ( don't forget of confirming it ). Your page will be similar to the one below:

8.2. Creating an API Key

  1. Now, click on API Keys on the menu on the left hands side of the console (this is NOT under your API. It is a "standalone" menu option).
  2. Click on Actions, and then on Create API key
    1. For Name*, input your own name
    2. For the field API key*, select Auto Generate
    3. For the Description field, input This is the key that I've created for myself
    4. Click on Save. You are going to be presented to a page where you have the details about the key.
  3. Click on the Show label which is at the right hand side of API key
  4. Copy the value to a helper text file. We will need it later to test the API.

8.3. Creating the Usage Plan to be used by the API

Think of a Usage Plan as a mechanism to control the way a certain authorized entity (represented by the API Key) can access certain resources of your API.

  1. On the main menu of API Gateway, click on Usage Plans (it is right above the API Keys option). You are going to be taken to the Create Usage Plan page
    1. For Name*, input Usage plan for <your name> usage plan  (replace <your name> with.... your name!)
    2. For Description, input This usage plan was created to learn about API Keys and usage plans
    3. Be sure that Enable throttling is unchecked (we are not testing throttling on this lab)
    4. Be sure that Enable quota is unchecked (we are not testing quota validation on this lab)
    5. Click on the Next button
  2. Now you must be on the page Associate API Stages
    1. Click on Add API Stage
    2. For the field API, select your API
    3. For Stage, select prod
    4. On the right, click on the check button to confirm the configuration
    5. Click on Next
  3. Now you must be on the Usage Plan API Keys
    1. Click on the button Add API Key to Usage Plan
    2. In the field Name, begin typing the name that you used to define your API Key (the field is case-sensitive). Select your key
    3. On the right, click on the check button to confirm the configuration
    4. Click on the button Done

Now your API Key is created, and associated to a Usage Plan, what makes it available to be used. Let's test it.


8.4. Testing your API

To test the API, we are going to use CURL. If you don't have it, you can download and install it from here.

After installing it, run the following commands on your terminal (or console prompt, depending on how you name it).

While running the tests below, be sure of:

1. Testing without any parameter (resulting in HTTP 400)
curl --verbose --header "x-api-key: <put your API key here>" https://<API URL>
2. Testing with an empty session (resulting in HTTP 400)
curl --verbose --header "x-api-key: <put your API key here>" https://<API URL>?sessionId=
3. Testing with a non-existing session (resulting in HTTP 404)
curl --verbose --header "x-api-key: <put your API key here>" https://<API URL>?sessionId=NONEXISTING
4. Testing with the session that we have inserted on DynamoDB  (resulting in HTTP 200)

curl --verbose --header x-api-key: <put your API key here>" https://<API URL>?sessionId=TheTestSession


Finishing the lab

You have finished the lab.

Clear your account by:

  1. Deleting the DynamoDB table
  2. Deleting the Lambda function
  3. Deleting the role associated to the Lambda function
  4. Deleting the API Gateway
  5. Deleting the Usage Plan
  6. Deleting the Key