re:Invent 2022 wishlist - at the front of my mind

I’ll be attending re:Invent this year for this first time. It’s fair to say that I’m more than a little excited, especially as it’s coupled with being my first time visiting Las Vegas. What I’ve decided to do this year is publish a series of 3 blog posts in the run up to re:Invent covering off my hopes and dreams for what the magical people at AWS have managed to conjure up over the last 12 months, ready for announcing at the event.

In this first blog post, I’m covering off those features that I think about existing most weeks. In later blog posts I’ll talk about some bigger wishes.

Let’s get started!

Preventing non-VPC Lambda function outbound network access

At present, the only way to prevent a Lambda function from being able to make outbound network requests to the public internet is by attaching it to a VPC. When so many architectures are going completely serverless by utilising orchestration services such as Step Functions and EventBridge - where a VPC isn’t required - the understanding and creation of a VPC and subnets is likely to be a barrier to some developers.

What I’d love to see is a way to disable outbound public network access (i.e. limit network requests to only AWS services) for a Lambda function. A potential attack vector that I see being closed by this feature is a scenario where a malicious actor gains access to change the application code that runs inside a Lambda function, changing it to send a copy of all the data that the function processes to their server silently.

For those wanting to avoid VPC configuration, three modes could be available. The default mode would remain as it is now for backwards compatibility purposes. At the other end of the spectrum, no outbound public access would be another mode. In the middle, an allow-list could be utilised as a lightweight alternative to full VPC configuration with security groups or NACLs. This would allow developers to limit where requests could be made to whilst still avoiding the full VPC set up process.

In terms of how this could be implemented from a user experience perspective, below are some ideas for how it could be interacted with via the AWS Management Console as well as via a CloudFormation (SAM) template.

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: >
  Creates Lambda functions with various levels  of outbound network access

Resources:
  NoOutboundAccessLambdaFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Runtime: 'python3.9'
      CodeUri: './hello_world'
      OutboundNetworkAccess:
        Enabled: False

  LimitedOutboundAccessLambdaFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Runtime: 'python3.9'
      CodeUri: './hello_world'
      OutboundNetworkAccess:
        Enabled: True
        AllowList:
          - Domain: 'domain1.com'
          - Domain: 'domain2.com'
          - Cidr: '1.2.3.4/32'

  FullOutboundAccessLambdaFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Runtime: 'python3.9'
      CodeUri: './hello_world'

Executing a Step Functions workflow task as an assumed role

The next item on my wishlist for this entry in the series are centred around AWS Step Functions.

This particular item is to be able to execute a Step Functions workflow task with an assumed role. This would enable patterns where a workflow needs to access something in a different account as part of its definition.

At present, to implement this you would need to create a Lambda function where its execution role has permission to assume the role in the external account. The action to be carried out would then need to be written with an AWS SDK into the application code of the function. The Step Functions workflow in our account could then invoke this Lambda function.

My wish is that this process is significantly simplified by allowing a workflow’s task to have a role assigned to it - similarly to how you can with actions in CodePipeline - which would result in the workflow’s execution role assuming the task’s specified role, cutting out the intermediary Lambda function that you need at the moment.

Below is an example of a how a Step Functions workflow could be defined. In this example: a UUID is generated with an intrinsic function (blog post to come on these soon); an external Lambda function is then invoked via the assumption of a role into the external account; finally the result of the function is stored in a DynamoDB table in our account.

Comment: >
  A workflow that assumes a role into another  account to carry out an action
StartAt: 'Generate unique ID'
States:
  Generate unique ID:
    Type: 'Pass'
    Parameters:
      UniqueIdentifier.$: 'States.UUID()'
    Next: Invoke external Lambda function

  Invoke external Lambda function:
    Type: 'Task'
    Resource: 'arn:aws:states:::lambda:invoke'
    RoleArn: 'arn:aws:iam::12345678901:role/MyExternalRole'
    Parameters:
      FunctionName: 'MyFunctionInAnotherAccount'
      Payload:
        UniqueIdentifier.$: '$.UniqueIdenfier.uuid'
    OutputPath: '$.Payload'
    ResultPath: '$.ExternalLambdaResult'
    Next: 'Store in my DynamoDB table'

  Store in my DynamoDB table:
    Type: 'Task'
    Resource: 'arn:aws:states:::dynamodb:putItem'
    Parameters:
      TableName: 'MyDynamoDbTable'
      Item:
        UniqueId:
          S.$: '$.UniqueIdenfier.uuid'
        LambdaResult:
          S.$: '$.ExternalLambdaResult.Value'
    End: True

Globally catching failures within a Step Function workflow

The final item on my wishlist that I’m going to talk about at the moment is around failure handling within a Step Functions workflow.

At present, to catch failures from any state within a Step Functions workflow it is required to add a Catch property to each state as shown in the code snippet below. The screenshot following shows this visualised. Whilst this may not be too much of an issue for short workflows; you can imagine that as it grows in number of states, the complexity also increases.

The alternative solution possible at present is to wrap the entire happy path of a workflow in a Parallel state with a single branch. The outer Parallel state can then catch and direct all errors where required. I’m not a fan of this approach as it can add confusion when reading through workflow definitions - as someone could, not unreasonably, expect to see more than one branch.

Comment: >
  A workflow where errors are caught by each state

StartAt: 'Invoke function 1'
States:
  Invoke function 1:
    Type: 'Task'
    Resource: 'arn:aws:states:::lambda:invoke'
    Parameters:
      FunctionName: 'MyFunction1'
    Catch:
      - ErrorEquals:
          - 'States.ALL'
        Next: Catch failure
    Next: 'Invoke function 2'

  Invoke function 2:
    Type: 'Task'
    Resource: 'arn:aws:states:::lambda:invoke'
    Parameters:
      FunctionName: 'MyFunction2'
    Catch:
      - ErrorEquals:
          - 'States.ALL'
        Next: Catch failure
    Next: 'Invoke function 3'

  Invoke function 3:
    Type: 'Task'
    Resource: 'arn:aws:states:::lambda:invoke'
    Parameters:
      FunctionName: 'MyFunction3'
    Catch:
      - ErrorEquals:
          - 'States.ALL'
        Next: Catch failure
    End: True

  Catch failure:
    Type: 'Succeed'

Of course, not every workflow is suited to having a single method of catching errors from any state. However, for those that would be suitable it would be really convenient to be able to define the Catch in one place and have it apply to all states.

In the example code snippet below, I could imagine being able to define at the top-level which state should catch any error, much like StartAt defines where to start.

For workflows where all errors should be caught and handled in the same way (e.g. a message published to an SNS topic), this would make building them considerably quicker.

Comment: >
  A workflow where catching errors is defined just once

StartAt: 'Invoke function 1'
Catch: Catch failure
States:
  Invoke function 1:
    Type: 'Task'
    Resource: 'arn:aws:states:::lambda:invoke'
    Parameters:
      FunctionName: 'MyFunction1'
    Next: 'Invoke function 2'

  Invoke function 2:
    Type: 'Task'
    Resource: 'arn:aws:states:::lambda:invoke'
    Parameters:
      FunctionName: 'MyFunction2'
    Next: 'Invoke function 3'

  Invoke function 3:
    Type: 'Task'
    Resource: 'arn:aws:states:::lambda:invoke'
    Parameters:
      FunctionName: 'MyFunction3'
    End: True

  Catch failure:
    Type: 'Succeed'

Wrapping up

This marks the end of the first of three blog posts covering my wishlist for new releases at re:Invent 2022. I’ll be back soon with the next three!