Category Archives: Uncategorized

How to combine Azure AD SSO with AWS programmatic access ?

Since we both use Azure AD (Office365 / LDAP) for the companies user management and AWS for our hosting we already enabled federation between these cloud providers. This works perfectly by assuming an IAM Role through the Azure AD credentials and have console access through single-sing-on.

However to use the AWS CLI a user would need programmatic access and it’s access and secret keys. Luckily for us there is an excellent open-source project for that called aws-azure-login. And since it’s on npm very easy to run on Mac, Linux and Windows.

What do we need ?

  • install the package as mentioned: npm install -g aws-azure-login (or not global if you fancy that)
  • you will need your Azure Tenant ID which the Azure administrator can supply ( Azure AD -> Properties -> Directory ID )
  • and the Azure Application ID and again the Azure administrator ( Enterprise Applications -> Amazon Web Services -> Properties -> Application ID )
  • you might want to know the AWS IAM Role max session duration to prevent some trial on error which your AWS administrator can supply (default is 1 hour)

First we will create an Azure profile:

jvzoggel$ aws-azure-login --configure --profile rubix
Configuring profile 'rubix'
? Azure Tenant ID: some-guid-you-will-need-part-1
? Azure App ID URI: some-guid-you-will-need-part-2
? Default Username:
? Default Role ARN (if multiple):
? Default Session Duration Hours (up to 12): 2

The max session duration on the AWS IAM Role is the last parameter, if you enter a to high value you will receive an error but only after the next step login. So ask first, or trial on error.

Then we actually login on Azure:

jvzoggel$ aws-azure-login --profile rubix --no-prompt
Logging in with profile 'rubix'...
? Password: [hidden]
Please type in the code displayed on your authenticator app from your device
? Verification Code: 424242
Assuming role arn:aws:iam::42:role/rbx-sso-myrole

Since we configured MFA on Azure AD there is also the Verification Code besides the password to provide. When we succeed the aws-azure-login actually generated a profile in the AWS credentials configuration which holds the access and secret key. So let’s check.

Connect to AWS

jvzoggel$ cat /Users/jvzoggel/.aws/credentials


jvzoggel$ export AWS_PROFILE=rubix
jvzoggel$ aws s3api list-buckets
    "Buckets": [
            "Name": "awsamplify-20181220092431-deployment",


Leave a comment

Posted by on 01-02-2019 in Uncategorized


Tags: , , , ,

Reminder: Example password complexity check in typescript

Since I mostly blog/commit to remember for my own future usage, another handy codesnip which I probably will need again. Requirement from our IAM provider for is user credentials was a certain password complexity of at least 8 characters (easy) and the combination of at least 3 out of 4 character sets (lowercase, uppercase, numbers and weird characters aka non-alphanumeric. The last criteria could be done with a blunt knife but we tried to be gentle.

    // Check password complexity
    const hasUpperCase = /[A-Z]/.test(password);
    const hasLowerCase = /[a-z]/.test(password);
    const hasNumbers = /\d/.test(password);
    const hasNonalphas = /\W/.test(password);
    const totalPassword = [ hasUpperCase, hasLowerCase, hasNumbers, hasNonalphas ].filter(v => v).length;
    if (password.length < 8 || totalPassword < 3) {
        console.error(`password length:${password.length} charsets:${totalPassword}`);
        // fancy error handling stuff

The regex for frontend if usable:



Leave a comment

Posted by on 31-01-2019 in Uncategorized


Tags: , , , ,

How to use AWS Simple Email Service (SES) from TypeScript on NodeJS example

In our application flows we use AWS Simple Email Service to send emails to our users. Since the documentation and examples of AWS SES are not that clear, it might take some trial and error to figure out which of the parameters are mandatory and, more important, obsolete. The AWS examples here crash if you actually configure the parameters with empty strings or null like mentioned there.

This TypeScript code below initiates a AWS SES connection and uses an earlied AWS CloudFront generated Email template (see example below somewhere). I found it quite surprising that you can actually refer to the template by name and not ARN only.

import * as AWS from 'aws-sdk';
import * as https from 'https';

const ses = new AWS.SES({
    httpOptions: {
        agent: new https.Agent({
            keepAlive: true

 * Send email through AWS SES Templates
export async function sendMail(email: string, name: string): Promise(String) {

    try {
        // Create SES sendTemplatedEmail templateData content
        const templatedata = {
            parameter_name: name
        // console.debug(`sendMail templatedata: ${JSON.stringify(templatedata)}`);

        // Create SES sendTemplatedEmail full message
        const params = {
            Destination: {
                ToAddresses: [ email ]
            Source: '',
            Template: 'myFanceTerra10EmailTemplate',
            TemplateData: JSON.stringify(templatedata)
        // console.debug(`sendMail param: ${JSON.stringify(params)}`);

        const sesResponse = await ses.sendTemplatedEmail(params).promise();
        console.log(`sendMail requestId: ${sesResponse.$response.requestId} and messageId: ${sesResponse.MessageId}`);
        return 'OK';
    } catch (e) {
        console.error(`sendMail unexpected: ${e.message}`);
        return 'something fancy error handling';

Here is an example AWS CloudFormation resource for the AWS SES email template. You can use the always handy sub function to prevent any complex character escaping or unreadable 1-line HTML. I love it for EC2 UserData and for stuff like this:

    Type: AWS::SES::Template
        TemplateName: myFanceTerra10EmailTemplate
        SubjectPart: My Subject
#       TextPart: "Nobody uses this anymore right ???"
          Fn::Sub: |
            <img src="">
            <h1>Sir/Madam {{parameter_name}},</h1>
            <p>Ho ya doin ?</p>
            <strong>the T10 crew</strong>

In case you use the Serverless Framework (you should) for Serverless deployments the following code is necessary in your Serverless.yaml. This allows your Lambda function to use the Email template on runtime. In our case the domain is hosted on AWS Route53 as well which saves you some problems.

- Effect: Allow
  - ses:SendTemplatedEmail
  - "arn:aws:ses:eu-west-1:*:identity/"

Hope it helps!


  • Link to the original article due to RSS feeds @
  • Sending Email Using Amazon SES
1 Comment

Posted by on 26-11-2018 in Uncategorized


Tags: , , , , , ,

How to setup unit testing for AWS Lambda serverless functions (on NodeJS) ?

We use AWS Lambda serverless functions combined with TypeScript and NodeJS which results in an extreme powerful developer toolset. Due to the fact that functions contain isolated logic they are ideal for automated unit testing in CI/CD pipelines. So eventually looking at our options we decided to use the features of mocha, chai and nock combined. This resulted in a very easy and powerful solution for unit testing.

I’m sharing this after a chat at a meetup where the use of EKS instead of Lambda even for really simple functions was advocated due to the fact that serverless was hard to isolate (run local) and was hard to setup any unit testing. I beg to differ because:

So let’s go …

Our example function

is a simple function for retrieving a single record from a AWS DynamoDB table

// Handler for serverless framework
export async function handler(event: APIGatewayProxyEvent, _context: Context, callback: Callback) {
    try {
        callback(undefined, await getWerkgever(event));
    } catch (err) {

// Main logic
export async function getRecord(event: APIGatewayProxyEvent) { 
  const id = '1'; // pointless, but good enough for this example
  const queryParams = { TableName: process.env.dynamotable, Key: { id } }; 
  const result = await documentClient.get(queryParams).promise(); 
  if (result.Item) { 
    return { statusCode: 200, headers, body: JSON.stringify(result.Item) }; 
  } else {
    return { statusCode: 404, headers, body: undefined };

So what happens here ?

  • We deliberately split the logic between handler (for the serverless framework) and the main logic
  • We need to export the main logic for use in our unit tests (and local development)

Using Mocha & Nock

Since we are running node we can use both mochaJS and nock for our unit testing. Setting up the specification file (.spec) for our simple function we first run the test with.

import { APIGatewayProxyEvent } from 'aws-lambda';
import { expect } from 'chai';
import * as nock from 'nock';
import { getRecord } from './getRecord';

process.env.dynamotable = 'myTable';

describe('getRecord', () => {

    it('UT001 - getRecord with valid response', async() => {
        const event: APIGatewayProxyEvent = {
            body: '',
            headers: {},
            httpMethod: 'GET',
            isBase64Encoded: false,
            path: '',
            pathParameters: {},
            queryStringParameters: undefined,
            stageVariables: {},
            requestContext: {},
            resource: '' };

        const response = await getRecord(event);


So what happened ?

  • We set the environment variables (like the DynamoDB table) which normally is done by AWS Lambda
  • We configure nock.recorder for auditing the upcoming execution
  • We define a dummy APIGatewayProxyEvent which has some mandatory elements which we leave mostly empty or undefined
  • We define the call to our AWS Lambda function and since we isolated the main logic we can call this directly
  • If your AWS profile used on your dev machine has enough IAM grants the code can execute against AWS DynamoDB (we use a special user for this to keep it clean)

By running mocha with the Nock recorder we can see the actual callout to AWS DynamoDB from our developer machine

<-- cut here -->

nock('', {"encodedQueryParams":true})
.get('/', {"TableName":"myTable","Key":{"id":{"S":"1"}}})
.reply(200,{Item: {name: {S: 'myName'}, id: {S: '1'}});
......... (much stuff)

So with nock we actually recorded the https call to DynamoDB. Which now, we can easily use with nock to mock the response during unit testing. So next change the code in our spec file with the info from nock recorder:

// nock.recorder.rec();
    .get('/' )
    .reply(200,{Item: {name: {S: 'myName'}, id: {S: '1'}});

So what happened ?

  • Disabled the recorder, we don’t need it anymore
  • Setup nock to catch the HTTPS GET call to the dynamoDB endpoint
  • Configured nock to reply with a 200 and the specified Item record (you can also use reply with files)

Using Chai

With this basis setup we can execute unit tests in our pipeline with Mocha where nock will handle the mocking of the endpoints. With some little Chai magic we can define expectations in our specification file to make sure all message logic of our function is done properly and the HTTP reply is as expected.

expect(response.headers).to.deep.include({ 'Content-Type': 'application/json' });
expect(JSON.parse(response.body)).to.deep.equal({ name: 'myName', id: '1'});

And there is more

With this it’s easy to catch all outbound HTTPS requests and mock different responses (0 records, multiple records, etc etc) for extensive unit testing. The possibilities are endless, so hope it helps …

Leave a comment

Posted by on 26-10-2018 in Uncategorized


Tags: , , , , , , , ,

How to determine your AWS Lambda@Edge regions and find your CloudWatch logs

You must review AWS CloudWatch log files in the correct region to see the log files created when CloudFront executed your Lambda function. I found this very usefull bash AWS CLI based bash command here which allows to determine a list of Regions where your Lambda@Edge function have received traffic so storing for future (personal) reference.

for region in $(aws --output text  ec2 describe-regions | cut -f 3) 
    for loggroup in $(aws --output text  logs describe-log-groups --log-group-name "/aws/lambda/us-east-1.$FUNCTION_NAME" --region $region --query 'logGroups[].logGroupName')
        echo $region $loggroup

You can just leave FUNCTION_NAME empty to get a list of all functions.


Leave a comment

Posted by on 25-10-2018 in Uncategorized


Tags: , , , , , ,

How to fix Git error ‘invalid active developer path’ after MacOS update

After upgrading to macOS Mojave,  I tried running GIT from Terminal or IDE but it kept giving the following error:

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun.

The problem is that you are using XCode and you explicit have to agree to the license agreement. So open Terminal, and run the following:

xcode-select --install

This will download and install xcode developer tools and fix the problem. As a follow on step, you may need to reset the path to Xcode if you have several versions or want the command line tools to run without Xcode.

xcode-select --switch /Applications/
xcode-select --switch /Library/Developer/CommandLineTools



Posted by on 27-09-2018 in Uncategorized


Tags: , ,

How to connect to the CEPH Object Gateway S3 API with Java

We use a CEPH storage solution and specifically want to use the Ceph Object Gateway with the S3 API through a Java client. The API is based on the AWS S3 standard however requires some special tweaking to work. Took me some effort to get a working connection, so here to share:


We can use either the new AmazonS3ClientBuilder

package nl.rubix.s3;

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.Protocol;
import com.amazonaws.SDKGlobalConfiguration;

public class AmazonS3ClientBuilder
  public static void main(String[] args)
    String accessKey = "XXXXX";
    String secretKey = "XXXXX";

    // Our firewall on DEV does some weird stuff so we disable SSL cert check
    if (SDKGlobalConfiguration.isCertCheckingDisabled())
      System.out.println("Cert checking is disabled");
    // S3 Client configuration
    ClientConfiguration config = new ClientConfiguration();
    // Not the standard "AWS3SignerType", maar expliciet signerTypeV2

    // S3 Credentials
    BasicAWSCredentials credentials = new BasicAWSCredentials(accessKey,secretKey);
    // S3 Endpoint
    AwsClientBuilder.EndpointConfiguration endpointConfiguration = new
      AwsClientBuilder.EndpointConfiguration("", "");
    AmazonS3 s3 =
      .withCredentials(new AWSStaticCredentialsProvider(credentials))
    System.out.println(" Connection to the Rubix S3 ");
    try { 
       * List of buckets and objects in our account
       System.out.println("Listing buckets and objects");
       for (Bucket bucket : s3.listBuckets())
         System.out.println(" - " + bucket.getName() +" "
           + "(owner = " + bucket.getOwner()
           + " "
           + "(creationDate = " + bucket.getCreationDate());
         ObjectListing objectListing = s3.listObjects(new ListObjectsRequest()
         for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) 
           System.out.println(" --- " + objectSummary.getKey() +" "
           + "(size = " + objectSummary.getSize() + ")" +" "
           + "(eTag = " + objectSummary.getETag() + ")");
     catch (AmazonServiceException ase)
       System.out.println("Caught an AmazonServiceException, which means your request made it to S3, but was rejected with an error response for some reason.");
       System.out.println("Error Message:    " + ase.getMessage());
       System.out.println("HTTP Status Code: " + ase.getStatusCode());
       System.out.println("AWS Error Code: " + ase.getErrorCode());
       System.out.println("Error Type: " + ase.getErrorType());
       System.out.println("Request ID: " + ase.getRequestId());
     catch (AmazonClientException ace)
       System.out.println("Caught an AmazonClientException, which means the client encountered "
       + "a serious internal problem while trying to communicate with S3,
       + "such as not being able to access the network.");
       System.out.println("Error Message: " + ace.getMessage());

or make it work with the older and depricated AmazonS3Client

package nl.rubix.s3;

import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.SDKGlobalConfiguration;

public class BasicAWSCredentials
    public static void main(String[] args)
        String accessKey = "XXXXXXX";
        String secretKey = "XXXXXXX";

    if (SDKGlobalConfiguration.isCertCheckingDisabled())
        System.out.println("Cert checking is disabled");
    AWSCredentials credentials = new com.amazonaws.auth.BasicAWSCredentials(accessKey,secretKey);

    ClientConfiguration clientConfig = new ClientConfiguration();

    AmazonS3 conn = new AmazonS3Client(credentials, clientConfig);

    for (Bucket bucket : conn.listBuckets())
      System.out.println(" - " + bucket.getName() 
        + " "
        + "(owner = " + bucket.getOwner()
        + " "
        + "(creationDate = " + bucket.getCreationDate());

Hope it helps!

Leave a comment

Posted by on 03-07-2018 in Uncategorized


Tags: , , , , ,