BD & BA – ISfBD Practical AWS

BD&BA – ISfBD Practical

Create AWS account


Continue with the signup process, choosing the “Basic (Free)” support plan. It will ask for a credit card number, which will not be billed during this course. Once you’re done, send the email address you have used to sign up for AWS to

to have charges taken over by ABS.

Accessing AWS console

Select the “US East” region

Configure Firewall

Go to EC2 in AWS Console

First, create a key pair for protected authorized access. Go to “Key Pair


Click on “Create Key Pair” and choose a name for your key pair

What is DevOps? Introduction to DevOps 6

Screenshot_2016-04-01_12-50-53.png Screenshot_2016-04-01_12-51-32.png

And save the key pair to your local machine following your browser’s download dialog — recall where, i.e., in which directory/location, you / your browser save(s) the text file containing your key pair; you’ll need it later.

Then, go to “Security Groups” and click “Create Security Group


The Hue Web interface runs on TCP port 8888, hence we need to create a firewall rule that allows access to this port. Please note that this is not suitable for a production environment. Name and description are of course arbitrary.


Start EMR Cluster

Go to EMR in AWS Console

Create Cluster

MQ on AWS: PoC of high availability using EFS

Select “advanced options“. We need to set some configuration.

This is the “Advanced cluster configuration screen“:


Step 1: Software & Steps

Select Amazon Software Release 3.9.0


Select Hue to be installed — most probably this is nowadays done automatically with software release 3.9.0; see the tick with “Hue 3.7.1” in above screenshot.

Step 2: Hardware:

Select instance type (m2.xlarge for now) and cluster size (2 in total, 1 master, 1 core).


Step 3: General Cluster Settings:

Just click “Next

Step 4: Security

First select the key pair you created earlier:


Then assign the security group we created earlier to the Master node:




This is how this should look like. Note that your security group ID will be different.


And finally, go for launch: click on “Create cluster”.

You will then see a screen like this:


You can check the “Hardware” tab to see instances launching. You can also check the EC2 service to see your instances starting.

Now you need to wait till you see “Waiting”. This can take a while, 15 mins are not unheard of.

While you’re waiting, calculate how much running this cluster will cost per hour


Since we opened the Firewall, we can access the Hue web interface directly. The address is the Master public DNS shown above, which will be different in your case. In this example,

Prerequisites for ABAP for HANA

You will see the Hue Web interface login screen:

Coming up with an arbitrary (simple, no special characters such as @) user name and a complex enough password is left as an exercise to the reader.

Afterwards, you will see the following screen.

Uploading data

We are going to use the King James version of the Bible as sample data. Download the text file from

Go to the file browser and upload the file

You will see your file uploaded if all goes well.

Running your first EMR Pig job – Wordcount

We are going to find the most common words in the Bible. To this end, we are going to run an Apache Pig script on the kjv.txt file using MapReduce.

AWS Interview Questions 6


Enter the following script:

a = LOAD ‘kjv.txt’;

b = FOREACH a GENERATE FLATTEN(TOKENIZE((chararray)$0)) AS word;

c = GROUP b BY word;

d = FOREACH c GENERATE COUNT(b) AS ct, group;

e = ORDER d BY ct DESC;

STORE e INTO ‘bible_wordcount’;

Then execute the script (what does it do? Can you tell?) by clicking the triangle:

The script will dutifully start executing and show the log:

Downloading results

Once your job has finished, you can inspect the results using the File Browser again. You will see a new folder, bible_wordcount.

Inside, there are files part-r-…, which have the word counts in it. Tadaa.

Before you shut down, you could look around a bit (if there is time)

  • Check the Hue job log to see the MapReduce jobs created from your Pig script. What were their statistics?
  • Check the EC2/EMR interfaces to see the various performance parameters
  • If you have time, you could try to change the pig script such that only the 100 most frequent words are shown and words with a length smaller than three are omitted.
Cloud Computing Infrastructure Tutorial

Cluster Shutdown

Do not forget to shut down your EMR cluster again, since it will create costs otherwise. Go back to the AWS Web interface, select the EMR service, select your cluster, and click “Terminate”.

Shutdown Verification

It is important to make sure no EC2 instances / EMR clusters at the end. So check the EC2 console to make sure 0 instances are running. Also check the EMR console to make sure no clusters are running.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.