I have tested the following solution on my side using the r4.4xlarge instance type
Please find the steps-
Step-1) Increase the root volume of the EMR master node
To navigate to the EMR cluster’s master node’s root EBS volume the following steps can be taken:
– Open the EMR cluster in the EMR console
– Expand the hardware dropdown
– Click on the Instance Group ID that is labelled as the MASTER
– Click on the EC2 instance ID shown in this table, which will open the master node in the EC2 console
– In the “Description” tab in the information panel at the bottom of the console, scroll down and click on the linked device for the “Root device” entry
– The EBS volume will now open in the EC2 console, this is the root EBS Volume for the master node, and should look like the screenshot below
– Now you should be able to choose the “Modify Volume” action from the “Actions” dropdown, and change the volume size!
In this case, I adjusted the size of the EBS volume from 10GB to 50GB, simple as that! Alternatively, you could give the customer the exact CLI command to do this rather than trying to guide them through this process on the console:
aws ec2 modify-volume –region us-east-1 –volume-id vol-xxxxxxxxxxxxxxxxxxxx –size 50 –volume-type gp2
Step-2) Login to the Master node with SSH and check run the following comand to check the newly attached size information under /dev/xvda.
Step-3) However, it is important to note that at this point you will still not see the additional space on the file system (root volume /dev/xvda1). Run the following command to add the following space to root volume.
sudo /usr/bin/cloud-init -d single -n growpart
sudo /usr/bin/cloud-init -d single -n resizefs
Step-4) Now run below command to see “/” volume is increased to 50GB.
Step-5) After the volume is increased, you can do a test create a sample test file and see the size of “/” volume increased its usage.
sudo fallocate -l 10G /test (This will create a test file inside “/” with 10GB in size)
df -h (Verify root volume mount point increased its usage)
Step-6) After verifying, just delete sample /test file.
sudo rm -rf /test
Note: Please back up any important file or configuration files before performing the operation.
Need to learn more about aws big data (demystified)?
- Contact me via linked in Omid Vahdaty
- website: https://amazon-aws-big-data-demystified.ninja/
- Join our meetup, FB group and youtube channel
- Join our meetup : https://www.meetup.com/AWS-Big-Data-Demystified/
- Join our facebook group https://www.facebook.com/groups/amazon.aws.big.data.demystified/
- subscribe to our youtube channel https://www.youtube.com/channel/UCzeGqhZIWU-hIDczWa8GtgQ?view_as=subscriber