New High-Performance Provisioned IOPS Storage for Amazon RDS

Posted in: Cloud Computing, Enterprise Java

Today AWS announced that it is extending the provisioned IOPS concept already available for EBS volumes to RDS instances. So just like with EBS volumes you can now not only provision the storage size associated with an RDS instance, but also the performance of the RDS instance. This is a great improvement for AWS customers who want to run high-performance applications on RDS as they can now pay for the performance they need.

Remember that RDS is a very easy service to leverage from enterprise Java applications because the databases can be made available through a standard JDBC interface. This means that there is absolutely no difference in terms of the Java code when using an RDS hosted MySQL instance, versus a MySQL instance hosted on your own in-house server, or even hosted on an EC2 instance for example.

Here is the official announcement:

We are excited to announce the availability of Amazon RDS Provisioned IOPS, a new high-performance storage option for the Amazon Relational Database Service (Amazon RDS). Amazon RDS makes it easy to set up, operate, and scale a MySQL, Oracle, or SQL Server database in the cloud — and now enables you to provision up to 10,000 IOPS (input/output operations per second) with 1TB of storage for your new database instances.

Amazon RDS Provisioned IOPS is optimized for I/O-intensive, transactional (OLTP) database workloads. We are delivering this functionality to you in two stages. Starting immediately, when you create new database instances using the AWS Management Console or the Amazon RDS APIs, you can provision from 1,000 IOPS to 10,000 IOPS with corresponding storage from 100GB to 1TB for MySQL and Oracle databases. If you are using SQL Server then the maximum IOPS you can provision is 7,000 IOPS.

In the near future, we plan to provide you with an automated way to migrate existing database instances to Provisioned IOPS storage for the MySQL and Oracle database engines. If you want to migrate an existing RDS database instance to Provisioned IOPS storage immediately, you can export the data from your existing database instance and import into a new database instance equipped with Provisioned IOPS storage.

Amazon RDS Provisioned IOPS can be used with all RDS features like Multi-AZ, Read Replicas, and Amazon Virtual Private Cloud (VPC), and with all RDS-supported database engines (MySQL, Oracle, and SQL Server). Amazon RDS Provisioned IOPS is immediately available for new database instances in the US East (N. Virginia), US West (N. California), and EU West (Ireland) Regions. We plan to launch in our other AWS Regions in the coming months.

Amazon Announces AWS Storage Gateway

Posted in: Cloud Computing

Today in an email to existing AWS customers, Amazon introduced a service known as AWS Storage Gateway which is an on-premise software appliance allowing customers to easily transfer data from on-premise storage to S3. This service could be used for offsite backup and capacity augmentation among other use cases.

From the email:

We’re excited to introduce the AWS Storage Gateway, a service that provides a new option to securely upload data to the AWS cloud for scalable, reliable, cost-effective storage.

The AWS Storage Gateway connects an on-premises software appliance with cloud-based storage for seamless integration between on-premises IT environments and AWS storage. The service supports a standard iSCSI interface, enabling you to take advantage of cloud based storage without re-architecting existing applications. The AWS Storage Gateway provides low-latency performance by maintaining data in your on-premises storage hardware while asynchronously uploading data over SSL to AWS, where it is encrypted and securely stored in the Amazon Simple Storage Service (Amazon S3).

The AWS Storage Gateway enables you to securely upload your data to the AWS cloud for cost-effective backup, storing point-in-time snapshots of your on-premises application data in Amazon S3 for future recovery. Your data in Amazon S3 is stored as Amazon EBS snapshots, which you can restore on-premises using the AWS Management Console.

The AWS Storage Gateway also makes it easy to leverage the on-demand capacity of Amazon Elastic Compute Cloud (Amazon EC2) for additional capacity during peak periods, as a more cost-effective way to run normal enterprise workloads, or for disaster recovery purposes. You can create Amazon EBS volumes from the snapshots you’ve taken using the AWS Storage Gateway, and attach these volumes to your Amazon EC2 compute instances. Once attached, your Amazon EC2 instances will have access to this data to do any processing or computation.

Pricing for the AWS Storage Gateway is $125/month per activated gateway and comes with a 60 day free trial. Snapshot storage pricing starts at only $0.14 per gigabyte per month.

Here is a video that describes Storage Gateway in more detail.

Degraded Amazon EC2 Instance

Posted in: Cloud Computing

I received an email from Amazon today that I have never seen before. Here it is in its entirety:

Hello,

We have noticed that one or more of your instances is running on a host degraded due to hardware failure.

i-XXXXXXXX

The risk of your instances failing is increased at this point. We cannot determine the health of any applications running on the instances. We recommend that you take appropriate action.

If your instance was launched from an EBS-backed AMI, issuing a stop and start from the AWS Management Console will migrate your instance to new hardware and help avoid any unforeseen downtime.

For more options to stop and start your instance please see:

http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/starting-stopping-instances.html

If your instance was launched from an instance store-backed AMI, you should launch a replacement instance from your most recent AMI and migrate all necessary data to the replacement instance.

Should have you have any additional questions, we offer AWS Basic Support via our Community Forums for free, or Premium Support for one-on-one assistance direct from an AWS Developer Support Engineer at http://aws.amazon.com/support.

Sincerely,

The Amazon EC2 Team

So I dutifully went and followed the instructions and stopped and started (not just rebooted) the specified instance using the EC2 Web Management Console.

PROBLEM: The instance came back up as expected in the Web Management Console, however I could not ping it or SSH to it or connect to it in any way for that matter using my DNS name. I could however connect to it using the Amazon assigned public DNS name. It took me a few minutes to figure it out (all the while my site was down of course), but I eventually noticed that the Elastic IP address assigned to that instance was no longer shown in the instance details view. I went over to the Elastic IP management screen and sure enough that Elastic IP address was shown as not being associated with any instances. I reassigned the Elastic IP address to the instance and a few moments later, everything was back up and running.

CONCLUSION: This scenario is exactly why you need to be using an Elastic Block Storage (EBS) backed EC2 instance for any of your important servers, so in the event that the hardware fails, your actual server image is still safe and can be restored on other hardware. It also proves that while “the cloud” is awesome, it can fail and you need to be prepared for it. Also, one last curious piece about Elastic IP addresses becoming disassociated with instances – not sure if this is related to the hardware failure, or to the stop/start of the instance, but definitely something to keep an eye out for in the future.