Friday, May 15, 2009

Basic Gentoo on Amazon EC2

Continued from Introduction to Amazon EC2 (Continued...)...

If you have come this far then I can assume you have an Amazon account, Firefox, Elasticfox extension on it. If not, well then you should get them to do this yourself or else just read through! Lets open Firefox and from the Tools menu on top you should see Elasticfox. On clicking it you should see a pretty detailed interface and a dialog box asking you to fill in your Amazon credentials. Go ahead and do that. Basically it needs a name (like a nick name, just for you to refer to that account), the Amazon Key, and Secret. The last two are string that you will find here after you login with your AWS account.

Now if that step is done we should be able to select our new Amazon profile name in Elasticfox (at the very top center in the Elasticfox extension's UI). Done. Now you will see many tabs in Elasticfox. They are Instances, Images, KeyPairs, etc. Right now we are only interested in the KeyPairs tab. Click on it.You should see a blank list unless you already have created a key for yourself. Anyways for our purposes we will use the small green icon which says "Create a new keypair". We will need to give a name for this, say "brainless". What happens now is that Amazon creates a keypair for us, and sends the private part of the keypair. If all goes well Elasticfox will let us save the private part of the keypair (named "id-brainless" if we used "brainless" for the name of the keypair).

Now we have a keypair. If you have no idea about this keypair thing, well just do a search on it, like here. Now in Elasticfox you see the tabs again. This time we need the Images tab. Click on the refresh button in the Images section. It will fetch a list of all images available at that moment. These images are basically like Live CD/DVD mediums. You use them to boot your own instance. Now there is plenty of choice here and you can make your own custom image too. You will notice a small search text box near the refresh button. Type in nginx. You should get an AMI with this AMI ID: ami-6138dd08. Notice that there is a Manifest name, which is pretty explanatory for many AMIs. Same for this one. It should say "", which means it comes with nginx, PHP and MySQL setup.

Now right click on that AMI. You will see a dialog box in Firefox. There are quite a few parameters, but we are right now interested in only a few. They are:

  • Instance Type (basically this defines how powerful a server you need, refer here). We choose m1.small here.
  • Minimum number of instances (in case you want more than one server or instance started at once). We keep this as 1.
  • Maximum number of instances (what is the maximum number of instances that Amazon should try to start at once). We keep this as 1.
  • KeyPair (choose the brainless key here, the one we created just above).
  • Just notice that in the Security Group section there is a "default" group in which this new instance will launch. This is the default parameter. We will come to this part soon.

Now click on Launch and pray! Well not really. Unless something is really wrong or Amazon is out of empty instance slots (very unlikely) you will have a shinny new instance all to yourself, with a fresh Gentoo booting on it as you are reading this!

Now notice the very important thing here. You did not do to a website to sign-up for a new server and got it in a few hours or maybe a day. Rather this is all through an API. In our case Elasticfox is consuming the APIs and we are using the GUI of Elasticfox. Later we will see the actual API and deal with it a bit to get an idea. And at an even later point I intend to blog on how to use Tarzan PHP AWS library to write a small scaling platform for your website. Yes auto scaling! Wallah!!!

Lets not get carried away though. So back to our instance. Where is it? Simple, check the Instances tab. You will notice a new row there with a lot of details. The most important one right now is State which I am sure you understand already. Well it simply tells what state the server is in: running, or pending (booting up), shutting-down, terminated (shutdown complete). Refer here.

Thursday, May 14, 2009

Introduction to Amazon EC2 (Continued...)

Continued from Introduction to Amazon Elastic Compute Cloud (Amazon EC2)...

So now that you have seen the feature lists and have some idea, lets start with the actual process. I will intentionally avoid going through details like EC2 Instance Types, Data Center Availabilities. These are relatively simple issues we can ignore now. We will come back to them maybe later. I will take you through a part by part tutorial of setting up a simple Drupal website on a fresh EC2 small instance. We will also setup WordPress on the same setup. We will deal with the full LAMP stack and our CMS apps including:

We will need a simple set of tools to do all this. If you are on Linux or similar you will not need an SSH client. On Windows you can get PuTTY or Bitvise Tunnelier. On Linux or Windows you will need Elastic Fox, which is an extension for Firefox web browser and helps manage EC2 and related stuff. And before you start you will need an Amazon account. You will need a Credit Card but do not worry, you will be paying only for the stuff you need. So if the installation takes 1 hour on a small EC2 then you are going to pay $0.10 for the server and about similar amount for the data transfer. Pretty cheap right? Heh, you could send me a donation if you want to though :P

So start with getting an account from Amazon WS, getting Firefox (if you do not already have it), install Elasticfox, set up the Amazon Access Key and Secret (you will get both of these under Your Account > Access Identifiers in the top navigation menu).

Now before I jump into action (and Amazon starts billing you), let me explain in plain English what I want to do. Also I am clubbing the tasks up so that I can write them over a few blog posts, taking 2-3 days time. The first time I will do Gentoo/nginx/Drupal. We will come to Debian, Apache, WordPress combo later.

  1. Create an SSH key-pair (from within Elasticfox). Create a small EC2 Instance from an existing AMI of Gentoo (using Elasticfox). Set simple Firewall stuff (using Elasticfox). Continue here...
  2. Login to the Instance (server) using the Public Domain name we see in Elasticfox (which is of course given by Amazon). Install any extra software we need using "emerge".
  3. Setup php-cgi in nginx. Get Drupal, setup nginx configuration for Drupal including rewrite rules (convert from the Apache rules supplied). Done!

Wednesday, May 13, 2009

Introduction to Amazon Elastic Compute Cloud (Amazon EC2)

Continued from Dummies Introduction to Amazon Web Services...

Imagine you need a web server. The hardware and the software, all together. You have a website which is growing in users and you want to make sure that the site grows along with the user numbers, that there is no downtime, and that the website stay as fast. You have a few things to do:

  • Optimize the codes that run your website
  • Optimize your software stack
  • Get a better hosting account or a bigger server hardware
  • Get more servers

Its the last two points we are interested in right now. Getting a better hosting account is easy if you have shared hosting account. If you have a dedicated server or a vds (virtual dedicated server) this is still easy task. But you may have to wait each time you want to upgrade your web server because it takes time to setup. Thus you need to plan ahead and take extra hardware so then when there are abrupt website peaks your site does not stop working.

Now since we are discussing EC2s which are like dedicated pieces of hardware we will assume the website does not run on shared hosting. So we are comparing with dedicated hosts or vds. Now getting extra servers is easy to say, but it costs if you want to predict usage (a digg effect, slashdot) and get extra servers. Also how much usage will you get? These maybe difficult numebrs to predict. It would be a lot easier if you could get servers when you need then, no hassle of installing you full software stack, and get it ready in a few minutes. Seems like the heaven of web serving? Well that is what EC2s are all about.

To start on what EC2s do in detail and how to use them from PHP (and other languages) its good to first read on Amazon's details about them (taken from their EC2 page, I was tired :P).

To use Amazon EC2, you simply:

  • Create an Amazon Machine Image (AMI) containing your applications, libraries, data and associated configuration settings. Or use pre-configured, templated images to get up and running immediately.
  • Upload the AMI into Amazon S3. Amazon EC2 provides tools that make storing the AMI simple. Amazon S3 provides a safe, reliable and fast repository to store your images.
  • Use Amazon EC2 web service to configure security and network access.
  • Choose which instance type(s) and operating system you want, then start, terminate, and monitor as many instances of your AMI as needed, using the web service APIs or the variety of management tools provided.
  • Determine whether you want to run in multiple locations, utilize static IP endpoints, or attach persistent block storage to your instances.
  • Pay only for the resources that you actually consume, like instance-hours or data transfer.

You will find those and other details on this page

Dummies Introduction to Amazon Web Services

Lot of you may have heard of this. Well certainly you have heard of and if you are into software/IT/web there is a good chance someone in your company is thinking of that mumbo jambo called cloud computing. Well what the hell is all this and then this web services?

Note: most of what I do is related to running websites. So whatever I write here will also relate to that. Beyond that I don't know nothing :|

Cloud Computing Well there is this thing called a computer. You know it right. And you know that you need them to run your websites or web apps or what else you call them. So what choices do you have? Well here are some:
  1. you can take a shared hosting account
  2. can take a dedicated server
  3. you can take a hosted option to run your website
  4. host the website from your own home/office
  5. get virtual dedicated server (its like a small slice out of a server, each slice is independent of the other at the OS layer) (short: vds)
  6. or you can take a cloud service

So you can assume the cloud service is maybe not radically different from the other options. I mean it still has to run your site or app. It just differs by the way you get the service, manage it, use it. Lets understand an example:

I need to host this blog.

I can take a shared hosting account which will allow me to run a site on a server that is being shared among many other users. Each user is running from his/her own folder. Each having maybe a few MySQL or similar database server users. Some hosts will allow to use a custom .htaccess file in your folder to use custom re-write rules etc. under Apache.

Now that seems a headache for a site which is just running a blog? Well take an account from or Blogger, LJ, etc. These are the hosted options.

I could also take a full dedicated server or a virtual server if I need, but I will to maintain it myself. Also getting a fully dedicated server setup takes sometime (1 day or more). The last option is that I take a cloud service. What I get is an independent slice like in a virtual dedicated host. But its more. A basic cloud service should give me access to the service fast. So a new slice fitted with 1GB or RAM/X Ghz of processor etc. should be up in just a few minutes maybe. This is same in vds. The most important part: all the setup, settings should be available through APIs. What was that? Well I should have a simple protocol (mostly http based) which I or my custom code or some app can access to create a new slice, or get a bigger slice if available, check status, and maybe other stuff. That is the most important part. The ability for software itself to create its own environment to run on! Confused? Even I am :P ... well don't worry, I will explain.

Now there are many examples of cloud services, but I have (sadly) enough exposure in just one: Amazon Web Services. So lets dig into it.

Amazon Web Services

Main website

Services listing:

Amazon Elastic Compute Cloud (Amazon EC2):

This is our primary service and we will inspect this first. In brief this is the service that allows you to get a server (hardwarde + software full stack as needed) up and running in little time, assign an IP address to it, set firewall and invite users to the website. Well except for the invite part the rest are all provided by Amazon and over an API. So my PHP code coulf do that! Of course I will be billed for it.

Amazon Simple Storage Service (Amazon S3):

Just store whatever you needed, setup access rules, and forget about it. Simple!

Amazon CloudFront:

This of these as the equivalent to you local post offices. So you do not need to go to the city central post office to fetch something. Local distribution of data! Its fast way to get things to the audience.

Well there are other services but I am not as good with them. When I experiment/use them enough I will blog on them.

I am back!

It's been a long while since I got lost. Well personal, professional and all other kinds o nals. But its good to be back. Have been doing a lot of Amazon WS these days, so will blog about them. Think I will start a series on Amazon WS with PHP. Oh ! and we shifted to Gentoo / nginx. Good combo. Will blog on that too.