Benchmarking Amazon micro instances

Posted on April 11, 2013 by

Amazon t1.micro instances are the most affordable EC2 instance types.  Their distinctive feature is that while allowing short bursts of high CPU power (around double that of a m1.small instance), but very soon throttle the CPU usage significantly.  Therefore they are well suited for uses where there is low constant load, but periodic bursts of high activity, such as web servers for low-volume sites.

However, Amazon does not provide any specification on the exact method of throttling nor how long you can use full CPU utilization before the VM throttles.  I couldn’t find any info on this elsewhere, so I did some testing myself.

I wrote a short program that generates a desired amount of CPU load and measures the amount of CPU time it receives (in generic units).  I ran it on a 64-bit micro instance running Ubuntu Linux in the EU Ireland region for 10 minutes at varying CPU utilization, idling for 10 minutes between each test.  Below are graphs on the allotted CPU time for various load levels.

Micro instance 100% load

Micro instance 100% load

Micro instance 75% load

Micro instance 75% load

Micro instance 50% load

Micro instance 50% load

Micro instance 25% load

Micro instance 25% load

Micro instance 20% load

Micro instance 20% load

Micro instance 15% load

Micro instance 15% load

Micro instance 10% load

Micro instance 10% load

Full load throttling

One interesting question is how the throttling performs when the CPU is fully utilized, which may occur for example during a system upgrade.

In this case the throttling is rather binary; the VM either allows using the entire CPU, or throttles to about 11% of the full capacity.  You get about 10-13 seconds of full CPU power every 2-2.5 minutes, though this duty cycle varies somewhat.

Constant load level

Micro instance 10% / 15% / 20% load

Micro instance 10% / 15% / 20% load

Another interesting matter is how much constant load you can provide a micro instance without it throttling.  This plot shows the results for 10%, 15% and 20% load levels.  From this it’s evident that the 20% load already throttles occasionally, but 15% load does not seem to have significant throttling.

Therefore if your micro instance web server’s load is going above 15%, you’re in a danger zone.  Once the throttling starts, response times will spike and requests may start piling up.

This limit is also evident when summing up the total CPU time obtained during the 10 minute tests.  From 20% load to 100% load the program got approximately the same amount of CPU time during the test, but for 10% and 15% it utilized less processing time than was available from the VM.

Another result of this is that if you have to do long processing on a micro instance, it’s approximately the same whether you hog up all the CPU you get, or slow down the processing yourself.

Conclusions

Micro instances are well suited for uses where there is little constant load.  A constant load level of 15% can be sustained indefinitely without CPU throttling.  However, it’s good to remember that if you’re constantly using all of your allotted CPU time, you may be throttled very quickly in CPU-intensive bursts.

During throttling, the available CPU is between 10-50% of the normal peak power.  The higher the load, the more aggressive the throttling.  With full load, you only get full power for about 10 seconds every 2 minutes, and 11% CPU time otherwise.

For long CPU-intensive tasks (lasting several minutes) it doesn’t matter whether you use all the CPU you get or regulate it, the task will take as long to finish anyhow.

Keep in mind, these are results for a single micro instance type in a single region.  Amazon doesn’t specify that the rules are the same everywhere.  They may even change according to the time of day and other region activity.  YMMV.

Posted in Amazon AWS | Tagged , , | 1 Comment

The Gimp is dead, long live the Gimp!

Posted on January 19, 2013 by

I have been an avid user of the Gimp for well over a decade now. I started using it in the mid 90′s around version 0.60.  I was promoting it to my friends as a great image manipulation tool, and recall laughing at PhotoShop that had only one level of undo at the time.

During that time Gimp evolved fast.  I anticipated every next release, and each one seemed to get closer to the functional level of PhotoShop.  Somewhere around version 1.2 – at least from my perspective – the only major deficiencies were lack of 16-bit color and CMYK support.

But then the development slowed down.  Major releases have been coming out every 1-3 years, but I haven’t noticed much change between them.  16-bit support and CMYK would be supported by GEGL, the do-it-all graphics library of the next-generation Gimp.  GEGL’s first release was in 2000, and Gimp has yet to see 16-bit support.

For many years now I have considered the Gimp to be at a dead end.  I have the utmost respect for the people who have worked – and still work – on it, and I use Gimp regularly.  But it seemed that to pull the next big thing into Gimp was too large an undertaking for a volunteer project.

Gimp 2.8

When I upgraded to Linux Mint and launched Gimp, I was in for a surprise.  After four years of development (think about it – that’s when Obama was first elected) Gimp 2.8 had been released.

My first reaction was that of disgust.  I couldn’t do anything in it.  Saving files was a pain, and even the standard brushes had been removed.  After cursing about it a while, I started looking around for a reason why these stupid changes had been made.

Ctrl+E is the new Ctrl+S

I came across the article GIMP 2.8: understanding UI changes.  I recommend every Gimp user reads it.  It explains the rationale the changes have been made, and even offers a simple “fix” to change the behavior to what it was earlier.  Even though I was looking for such a solution, after reading the article I decided to leave it as it was.

In Gimp 2.8 you essentially can have two “save files” at the same time, one XCF file and one export file (PNG, JPG, etc).  Ctrl+S saves the XCF file, Ctrl+E saves the export file.  Using Ctrl+Shift will prompt for the file (Save as, Export as).  Once you get used to this, it does save a lot of hassle.  You can independently save the original XCF and export a file to be used outside.  When you do changes, you don’t have to worry about remembering to save both the exported and original versions.

Brush on the world

The other thing I was missing was the 1px, 3px, 5px, etc. brushes from the old Gimp versions.  After creating several new brushes for the purpose, I found out that these have been changed to a “Size” parameter in the tool options.  Now all brushes are resizable from one place.

One gripe still is that the size control slider is very hard to use precisely with small brushes.  I often work with brushes below 10px in size, and controlling the size using a linear slider than goes up to 1000px is not very user-friendly.  However, according to the UI change article, the tool options are going to have an overhaul soon as well.

Gimp 2.8 also has a number of other nice changes, such as expressions in entry fields (how often have you whipped open a calculator to divide a size?  Now just type “1234/3″), layer groups, and single window mode with image tabs.

The future

Looking past 2.8, the Gimp core has finally been ported to use GEGL by a hero effort of Øyvind Kolås and Mike Natterer.  This means that after waiting 15 years, Gimp 2.10 will finally have 16-bit color support.  Hooray!

The important question is, when will 2.10 be released?  Martin Nordholts has written about the problems in Gimp development cycles.  He states many of the problems that I have pondered and struggled with during OpenRocket development.  Gimp 2.10 development will utilize more feature branches, which are merged to the master branch only once they’re ready, so as not to hinder the release of other features.  This will hopefully allow more regular feature releases and more motivation for new developers to jump in.

I have yet to explore many of the features in 2.8, but for the first time in a long time I have a heightened sense of excitement about Gimp.  There is finally clear and undeniable improvement happening.

Finally, I have to restate my admiration at the people who have made Gimp a reality, from its early stages and through its troublesome times.  A huge thanks to you all, and hoping for a great 2.10 development cycle.  I am rooting for you once again!

Posted in Gimp, Linux | Tagged , , , | Leave a comment

Switching to Linux Mint

Posted on January 19, 2013 by

This is a description of my migration from Ubuntu to Linux Mint 14.  As this was my work laptop, there are some additional requirements that need to be met by the system.  In particular, I review:

  • Configurability and ease of use
  • Hardware support for a ThinkPad T430s
  • Printer and display configuration
  • Full disk encryption
  • Running Windows in VirtualBox

Continue reading →

Posted in Linux, Linux Mint, VirtualBox | Tagged , , , , , , , , | 1 Comment

No-nonsense ASCII chart

Posted on December 28, 2012 by

There are a lot of ASCII charts floating around the Internet.  However all of them seem to have a few fundamental problems:  They include the whole ASCII set in the chart and cram too much information of each char, making the chart illegible.  What I want is a clean, readable chart that I can glance at from a distance.

Therefore I made my own, the No-nonsense ASCII chart.  It features:

  • the hex code and character, easily legible from two meters away
  • the decimal code, since this is occasionally needed
  • only the control characters I have ever heard about  (NUL excluded – everyone knows that anyway)
  • numbers and letters as a range, not individually printed out
  • no nonsense

The chart is available in PDF and OpenDocument formats sized for A4 and Letter.  It’s licensed as CC-BY.  Customize as you will.

I recommend highlighting the most often used characters with a yellow highlighting pen (for web developers %, &, ? and =).

ASCII chart

Posted in Coding | Tagged | Leave a comment

Android Torture

Posted on December 12, 2012 by

Okay, this is my first rant post.  It’s 5:30 in the morning and I’m writing a blog post thanks to my new-and-shiny Samsung Galaxy S3.

I was recently force-upgraded from my Nokia E7 because its USB-adapter broke off – again.  Long story short, I’m now adapting to life as a new Android user.

One of the most frustrating things immediately visible is how every application on the device believes it’s the center of the universe craving for attention.  Even though I haven’t installed many apps, my notification board is often crammed with stuff.  Right out of the box the phone had started beeping to the beat of incoming email, and it took me quite a while to figure out how to turn it off.

But tonight has been a nightmare.

At around 4 AM the phone bleeped.  I probably hardly stirred.  Then it bleeped again.  And again.  And again.  Slowly but surely it was wrenching me from sleep.

I tried ignoring it.  It was rare, maybe once every 5-10 minutes.  But each time I was just falling asleep, it bleeped again.  And again.  It was the electronic form of water torture.

Finally, after about an hour of futile resistance, I succumbed to the device’s will.  I opened it and found a Facebook notification.  I had just last evening installed the Facebook app, and figured it was causing the havoc – though I have only 1-2 friends abroad that should be awake at this hour.  I browsed through the app settings and turned notifications off, thinking I had slain the monster, and went back to sleep.  And then it bleeped again.

I recalled seeing some options for preventing notifications in the general settings, so I started scanning the menus.  I finally found the Blocking mode, turned it on, and configured it to block all notifications and the LED indicator.  For good measure I set the phone to vibration – I was afraid that muting it would mute the alarm I had set for the morning.  And then it vibrated.

At this point I was ready to drop the phone in a glass of water had it only fit in.

I was unsure was it a fluke.  As time passed I was starting to doubt myself that had I just dreamed it.  I turned sounds back on just in case there was some weird vibration alerts.  Time passed, and I was finally confident that the beast had been silenced and I tried frantically to get back to sleep.  And then it bleeped again.

Had this been my trusty Nokia, I would have turned it off an hour ago, confident in the fact that it will wake me up even if it’s turned off.  Hell, a Nokia phone will sound the alarm even if it has run out of battery during the night!  That’s bloody engineering!

Now I’m up, sure that I won’t get any more sleep tonight and knowing it will be a long day ahead.  I’m already panicking about what will happen tomorrow night, will the torture ensue again.  Maybe I have to go buy an alarm clock…

Post-mortem:

  • I still have no idea what caused the bleeping during the night.  My best guess is that something in the charger or charging logic got screwed up and it starting notifying of a full battery every 5-10 minutes.
  • I have had the phone multiple times emit varying beep noises, without any indication of what caused it.  It has a life of its own.
  • A timed blocking mode seems to prevent unnecessary beeping during the night time.
  • I verified that the Samsung will not wake you up if it’s turned off.  So much for conserving battery when you’re in a tight spot.  Similarly if you forget to plug the phone in when going to sleep and the battery runs out, you’re screwed.
  • I found out that it will sound the alarm even if the device is set to mute.  I’m torn on this one – it would have saved me a night’s sleep, but I’d really want a button that ensures the phone will stay quiet no matter what.
Posted in Android | Tagged , , | Leave a comment

Documenting Cucumber steps

Posted on December 8, 2012 by

One of the great things in Cucumber testing is that as you go along, you start to accumulate ready step definitions, which makes writing new tests even faster.  Several times I’ve actually been surprised that a definition for a step I wrote already existed and worked as I expected it to.

However, I noticed that developers in our project were also writing duplicate step definitions, because they didn’t know the definition already existed with a different name.  We defined some scenario writing guidelines and nomenclature to be used, which helped somewhat.  Still, in the mass of test code it was often hard to find whether the step you needed already existed.

Things like Wally and Relish are great for converting the feature files into pretty documentation, but I couldn’t find anything similar for the step definitions themselves.  So I wrote my own:  cuke-steps

The script parses the step definition files and generates a single page document containing all the step definitions it finds, in a pretty-printed fashion.  By clicking on any definition you can see the file and row number where the definition is and the code for the step.

Originally I wrote the script to create Confluence wiki markup.  We have a Jenkins job run every night that generates the documentation and pushes it into our wiki using the Confluence Publisher Plugin.  I wrote a simple HTML outputter when publishing the script.  You can check out a sample output page generated from Alister Scott’s wmf-custom-page-object.

There’s still some work to be done, like prettifying the HTML output, creating a gem out of it and making it link directly to a source repo.  Any updates are appreciated.

Posted in Cucumber, Testing | Tagged , , | Leave a comment

Dump World on failure

Posted on October 7, 2012 by

One of the most important aspects of test reports is that when something fails, you can easily debug what went wrong.  Splitting tests into logical, small tests that exercise a single functionality in the system goes a long way toward this goal.  Inevitably, you still occasionally get a weird failure where you have no idea what’s going wrong.

In such circumstances it’s vital to have sufficient information included in the test report to understand what happened — otherwise you’re stuck trying to do an autopsy without a corpse.  When running Cucumber tests, we have found that a dump of the World is often extremely useful information.

The following piece of code does just that.  Whenever a Cucumber scenario fails, it dumps the contents of the World to the console:

After do |scenario|
  if scenario.failed?
    Kernel.puts "Scenario failed, world contents:"
    vars = self.instance_variables.select { |v| !v.to_s.include?("__") }
    vars.sort.each do |v|
      value = instance_variable_get(v)
      Kernel.puts "    #{v}=#{value}"
    end
  end
end

This iterates through the variables and writes them out one by one.  The select statement on line 4 filters out some Cucumber-specific variables stored in the World.

You can easily customize this to output elsewhere (we’ve found Kernel.puts to be the most reliable way to get the info out) or how to output certain data types.  For example, we use the rest_client gem extensively, so we’ve customized the hook to detect the response object and to output value.code, value.headers and value.body in that case.

Posted in Coding, Cucumber, Testing | Tagged , , , | 1 Comment