Monthly Archives: December 2013

CMIPS v.1.03 benchmarks updated with CloudSigma and Azure

Here are provided the CMIPS results after the past tests with Microsoft Azure and CloudSigma Cloud.

CMIPS v.1.03 benchmarks* Please note that values published in the graph for CloudSigma’s CMIPS are the max obtained, as results fluctuated during the tests, as indicated in the review. The range is below, on the html table.

*2 Costs for CloudSigma came in EUR, so have been exchanged from EUR to USD using google. Prices show subscription (not burst mode)

And the CUP (Cost per Unit Process), the orange bar.

cmips-cup-2013-12-cloudsigma-aws-rackspace-linode

Detailed results:

Type of Service Provider Name of the product Codename Zone Processor Ghz Processor Cores (from htop) RAM (GB) Os tested CMIPS Execution time (seconds) USD /hour USD /month
Cloud Amazon T1 Micro t1.micro US East Intel Xeon E5-2650 2 1 0.613 Ubuntu Server 13.04 64 bits 49 20,036.7 $0.02 $14.40
Cloud Amazon M1 Small m1.small US East Intel Xeon E5-2650 2 1 1.6 Ubuntu Server 13.04 64 bits 203 4,909.89 $0.06 $43.20
Cloud GoGrid Extra Small (512 MB) Extra Small US-East-1 Intel Xeon E5520 2.27 1 0.5 Ubuntu Server 12.04 64 bits 441 2,265.14 $0.04 $18.13
Physical (laptop) Intel SU4100 1.4 2 4 Ubuntu Desktop 12.04 64 bits 460 2,170.32
Cloud CloudSigma 1 Core / 1 Ghz 1 Core / 1 Ghz Zurich (Europe) Amd Opteron 6380 2.5 Ghz to 3.4 Ghz with Turbo 1 1 Ubuntu Server 12.04.3 64 bits 565 to 440 1,800 $0.04475 $32,22
Cloud Amazon M1 Large m1.large US East Intel Xeon E5-2650 2 2 7.5 Ubuntu 13.04 64 bits 817 1,223.67 $0.24 $172.80
Cloud Linode 1x priority (smallest) 1x priority London Intel Xeon E5-2670 2.6 8 1 Ubuntu Server 12.04 64 bits 1,427 700.348 n/a $20
Cloud Amazon M1 Extra Large m1.xlarge US East Intel Xeon E5-2650 2 4 15 Ubuntu 13.04 64 bits 1,635 606.6 $0.48 $345.60
Cloud CloudSigma 3 Core / 1,667 Ghz each / 5 Ghz Total 3 Core / 1,667 Ghz each / 5 Ghz Total Zurich (Europe) Amd Opteron 6380 2.5 Ghz to 3.4 Ghz with Turbo 3 1 Ubuntu Server 13.10 64 bits 1928 to 1675 518.64 $0.1875 $135
Cloud Amazon M3 Extra Large m3.xlarge US East Intel Xeon E5-2670 2.6 4 15 Ubuntu 13.04 64 bits 2,065 484.1 $0.50 $360
Cloud Linode 2x priority 2x priority Dallas, Texas, US Intel Xeon E5-2670 2.6 2 Ubuntu Server 12.04 64 bits 2,556 391.19 n/a $40
Cloud GoGrid Extra Large (8GB) Extra Large US-East-1 Intel Xeon E5520 2.27 8 8 Ubuntu Server 12.04 64 bits 2,965 327.226 $0.64 $290
Cloud Amazon C1 High CPU Extra Large c1.xlarge US East Intel Xeon E5506 2.13 8 7 Ubuntu Server 13.04 64 bits 3,101 322.39 $0.58 $417.60
Dedicated OVH Server EG 24G EG 24G France Intel Xeon W3530 2.8 8 24 Ubuntu Server 13.04 64 bits 3,881 257.01 n/a $99
Cloud Amazon M2 High Memory Quadruple Extra Large m2.4xlarge US East Intel Xeon E5-2665 2.4 8 68.4 Ubuntu Server 13.04 64 bits 4,281 233.545 $1.64 $1,180.80
Cloud Rackspace RackSpace First Generation 30 GB RAM – 8 Cores – 1200 GB US Quad-Core AMD Opteron(tm) Processor 2374 HE 2.2 8 30 Ubuntu Server 12.04 64 bits 4,539 220.89 $1.98 $1,425.60
Physical (desktop workstation) Intel Core i7-4770S 3.1 (to 3.9 with turbo) 8 32 Ubuntu Desktop 13.04 64 bits 5,842 171.56
Cloud Digital Ocean Digital Ocean 48GB RAM – 16 Cores – 480 GB SSD Amsterdam 1 QEMU Virtual CPU version 1.0 16 48 Ubuntu Server 13.04 64 bits 6,172 161.996 $0.705 $480
Cloud Amazon High I/O Quadruple Extra Large hi1.4xlarge US East Intel Xeon E5620 2.4 16 60.5 Ubuntu Server 13.04 64 bits 6,263 159.65 $3.1 $2,232
Cloud Digital Ocean Digital Ocean 64GB RAM – 20 Cores – 640 GB SSD Amsterdam 1 QEMU Virtual CPU version 1.0 20 64 Ubuntu Server 13.04 64 bits 8,116 123.2 $0.941 $640
Cloud Digital Ocean Digital Ocean 96GB RAM – 24 Cores – 960 GB SSD New York 2 QEMU Virtual CPU version 1.0 24 96 Ubuntu Server 13.04 64 bits 9,733 102.743 $1.411 $960
Cloud GoGrid XXX Large (24GB) XXX Large US-East-1 Intel Xeon X5650 2.67 32 24 Ubuntu Server 12.04 64 bits 10,037 99.6226 $1.92 $870
Cloud CloudSigma 24 Core / 52 Ghz Total 24 Core / 52 Ghz Total Zurich (Europe) Amd Opteron 6380 2.5 Ghz to 3.4 Ghz with Turbo 24 1 Ubuntu Server 13.10 64 bits 10979 to 8530 98 $0.9975 $718.20
Cloud Amazon Memory Optimized CR1 Cluster 8xlarge cr1.8xlarge US East Intel Xeon E5-2670 2.6 32 244 Ubuntu Server 13.04 64 bits for HVM instances (Cluster) 16,468 60.721 $3.5 $2,520
Cloud Amazon Compute Optimized CC2 Cluster 8xlarge cc2.8xlarge US East Intel Xeon E5-2670 2.6 32 60.5 Ubuntu Server 13.04 64 bits for HVM instances (Cluster) 16,608 60.21 $2.4 $1,728
Cloud CloudSigma 37 Core / 2.16 Ghz each / 80 Ghz Total 37 Core / 2.16 Ghz each / 80 Ghz Total Zurich (Europe) Amd Opteron 6380 2.5 Ghz to 3.4 Ghz with Turbo 37 1 Ubuntu Server 13.10 64 bits 17136 to 8539 58 $1.5195 $1,094.10

 

Evaluating Cloud Sigma

CloudSigma has been one of the most difficult-to-test providers to the date for me.

Mainly because they have several differences with the average Cloud provider and the common way to perform of the vast majority of projects I’ve evaluated, and because the user interface is a bit ambiguous.

I’ll explain those differences and how to perform with CloudSigma.

First of all I want to thank to the Company and to the Customer Support that has been very nice.

CloudSigma assigned one of the support representatives to me, and she has been replying all my in deep questions, they provided 200 € in free credit so I can evaluate their platform at full level, they have provided a lot of technical information on their internals and platform, and have forwarded the bugs I’ve found to their Engineers and kept up me up to date on the development cycle and when they were going to fix the bugs I reported.

I have to clarify that I have found bugs in all the Cloud platforms I’ve tested, and in fact that’s one of my skills, finding bugs, that my employers love so much, but the main difference is that CloudSigma has been the only provider that has informed me about when they would fix the bugs and provided details on the Sprint (agile) and on the schedule for the deployment of the fixes.

I’ve evaluated their API and reported doubts and missing info on the documentation, and they have been transparent to reply all my questions and to recognize the points that I reported to improve. So big congratulations to their wonderful team, and for the good attitude.

For the development lovers I would say that their user interface is new, and works with Html5 WebSockets.

CloudSigma allows you to work with subscription and with burst mode. In burst mode you can pre-charge money to your account and work. To start an instance in burst mode you have to have enough credit to run the instance for 5 days.

One of the differences is that you can create an instance without a disk. This is a bit confusing the first time.

You have to create an instance, then assign a disk.

The disk can be assigned from, and that’s very good, one ISO Raw image that you upload.

It can be even created from a Marketplace’s image.

You can clone and use from the Marketplace OpenBSD, RedHat, Slackware, Debian, Ubuntu… pre-installed images (although Ubuntu 13.10 Server was not still in the list, I must say that after the publication of this article it was immediately added. Great!) or you can assign installation CD’s in ISO format and do a clean install. You can also use Windows and Sql Server images.

There are images, in the marketplace, for the drivers.

A positive thing also is that you have all the variety of Linux Desktops. So if you want to install Ubuntu 13.10 Desktop, Debian, or Fedora… you can do it and access through VNC.

In the other hand you can move the main or system disk from one server to another, that’s pretty cool.

The disks are SSD, and that’s very good for performance. You can create a Hard disk from 1 GB to 3 TB SSD.

CloudSigma platform is different from providers that have fixed instance-size, and here you can define how many Ghz (processor frequency) and how many RAM GB you want.

That’s a very interesting feature, as if you want a really powerful instance, but you don’t need much RAM, then you don’t have to pay for it!. Also if you want to create a cluster of Cassandra servers with not much CPU but a lot of RAM you can do it.

You can stop your server and change the amounts later and restart. That easy.

That’s really a very cool provisioning feature.

In terms of Zones CloudSigma has Datacenters in Zurich (Europe) and Las Vegas (USA), they run double 10GigE lines and they offer hybrid private patching (direct cable) that is a direct connection from your co-location servers to the public cloud, eliminating the need of a VPN and offering the possibility to contract advanced tools for security, backups…

As me they believe that hybrid cloud is key.

CloudSigma is a Cloud provider that remembers a lot to a familiar company. They have advanced services, but the relation with the customers is exquisite. They care about the companies being happy there like if it was a very small company. Some of my friend’s Start ups work with CloudSigma and they’re very satisfied.

I found and they confirmed, that are running on latest AMD 6380 processors (16 cores from 2.5 Ghz to 3.4 Ghz with Turbo) and they can offer up to 80 Ghz (80 virtual cores at 1 Ghz) but unless I’m missing something I don’t see the maths working (3.4 * 16 = 54.4 Ghz).

The RAM can be from 1 GB to 128 GB, that’s very good.

Another cool feature is that you can balance the number of cores seen by the guest OS, and play with the Ghz of that cores. So you can have 80 cores at 1 Ghz or 37 cores at 2.16 Ghz.

This is a very cool feature.

However, in the other hand the CPU performance detected by CMIPS is not always the same. It varies a lot from one test to the next and the next.

For a full 80 Ghz (37 cores at 2.16 Ghz each) instance, I got measurements from astonishing 17136 to 8539 CMIPS. That is shocking as I got 10770 to 8530 CMIPS with a 52 Ghz (24 virtual cores) instance.

I believe that this is related to AMD power-consumption features, and not related to CloudSigma itself, but right now I’m not sure and I would need a lot more testing to be.

Please note 17136 is the highest score achieved, even better than the powerful Amazon Compute Optimized CC2 Cluster 8xlarge.

In CloudSigma you can also use all the available bandwidth. Your instances are only limited by the ability of the server to handle network packets. So you don’t have a limit like in the case of other Cloud providers capping the outbound to 150 Mbits or more depending on the instance size.

Launching instances with CloudSigma is really really fast. The drive images are cloned at light speed (10 GB 3 seconds), and the servers are created in seconds and started even faster. Unlikely other Cloud providers the servers are created stopped.

Another cool feature is the incorporated costs estimation:

cmips-cloudsigma-estimated-costs-over-timeThe image shows the price for a 80 Ghz instance, with 1 GB of RAM.

As you can see prices per subscription are cheaper than with the burst mode.

In fact, CloudSigma has the most clear and predictable pricing model. You can check everything from the profile:

cmips-cloudsigma-pricing

Using a pre-installed image has the advantage that all the drivers are pre-installed.

Cloud Sigma Properties of a CMIPS Instance Another confusing option is the public key option, that you can define when you create your instance or modify it later. As I’ve been using images from the Marketplace, that are user (cloudsigma) and password based, not certificate-based, it seems like this is not an useful option for this scenario and is really confusing.

cmips-cloudsigma-properties-tab2

You can access their API from here: https://zrh.cloudsigma.com/docs/

Something that I dislike from the user interface is that the STOP button has no warning. You press STOP and you immediately stop the instance (like power off), and shutdown has a 90 seconds count down.

After you have created an instance, assigned the disk, and started it, you have to:

cmips-cloudsigma-actions-open-vnc

– Edit the instance Properties

– Go to menu Actions

– Select Open VNC

This opens a set of options at the bottom of the same section

– Log in via SSH or via the embedded VNC client (very nice)

 

 

CloudSigma open VNC in a new browser windowThe VNC client works well, unfortunately the key map from the keyboard is different, so I am not able to use certain characters like # or | that are generated with AltGr.

This is a problem for the default password of Ubuntu 12.04 Server, that user a #.

Specially painful with the laptop.

But you can access via ssh instead, and the keyboard mapping is right.

Then you must change the password for the user cloudsigma, via ssh it kicks you out after, and you must relogin with the new password.

The network polices (firewall) can be applied to several servers, but if they’re changed the new ones don’t apply until restart.

Fortunately you can access through the VNC web client (QEMU) even if you messed it up and blocked all the firewall ports.

You can see in real time your bandwidth consumption.

cmips-cloudsigma-outgoing-bandwidth-usage