Author Archives: admin

Introducing CUPS

CUP – Cost of Unit Process, CUP is the cost in USD to get 1 CMIP for a given instance.

CUP is get by diving cost of the instance per-hour-running by the number of CMIPS that the instance achieved in the tests.

So we get the next table, sorted from cheapest to expensive:

Instance name CMIPS Price/CMIP
Linode 1x priority (smallest) 1427 0.0000194658568870202
Linode 2x priority 2556 0.000021735350373848
OVH Server EG 24G 3881 0.000035429
Digital Ocean 48GB RAM – 16 Cores – 480 GB SSD 6172 0.0001142255
Digital Ocean 64GB RAM – 20 Cores – 640 GB SSD 8116 0.0001159438
Amazon CC2 Cluster Compute 16608 0.0001445087
Digital Ocean 96GB RAM – 24 Cores – 960 GB SSD 9733 0.0001449707
Amazon C1 High CPU Extra Large 3101 0.0001870364
Amazon Memory Optimized CR1 Cluster 8xlarge 16468 0.0002125334
Amazon Second Generation M3 Extra Large 2065 0,0002421308
Amazon M1 Extra Large 1635 0.000293578
Amazon M1 Large 817 0.0002937576
Amazon M1 Small 203 0.0002955665
Amazon M2 High Memory Quadruple Extra Large 4281 0.0003830881
Amazon T1 Micro 49 0.0004081633
RackSpace First Generation 30 GB RAM – 8 Cores – 1200 GB 4539 0.0004362194
Amazon High I/O Quadruple Extra Large 6263 0.0004949705

And the graphic, sorted from expensive to cheap.

Orange bars reflect CUP (multiplied by 10M to fit well in the graph), and blue bars show MIPS achieved by the instance.

cmips-cups-2013-09-21-cost-unit-power

Desktop: Intel i7-4770S and Laptop: Intel SU4100 have no CUP associated as are included in the graph only to compare CPU performance.

So, having also all the other parameters in mind (bandwidth per instance, cost per exceeded Gigabyte, storage, speed of the storage, time to launch a new instance, if provisioning is paid by hour, by day or by month, etc…) most Startups that use webservers and the Cloud to scale up Arrays of instances to deal with peaks at highly traffic hours, and scale down the number of instances at valley hours or when number or users/traffic decreases, will consider using the instances with lower CUP, or find a balance between instance with enough performance and low Cost of Unit Process.

Updated CMIPS V1.0.3 Stats with Cluster Amazon Instances

Update: Article updated and more data provided here.

I update the results, adding newly introduced Amazon Cluster instances on January 2013:

  • Compute Optimized CC2 Cluster 8xlarge  – cc2.8xlarge – Available in US East (North Virginia) – with 88 ECUS (Amazon computing unit) and 16 cores (but 32 cores detected by htop), 60 GB RAM, featuring an amazing 16,608 CMIPS and taking 60.21 seconds to complete the tests
  • Memory Optimized CR1 Cluster 8xlarge – cr1.8xlarge – with 88 ECUS also (32 cores shown by htop), 244 GB RAM, and featuring 16,468 CMIPS despite having the same CPU Intel Xeon E5-2670 at 2.6 Ghz, taking 60,721 seconds, and equipped with “local” SSD disks. I’m taking complete tests on disk performance that will be added soon

Stats have been updated to show (public today) price per hour and per month of the instances.

cmips-2013-09-21-amazon-cluster-linode-rackspace-first-gen

Detailed results:

Type of Service Provider Name of the product Codename Zone Processor Ghz Processor Cores (from htop) RAM (GB) Os tested CMIPS Execution time (seconds) USD /hour USD /month
Cloud Amazon T1 Micro t1.micro US East Intel Xeon E5-2650 2 1 0.613 Ubuntu Server 13.04 64 bits 49 20,036.7 $0.02 $14.40
Cloud Amazon M1 Small m1.small US East Intel Xeon E5-2650 2 1 1.6 Ubuntu Server 13.04 64 bits 203 4,909.89 $0.06 $43.20
Physical (laptop) Intel SU4100 1.4 2 4 Ubuntu Desktop 12.04 64 bits 460 2,170.32
Cloud Amazon M1 Large m1.large US East Intel Xeon E5-2650 2 2 7.5 Ubuntu 13.04 64 bits 817 1,223.67 $0.24 $172.80
Cloud Linode 1x priority (smallest) 1x priority London Intel Xeon E5-2670 2.6 8 1 Ubuntu Server 12.04 64 bits 1,427 700.348 n/a $20
Cloud Amazon M1 Extra Large m1.xlarge US East Intel Xeon E5-2650 2 4 15 Ubuntu 13.04 64 bits 1,635 606.6 $0.48 $345.60
Cloud Amazon M3 Extra Large m3.xlarge US East Intel Xeon E5-2670 2.6 4 15 Ubuntu 13.04 64 bits 2,065 484.1 $0.50 $360
Cloud Linode 2x priority 2x priority Dallas, Texas, US Intel Xeon E5-2670 2.6 2 Ubuntu Server 12.04 64 bits 2,556 391.19 n/a $40
Cloud Amazon C1 High CPU Extra Large c1.xlarge US East Intel Xeon E5506 2.13 8 7 Ubuntu Server 13.04 64 bits 3,101 322.39 $0.58 $417.60
Dedicated OVH Server EG 24G EG 24G France Intel Xeon W3530 2.8 8 24 Ubuntu Server 13.04 64 bits 3,881 257.01 n/a $99
Cloud Amazon M2 High Memory Quadruple Extra Large m2.4xlarge US East Intel Xeon E5-2665 2.4 8 68.4 Ubuntu Server 13.04 64 bits 4,281 233.545 $1.64 $1,180.80
Cloud Rackspace RackSpace First Generation 30 GB RAM – 8 Cores – 1200 GB US Quad-Core AMD Opteron(tm) Processor 2374 HE 2.2 8 30 Ubuntu Server 12.04 64 bits 4,539 220.89 $1.98 $1,425.60
Physical (desktop workstation) Intel Core i7-4770S 3.1 (to 3.9 with turbo) 8 32 Ubuntu Desktop 13.04 64 bits 5,842 171.56
Cloud Digital Ocean Digital Ocean 48GB RAM – 16 Cores – 480 GB SSD Amsterdam 1 QEMU Virtual CPU version 1.0 16 48 Ubuntu Server 13.04 64 bits 6,172 161.996 $0.705 $480
Cloud Amazon High I/O Quadruple Extra Large hi1.4xlarge US East Intel Xeon E5620 2.4 16 60.5 Ubuntu Server 13.04 64 bits 6,263 159.65 $3.1 $2,232
Cloud Digital Ocean Digital Ocean 64GB RAM – 20 Cores – 640 GB SSD Amsterdam 1 QEMU Virtual CPU version 1.0 20 64 Ubuntu Server 13.04 64 bits 8,116 123.2 $0.941 $640
Cloud Digital Ocean Digital Ocean 96GB RAM – 24 Cores – 960 GB SSD New York 2 QEMU Virtual CPU version 1.0 24 96 Ubuntu Server 13.04 64 bits 9,733 102.743 $1.411 $960
Cloud Amazon Memory Optimized CR1 Cluster 8xlarge cr1.8xlarge US East Intel Xeon E5-2670 2.6 32 244 Ubuntu Server 13.04 64 bits for HVM instances (Cluster) 16,468 60.721 $3.5 $2,520
Cloud Amazon Compute Optimized CC2 Cluster 8xlarge cc2.8xlarge US East Intel Xeon E5-2670 2.6 32 60.5 Ubuntu Server 13.04 64 bits for HVM instances (Cluster) 16,608 60.21 $2.4 $1,728

 

Updated Stats with RackSpace (first gen), Linode, Digital Ocean, Core I7-4770S

Update: Article updated and more data provided here.

New CMIPS v. 1.0.3 measurements with:

  • RackSpace First Generation instance. 30 GB RAM – 8 Cores – 1200 GB Disk with a Processor Quad-Core AMD Opteron(tm) Processor 2374 HE and performing 4,539 CMIPS, and taking 220.289 seconds to complete the test
  • Digital Ocean 96GB RAM – 24 Cores – 960 GB SSD with unknown processor due to virtualization layer (identified as QEMU Virtual CPU version 1.0), performing 9,733 CMIPS, taking only 102.743 seconds
  • Other Digital Ocean medium instances
  • Linode, the smallest one with an Intel Xeon E5-2670 at 2.6 Ghz, 1,427 CMIPS and 700.34 seconds to complete tests
  • Linode, 2x priority with same CPU and 2,556 CMIPS, 391.18 seconds
  • A new Intel Desktop CPU, Core i7-4770S, performing an astonishing 5,842 CMIPS, 171.56 seconds to complete the tests (is my work’s computer)

More CMIPS means more performance, and time is the number of seconds needed to do the tests, so less is better.

cmips-compare-linode-digitalocean-rackspace-amazon-corei7
Detailed data

 

Type of Service Provider Name of the product Codename Zone Processor Ghz Processor Cores (from htop) RAM (GB) Os tested CMIPS Execution time (seconds)
Cloud Amazon T1 Micro t1.micro US East Intel Xeon E5-2650 2 1 0.613 Ubuntu Server 13.04 64 bits 49 20,036.7
Cloud Amazon M1 Small m1.small US East Intel Xeon E5-2650 2 1 1.6 Ubuntu Server 13.04 64 bits 203 4,909.89
Physical (laptop) Intel SU4100 1.4 2 4 Ubuntu Desktop 12.04 64 bits 460 2,170.32
Cloud Amazon M1 Large m1.large US East Intel Xeon E5-2650 2 2 7.5 Ubuntu 13.04 64 bits 817 1,223.67
Cloud Linode Smallest London Intel Xeon E5-2670 2.6 2 Ubuntu Server 12.04 64 bits 1,427 700.348
Cloud Amazon M1 Extra Large m1.xlarge US East Intel Xeon E5-2650 2 4 15 Ubuntu 13.04 64 bits 1,635 606.6
Cloud Amazon M3 Extra Large m3.xlarge US East Intel Xeon E5-2670 2.6 4 15 Ubuntu 13.04 64 bits 2,065 484.1
Cloud Linode 2x priority 2x priority Dallas, Texas, US Intel Xeon E5-2670 2.6 2 Ubuntu Server 12.04 64 bits 2,556 391.19
Cloud Amazon C1 High CPU Extra Large c1.xlarge US East Intel Xeon E5506 2.13 8 7 Ubuntu Server 13.04 64 bits 3,101 322.39
Dedicated OVH Server EG 24G EG 24G France Intel Xeon W3530 2.8 8 24 Ubuntu Server 13.04 64 bits 3,881 257.01
Cloud Amazon M2 High Memory Quadruple Extra Large m2.4xlarge US East Intel Xeon E5-2665 2.4 8 68.4 Ubuntu Server 13.04 64 bits 4,281 233.545
Cloud Rackspace RackSpace First Generation 30 GB RAM – 8 Cores – 1200 GB US Quad-Core AMD Opteron(tm) Processor 2374 HE 2.2 8 30 Ubuntu Server 12.04 64 bits 4,539 220.89
Physical (desktop workstation) Intel Core i7-4770S 3.1 (to 3.9 with turbo) 8 32 Ubuntu Desktop 13.04 64 bits 5,842 171.56
Cloud Digital Ocean Digital Ocean 48GB RAM – 16 Cores – 480 GB SSD Amsterdam 1 QEMU Virtual CPU version 1.0 16 48 Ubuntu Server 13.04 64 bits 6,172 161.996
Cloud Amazon High I/O Quadruple Extra Large hi1.4xlarge US East Intel Xeon E5620 2.4 16 60.5 Ubuntu Server 13.04 64 bits 6,263 159.65
Cloud Digital Ocean Digital Ocean 64GB RAM – 20 Cores – 640 GB SSD Amsterdam 1 QEMU Virtual CPU version 1.0 20 64 Ubuntu Server 13.04 64 bits 8,116 123.2
Cloud Digital Ocean Digital Ocean 96GB RAM – 24 Cores – 960 GB SSD New York 2 QEMU Virtual CPU version 1.0 24 96 Ubuntu Server 13.04 64 bits 9,733 102.743

 

How measurements are done

To measure the performance of an instance or a server CPU, I do several things.

After I create the instance I block the Firewall, so no one trying to discover vulnerabilities or playing with other’s servers can deviate the tests by overloading the CPU.

I htop to be sure that the server is between 0% and 2% CPU usage. After I ensured that, I do two tests. One, I note the result, and another a bit later, I note the result and compare to be sure that are accurate and no process caused a deviation.

To see the model of the CPU in the host I run the command:

grep -i --color "model name" /proc/cpuinfo

/proc/cpuinfo of an Intel Core i7-4470S CPU at 3.10GHz - 8 Cores

I note this info with the number of cores provided by htop.

I monitor and have more much info, like the time that take to start an instance, size of the disk, price per hour, position relative in performance to that provider’s offer… but I do not provide all this information by the moment in the web page.

Other cool indicators are speed to write/read to disk, outboud bandwidth pipe allowed per instance size, price per extra Gigabyte, if inbound packets are charged also, speed in connection to the same Lan instances (Gigabit, 10 Gigabit, etcetera), latency to Europe and US…

To register disk write speed I use random data and big files with this dd command:

dd if=/dev/urandom of=cmips-speed-test.000 bs=1024 count=5000000

 

 

First results with cmips v.1.0.3

Update: Article updated and more data provided here.

Here are the first results comparing CPU + Ram performance from different models of Cloud virtual machines, with CMIPS v.1.0.3.
There are also one dedicated server from OVH and a Intel Core SU4100 laptop to provide some data to compare.

cmips-net-cmips1-0-3-ec2-ovh-linode-laptop

Detailed data

 

Type of Service Provider Name of the product Codename Zone Processor Ghz Processor Cores (from htop) RAM (GB) Os tested CMIPS Execution time (seconds)
Cloud Amazon T1 Micro t1.micro US East Intel Xeon E5-2650 2 1 0.613 Ubuntu Server 13.04 64 bits 49 20,036.7
Cloud Amazon M1 Small m1.small US East Intel Xeon E5-2650 2 1 1.6 Ubuntu Server 13.04 64 bits 203 4,909.89
Physical (laptop) Intel SU4100 1.4 2 4 Ubuntu Desktop 12.04 64 bits 460 2,170.32
Cloud Amazon M1 Large m1.large US East Intel Xeon E5-2650 2 2 7.5 Ubuntu 13.04 64 bits 817 1,223.67
Cloud Amazon M1 Extra Large m1.xlarge US East Intel Xeon E5-2650 2 4 15 Ubuntu 13.04 64 bits 1,635 606.6
Cloud Amazon M3 Extra Large m3.xlarge US East Intel Xeon E5-2670 2.6 4 15 Ubuntu 13.04 64 bits 2,065 484.1
Cloud Linode 2x priority 2x priority Dallas, Texas, US Intel Xeon E5-2670 2.6 2 Ubuntu Server 12.04 64 bits 2,556 391.19
Cloud Amazon C1 High CPU Extra Large c1.xlarge US East Intel Xeon E5506 2.13 8 7 Ubuntu Server 13.04 64 bits 3,101 322.39
Dedicated OVH Server EG 24G EG 24G France Intel Xeon W3530 2.8 8 24 Ubuntu Server 13.04 64 bits 3,881 257.01
Cloud Amazon M2 High Memory Quadruple Extra Large m2.4xlarge US East Intel Xeon E5-2665 2.4 8 68.4 Ubuntu Server 13.04 64 bits 4,281 233.545
Cloud Amazon High I/O Quadruple Extra Large hi1.4xlarge US East Intel Xeon E5620 2.4 16 60.5 Ubuntu Server 13.04 64 bits 6,263 159.65

Welcome to cmips

Welcome to CMIPS, Carles MIPS.

CMIPS is a program written in C++ by Carles Mateo with the purpose of sharing the different measurements of performance from Cloud Providers.

I’ve seen great differences in performance, time serving dynamic web pages (mostly PHP), and in price per hour from different Cloud providers, so often I asked myself what real speed one of the Instances/Virtual machines had, what was the best instance size suitable for my needs and what was the cost per unit of power, and what was the best price/performance provider. So I decided to write cmips and run in all the Cloud Providers instance sizes, and in some Physical Dedicated Servers as well to compare performances.

CMIPS is written in C++ and compiled in 64 bits. I’ve also a 32 bit version but I did only for testing Raspberry Pi. And a support for windows also, that I’ve never tested. 🙂

CMIPS is multithread and V.1.0.3 launches 100 threads to get an accurate idea of the servers performance of those servers with so many cores.