Customer Experiences Adopting Private Cloud Building Elastic, Scalable Applications on Virtualized Infrastructures

Snehal S. Antani

IBM Private Cloud Strategy [email protected] http://snehalantani.googlepages.com

1

Post-Virtualization Band-aid to improve CPU utilization Observation: after applying virtualization, customers are not achieving the CPU utilization promised because they are quickly running out of memory (RAM) Cause: • • • •

1 GB of app server JVM Heap = 1.4 GB of OS memory (due to OS overhead, varies per OS) Application server best practices recommends JVM min & max heap size to be the same (Min=Max= 1GB) To avoid memory page thrashing by the OS, performance best practices recommends application servers not run on virtualized or overcommitted memory (except on System z) example: 700 Application Server JVM’s each with 1 GB heap requires ~1 TB of dedicated RAM!

Solution: •

Apply Application Virtualization technology (WebSphere Virtual Enterprise) • Automatically start/stop JVM’s based on application demand and priority • Reduces the average # of running JVM’s

•

Apply Elastic Caching technology (WebSphere eXtreme Scale) • Cached application data (using EHCache, Hibernate, etc) is stored in a global cache • HTTP session data is stored in a global cache • Reduces the average JVM size • Global cache can be stored on hardware optimized for memory-intensive workloads (WebSphere XC10, nonvirtualized x86, etc)

Example results: • •

Customer reduce 700 running JVM’s to 400 on average; average JVM size reduced from 1GB to 750 MB Total required RAM reduced from 1TB to 420 MB, CPU utilization increased significantly 2

Understanding Application Virtualization

3

Application Virtualization - Dynamic, policy-based middleware systems - Provides dynamic infrastructure services for WAS and non-WAS (Tomcat, WebLogic, etc) across VMware (linux, windows), Sun zones (solaris), PowerVM (AIX), zVM (z/Linux, z/OS) - Dynamically obtain & release server resources to meet demands - Determine request priorities based on business objectives - Monitor the health of the application infrastructure -

Route requests away from sick application servers,

-

Take actions to repair unhealthy servers

- Driven by policies, reducing admin intervention Classification Policies aligned with business objectives

Policies to manage server & application health

Performance Metrics

Intelligent Routing Tier

HTTP Traffic Web Server Tier

• Request Flow Control • Cluster/Server Optimization Instructions

Pool of Application Servers (WAS, Non-WAS)

4

WebSphere Virtual Enterprise optimizes the middleware system with dynamic clusters - Applications are installed to a Dynamic Cluster - ODR uses policies to determine when to start/stop dynamic cluster members - Dynamic Cluster member = App Server JVM

Frame 1

Frame 2

- You can convert your existing WAS clusters into dynamic clusters App 1 Dynamic Cluster Min = 1, Max=2

HTTP Traffic Web Server Tier

= App Server JVM

On-Demand Router (ODR)

App 2 Dynamic Cluster Min = 1, Max=2 5

Understanding Elastic Caching via Data Grids

6

Architecture Web Server Tier

Application Server Tier

Elastic Data Grid

A

D’

B

C’

C

B’

D

A’

Database Tier

7

WebSphere eXtreme Scale – A Data Grid Application Server WXS Data Grid Client

App BankRecordGrid. get(0123456789);

(Access Services)

Get record 0123456789 from the Bank Record data grid

Bank Record Data Partition Http Session Data Partition Health Record Data Partition WebSphere eXtreme Scale Data Grid (Placement, Replication, Elasticity Services)

JVM

JVM

JVM 8

Reducing the Average JVM footprint with a global cache • Reduce the JVM memory footprint with smarter caching

• Data stored as key-value pairs. (Think: hashmap if ‘infinite’ size) • Can be horizontally partitioned – list-based, hash-based, range-banged partitioning schemes can be applied to the data

• Often transient or referential data: – HTTP sessions, user profile, – Read-only or read-mostly – Can tolerate some staleness Large app JVM’s with duplicate cached data

Smaller app JVM’s that connect to a fault-tolerant, centralized distributed cache

JVM 1

JVM 2

JVM 3

JVM 4

App

App

App

App

Cache

Cache

Cache

Cache

JVM 1

JVM 2

App

App

JVM 3

App

WebSphere eXtreme Scale cache

JVM 4

App

9

Global data cache enables Data Virtualization across the Enterprise & Clouds • Expose cached data to all applications running within the cloud • Improve fault-tolerance, performance, and reduce the application memory footprint • Offer simple access mechanisms to data via:

– Map-based query interface – Entity Manager query interface – ESB Mediations

Cluster

Cluster

App 2

App 3

Cluster

Cluster

App 1

App 4

WebSphere eXtreme Scale Fault-Tolerant, Elastic Data Store

ESB App 5 Cluster

DB

DB

DB 10

The cache spans datacenters for higher availability Datacenter 2 AS WXS WXS AS datacenter 1

Multi-datacenter replication WXS

SoR

AS

datacenter 3

1. Store data in a geographically distributed cache 2. Cache offloads data from the back-end 3. Replication in the cache keeps the back-end from getting overloaded in case of failures 4. Replicated cache improves application availability

11

Combining Application Virtualization & Elastic Caching

12

Web Application Challenges

1. Web Application Infrastructures don’t scale well

2. New application infrastructure designs enable scaling to tolerate huge transaction volumes 1.

Think: Facebook, Google, Amazon, Ebay

3. WebSphere eXtreme Scale delivers technology to build highly scalable web application infrastructures

13

Traditional Application Infrastructures Add items to MY shopping cart

Route Request to JVM that holds MY shopping cart

Add item to a shopping cart My Shopping Cart Contents

Web Server

Application

Application

Conversational State

Conversational State

App Server

App Server

- Business Transactions and Conversational State are located in the same JVM - Conversational State (HTTP Sessions) require session affinity & special failover - App Server required complex HA and failover - As a result, scalability was limited 14

Scalable Web Application Infrastructures Add items to MY shopping cart Simple round-robin load balancing

Add item to a shopping cart Get My Shopping Cart Contents

Web Server

Application

Application

App Server

App Server

Conversational State WebSphere eXtreme Scale Data Grid JVM

JVM

JVM

JVM

JVM

JVM

- Business Transactions and Conversational State are in separate tiers - Business Transactions are stateless - Conversational State is fault tolerant - Each tier can be scaled independently 15

Next-Generation Application Infrastructures

SLA-Driven Load Balancing

SLA’s If URL = “/checkout” • priority = 1, Response time < 250 ms If URL = “/catalog” • priority = 3, Response time < 500 ms

On-Demand On-DemandRouter Router

Health Policies Memory leak, Storm-drain, Excessive Time out, Custom defined

Dynamic Cluster

Application

.....

Application

Cache Policies Synchronous Replication, Async Replication

App Server

App Server

Conversational State WebSphere eXtreme Scale Data Grid JVM

JVM

JVM

JVM

JVM

JVM

- On-demand Router auto-scales the application to meet service-level agreements - Web sessions (conversational state) are fault-tolerant and linearly scalable - Applications & Conversational State are resilient, self-healing, and span datacenters - Significantly increases the “density” of the system (increasing CPU utilization) - No Application Code changes necessary 16

IBM’s private cloud foundation addresses the needs of the most diverse enterprises Hybrid Cloud Service Automation, Provisioning, & Monitoring (IBM Service Delivery Manager)

WebSphere Application Server Dynamic Infrastructure Services (WebSphere Virtual Enterprise)

Oracle Weblogic

Integration (Patterns deployed to IBM SmartCloud)

Apache Tomcat

JBoss, .Net, etc

HTTP Session Cache (WebSphere eXtreme Scale, WebSphere XC10) VMWare (Linux, Win)

KVM (Linux, Win)

PowerVM (AIX)

z/VM (z/Linux, z/OS)

•

Elastic application runtimes

•

Application health management & self-healing

•

Elastic data services

•

Continuously available applications

•

Pattern-based middleware

•

Heterogeneous platform management

•

Dynamic workload management

•

Optimized workload placement

•

Fine-grained SLA enforcement

•

Service automation & governance

•

Fine-grained chargeback 17

Questions?

Snehal S. Antani

IBM Private Cloud Strategy [email protected] http://snehalantani.googlepages.com

18