AJAX Application Server Performance Clinton W. Smullen III
Stephanie A. Smullen The University of Tennessee at Chattanooga
615 McCallie Avenue, Chattanooga, TN 37415 U.S.A.
Contents • AJAX • Application used in study – HTML app, Ajax app
• Server Performance comparison • Stochastic effects • Conclusions
AJAX Asynchronous Java And XML Designed to improve web application responsiveness provide desktop-like user interfaces reduce network overhead reduce server loads
AJAX A set of technologies: HTML/XHTML, CSS, Dynamic HTML,
Client scripting in Java or JavaScript, Document object model,
XML data exchange, Synchronous or asynchronous data retrieval using XMLHttpRequest objects
AJAX Commercial applications include Google Suggest, Maps, Gmail Yahoo Flickr Backbase RSS reader and Portal Writely Kiko and many, many more
See http://adaptivepath.com/
│Ref: │”Modelling AJAX Application │ Performance” │Web Technologies and Services, 2006 │ │This Study │ │
AJAX Performance Study White Dec 2005 • Parallel HTML and AJAX applications • 5 users each performed 3 tasks
Bytes transferred Time (secs) Est transmit time to W Coast
Performance Increase (%)
HTML (Avg)
Ajax (Avg)
1,737,607
460,799
73%
115
78
32%
293.45
94.44
68%
White’s Performance Study • Small sample size • Two systems did not implement same user interface
• Large savings from use of long link
Our study application • Existing production application – Used daily by students and staff
• Supplies real-time class data from SIS – Not “test” data
• Based on – Apache web server – PHP 5.05 and custom database code – All pages are validated XHTML 1.1
The HTML application • Significant “branding” overhead • Uses common elements – Navigation elements, CSS, graphics – Linked to HTML page
• Common elements total 15.2K bytes • HTML query form totals 27K bytes
The HTML application Typical use: Client loads HTML common elements Client loads HTML query page User enters query, submits to server Server queries SIS SIS produces XML data Server reads XML data, applies XSLT to produce HTML Server returns HTML to client
The HTML application HTML page returned • Links to common elements • Contains the query form • Is a complete XHTML page A query with no results still returns 27KB
AJAX Application • Creates same “look and feel” – Uses same common elements as before – Uses only synchronous requests
• Appears to user as the same • JavaScript runs user interface – Implemented with Sarissa open-source library – AJAX code size 57K bytes
The AJAX application Typical use: Client loads HTML common elements Client loads JavaScript and HTML User enters query, submits JavaScript sends XMLHttpRequest Server queries SIS SIS produces XML data Server sends XML data to client JavaScript displays results as HTML
The AJAX application • Uses the same server process as HTML application – No modifications were made to server end – Server responds with XML data
• Caused “extra” overhead bytes – Needed by HTML but not by AJAX code – 7140 bytes extra returned to AJAX code – Eliminating this studied in “Modelling AJAX…”
Response Sizes •
Initial HTML load is 42.2K bytes 15.2K common elements 27K HTML query form
•
Each response is complete HTML page
•
Initial AJAX load is 99.2K bytes 15.2K common elements 27K query form data 57K JavaScript code
•
Each response is XML data
Data Collection 13,260 queries posed to each system • Generated by Perl scripts – User-agent field tagged • Measured response bytes, service time – From Apache server logs – This study ignores common elements – See “Modelling AJAX…” for client-side analysis
Data Collection • Number of bytes returned measures network impact of the application – Affects responsiveness of system to a user – Especially over dial-up connections
• Response bytes and service time measures impact on servers – Affects responsiveness of system to a user
Data Collection • Data collected querying production server – HTML averaged 180 page-views/hour – HTML peak was 676 page-views/hour
• Querying times spread out over 3 months – Daytimes, evenings, nighttimes – Weekdays, weekends, holidays
• Clients spread out – On campus, off-campus – Windows, Linux, MacOS
HTML vs AJAX Bytes Summary of 13,260 queries to each Response bytes
HTML
AJAX
Min
27,650
7,746
Max
2,017,914
1,196,618
Mean
129,828
68,234
Median
63,996
28,996
Total
1,721,523,273
904,776,578
HTML vs AJAX Responses • Bytes returned value is deterministic – Does not vary with load on servers
• Seconds to process request is stochastic – Recorded in microsecs from Apache logs – Wide variations observed due to exogenous load on servers
HTML vs AJAX Seconds Summary of 13,260 queries to each Response seconds
HTML
AJAX
Min
0.308
0.254
Max
37.584
34.421
Mean
1.527
1.289
Median
0.887
0.731
Total
20,252.32
17,090.15
Service Times • Service times are not statistically linearly related to response bytes – Additional processing performed on servers – Can’t model as a “slow pipe” between SIS and web server
Service Times Effect of HTTP protocol type • Group of queries used HTTP/1.0 – One query and response per connection
• Group of queries used HTTP/1.1 with “keepalive” – Several query/response pairs per connection
No statistically significant differences between service times for these groups – Time to generate data dominates service time
Service Times • Service times vary due to contention for server resources – From other users of production servers
• Random nature of poor response times • Hence represents “real-life” responses
Service Time Variations • 110 trials of an AJAX query – “List all courses” query – Returns 1,196,681 bytes
• Response times for 110 trials Mean
12.69
Std deviation
2.59
Min
10.62
Max
31.02
Response Time (sec) for 110 Trials
Service Time Variations • Most queries received reasonable service • A few received very bad service • Seemingly random – Not repeatable – Not predictable
• No statistical correlation between poor HTML service and poor AJAX service • No statistical correlation between size of response and poor service
Service Time Variations • Large, random variations in service time • To reduce the effects of these random variations: – Identified a set of 120 queries – Repeated this set 110 times for each system – 110 * 120 = 13200 – 13,200 for HTML and 13,200 for AJAX
– Additional 60 queries for each makes 13,260
Service Time Summaries Listed in groups • “ALL” group includes all trials • “99%” group excludes the 1% of trials that received the longest service • “95%” group excludes the 5% of trials that received the longest service • “90%” group excludes the 10% of trials that received the longest service
Service Times in Secs HTML
All
99%
95%
90%
13,260
13,129
12,597
11,935
Min
0.31
0.31
0.31
0.31
Max
37.58
15.38
4.36
2.65
Mean
1.53
1.34
1.10
0.98
Median
0.89
0.87
0.81
0.75
Std Dev
2.44
1.53
0.82
0.64
C.V.
159.7
114.1
74.0
65.0
Total
20,252
17,618
13,872
11,685
28.6%
53.7%
59.3%
N
% CV Improvement
Service Times in Secs AJAX
All
99%
95%
90%
13,260
13,128
12,598
11,935
Min
0.25
0.25
0.25
0.25
Max
34.42
11.26
3.48
2.19
Mean
1.29
1.11
0.92
0.82
Median
0.73
0.71
0.65
0.60
Std Dev
2.26
1.24
0.67
0.59
C.V.
175.2
111.6
73.4
65.9
Total
17,090
14,541
11,545
9,775
36.3%
58.1%
62.4%
N
% CV Improvement
Example • 95% of HTML requests received service in 4.36 secs or less – One request received service 9x as long
• 95% of all AJAX requests received service in 3.48 secs or less – One request received service 10x as long
CV Improvement • CV = std dev / mean * 100 – Measure of variation per average value
• A measure of the volatility of a data set – Measure of dispersion of prob distribution – “Unitized risk”
• Lower CV indicates lower variability • For this data, indicates improvement going from ALL to 99%, to 95%, or to 90%
CV Improvement HTML
All
99%
95%
90%
37.58
15.38
4.36
2.65
Mean
1.53
1.34
1.10
0.98
Std Dev
2.44
1.53
0.82
0.64
159.7
114.1
74.0
65.0
28.6%
53.7%
59.3%
All
99%
95%
90%
34.42
11.26
3.48
2.19
Mean
1.29
1.11
0.92
0.82
Std Dev
2.26
1.24
0.67
0.59
175.2
111.6
73.4
65.9
36.3%
58.1%
62.4%
Max
C.V. % CV Improvement from All AJAX Max
C.V. % CV Improvement from All
Performance Increase Following White, Smullen/Smullen define Performance Increase = (HTML value - AJAX value) ──────────────── (HTML value)
* 100%
• The percentage improvement of AJAX compared to HTML – Savings achieved by using AJAX – 0% means same as HTML
• Positive is good, Larger is better
Size Performance Increase Mean percentage improvement 54.7% – Calculate % improvement for each query – Calculate mean of these values
• Response size is deterministic – Every trial for a query gives same % value
• Percent improvement is nonlinear – See “Modelling AJAX…” for analysis of this
Performance increase vs HTML response size
Time Performance Increase • Service time is stochastic – Different trials for a query give different service times
• “Performance increase” for a query varies widely with each trial – Mean performance increase is 2.4% – Compute increase for each query and use mean of these values
Time Performance Increase • Mean increase over all queries is 2.4% • For 4% of queries AJAXsec > HTMLsec – One has an increase of -7135%
• Using means of full data 15.6% – Calculate using means of HTML and AJAX
• Using means of 95% groups 16.8% – Calculate using means of two 95% groups
Time performance increase vs HTML size
Results for typical user • Typical user would not issue 110 queries – Or 13,260 queries
• Typical user of this application issues about 7 queries
• Use average query response values from full data, estimate effects on typical user
Results for typical user Based on 7 average queries 7 queries
Response bytes
Response secs
HTML
447,973
6.209
AJAX
202,972
5.117
Improvement
54.7%
17.59%
Results for typical 95% user Based on 7 average queries from 95% group 7 queries
Response bytes
Response secs
HTML 95%
447,973
5.733
AJAX 95%
202,972
4.543
Improvement
54.7%
20.76%
Results for typical user • Using AJAX, a typical user sees 56% improvement in bytes 18% improvement in speed • Using AJAX, 95% of typical users see 56% improvement in bytes 21% improvement in speed
Conclusions • AJAX byte savings averaged 55% – Significant bandwidth reduction – Or improvement in responsiveness of app
• AJAX server time savings 18% – AJAX is less useful as server productivity tool – Improving database querying would provide better performance for all
