Several sites on the Net offer so-called speed tests or bandwidth tests. These free services promise to measure the "real time" performance of one's Internet connection. But do these tests really work? Do they present more meaningful and accurate data than one's network operating system (NOS)?
This article discusses the concepts behind online speed measurement. Follow-up articles (see sidebar) critique the effectiveness of individual testing services based on actual trials.

Bandwidth and Speed
In computer networking, the term bandwidth refers to the rate of data transfer. Very often the bandwidth numbers one sees and hears represent theoretical or peak performance of a device. A 100 Mbps Ethernet adapter installed in a computer with a slow processor, for example, rarely exceeds 10 Mbps in actual performance. Traditional dial-up modems, advertised as capable of 56 Kbps, never actually perform at that rate in practice.
The term speed is often used synonymously with data rate in networking. Technically speaking, speed refers to the user-perceived performance of the network application. Speed correlates positively with bandwidth in many cases, but not always.
One's Web browser, for example, may perform "slowly" on a high-bandwidth network for several reasons, such as a bottleneck at the Web server or at one's ISP. Those who have upgraded from traditional dial-up to broadband may already be familiar with the phenomenon. It's common to attribute performance problems to a "slow network" even though the local area data transfer rates (when transfers occur) remain high.

The Need for Speed Tests

Network performance depends on many factors. To help a person understand the state of their network, modern operating systems support various performance monitoring mechanisms. Though useful, these features tend to drastically oversimplify or "dumb down" the analysis of network performance.

Dial-Up Networking

Windows maintains a network icon on the taskbar whenever a dial-up connection exists. As shown below for Windows 2000, the Dial-Up Networking (DUN) Monitor icon uses color-coded indicators to distinguish between sent and received packets. The icon updates dynamically approximately once per second.

The dynamic icon appears again on the DUN status page, shown below. This page reveals more session parameters, including connection state and duration, total bytes sent and received, errors, and compression ratios.

Dial-Up Networking Status Window


Marked in red for illustrative purposes, the Speed entry in the status window displays a data rate of 21.6 Kbps. This same statistic also appears in the DUN tooltips displayed at connect time (left) and throughout the life of the connection (right).

Dial-Up Networking Tooltip



Windows receives this statistic from the modem at the time the modem protocol establishes the connection. The number represents the expected performance of the connection, not based on any data transfer history but rather on the design of the protocol in use.
Several factors can cause actual network performance to vary significantly from this theoretical value. As mentioned earlier, a high processor load on the computer on either end of the connection can create a bottleneck that lowers the data rate. On wide-area networks (WANs) such as the Internet, an overloaded router on the path from source to destination can also prevent connections from attaining optimal performance.
On the positive side, data compression can result in much higher speeds than reported on by Windows DUN. Compression works especially well with large amounts of text, such as HTML pages, though it handles images and other binary data much less effectively. Compressed data obviously cannot magically increase one's bandwidth, but it can utilize the same bandwidth more intelligently, resulting in higher user-perceived speed.

Windows Performance Monitor

A lesser-known but useful network performance tool in Windows (2000 and NT) is the Performance Monitor. Accessible from the Control Panel (Administrative Tools) or the command-line name perfmon, the Performance Monitor provides more detailed and dynamic, near real-time network statistics.
Perfmon presents statistics in the form of counters and counter groups. Windows maintains several dozen counter groups including several related specifically to networking:

IP
Network Interface
Remote Access Service (RAS)
TCP
UDP
ICMP

Windows Performance Monitor


Network groups contain counters such as the following:
> datagrams, segments, packets, or bytes sent and/or received
> number of active/established/passive/reset connections
> error/failure counts

Compared to Dial-Up Networking, where statistics are displayed and updated automatically, Perfmon displays no data automatically and requires a fair amount of configuration time and effort to extract data in the form of stripcharts.

Windows Performance Monitor Chart


The Speed Test Alternative

Tools such as Dial-Up Networking and Performance Monitor offer good insight into a computer's network performance. However, these tools generally lack two key features:
1. Measure of performance at the application level rather than the network protocol level.

2. Ability to create a synthetic (artificially controlled) workload.

Online speed testers generally offer both of these features. A typical speed testing service allows one to specify their connection type (traditional dial-up, broadband, and so on) and a workload, usually represented by a file download of a fixed size. To the extent speed testers actually "work," they serve as a useful complement to other network monitoring tools.

Benchmarking the Testing Services

Many technical factors affect a person's network performance. From the current workload, processing power and amount of memory on either end of the connection, to the protocols supported by the modem, to bottlenecks at intermediate devices or "hops," performance monitoring involves many variables. No one monitoring tool can fully capture all of these nuances.
On subsequent pages, results of benchmarking for one of the available speed testers will reveal the strengths and weaknesses of this type of tool.

Evaluation 1 - Toast.net
As discussed , so-called bandwidth/speed test services provide one way to measure the performance of an Internet connection. Some tests give more accurate results than others, but the merits of these services are difficult to compare directly because each offers a unique implementation and set of features.
This article presents the results of evaluation one such service, the Toast.net speed test.

Methodology
The evaluation employed a V.90 standard 56Kbps modem and Internet access via Earthlink, a reliable ISP in the USA. Tests were conducted at off-peak hours (between 1:00 AM and 2:00 AM EDT) on a week-day (Wednesday 9 May 2001). These parameters were chosen because they represent a common configuration with minimal Internet service disruptions.

Toast.net Configuration
Toast.net presents several configuration options on its main page as shown below. First, a person may enter the name of their ISP in the text field. Toast.net apparently uses this data for tracking purposes only; the field may be left blank or any nonsense value can be entered without affecting the results of the test.

Toast.Net Configuration - ISP/Connection Type


Next, a person selects their connection type from the list below. These choices represent the variety of connections that Toast.net should be able to analyze. As with the ISP setting, Toast.net provides this option for tracking purposes only.

Toast.Net Configuration - Supported Connection Types


Finally, a person must specify the Web server to be used in the test. As shown below, Toast.net offers several choices optimized for their customers. As a non-customer, the choice of Web server can still affect test results, due to geographic location or server load, as is true with all speed tests of this type. For benchmark purposes, these tests used the Toast.net Basic Hosting option.

Toast.Net Configuration - Server Types



Evaluation 1 - Toast.net (continued)

Test Runs
For dial-up connections, Toast.net provides two workloads -- a single large text file, and a single large image. Toast.net also provides two additional workloads suited to high-speed broadband connections -- a very large text file, and a streaming video. A series of trials were run for each of the two dial-up workloads.

Toast.Net Configuration - Test Workloads


Text File Download
The text file test is meant to simulate downloads of HTML pages to a Web browser. The test utilizes a single file of size 340,646 bytes
In the initial trial, the file download completed in 62.25 seconds, yielding an average connection speed of 44 Kbps. Five additional trials were run, with 30-second waits between each, with the results shown below.

Toast.Net Text Download Performance
TRIAL TIME SPEED
1 62.25 44KBPS
2 73.586 37KBPS
3 72.965 37KBPS
4 70.892 38KBPS
5 62.791 43KBPS
6 67.567 40KBPS

To aid in interpreting the results, Toast.net provides a chart that compares the performance acheived in the trial against typical results

Toast.Net Expected Performance


Test Analysis
Toast.net claims that modems with an optimal 56K connection can achieve speeds of 70-90 Kbps in this benchmark. These results are possible with compression features in the modem protocol. Compression works especially well for text files like HTML pages, and it is not unusual to see a doubling of effective speed due to compression alone.
The V.90 modem used in this test connected to Earthlink at 21.6 Kbps. The results from Toast.net accurately point out that connection speed alone does not tell the whole performance story. Not only can much better performance be attained, but the actual speed a person sees on the Internet can vary significantly (up to 20% in these trials) from one minute to the next, even at a relatively low-traffic time of day.
Internet users typically download multiple small HTML pages rather than a single large file as done in the Toast.net benchmark. Multiple small downloads generally result in lower aggregate data rates, as the connection management overhead (messages to establish and teardown connections) take bandwidth away from the actual data transfer. For this reason, this Toast.net benchmark will likely report higher speeds than Internet users will see in practice.

Image Download
The image file test is meant to simulate downloads of graphics (GIF or JPEG images) contained in Web pages. The test utilizes a single JPEG file of size 225,455 bytes. The snapshot below illustrates the page as it is loading..


Toast.Net Expected Performance



Test Analysis
Typical Web pages incorporate a mix of text and graphics; users rarely download a single image in isolation. However, the Toast.net graphic test demonstrates the network-intensive nature of large images such as maps and diagrams. As compression works very poorly with large and complex graphics, the effective network speed often drops to one-half or lower the performance of text downloads. Image download performance consistently fails to reach the rating advertisted by the modem connection.
n these trials, image download performance attained only 40% of text download performance. The image trials yielded an effective speed 25% slower than the rated connection speed (21.6 Kbps in these trials). As in the text download trials, performance from run to run varied by roughly +/- 20%.

Toast.net Summary

Overall, Toast.net offers a useful performance measurement service. By providing both text and image download tests, users can accurately gauge the effective speed of their connection on the two most basic forms of Internet workloads. The dramatic difference in performance between the two also serves to educate users on the importance of compression in networking and the wide variation to expect in Web "surfing" performance depending on the content of a given page. Comparison charts allow one to compare their performance against other users with that their same type of connection.

About Computer Networking Ratings for Toast.net

Accuracy / Reliability A-
Features B-
Informativeness B
Overall B

Toast.net offers download testing of single files only. The service would benefit from adding additional workloads that more closely simulate typical usage patterns. For example, a mixed workload of multiple small test file and images would more accurately represent Web page downloads. Upload tests would benefit Web site publishers and would also greatly educate the general user on the difference between upstream and downstream performance (particularly for broadband connections).

- SRIKANTH DHANWADA