Speech and multimedia Transmission Quality (STQ) Testing

Introduction

Since years, several test campaigns have been designed to measure, analyze and compare speech quality parameters for speech terminal equipment, in narrowband (up to 3.4 kHz) and more recently wideband (up to 7 kHz).

For IP terminals and gateways all conversational aspects, including speech sound quality, echo measurements, double talk performance and the transmission quality in the presence of background noise, are considered.

An important characteristic of these Plugtests™ events or test events is that quality is considered from the user's point of view, including the acoustic interfaces for the speech terminal. As traditional speech communication evolves into innovative services such as multimedia conferencing, and as new means of delivery are introduced (e.g. VoIP), verification of acceptable levels of quality is vital. The distinctive Plugtests™ approach implies specific environments and processes; through a special analysis of test logs and diagnosis of the equipment behaviour, it also generates information that is useful for system optimization. The approach significantly reduces the time needed for manufacturers to develop and test their products, thus both speeding up their availability and improving their quality. Each participating manufacturer receives all his individual results, including detailed information about potential improvements, and is able to compare these results with those of all the other participants which are published in an anonymous test report at the end of the event.

For the test events implemented for multiplay services, the most of the tests are done without the acoustical interface for speech quality and without the audio/video terminal for IPTV. However these tests are done using actual offerings from ISPs.

Our Role & Activities

Our role is both to define the test methodologies and the relevant test set-ups to be implemented for the test events.

The standards produced by TC STQ are implemented for these plugtests/ events, giving first the opportunity to manufacturers not equipped for these kinds of innovative test equipments to become acquainted with, and trained in, these new processes and new standards.

The results of the tests provide, through anonymous reports, a feedback on the real performance of equipments or services. This may consolidate the requirements available in the standards or may provide change requests for revised versions of the standards.
It should also be needed in some cases to integrate quality tests with interoperability events, because from the user's point of view the expectation is to have not only interoperable services but also offering a good QoS/QoE. 

Several documents from STQ are available:

For Plugtests / Speech Quality Test Events

TR 102 648-Series: Test methodologies for ETSI test events and results
Part 1: VoIP Speech Quality Testing
Parts 2, 3,…: ETSI Plugtests Speech Quality Test Event Reports

For Plugtests/events for multiplay services

TR 102 716 Series: Guidelines, objectives and results of speech quality analysis in the context of interworking plugtests for multiplay services
Part 1: Guidelines and objectives
Part 2: Results

Standards

The following is a list of the 20 latest published ETSI standards on STQ testing.

A full list of related standards in the public domain is accessible via the ETSI standards search. Via this interface you can also subscribe for alerts on updates of ETSI standards.

For work in progress see the ETSI Work Programme on the Portal

Standard No. Standard title.
TS 102 250-2 Speech and multimedia Transmission Quality (STQ); QoS aspects for popular services in mobile networks; Part 2: Definition of Quality of Service parameters and their computation
ES 202 718 Speech and multimedia Transmission Quality (STQ); Transmission Requirements for IP-based Narrowband and Wideband Home and Network Media Gateways from a QoS Perspective as Perceived by the User
ES 202 737 Speech and multimedia Transmission Quality (STQ); Transmission requirements for narrowband VoIP terminals (handset and headset) from a QoS perspective as perceived by the user
ES 202 738 Speech and multimedia Transmission Quality (STQ); Transmission requirements for narrowband VoIP loudspeaking and handsfree terminals from a QoS perspective as perceived by the user
ES 202 739 Speech and multimedia Transmission Quality (STQ); Transmission requirements for wideband VoIP terminals (handset and headset) from a QoS perspective as perceived by the user
ES 202 740 Speech and multimedia Transmission Quality (STQ); Transmission requirements for wideband VoIP loudspeaking and handsfree terminals from a QoS perspective as perceived by the user
TS 103 737 Speech and multimedia Transmission Quality (STQ); Transmission requirements for narrowband wireless terminals (handset and headset) from a QoS perspective as perceived by the user
TS 103 224 Speech and multimedia Transmission Quality (STQ); A sound field reproduction method for terminal testing including a background noise database
TS 103 738 Speech and multimedia Transmission Quality (STQ); Transmission requirements for narrowband wireless terminals (handsfree) from a QoS perspective as perceived by the user
TS 103 739 Speech and multimedia Transmission Quality (STQ); Transmission requirements for wideband wireless terminals (handset and headset) from a QoS perspective as perceived by the user
TS 103 740 Speech and multimedia Transmission Quality (STQ); Transmission requirements for wideband wireless terminals (handsfree) from a QoS perspective as perceived by the user
TR 102 493 Speech and multimedia Transmission Quality (STQ); Guidelines for the use of Video Quality Algorithms for Mobile Applications
TS 103 281 Speech and multimedia Transmission Quality (STQ); Speech quality in the presence of background noise: Objective test methods for super-wideband and fullband terminals
ES 202 738 Speech and multimedia Transmission Quality (STQ); Transmission requirements for narrowband VoIP loudspeaking and handsfree terminals from a QoS perspective as perceived by the user
ES 202 740 Speech and multimedia Transmission Quality (STQ); Transmission requirements for wideband VoIP loudspeaking and handsfree terminals from a QoS perspective as perceived by the user
EG 202 396-3 Speech and multimedia Transmission Quality (STQ); Speech Quality performance in the presence of background noise; Part 3: Background noise transmission - Objective test methods
ES 202 737 Speech and multimedia Transmission Quality (STQ); Transmission requirements for narrowband VoIP terminals (handset and headset) from a QoS perspective as perceived by the user
ES 202 738 Speech and multimedia Transmission Quality (STQ); Transmission requirements for narrowband VoIP loudspeaking and handsfree terminals from a QoS perspective as perceived by the user
ES 202 739 Speech and multimedia Transmission Quality (STQ); Transmission requirements for wideband VoIP terminals (handset and headset) from a QoS perspective as perceived by the user
ES 202 740 Speech and multimedia Transmission Quality (STQ); Transmission requirements for wideband VoIP loudspeaking and handsfree terminals from a QoS perspective as perceived by the user