The Centera Approach to Testing a Complex Distributed Storage System

Organizers: Koen De Hondt & Wouter Soubry

Objectives:
Show the community what kind of investment is needed to test a complex distributed system that is being used as a TIER-1 storage device. Share how the Centera development team fits not-so-fast-feedback into its agile software development process.

Contents:
Centera is a device for online storage. It consists of 4 to 128 computers in a RAIN architecture. Centera is a device for content addressable storage (CAS): the users of the system do not know the location of their files, but they get a key for a file with which they can retrieve the file later. Centera has self healing capability: if a drive, a network card or a CPU fails to work, the system is able to recover and ensure that no data is lost. The presentation shows how the Centera division of EMC tests its product. The presentation will not only show the technological highlights of our testing infrastructure. It will also indicate where the weak spots are and it will indicate which challenges we still face. An important message that we want to convey is that development teams should not sit idle when a system appears hard to test, but instead should look for ways to test at different levels to get the testability problem under control.

Brief overview of the presentation:
  • Short introduction to EMC and Centera, including the quality challenges we face for a tier-1 product
  • History: where does Centera testing come from and where does it stand today
  • Requirements for testing a complex distributed storage system being developed by multiple teams in multiple locations
  • The Centera approach to meet the testing requirements
  • Testing at the right level: unit testing, component testing, system testing
  • Lessons learnt and challenges we still face

Process & Timetable:
We will keep the introduction very short, so that we can focus on the solutions and the issues that we still face. This presentation is not very interactive, although we welcome questions at any time. We like to do this presentation at the end of the day (preferably the first day), so that people can talk to us "tussen pot en pint" after the presentation if they like to know more.

Session Format:
Case study

Intended Audience:
All. No experience required.

Benefits of participating:
Getting insight in how to test complex distributed systems, how it affects the development process and what the cost is. People not involved in the development of distributed systems will learn how dividing the testing effort into different levels of testing (unit testing, component testing, system testing) helps to get feedback as fast as possible without wasting resources (computer time).

Benefits of organizing:
Any feedback on our current way of working is beneficial to improve the organizer's testing activities