October 3, 2015 22:15:41

ScaDaVer – Schema and Data Versioning System

Managing and accommodating the evolution of database schema (the metadata describing the structure of a database) poses a number of interesting problems. Some of these problems are particularly acute in Online Transaction Processing (OLTP) databases that serve as the data store for large, extremely active data processing systems, especially in systems that use a Software as a Service (SaaS) delivery model. We propose a framework to be incorporated into such databases to efficiently manage the inevitable schema evolution. This framework is based on common practices used to manage source code changes in software development. It allows administrators and users of the database to create sandboxes in which changes to the database are isolated from the main database and from other sandboxes. Schema versioning techniques are used to isolate schema changes made within sandboxes and to allow queries executed in a sandbox to retrieve data from the main database without transforming all data in the system to conform to the sandbox’s schema. The framework also includes data versioning mechanisms to maintain isolation of data that is added, updated, or deleted in the sandbox. A merge facility allows changes made in a sandbox to be integrated into the main database while it is online, with minimal disruption to ongoing transaction processing. These two versioning techniques together with the merge facility create a database infrastructure that will significantly reduce the time, manpower, opportunity for errors, storage capacity, and infrastructure required to perform development, maintenance, and testing of schema and data changes against a high-volume online database.