Database Systems

A Pragmatic Approach

Elvis C. Foster

Shripad V. Godbole

Database Systems

Copyright © 2014 by Elvis C. Foster with Shripad V. Godbole

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.

ISBN-13 (pbk): 978-1-4842-0878-6

ISBN-13 (electronic): 978-1-4842-0877-9

Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.

The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.

While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.

Managing Director: Welmoed Spahr

Lead Editor: Jonathan Gennick

Editorial Board: Steve Anglin, Mark Beckner, Gary Cornell, Louise Corrigan, Jim DeWolf, Jonathan Gennick, Robert Hutchinson, Michelle Lowman, James Markham, Matthew Moodie, Jeff Olson, Jeffrey Pepper, Douglas Pundick, Ben Renow-Clarke, Gwenan Spearing, Matt Wade, Steve Weiss

Coordinating Editor: Jill Balzano

Compositor: SPi Global

Indexer: SPi Global

Artist: SPi Global

Cover Designer: Anna Ishchenko

Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.

For information on translations, please e-mail rights@apress.com, or visit www.apress.com.

Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales–eBook Licensing web page at www.apress.com/bulk-sales.

Any source code or other supplementary material referenced by the author in this text is available to readers at www.apress.com. For detailed information about how to locate your book’s source code, go to www.apress.com/source-code/.

This book is dedicated to my late father, Claudius Foster, who taught me the discipline of being a responsible person. The book is also dedicated to my students — past, present, and future. You are the object of my inspiration and motivation; you are the reason for this book.

Contents at a Glance

About the Authors

Preface

Acknowledgements

 Part A: Preliminary Topics

 Chapter 1: Introduction to Database Systems

 Chapter 2: The Database System Environment

 Part B: The Relational Database Model

 Chapter 3: The Relational Model

 Chapter 4: Integrity Rules and Normalization

 Chapter 5: Database Modeling and Design

 Chapter 6: Database User Interface Design

 Chapter 7: Relational Algebra

 Chapter 8: Relational Calculus

 Chapter 9: Relational System — a Closer Look

 Part C: The Structured Query Language

 Chapter 10: Overview of SQL

 Chapter 11: SQL Data Definition Statements

 Chapter 12: SQL Data Manipulation Statements

 Chapter 13: SQL Views and System Security

 Chapter 14: The System Catalog

 Chapter 15: Some Limitations of SQL

 Part D: Some Commonly Used DBMS Suites

 Chapter 16: Overview of Oracle

 Chapter 17: Overview of DB2

 Chapter 18: Overview of MS SQL Server

 Chapter 19: Overview of MySQL

 Chapter 20: Overview of Delphi

 Part E: Advanced Topics

 Chapter 21: Database Administration

 Chapter 22: Distributed Database Systems

 Chapter 23: Object Databases

 Chapter 24: Data Warehousing

 Chapter 25: Web-Accessible Databases

 Part F: Final Preparations

 Chapter 26: Sample Exercises and Examination Questions

 Part G: Appendices

 Appendix 1: Review of Trees

 Appendix 2: Review of Hashing

 Appendix 3: Review of Information Gathering Techniques

Index

About the Authors

Elvis C. Foster is Associate Professor of Computer Science at Keene State College, New Hampshire. He holds a Bachelor of Science (BS.) in Computer Science and Electronics, as well as a Doctor of Philosophy (PhD) in Computer Science (specializing in strategic information systems and database systems) from University of the West Indies, Mona Jamaica. Dr. Foster has over 25 years of combined experience as a software engineer, database expert, information technology executive and consultant, and computer science educator. He has lectured at the higher education level in three different countries, including the United States, and has produced several outstanding computer science and information technology professionals, many of whom have excelled at graduate school as well as in the workplace.

Shripad V. Godbole is an independent database administrator/consultant with over 20 years of experience in diverse business environments, information infrastructure planning, diagnostics, and administration. His qualifications include Bachelor of Science (BS) in Physics, Bachelor of Computer Science (BCS), Master of Science (MS) in Physics with Specialization in Electronics — all from Poona University, Pune, India. He is also an Oracle Certified Professional Database Administrator (OCPDBA), and holds a Master of Business Administration (MBA) in Technology Management from University of Phoenix.

Preface

This book has been compiled with three target groups in mind: The book is best suited for undergraduate students who are pursuing a course in database systems. Graduate students who are pursuing an introductory course in the subject may also find it useful. Finally, practicing software engineers who need a quick reference on database design may find it useful.

The motivation that drove this work was a desire to provide a concise but comprehensive guide to the discipline of database design and management. Having worked in the information technology (IT) and software engineering industries for several years before making a career switch to academia, it has been my observation that many IT professionals and software engineers tend to pay little attention to their database design skills; this is often reflected in the proliferation of software applications with inadequately designed underlying databases. In this text, the discipline of database systems design and management is discussed within the context of a bigger picture — that of software engineering. The student is led to understand from the outset that a database is a critical component of a software system, and that proper database design and management is integral to the success of the software product. Additionally and simultaneously, the student is led to appreciate the huge value of a properly designed database to the success of a business enterprise.

The text draws from lectures notes that have been compiled and tested over several years with outstanding results. They draw on personal experiences gained in industry over the years, as well as the suggestions of various professionals and students. The chapters are organized in a manner that reflects my own approach in lecturing the course, but each chapter may be read on its own.

The text has been prepared specifically to meet three objectives: comprehensive coverage, brevity, and relevance. Comprehensive coverage and brevity often operate as competing goals. In order to achieve both, I have adopted a pragmatic approach that gets straight to the critical issues for each topic, and avoids unnecessary fluff, while using the question of relevance as the balancing force. Additionally, readers should find the following features quite convenient:

Short paragraphs that express the salient aspects of the subject matter being discussed

Bullet points or numbers to itemize important things to be remembered

Diagrams and illustrations to enhance the reader’s understanding

Real-to-life examples

Introduction of a number of original methodologies for database modeling and design

Step-by-step, reader-friendly guidelines for solving generic database systems problems

Each chapter begins with an overview and ends with a summary

A chapter with sample examination questions (for the student) and case studies (for the student as well as the practitioner)

Organization of the Text

The text is organized in twenty-five (25) chapters, and an additional chapter consisting of sample examination questions and case studies. The chapters are placed into six divisions, with a seventh division for the appendices. The chapters and related divisions are as follows:

Part A: Preliminary Topics

Chapter 1, Introduction to Database Systems, introduces the database system (DBS) as an essential resource in the business organization. It also identifies the objectives and advantages of a DBS.

Chapter 2, The Database System Environment, discusses the environment of a database system. This includes the components, architecture, and personnel.

Part B: The Relational Database Model

Chapter 3, The Relational Model, discusses the fundamentals of the relational model for databases. It provides the foundation for subsequent chapters.

Chapter 4, Integrity Rules and Normalization, builds on chapter 3 to discuss the database normalization and other related topics.

Chapter 5, Database Modeling and Design, builds on chapters 3 and 4, and examines alternate methodologies for modeling and designing a database.

Chapter 6, Database Application Design, summarizes the process of application design and development for a database environment, within the larger context of software engineering.

Chapter 7, Relational Algebra, introduces relational algebra as the foundation for an understanding of databases languages such as Structured Query Language (SQL).

Chapter 8, Relational Calculus, introduces relational calculus as an alternate foundation that is equivalent to relational algebra.

Chapter 9, Relational System — a Closer Look, discusses established benchmarks for relational database management systems.

Part C: Structured Query Language

Chapter 10, Overview of SQL, introduces SQL as the universal database language.

Chapter 11, SQL Definition Statements, discusses the SQL statements that are used for creation and management of objects comprising the database.

Chapter 12, SQL Data Manipulation Statements, discusses SQL statements that facilitation the manipulation of data contained in the database.

Chapter 13, Logical Views and Security, discusses the importance of logical views, and how to create and use them. It also discusses database security, and the various SQL statements that can be used to enforce security constraints in the database.

Chapter 14, The System Catalog, discusses the system catalog as a vital resource of the database that can be used in its management.

Chapter 15, Some Limitations of SQL, discusses some limitations of SQL, and how they can be circumvented.

Part D: Some Commonly Used DBMS Suites

Chapters 16 – 20 provide an overview of some commonly used DBMs suites. This includes Oracle, DB2, Microsoft SQL Server, MySQL, and Delphi.

Part E: Advanced Topics

Chapter 21, Database Administration, provides an overview of database administration. This includes issues such as security, mmanagement, backup and recovery, tuning, among others.

Chapter 22, Distributed Database Systems, discusses the importance of distributed databases, and the challenges of maintaining them.

Chapter 23, Object Databases, discusses the importance of object databases, and the challenges of achieving them.

Chapter 24, Data Warehousing, provides an overview of data warehousing. This includes a discussion of the rationale for data warehousing, characteristic features of a data warehouse, architecture, and construction of a data warehouse.

Chapter 25, Web-Accessible Databases, discusses the relevance, architecture, supporting technologies, and implementation alternatives for Web-accessible databases.

Part F: Final Preparations

Chapter 26, Sample Exercises and Examination Questions is self-explanatory.

Part G: Appendices

Appendix 1, Review of Trees , gives you a chance to review information that you would have covered in your course in data structures. The same is true for appendix 2; Review of Hashing.

Finally, appendix 3, Review of Information Gathering Techniques, pulls some relevant information from the field of software engineering.

Text Usage

The text could be used as a one-semester or two-semester course in database systems, augmented with material from a specific database management system. However, it must be stated that it is highly unlikely that a one-semester course will cover all twenty-five chapters. The preferred scenario therefore is a two-semester course. Below are two suggested schedules for using the text; one assumes a one-semester course; the other assumes a two-semester course. The schedule for a one-semester course is a very aggressive one that assumes adequate preparation on the part of the participants. The schedule for a two-semester course gives the student more time to absorb the material, and also engage in a meaningful project.

Approach and Notations

As can be observed, I have employed a principle-then-example approach throughout the course. All principles and theories are first explained, and then clarified by examples. The reason for this approach is that I firmly believe one needs to have a firm grasp of database principles and theories in order to do well as a database designer or administrator. Database design is emphasized as a critical component of good software engineering, as well as the key to successful company databases.

Chapters 8 and 9 discuss relational algebra and relational calculus respectively, as the basis for modern database languages. Then chapters 11-15 cover the salient features of SQL, the universal database language. In these chapters, the BNF notation is extensively used, primarily because of its convenience and brevity, without sacrificing comprehensive coverage.

Feedback and Support

It is hoped that you will have as much fun using this book as I had preparing it. Additional support information can be obtained from the Web site http://www.elcfos.com or http://www.elcfos.net. Also, your comments will be appreciated.

Acknowledgements

My profound gratitude is owed to my wife, Jacqueline, and children Chris-Ann and Rhoden, for putting up with me during the periods of preparation of this text. Also, I must recognize several of my past and current students (from four different institutions and several different countries) who at various stages have encouraged me to publish my notes, and have helped to make it happen. In this regard, I would like to make special mention of Dionne Jackson, Kerron Hislop, Brigid Winter, Sheldon Kennedy, and Ruth Del Rosario.

Special appreciation is offered to my colleague Shripad Godbole, who has coauthored some of the chapters with me, particularly in divisions D and E. Being a practicing database administrator, Shripad has also served as a valuable resource in these areas. And through Shripad, heartfelt gratitude is extended to his wife, Smita, who has given him unwavering support in everything he has done over the years, including his involvement in this project.

I offer a big thank you to Dr. Han Reichgelt at Southern Polytechnic State University, who in many ways has been my professional mentor. As on previous occasions, I have relied on him for critical evaluations and advice. Speaking of critical evaluations, the contribution of my relatively new friend and colleague, Dr. Jared Bruckner at Southern Adventist University was also significant.

The editorial and production teams at Xlibris Corporation deserve mention for their work in facilitating initial publication of the volume. An equally significant level of gratitude is extended to the editorial team at Apress Publishing for recognizing the work’s value and for investing the time and effort in the project. Thanks to everyone involved.

Finally, I should also make mention of reviewers Jared Bruckner, Marlon Moncrieffe, Jacob Mangal, Abrams O’Buyonge, and Han Reichgelt, each a practicing software engineer, information technology consult, or computer science professor who has taken time to review the manuscript and provide useful feedback. Thanks, gentlemen.

—Elvis C. Foster, PhD

Keene State College

Keene, New Hampshire, USA

PART A

Preliminary Topics

This preliminary division of the course is designed to cover some fundamentals. The objectives are

to define and provide a rationale for database systems;

to identify the many objectives, advantages, and desirable features of a database system;

to discuss the salient features of a database system environment.

The division consists of two chapters:

Chapter 1 — Introduction to Database Systems

Chapter 2 — The Database System Environment

CHAPTER 1

Introduction to Database Systems

Welcome and congratulations on your entry to this course in database systems. The fact that you are in this course means that you have covered several fundamental topics in programming, data structures, user interface, and software engineering. Now you want to learn about databases — their significance, the underlying theoretical principles that govern them, how they are constructed, and their management. You are at the right place. This chapter addresses the first issue: the significance of database systems. Topics covered include the following:

Definition and Rationale

Objectives of a Database System

Advantages of a Database System

Approaches to Database Design

Desirable Features of a Database System

Database Development Life Cycle

Summary and Concluding Remarks

1.1 Definitions and Rationale

A database system (DBS) is a computerized record keeping system with the overall purpose of maintaining information and making it available whenever required. The database typically stores related data in a computer system.

A database management system (DBMS) is a set of programs that allow for the management of a database. Starting in chapter 2 and extending to subsequent chapters, we will cover several of the critical functions of a DBMS. Some of the more obvious ones are the following:

Data definition (relation, dependencies, integrity constraints, views, etc.)

Data manipulation (adding, updating, deleting, retrieving, reorganizing, and aggregating data)

Data security and integrity checks

Programming language support

Components of a DBS include:

Hardware and operating system

DBMS

Database

Related software systems and/or applications

End users

End users communicate with the software systems/applications, which in turn, communicate (through the programming interface) with the DBMS. The DBMS communicates with the operating system (which in turn communicates with the hardware) to store data in and/or extract data from the database. Figure 1-1 illustrates.

Figure 1-1. Simplified Representation of a DBS

Databases are essential to software engineering; many software systems have underlying databases that are constantly accessed, though in a manner that is transparent to the end user. Figure 1-2 provides some examples. Companies that compete in the marketplace need databases to store and manage their mission critical and other essential data.

Figure 1-2. Illustrations of the Importance of Database

In this course you will learn how to design, implement and manage databases. In so doing, you will be exposed to various database technologies and methodologies that are common in the software engineering industry.

1.2 Objectives of a Database System

There are several primary and secondary objectives of a database system that should concern the computer science (CS) professional. Whether you are planning to design, construct, develop and implement a DBS, or you are simply shopping around for a DBMS, these objectives help you to develop an early appreciation for the field; they should also provide useful insight into where the course is heading. As you will soon see, these objectives are lofty, and it is by no means easy to achieve them all.

The primary objectives of a database system include the following:

Security and protection — prevention of unauthorized users; protection from inter-process interference

Reliability — assurance of stable, predictable performance

Facilitation of multiple users

Flexibility — the ability to obtain data and effect action via various methods

Ease of data access and data change

Accuracy and consistency

Clarity — standardization of data to avoid ambiguity

Ability to service unanticipated requests

Protection of intellectual Investment

Minimization of data proliferation — new application needs may be met with existing data rather than creating new files and programs

Availability — data is available to users whenever it is required

Among the significant secondary objectives of a database system are the following:

Physical data independence — storage hardware and storage techniques are insulated from application programs

Logical data independence — data items can be added or subtracted or the overall logical structure modified without existing programs being affected

Control of redundancy

Integrity controls — range checks and other controls must prevent invalid data from entering the system

Clear data definition — a data dictionary is usually kept

Suitably friendly user interface

Tunability — easy reorganizing the database to improve performance without changing the application programs

Automatic reorganization of migration to improve performance

Clarification on Data Independence

Data independence is an important concept that needs further clarification: Data independence is the immunity of application programs to changes in structure and access strategy of data. It is necessary for the following reasons:

Different applications and users will need to have different logical views (interpretation) of data.

The tuning of the system should not affect the application programs.

Physical data independence implies that the user's view is independent of physical file organization, machine or storage medium. Logical data independence implies that each user (or application program) can have his/her (its) own logical view and does not need a global view of the database.

1.3 Advantages of a Database System

A database system brings a number of advantages to its end users as well as the company that owns it. Some of the advantages are mentioned below:

Redundancy can be reduced.

Inconsistencies can be avoided.

Data can be shared.

Standards can be enforced.

Security restrictions can be applied.

Integrity can be maintained.

Conflicting requirements can be balanced.

Improved performance due to speed of processing, reduction in paperwork, etc.

Maintenance and retrieval of data are very easy — no complicated application program needed.

Is not solely dependent on the high level language (HLL) programming for use.

Logical views of data stored can be easily created.

Record structures can change without any adverse effect on data retrieval (due to data independence).

1.4 Approaches to Database Design

Analysis of the management of data via computerized systems reveals five approaches that have been pursued over the past forty years:

Instant small system — uses one file

File processing systems — involve many files

Other non-relational systems e.g. hierarchical, inverted, and network approaches

Relational databases (the focus of this course) — pioneered by prominent individuals such as Edgar Codd, Ronald Fagin, Christopher Date, among others

Object databases — a contemporary approach (also discussed later in the course)

Since the 1970s, relational databases have dominated the field of database systems. With the advancement of object databases (notably since the 1990s), relational databases continue to maintain dominance. Later in the course, you will understand why this dominance is likely to continue. For now, we assert that relational databases will be around for a long time in the foreseeable future, and they will be complemented (rather than challenged) by object databases.

1.5 Desirable Features of a DBS

Contemporary database systems must live up to de facto standards set by the software engineering industry. Roughly speaking, a well-designed database system must exhibit the following features (more specific standards will be discussed later in the course):

Provide most (at least 70%) of the advantages mentioned earlier

Meet most (at least 70%) of the objectives mentioned earlier

Provide for easy communication with other systems

Be platform independent

Have a friendly user interface

Be thoroughly documented

1.6 Database Development Life Cycle

You are no doubt familiar with the software development life cycle (SDLC). For the purpose of review, it is presented in Figure 1-3.

Figure 1-3. Software Development Life Cycle

As mentioned earlier (section 1.1), databases do not exist in a vacuum, but are usually part of a software system. A database development life cycle (DDLC) may therefore be perceived from two perspectives:

It may be viewed as being identical and concurrent with the SDLC. At each phase in the SDLC, consideration is given to the database as an integral part of the software product.

If we consider that in many cases, the database has to be constructed and implemented, and managed as a separate resource that various software systems can tap into, then we may construct a similar but different life cycle for the database as illustrated in Figure 1-4.

Figure 1-4. Database Development Life Cycle

Note:

Applying basic investigation skills that you would have acquired in your software engineering course covers the database investigation and analysis phase. This course assumes that you have acquired those skills. The course therefore concentrates on the other phases.

With experience, the database modeling and database designing phases can be merged into one phase. This will be further clarified in chapters 3 through 5.

Once the database is in implementation phase, management of it becomes an ongoing experience, until the database becomes irrelevant to the organization.

1.7 Summary and Concluding Remarks

Let us summarize what we have covered in this chapter:

A database system is a computerized record keeping system with the overall purpose of maintaining information and making it available on demand.

The DBMS is the software that facilitates creation and administration of the database.

The DBS is made up of the hardware, the operating system, the DBMS, the actual database, the application programs and the end users.

There are several primary and secondary objectives of a DBS, which are of importance to the CS professional.

A DBS brings a number of significant advantages to the business environment.

There are five approaches to constructing a DBS. Three of them are traditional and are no longer used. The two contemporary approaches are the relational approach and the object-oriented approach. For various reasons, the relational approach is fundamental to a course in database systems.

In striving to acquire a DBS, it is advisable to aspire for a minimum or 70% of the objectives and advantages. Additionally, one should strive for platform independence, user-friendliness, and thorough documentation.

The database development life cycle outlines the main activities in the useful life of a DBS.

Interested? We have just begun to touch the surface. There is a lot more to cover. Most successful software systems are characterized by carefully designed databases. In fact, it is safe to say that the efficacy of the software system is a function of its underlying database. So stay tuned: the next chapter provides more clarification on the database environment.

1.8 Review Questions

What is a database system?

Why are database systems important?

What is a database management system (DBMS)?

What are the objectives (primary and secondary) of a DBS?

What is data independence, and how important is it?

What are the advantages of a DBS?

What are the possible approaches to acquiring a DBS?

How do database systems relate to software engineering?

Compare the software development life cycle to the database development life cycle.

1.9 References and/or Recommended Readings

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York, NY: Addison-Wesley, 2002. See chapter 1.

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8th ed. Menlo Park, CA: Addison-Wesley, 2004. See chapter 1.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See chapter 1.

[Garcia-Molina, 2002] Garcia-Molina, Hector, Jeffrey Ullman and Jennifer Widom. Database Systems: The Complete Book. Upper Saddle River, NJ: Prentice Hall, 2002. See chapter 1.

[Hoffer, 2007] Hoffer, Jeffrey A., Mary B. Prescott and Fred R. McFadden. Modern Database Management 8th ed. Upper Saddle River, NJ: Prentice Hall, 2007. See chapters 1&2.

[Lewis, 2002] Lewis. Phillip M., Arthur Bernstein and Michael Kifer. Databases and Transaction Processing: An Application Oriented Approach. New York, NY: Addison-Wesley, 2002. See chapter 1.

[Martin, 1995] Martin, James, and Joe Leben. Client/Server Databases: Enterprise Computing. Upper Saddle River, NJ: Prentice Hall, 1995. See chapter 1.

[Pratt, 2002] Pratt, Phillip J. and Joseph J. Adamski. Concepts of Database Management 4th ed. Boston, Massachusetts: Course Technology, 2002. See chapter 1.

[Riccardi, 2003] Riccardi, Greg. Database Management With Web Site Development Applications. Boston, MA: Addison-Wesley, 2003. See chapter 1.

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, MA: Course Technology, 2007. See chapter 1.

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, New Jersey: Prentice Hall, 1997. See chapter 1.

CHAPTER 2

The Database System Environment

This chapter discusses the environment of a database system. Topics covered include:

Levels of Architecture

Inter-level Mappings

Database Administrator

Database Management System

Components of the DBMS Suite

Front-end and Back-end Perspectives

Database System Architecture

Summary and Concluding Remarks

2.1 Levels of Architecture

In [Date, 2004], Christopher Date describes three levels of architecture of a database system, namely, the external level, the conceptual level, and the internal level. These levels are illustrated in Figure 2-1; we will briefly discuss each.

Figure 2-1. Levels of DBS Architecture

2.1.1 External Level

The external level is concerned with individual user views. It therefore varies according to users’ perspectives. The external level is defined by the external schema.

Typically, the database is accessed through its external schema. The application programmer uses both the host language and the data sublanguage (DSL) to create a user interface that end users use to access the system:

The DSL is the language that is concerned specifically with database objects and operations. To illustrate, SQL (structured query language) is the industry’s standard DSL. Other examples of data sub-languages are QUEL and KQL (knowledge query language). These languages will be further discussed later in the course.

The host language is that which supports the DSL in addition to other non-database facilities such as manipulation of variables, computations and Boolean logic. Host languages are typically high level languages (HLL); examples include COBOL, C, C++, Java, Pascal, RPG-400, etc.

Typically, the sublanguage consists of a data definition language (DDL), a data manipulation language (DML), and a data control language (DCL). These components are not necessarily distinct entities, but are typically part of a single coherent product.

The above-mentioned facilities allow users to define assorted logical views of data in the database. In summary, the external schema is the user interpretation of the database, but facilitated by the DSL.

2.1.2 Conceptual Level

The conceptual level is an abstract representation of the entire information content of the database; it also referred to as the logical or community user view. It is defined by means of the conceptual schema, which includes definition of each of the various types of conceptual records.

The conceptual schema includes defining the structure of the database, security constraints, operational constraints and integrity checks. It represents a closer picture of how data will be actually stored and managed, and is the level that most technical user will relate.

The conceptual schema must adhere to the data independence requirement. Also, it must be comprehensive since it represents the realization of the entire database design.

2.1.3 Internal Level

Also called the storage view, the internal level is the low level representation of the database. It is one level above the physical level, which deals with pages, cylinders and tracks on the storage device.

The internal level is defined by the internal schema, which addresses issues such as record types, indexes, field representation, physical storage sequence of records, data access, etc., and written in the internal DDL.

2.2 Inter-level Mappings

Continuing from the previous section, the literature also describes two levels of mappings that connect the three schemas (again, see [Date, 2004]). Figure 2-1 illustrates the different schemas and their interrelationships with respect to the DBMS. From the figure, observe that there are two levels of mapping — the external-conceptual mapping and the conceptual-internal mapping:

The conceptual-internal mapping specifies how conceptual records are represented at the internal level. If changes are made at the internal level, this mapping must be updated. Traditionally, the database administrator (DBA) maintains this mapping in order to preserve data independence (the DBA is discussed in the next section). In contemporary systems, the DBMS automatically updates and maintains this mapping, transparent to the user.

The external-conceptual mapping specifies how external views are linked to the conceptual level. In effect, this is achieved by application programs and logical views via the host language and the DSL.

It must be borne in mind that these levels are abstractions that facilitate understanding of the DBS environment. As an end user, you will most likely not visibly observe these levels of architecture. However, if as a software engineer, you find yourself on a software engineering team that is constructing or maintaining a DBMS (a huge undertaking), knowledge of these abstractions becomes critical.

2.3 The Database Administrator

The database administrator (DBA) has overall responsibility for the control of the system at the technical level. Some of the functions of the DBA include:

Defining the conceptual schema (i.e. logical database design)

Defining the internal schema (i.e. physical database design)

Liaising with users and identifying / defining views to facilitate the external schema

Defining security and integrity checks

Defining backup and recovery procedures

Monitoring performance and responding to changing requirements

In many organizations, the tendency is to include these functions in the job description of the software engineer. This is quite rational and prudent, since good software engineering includes good database design. However, large corporations that rely on company database(s) on an on-going basis, usually employ the services of one or more DBAs. Because of the importance of having reliable databases, DBAs are among the highest paid information technology (IT) professionals.

2.4 The Database Management System

The database management system (DBMS) is the software that facilitates management of the database. When a user issues a request via some DSL (for example SQL), it is the DBMS that interprets the request, executes the appropriate instructions and responds to the user.

Functions of the DBMS include the following:

Data definition (relation, dependencies, integrity constraints, views, etc.)

Data manipulation (adding, updating, deleting, retrieving, reorganizing, and aggregating data)

Data security and integrity checks

Management of data access (including query optimization), data recovery and concurrency

Maintenance of a user-accessible system catalog (data dictionary)

Support of miscellaneous non-database functions (e.g. utilities such as copy)

Programming language support

Transaction management (either all updates are made or none is made)

Backup and recovery services

Communication support (allow the DBMS to integrate with underlying communications software)

Support for interoperability including open database connectivity (ODBC), Java database connectivity (JDBC), and other related issues

Optimum efficiency and performance are the hallmarks of a good DBMS. To illustrate the critical role of the DBMS, consider the steps involved when an application program accesses the database:

Program-A issues a request to the DBMS (expressed in terms of sub-schema language);

DBMS looks at Program-A sub-schema, schema and physical description (these information are stored in tables);

DBMS determines which files must be accessed, which records are needed and how access is done;

DBMS issues instruction(s) (reads or writes) to the operating system;

Operating system causes data transfer between disk storage and main memory;

DBMS issues moves to transfer required fields;

DBMS returns control to Program-A (possibly with a completion code).

Figure 2-2 provides a graphic representation, but bear in mind that these steps are carried out automatically, in a manner that is transparent to the user.

Figure 2-2. Steps Involved When Application Programs Access a Database

2.5 Components of DBMS Suite

The DBMS is actually a complex conglomeration of software components working together for a set of common objectives. For the purpose of illustration, we may represent the essential components of the DBMS as the following:

DBMS Engine

Data Definition Subsystem

User Interface Subsystem

Application Development Subsystem

Data Administration Subsystem

Data Dictionary Subsystem

Data Communications Manager

Utilities Subsystem

These functional components (illustrated in Figure 2-3) are not necessarily tangibly identifiable, but they exist to ensure the acceptable performance of the DBMS.

Figure 2-3. Functional Components of a DBMS

2.5.1 The DBMS Engine

The DBMS engine is the link between all other subsystems and the physical device (the computer) via the operating system. Some important functions are as follows:

Provision of direct access to operating system utilities and programs (e.g. I/O requests, data compaction requests, communication requests etc.)

Management of file access (and data management) via the operating system

Management of data transfer between memory and the system buffer(s) in order to effect user requests

Maintenance of overhead data and metadata stored in the data dictionary (system catalog)

2.5.2 Definition Tools Subsystem

The data definition subsystem (DDS or its equivalent) consists of tools and utilities for defining and changing the structure of the database. The structure includes relational tables, relationships, constraints, user profiles, overhead data structures, etc.

The DDL (data definition language) is used to define all database objects that make up the conceptual schema (relations, relationships, constraints, etc.). The DML (data manipulation language) is used to facilitate manipulation (insert, remove, update, find, query etc.) of data. The DML usually includes a query language. The DCL (data control language) is used to set up control environments for data management by the end user. As mentioned earlier, the DDL, DML and DCL comprise the DSL.

2.5.3 The User Interface Subsystem

The user interface subsystem (UIS or its equivalent) allows users and programs to access the database via an interactive query language such as SQL and/or the host language. The traditional interface is command based; however in recent times menus and graphical user interfaces (GUI) have become more prevalent. Of course, it is not uncommon for a product to provide the user with all three interfaces (for example Oracle). Other more sophisticated DBMS suites may use natural language interface.

The user interface may also include a DBMS-specific programming language (e.g. FoxPro, Scalable Application Language, and Oracle’s PL/SQL). These languages pertain only to the DBMS in which they are used. Additionally, the DBMS may support multiple high level languages such as C++, Java, etc., thus making it more flexible and marketable.

2.5.4 Application Development Subsystem

The application development subsystem (ADS or its equivalent) contains tools for developing application components such as forms, reports, and menus. In some cases, it may be merged with the user interface subsystem. Typically, this subsystem provides a graphical user interface (GUI), which is superimposed on an underlying host language. The suite may include an automatic code generator (as in Delphi and Team Developer), or seamless access of the compiler of the host language (as in Oracle).

Other facilities that may be included such as:

Report writer

Project manager

Menu builder

Graphic data interpreter

2.5.5 Data Administration Subsystem

The data administration subsystem (DAS) consists of a collection of utilities that facilitate effective management of the database. Included in this subsystem are facilities for backup and recovery, database tuning, and storage management. It is typically used by DBAs as well as software engineers.

2.5.6 Data Dictionary Subsystem

Also called the system catalog in many systems, the data dictionary (DD) contains information on the database structure as well relationships among database objects. It is automatically created and maintained by the DBMS.

The system catalog contains all metadata for the database. It can be queried using the same commands used to manipulate source data; it is therefore of inestimable value to the DBAs and software engineers. More will be said about the system catalog later in the course.

2.5.7 Data Communications Manager

Traditionally, a separate system that is linked to the DBMS, the data communications manager (DCM) caries out functions such as:

Handling communication to remote users in a distributed environment

Handling messages to and from the DBMS

Communication with other DBMS suites

Modern systems tend to have this subsystem as an integral part of the DBMS suite. In short, the data communications manager ensures that the database communicates effectively with all client requests in a client-server-based environment. Typically, the server-based portions of the DBMS will be running on machines designated as servers in the network. All other nodes are then deemed as client nodes that can request database services from a server. There may be several database servers in the network; also, a node may act as both a server and a client (provided the essential software components are in place).

2.5.8 Utilities Subsystem

Utilities are programs that perform various administrative tasks. The utilities subsystem consists of various utility programs that are applicable to the database environment. Examples of utilities are as follows:

Load routines to create initial version of a database from non-database files

Copy routines for duplicating information

Reorganization routines to reorganize data in the database

File deletion routine(s)

Statistics routines to compute and store file statistics

Backup and recovery utilities

Other utilities (that might have been) developed by application programmers

2.6 The Front-end and Back-end Perspectives

A DBS can be perceived as a simple two-part structure:

The Front-end consists of end users, applications and a programming interface.

The Back-end consists of the actual DBMS and the database.

The front-end system may be on a different machine from the back-end system and the two connected by a communication network. For example, we may have a front-end system in Delphi or Java NetBeans, and a back-end system in Oracle, Sybase, MySQL, or DB2. Figure 2-4 illustrates.

Figure 2-4. Front-end / Back-end Perspective

2.7 Database System Architecture

There may be added benefits of using different machines for the back-end and front-end system. Figures 2-5 – 2-7 show three possible configurations. Please note also that various network topologies are applicable to any computer network (network topology is outside of the scope of this course; however, it is assumed that you are familiar with such information).

Figure 2-5. Back-end and Front-end Running on Different Machines

Figure 2-6. One Back-end, Multiple Front-ends

Figure 2-7. Distributed System Where Each Machine has both Front-end and Back-end

Discuss: What are some of the advantages of using distributive processing in a DBS environment?

2.8 Summary and Concluding Remarks

Here is a summary of what has been covered in this chapter:

A database system can be construed as having three levels of architecture: the external, the conceptual and the internal. These levels are seamlessly interlinked by the DBMS.

The external level constitutes all the external views that end users have of the database.

The conceptual level relates to the logical structure of the database.

The internal level relates to the physical structure of the files making up the database.

The DBA is the official responsible for the planning, construction, Implementation and administration of the database.

The DBMS is the software that facilitates creation and administration of the database.

A database system can be construed as being comprised of a front-end system and a back-end system. The back-end system relates to the actual creation and administration of the database. The front-end system relates to the creation and administration of the user interface through which end users access the system.

By applying the principle of separating front-end from back-end, we can conceive of various database architectures.

With this background, we are now ready to move ahead and learn more about the relational database model. You will learn the foundations of the model, and why it is so important.

2.9 Review Questions

With the use of a diagram, explain the different levels of architecture of a database system.

Explain the acronyms DSL, DML, DCL, and DDL. How are they all related?

What are the primary functions of the DBA?

What are the main functions of the DBMS?

With the aid of an appropriate diagram, explain how the DBMS ensures that requests from end-users are satisfactorily addressed.

Discuss the functional components of a DBMS. Use an appropriate diagram to illustrate.

Explain the concept of front-end and back-end systems, and show how they add flexibility to the implementation of distributed database systems.

2.10 References and/or Recommended Readings

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York, NY: Addison-Wesley, 2002. See chapter 2.

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8h ed. Menlo Park, CA: Addison-Wesley, 2004. See chapter 2.

[Elmasri, 2007] Elmasri, Ramez and Shamkanf B. Navathe. Fundamentals of Database Systems 4th ed. Reading, MA: Addison-Wesley, 2004. See chapter 2.

[Garcia-Molina, 2002] Garcia-Molina, Hector, Jeffrey Ullman and Jennifer Widom. Database Systems: The Complete Book. Upper Saddle River, NJ: Prentice Hall, 2002. See chapter 1.

[Hoffer, 2007] Hoffer, Jeffrey A., Mary B. Prescott and Fred R. McFadden. Modern Database Management 8th ed. Upper Saddle River, NJ: Prentice Hall, 2007. See chapters 1&2.

[Martin, 1995] Martin, James, and Joe Leben. Client/Server Databases: Enterprise Computing. Upper Saddle River, NJ: Prentice Hall, 1995. See chapter 2.

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 5th ed. Boston, MA: Course Technology, 2002. See chapter 1.

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, NJ: Prentice Hall, 1997. See chapter 1.

PART B

The Relational Database Model

The next seven chapters will focus on the relational database model. As pointed out in chapter 1, there are other approaches to database design, but the relational model reigns supreme: it is superior to other traditional approaches; it remains a strong, viable alternative to or complement of (depending on your perspective) the more contemporary object-oriented model. Even if your choice is to construct an object database, a working knowledge of the relational model will still be required. For these and other reasons, mastery of the relational model is essential to good database administration and software engineering. The objectives of this division are

to clearly define, describe and discuss the relational database model;

to discuss how databases are planned, represented and implemented;

to discuss the theory, rationale, and practical ramifications of normalization;

to discuss important database integrity rules;

to discuss relational algebra and relational calculus as the foundations to modern database languages;

to discuss the standards to which database management systems ought to attain.

Chapters to be covered include:

Chapter 3 — The Relational Model

Chapter 4 — Integrity Rules & Normalization

Chapter 5 — Database Modeling and Design

Chapter 6 — Database User Interface Design

Chapter 7 — Relational Algebra

Chapter 8 — Relational Calculus

Chapter 9 — Relational System — a Closer Look

CHAPTER 3

The Relational Model

This chapter introduces you to the fundamental principles and concepts upon which subsequent chapters will be built. Discussion advances under the following captions:

Basic Concepts

Domains

Relations

Relational Database System

Identifying, Representing and Implementing Relationships

Relation-Attributes List and Relationship List

Non-relational Approaches

Summary and Concluding Remarks

3.1 Basic Concepts

The relational model is by far the most widely used model for database design. The model owes its success to the fact that it is firmly founded on mathematical principles (set theory and linear algebra) which have been tested and proven; like the underlying principles, the model itself has been tested and proven oven the years. Before we can proceed, there are some fundamental concepts to be introduced (Figure 3-1):

Figure 3-1. Basic Concepts

Figure 3-2 provides a list of commonly used relational terms and their informal equivalents:

Figure 3-2. Relational Terms and their Informal Equivalents

Figure 3-3 illustrates how the terms are applied. Observe that the idea of entity and relation seem to be similar. This will be clarified later. For now, assume that similarity.

Figure 3-3. Illustrating Basic Terms

3.2 Domains

A domain is a named set of scalar values from which attribute values are drawn. Scalar values are non-decomposable (atomic) values.

Each attribute of a relation must be defined on an underlying domain. The attribute values must come from the underlying domain (as illustrated in Figure 3-3).

Domains are conceptual; they may not be (and usually are not) actually stored in the database. Rather, the subsets of the domains containing actual attribute values are stored. Domains are analogous to data types in high level programming languages such as Pascal, C++, Java, etc.

A composite domain is a combination of simple domains. Whether a composite domain is used or is replaced by its constituent simple domains, is a design decision that should follow thoughtful consideration.

Example 1: Date is an excellent illustration of a composite domain, as explained below:

There can therefore be composite attributes. Composite domains are analogous to Pascal records and C++ structures. Few systems support composite domains and composite attributes.

Significance of Domains

An understanding of domains is critical for the following reason: If attributes of different relations (entities) come from the same domain, then comparisons can be made; otherwise, comparisons are meaningless.

Example 2: The following illustrations should emphasize the importance of domains:

3.3 Relations

A relation R on a collection of domains D1, D2, . . . Dn (not necessarily distinct) consists of two parts — a heading and a body.

The heading consists of a fixed set of attributes, or more precisely, attribute-domain pairs,

{(A1:D1), (A2:D2), . . . (An:Dn)}

such that each attribute corresponds to exactly one domain and n is the degree of the relation. Another term used to describe the heading of a relation is the predicate of the relation.

The body consists of a time-varying set of tuples where each tuple consists of a set of attribute-value pairs

{(A1:Vi1), (A2:Vi2), . . . (An:Vin)} (i = 1 . . . m)

where m is the number of tuples (cardinality) in the set. The body of the relation is also sometimes referred to as the proposition of the relation. The proposition defines a set of tuples whereby for each row in the relation, the respective attributes take on legal values from their respective domains.

Observe that the definition of a relation appears to be similar to that of an entity. There are two subtle differences:

The term relation as used, belongs to the field of relational systems. We talk about relations because we are discussing the relational model. Entities on the other hand, describe identifiable objects and/or situations.

Entity, as defined does not account for relationships. Relation on the other hand, accounts for entities as well as relationships. Thus in the relational model, we represent entities as relations and (M:M) relationships (between entities) as relations. A binary relation for instance, has two attributes. If both attributes are foreign keys and they both constitute the primary key, this binary relation actually represents a many-to-many relationship between two referenced relations; otherwise it is (a relation that can be construed as) an entity. This point will become clear as we proceed.

The foregoing underscores the point that entities can be construed as special kinds of relations. In designing a database, the software engineer or database designer commences by identifying entities during the requirements specification. After further analysis, these entities are eventually implemented by normalized relations.

Note: A unary relation differs from a domain in the sense that former is dynamic and the latter static.

3.3.1 Properties of a Relation

Based on the relational model, all relations have the following properties:

No duplicate tuples (records)

Records are unordered

Attributes are unordered

Attribute values are atomic

The first and last properties are constraints that both end users and software engineers should be cognizant of, since they have to manage data contained in the database; they are also of interest to the database designer. The second and third properties on the surface are immaterial to end users as well as software engineers; they are usually enforced by the DBMS in a manner that is transparent to the end user. However, when the DBMS is written, concern has to be given to accessing of records. Further, DBMS suites are typically written to give the illusion that the attributes of a relation are ordered.

3.3.2 Kinds of Relations

A database will consist of various types of relations, some of them at different stages of the system. The common categories of relations are mentioned below:

Base Relations are named and permanently represented in the database. They make up the conceptual schema of the database; they form the foundation of the database.

Views (virtual relations) are derived from named (base) relations. A view stores the definition of the virtual relation (derived from base relations), but stores no physical data. It is simply a logical (conceptual/external) interpretation of data stored in base relations. SQL views and System i logical files are good examples of views.

Snapshots are named, derived relations. They differ from logical views in that they are represented in terms of definition as well as physically stored data. From the perspective of the end user, a snapshot relation is typically (but not necessarily) read-only. To illustrate, consider two systems — System-A and System-B — which both need to access a database table, Table-X. Suppose that System-A has update rights to Table-X, but System-B does not. Table-X is therefore stored in System-A’s database; a duplicate version for read-only purposes, is stored in System-B, and is periodically updated (without user interference) from System-A.

Query Results: Queries are typically entered at a command prompt (they may be also embedded in high level language programs or stored in special query files). Results may be directed to screen, printer, or a named relation. An important principle to note is that a query when executed always results in a new relation. This principle will be elucidated later in the course.

Intermediate Results: The DBMS may create an intermediate relation to assist in furnishing a final answer to a complex query request. This will also be elucidated later in the course.

Temporary Relations are named relations that are destroyed at some point in time.

3.4 Relational Database System

A relational database system (RDBS) is a collection of time-varying normalized relations, managed through an appropriate user interface, and with desirable constraints and features that enhance the effective, efficient management of the database. These desirable features and constraints will be discussed (see chapter 9) as we progress through the course. The term normalized relations will be fully clarified in chapter 4; for now, just consider it to mean that the relations are designed to promote efficiency and accessibility.

The relations are conceptualized as tables and ultimately implemented as files. Each relation contains one and only one record type. Each relation has a primary key (chosen from a set of candidate keys). In many cases, the primary key is obvious and can be identified intuitively. In situations where this is not the case, the database designer, based on principles to be discussed in the next chapter, typically takes decision about the primary key.

Each record type is made up of atomic attributes. This means that each attribute is defined on a single domain, and can only have a value from that domain. Moreover, when data is loaded into the database, each record from any given table has a unique primary key value.

Superimposed on the database is a user interface that facilitates access of the database by end users. The database and the user interface are designed to ensure that certain objectives are met (section 1.2) and established standards are conformed to.

Steps in Building a Relational Database System

In constructing a RDBS, the following steps may bec pursued:

Identify entities

Identify relationships

Eliminate unnecessary relationships

Develop entity-relationship diagram (ERD), object-relationship diagram (ORD) or some equivalent model

Normalize the database

Revise E-R diagram, O-R diagram, or the equivalent model used

Design the user interface

Proceed to development phase

Note that these steps are to be pursued within the context of a software engineering project. Accordingly, this course assumes that the reader is familiar with steps a-d, and h. The rest of this chapter will review steps b-d, while providing some (additional) insights probably not covered in your (introductory) software engineering course. Chapters 4 and 5 will focus on steps e and f; step g is covered in chapter 6, and step h is covered in chapters 10–14.

3.5 Identifying, Representing, and Implementing Relationships

As mentioned earlier, a relationship is an inherent mapping involving two or more relations. In planning a relational database, it is very important to know how to identify and represent relationships. Of course, the ultimate objective is successful implementation of the model. Let us take some time to discuss these issues:

3.5.1 Identifying Relationships

To identify relationships, you have to know what a relationship is (review section 3.1) and what types of relationships there are. There are six types of relationships:

One-to-one (1:1) Relationship

One-to-many (1:M) Relationship

Many-to-one (M:1) Relationship

Many-to-many (M:M) Relationship

Component Relationship

Subtype Relationship

The first four types of relationships are referred to as traditional relationships because up until object model (for database design) gained preeminence, they were essentially the kinds of relationships that were facilitated by the relational model. Observe also, that the only difference between a 1:M relationship and an M:1 relation is a matter of perspective; thus, a 1:M relationship may also be described as an M:1 relationship (so that in practice, there are really three types of traditional relationships). Put another way:

For traditional relationships, to determine the type of relationship between two relations (entities) R1 and R2, ask and determine the answer to the following questions:

How many records of R1 can reference a single record of R2?

How many records of R2 can reference a single record of R1?

To test for a component relationship between any two relations R1 and R2, ask and determine the answer to the following questions:

Is (a record of) R1 composed of (a record of) R2?

Is (a record of) R2 composed of (a record of) R1?

For a subtype relationship, the test is a bit more detailed; for relations R1 and R2, ask and determine the answer to the following questions:

Is (a record of) R1 also a (a record of) R2?

Is (a record of) R2 also a (a record of) R1?

Possible answers to these questions are always, sometimes, or never. The possibilities are shown below:

3.5.2 Representing Relationships

Having identified the entities and relationships, the next logical question is, how do we represent them? Four approaches have been used: database hierarchies, simple networks, complex networks, the entity-relationship model and the object-relationship model. The first three approaches are traditional approaches that have made way for the more reputed latter two approaches. We will therefore start by discussing the latter two approaches.

The Entity-Relationship Model

The popular answer to this challenge of database representation is the entity-relationship diagram (ERD or E-R diagram). Figure 3-4a shows the symbols used in an ERD, while Figure 3-4b provides an illustration based on the Crows-Foot notation. In the diagram, the convention to show attributes of each entity has been relaxed, thus avoiding clutter. Note also that relationships are labeled as verbs so that in linking one entity to another, one can read an entity-verb-entity formulation. If the verb is on the right or above the relationship line, the convention is to read from top-to-bottom or left-to-right. If the verb is on the left or below the relationship line, the convention is to read from bottom-to-top or right-to-left.

Figure 3-4a. Symbols Used in E-R Diagrams

Figure 3-4b. Partial ERD for Manufacturing Firm

The ERD is normally used to show binary relationships but can also show n-ary relationships. In many cases, E-R diagrams show only binary relationships. For example, a possible ternary relationship not shown in Figure 3-4b is Supplier-Schedule (linking Supplier, InventoryItem and Project). The reason for this is the following principle:

The proof for this principle is beyond the scope of this course. However, we shall revisit it later, and provide additional clarifications. For now, a simple illustration will suffice: Figure 3-5 shows how the ternary Supplier-Schedule relationship may be broken down into three binary relationships. Since care must be taken in applying this principle, it will be further discussed in the next chapter.

Figure 3-5. Decomposing a Ternary Relationship

The Object-Relationship Model

As you are aware, or will soon learn (from your software engineering course), there are, broadly speaking, two alternate paradigms for software construction: the functional approach (which is the traditional approach) and the object-oriented (OO) approach. In an object-oriented environment, the comparative methodology for the E-R diagram is the object-relationship diagram (ORD or O-R diagram). The concept of an ORD is similar to that of an ERD, and the diagrams are also similar, but there are a few exceptions:

In the OO paradigm, the object type replaces the entity (type) of the relational model. Like the entity, an object type is a concept or thing about which data is stored. Additionally, the object type defines a set of operations, which will be applicable to all objects (instances) of that type.

The symbol used to denote an object type is similar to an entity symbol, except that it has two extended areas — one for the attributes of the object type, and the other for its defined operations.

The preferred diagramming convention is the UML (Unified Modeling Language) notation.

Depending on the OO development tool, there might be additional notations regarding the cardinality (more precisely, multiplicity) of the relationships represented.

A full treatment of the OO approach is beyond the scope of this course. You are no doubt familiar with using UML diagrams in your OO programming courses. For a quick review of the fundamentals, please see references [Lee, 2002] and [Martin, 1993]. However, in the interest of comprehensive coverage, an overview of the approach is provided in chapters 5 and 23.

Database Tree

A database tree (hierarchy) is a traditional alternative, which used to be employed prior to the introduction or the E-R model; it was successfully employed in a system called RAMIS (the original acronym stands for “Random Access Management Information System”). A database tree (hierarchy) is a collection of entities and 1:M relationships arranged such that the following conditions hold:

The root has no parent

Each subsequent node has a single parent

Figure 3-6 illustrates a database hierarchy. Observe that it looks like a general tree (review your data structures). Except for the root (node A), each node has a parent node that it references. Note also that all the relationships are 1:M relationships (traditionally referred to as parent-child relationships).

Figure 3-6. Example of a Hierarchy (Tree)

Database Networks

The database network approach is another traditional approach that is no longer employed. In the interest of historical context, a brief overview is provided here. A simple database network is a collection of entities and 1:M relationships arranged such that any member can have multiple parents, providing that the parents are different entities. Figure 3-7 illustrates the approach. It was successfully employed in a DBMS called the CODASYL system (the original CODASYL acronym stands for “Conference on Data Systems Languages”).

Figure 3-7. A Simple Network

A complex database network is a collection of entities and relationships, at least one of the relationships being an M:M relationship. Figure 3-8 illustrates. The complex network can be reduced to a simple network by replacing all M:M relationships with M:1 relationships. The technique for replacing M:M relationships will be discussed in the upcoming subsection.

Figure 3-8. Complex Network

3.5.3 Multiplicity of Relationships

It is customary to indicate on the ERD (or ORD), the multiplicity (also called the cardinality) of each relationship. By this we mean, how many occurrences of one entity (or object type) can be associated with one occurrence of the other entity (or object type). This information is particularly useful when the system is being constructed. Moreover, violation of multiplicity constraints could put the integrity of the system is question, which of course is undesirable. Usually, the DBMS does not facilitate enforcement of multiplicity constraints at the database level. Rather, they are typically enforced at the application level by the software engineer.

Several notations for multiplicity have been proposed, but the Chen notation (first published in 1976, and reiterated in [Chen, 1994]) is particularly clear; it is paraphrased here: Place beside each entity (or object type), two numbers [x,y]. The first number (x) indicates the minimum participation, while the second (y) indicates the maximum participation.

An alternate notation is to use two additional symbols along with the Crow’s Foot notation: an open circle to indicate a participation of zero, and a stroke (|) to indicate a participation of 1. The maximum participation is always indicated nearest to the entity (or object type) box.

For convenience, you could also use the Chen’s notation for multiplicity, along with the Crow’s Foot notation for representing the relationships. The Chen notation is preferred because of its clarity and the amount of information it conveys. Figure 3-9 provides an illustrative comparison of the two notations.

Figure 3-9. Illustrating Multiplicity Notations

3.5.4 Implementing Relationships

Assuming the E-R model, relationships can be implemented by following a set of guidelines as outlined below:

The foregoing strategies should underscore forcefully in your mind, the importance of foreign keys in database design. In fact, foreign keys are referred to as the “glue” that holds the database together. We shall revisit this concept in the next chapter.

In many textbooks and database environment, you will see and/or hear the term parent-child relationship. This is a rather lame term, borrowed from preexisting hierarchical database systems, to describe 1:1 and 1:M relationships. In a parent-child relationship, the parent relation is the referenced relation; the child relation is the referencing relation. Throughout this course, these terms are avoided because they are rather confusing, and do not accurately describe several scenarios involving 1:1 and/or 1:M relationships. Alternately, we will use no euphemism for 1:1 and 1:M relationships; instead of parent relation, we’ll say the referenced relation; instead of child relation, we say primary relation or referencing relation.

Figure 3-10. Implementing M:M Relationships

Figure 3-11. Illustrating Subtype and Component Relationships

Figure 3-12. Illustrating the Implementation of Subtype and Component Relationships

3.6 The Relation-Attributes List and Relationship List

In large, complex information systems projects, it is often impractical to attempt to develop and maintain ERDs, unless they are automatically generated and maintained by computer-aided software engineering (CASE) tools (more on these in chapter 5). Even when maintained by CASE tools, an ERD for such a project could become large, spanning several pages. Reading and interpretation then becomes difficult.

To circumvent the above challenges, a Relation-Attribute List (RAL) and a Relationship List (RL) may be constructed and maintained. The former maintains information on all relations of the system and the latter maintains information on all relationships implemented in the system. Figures 3-13 and 3-14 illustrate partial RAL and RL for the database model of Figure 3-4b. As you examine these figures, please note the following:

In practice, the format of the RL shown in Figure 3-14b is used as the final list over the format shown in Figure 3-14a (the format used in Figure 3-14a can be deduced by simply identifying all possible relationships among entities; hence, it may contain optional relationships and is therefore useful as a first draft).

The revised RL of Figure 3-14b has been stripped of all M:M relationships (review section 3.5.4 on treating M:M relationships).

The relations PurchaseInvSummary and PurchaseInvDetail of Figure 3-13, are used to replace the M:M relationship between PurchaseInvoice and InventoryItem in Figure 3-4b. Similarly, the relations PurchaseOrdSummary and PurchaseOrdDetail are used to replace the M:M relationship between PurchaseOrder and InventoryItem in Figure 3-4b.

In constructing the RAL and RL, it is sometimes useful to use the RL to refine the RAL and vice versa. In particular, once you have identified all the (mandatory) relationships, you may use this along with the principles outlined in section 3.5.4 to refine the RAL.

Remember, the model of Figure 3-4b, RAL of Figure 3-13, and the RL of Figure 3-14 do not represent a comprehensive coverage of the database requirements of a manufacturing firm; neither are they intended to be. Rather, they serve as useful illustrations.

Figure 3-13. Partial Relation-Attributes List for a Manufacturing Firm’s Database

Figure 3-14a. Relationships List for a Manufacturing Firm’s Database

Figure 3-14b. Refined Relationships List for a Manufacturing Firm’s Database

3.7 Non-Relational Approaches

Prior to development of the relational model, the following approaches used to be employed:

Inverted List Approach: Exemplified by DATACOM/DB

Hierarchical Approach: Exemplified by Information Management System (IMS) database and the Random Access Management Information System (RAMIS)

Network Approach: Exemplified by Conference on Data Systems Languages (CODASYL) initiative and Integrated Database Management System (IDMS) initiative

The strengths of the relational approach when compared to these approaches are its sound mathematical base, its flexibility, robustness, and simplicity.

In recent times, the object oriented (OO) model has been challenging the relational model on performance and efficiency for certain scenarios. Nonetheless, we expect that the two technologies will continue to peaceably coexist; huge investments have been made in relational database systems, and it is not likely that these will be abandoned. What is more likely to happen is that systems will be built based on relational databases, with object oriented user interfaces superimposed.

3.8 Summary and Concluding Remarks

Let us summarize what we have covered in this very important chapter:

The relational database model is based on a number of fundamental concepts relating to the following: entity, entity set, relation, relationship, tuple, candidate key, primary key, alternate key, foreign key, domain, cardinality, degree.

A domain is a named set of scalar values from which attribute values are drawn.

A relation consists of a heading and a body. The heading consists of atomic attributes defined on specific domains. The body consists of a set of attribute-values pairs, where each attribute has a value drawn from its domain.

In a database system, you are likely to find any combination of the following types of relations: base relations, logical views, snapshots, query results, intermediate results, and temporary relations.

A relational database system (RDBS) is a collection of time-varying normalized relations, managed through an appropriate user interface, and with desirable constraints and features that enhance the effective, efficient management of the database.

A relationship is an inherent mapping involving two or more relations. There are six types of relationships: one-to-one (1:1)relationship, one-to-many (1:M) relationship, many-to-one (M:1) relationship, many-to-many (M:M) relationship, component relationship, and subtype relationship.

An E-R diagram (ERD) is a graphical representation of a database model. It is important to know how to represent relations/entities and relationships on the ERD.

It is important to know how to implement the various types of relationships in the actual database design.

The relation-attributes list (RAL) and relationship list (RL) are two useful alternatives to the E-R diagram, especially for large, complex systems.

Database approaches that preexisted the relational approach include the inverted-list approach, the hierarchical approach and the network approach.

A contemporary alternative to the relational approach is the object-oriented approach. However, given the efficacy of both approaches, it is more likely that they will complement each other in the future, rather than compete against each other.

Take the time to go over this chapter more than once if you need to, and make sure that you are comfortable with the concepts covered. In the upcoming chapter, we will build on the information covered in this chapter, as we discuss integrity rules and normalization. These two topics form the foundation for the rest of the course.

3.9 Review Questions

Clarify the following terms:

Entity

Attributes

Entity set

Relation

Relationship

Tuples

Primary key

Candidate key

Foreign key

Domain

Cardinality

Multiplicity

Degree

Develop an illustrative data model to explain how these concepts are related.

Give the formal definition of a relation. Explain the different kinds of relations that may be found in a database.

Outline the steps to be observed in constructing a relational database system.

What are the types of relationships that may exist among information entities? For each, explain how it is identified, and how it is implemented.

Think about a problem that requires a small database system (requiring six to twelve entities or object types). Do the following:

Identity all required entities (object types).

Identify relationships among the entities (object types).

Develop an ERD or ORD.

Propose a relation-attributes list (RAL) and a relationship list (RL) to represent your model.

3.10 References and/or Recommended Readings

[Bruegge, 2002] Bruegge, Bernard and Allen H. Dutdt. Object-Oriented Software Engineering: Conquering Complex and Changing Systems. Upper Saddle River, NJ: Prentice Hall, 2002.

[Chen, 1994] Chen, Peter. “The Entity-Relationship Model – Toward a Unified View of Data,” In Readings in Database Systems 2nd ed., pages 741-754. San Francisco, CA: Morgan Kaufmann, 1994.

[Chen, 2007] Chen, Peter. http://bit.csc.lsu.edu/~chen/display.html

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York: Addison-Wesley, 2002. See chapters 3 and 11.

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8th ed. Menlo Park, California: Addison-Wesley, 2004. See chapters 3 and 6.

[Edward, 1992] Edward, David. Information Modeling: Specification and Implementation. Eaglewood Cliffs, NJ: Prentice Hall, 1992. See chapters 2-4.

[Elmasri, 2007] Elmasri, Ramez and Shamkanf B. Navathe. Fundamentals of Database Systems 5th ed. Reading, Massachusetts: Addison-Wesley, 2007. See chapters 3 and 7.

[Foster, 2010] Foster, Elvis C. Software Engineering — a Methodical Approach. Bloomington, IN: Xlibris Publishing, 2010. See chapter 10.

[Garcia-Molina, 2002] Garcia-Molina, Hector, Jeffrey Ullman and Jennifer Widom. Database Systems: The Complete Book. Upper Saddle River, New Jersey: Prentice Hall, 2002. See chapters 2 and 3.

[Hoffer, 2007] Hoffer, Jeffrey A., Mary B. Prescott and Fred R. McFadden. Modern Database Management 8th ed. Upper Saddle River, New Jersey: Prentice Hall, 2007. See chapters 3 and 4.

[Kifer, 2005] Kifer, Michael, Arthur Bernstein and Philip M. Lewis. Database Systems: An Application-Oriented Approach 2nd ed. New York: Addison-Wesley, 2005. See chapters 3 and 4.

[Kroenke, 2004] Kroenke, David M. Database Processing: Fundamentals, Design and Implementation 9th ed. Upper Saddle River, New Jersey: Prentice Hall, 2004. See chapters 2 and 3.

[Lee, 2002] Lee, Richard C. and William M. Tepfenhart. Practical Object-Oriented Development With UML and Java. Upper Saddle River, NJ: Prentice Hall, 2002.

[Lewis, 2002] Lewis. Phillip M., Arthur Bernstein and Michael Kifer. Databases and Transaction Processing: An Application Oriented Approach. New York: Addison-Wesley, 2002. See chapters 4 and 5.

[Martin, 1993] Martin, James and James Odell. Principles of Object-Oriented Analysis and Design. Upper Saddle River, New Jersey: Prentice Hall, 1993. See chapters 6 and 7, and appendix A.

[Martin, 1995] Martin, James. Client/Server Databases: Enterprise Computing. Upper Saddle River, New Jersey: Prentice Hall, 1995. See chapter 3.

[Riccardi, 2003] Riccardi, Greg. Database Management With Web Site Development Applications. Boston, Massachusetts: Addison-Wesley, 2003. See chapters 3 and 4.

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, Massachusetts: Course Technology, 2007. See chapters 2 and 3.

[Silberschatz, 2006] Silberschatz, Abraham, Henry F. Korth and S. Sudarshan. Database Systems Concepts 5th ed. New York, New York: McGraw-Hill, 2006. See chapters 2 and 6.

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, New Jersey: Prentice Hall, 1997. See chapter 2.

CHAPTER 4

Integrity Rules and Normalization

In order to design high quality databases, you need to be cognizant of the fundamental integrity and normalization rules. We will discuss these rules in this chapter. Sub-topics to be discussed include:

Fundamental Integrity Rules

Foreign Key Concept

Rationale for Normalization

Functional Dependence and Non-loss Decomposition

First Normal Form

Second Normal Form

Third Normal Form

Boyce/Codd Normal Form

Fourth Normal Form

Fifth Normal Form

Other Normal Forms

Summary and Concluding Remarks

4.1 Fundamental Integrity Rules

Two fundamental integrity rules that the database designer must be cognizant of are the entity integrity rule and the referential integrity rule.

Entity Integrity Rule: The entity integrity rule states that no component of the primary key in a base relation is allowed to accept nulls. Put another way, in a relational model, we never record information about something that we cannot identify. Three points are worth noting here:

The rule applies to base relations.

The rule applies to primary key, not alternate keys.

The primary key must be wholly non-null.

Referential Integrity Rule: The referential integrity rule states that the database must not contain unmatched foreign key values. By unmatched foreign key value, we mean a non-null foreign key value for which there is no match in the referenced (target) relation. Put another way, if B references A then A must exist. The following points should be noted:

The rule requires that foreign keys must match primary keys, not alternate keys.

Foreign key and referential integrity are defined in terms of each other. It is not possible to explain one without mentioning the other.

4.2 Foreign Key Concept

The concept of a foreign key was introduced in the previous chapter. Let us revisit this concept, by introducing a more formal definition:

We will use the notation R1 → R2 to mean, the relation R1 references the relation R2. In this case, R1 is the referencing (primary) relation and R2 is the referenced relation. Since R1 is the referencing relation, it contains a foreign key. We will also use the notion R1{A, B, C, ...} to mean, the relation R1 contains attributes A, B, C, and so on. Where specific examples are given, the relation-name will be highlighted or placed in upper case; attribute-names of specific examples will not be highlighted when stated with the related relation; however, they will be highlighted when reference is made to them from the body of the text.

Based on the definition of a foreign key, the following consequential points should be noted:

The foreign key and the referenced primary key must be defined on the same domain. However, the attribute-names can be different (some implementations of SQL may require that they be identical).

The foreign key need not be a component of the primary key for the host (holding) relation (in which case nulls may be accepted, but only with the understanding that they will be subsequently updated).

If relations Rn, R(n-1), R(n-2) .... R1 are such that Rn → R(n-1) → R(n-2) → .... R2 → R1 then the chain Rn to R1 forms a referential path.

A relation can be both referenced and referencing. Consider the referential path R3 → R2 → R1. In this case, R2 is both a referenced and a referencing relation.

We can have a self-referencing relation. Consider for example, the relation Employee{Emp#, EmpName .... MgrEmployee#} with primary key [Emp#]where attribute MgrEmp# is a foreign key defined on the relation Employee. In this case we have a self-referencing relation.

More generally, a referential cycle exists when there is a referential path from Rn to itself: Rn → R(n-1) → .... R1 → Rn

Foreign-to-primary-key matches are said to be the “glue” that holds the database together. As you will see, relations are joined based on foreign keys.

Deletion of Referenced Tuples

Now that we have established the importance of foreign keys, we need to address a question: How will we treat deletion of referenced tuples? Three alternatives exist.

Restrict deletion to tuples that are not referenced.

Cascade deletion to all referencing tuples in referencing relations.

Allow the deletion but nullify all referencing foreign keys in referencing relations.

The third approach is particularly irresponsible, as it could quite quickly put the integrity of the database in question, by introducing orphan records. Traditionally, DBMS suites implement the restriction strategy (and for good reasons). The cascading strategy has been surfacing in contemporary systems, as an optional feature. It must be used with much care, as it is potentially dangerous when used without discretion.

4.3 Rationale for Normalization

Normalization is the process of ensuring that the database (conceptual schema) is defined in such a manner as to ensure efficiency and ease of data access. Normalization ensures the following:

Data integrity

Control of redundancy

Logical data independence

Avoidance of modification anomalies

The following problems can be experienced from having un-normalized files in a system:

Data redundancy that leads to the modification anomalies

Modification anomalies which include:

✓    Insertion anomaly: Data cannot be inserted when it is desirable; one has to wait on some future data, due to organization of the data structure

✓    Deletion anomaly: Deletion of some undesirable aspect(s) of data necessarily means deletion of some other desirable aspect(s) of data

✓    Update anomaly: Update of some aspect(s) of data necessarily means update of other aspect(s) of data

Inefficient file manipulation; lack of ease of data access

Inhibition to the achievement of logical data independence

Compromise on the integrity of data

Pressure on programming effort to make up for the poor design

Figure 4-1 indicates the six most commonly used normal forms. The hierarchy is such that a relation in a given normal form is automatically in all normal forms prior to it. Thus a relation in the second normal form (2NF) is automatically in the first normal form (1NF); a relation in the third normal form (3NF) is in 2NF and so on. Edgar Frank Codd defined the first three normal forms in the early 1970s; the Boyce-Codd normal form (BCNF) was subsequently deduced from his work. The fourth and fifth normal forms (4NF and 5NF) were subsequently defined by Ronald Fagin in the late 1970s.

Figure 4-1. Normal Forms

The normalization procedure involves decomposing relations into other relations of repeatedly higher normal forms. The process is reversible. Moreover, normalization must be conducted in a manner that ensures that there is no loss of information.

4.4 Functional Dependence and Non-loss Decomposition

Before discussion of the normal forms, we need to define and clarify two fundamental concepts: functional dependence and non-loss decomposition.

4.4.1 Functional Dependence

Given a relation, R{A, B, C, ...}, then attribute B is functionally dependent on attribute A, written A → B (read as “A determines B”) if and only if (denoted iff from this point onwards) each value of A in R has precisely one B-value in R at any point in time. Attributes A and B may or may not be composite.

An alternate way to describe functional dependence (FD) is as follows: Given a value of attribute A, one can deduce a value for attribute B since any two tuples which agree on A must necessarily agree on B.

Example 1:

From definition of primary key, all attributes of a relation are functionally dependent on the primary key. This is precisely what is required; in fact, an attribute (or group of attributes) qualifies as a candidate key iff all other attributes of the entity are dependent on it.

We need to further refine our concept of FD by introducing another term — full functional dependence: Attribute B is said to be fully functionally dependent on attribute A if it is functionally dependent on A and not functionally dependent on any proper subset of A.

As a spinoff from the definition of functional dependence, please note the following:

FD constraints have similarities with referential constraints, except that here, reference is internal to the relation.

FDs help us to determine primary keys.

Each FD defines a determinant in a relation: the attribute(s) on the right are dependent on the attribute(s) on the left; the attribute(s) on the left constitute(s) a determinant.

4.4.2 Non-loss Decomposition

Suppose we have a relation R0 as follows: R0{Suppl#, SuplName, Item#, ItemName, Quantity, SuplStatus, Location}

Functional dependencies of R0 are illustrated in Figure 4-2; they may also be listed as follows:

[Suppl# , Item#] → {Quantity, SuplName, SuplStatus, Location, ItemName}

Suppl# → {SuplName, SuplStatus, Location}

Item# → ItemName

Figure 4-2. FD Diagram for Relation R0

Storing R0 this way causes duplication. The reason is that R0 is not sufficiently normalized. As an alternate, we could have the following:

R1{Supl#, SuplNam, Location, SuplStatus}

R2{Item#, ItemName}

R3{Supl#, Item#, Quantity}

R1, R2, and R3 constitute (an example of) a non-loss decomposition (NLD) of R0.

Here is a formal definition of an NLD:

Notice that we have glided into two new terms, projection and join. These will be formerly treated later in the course. Suffice it now to say that a projection on a relation is an extraction (into a new relation) of some attributes of the relation; a join requires at least two relations and may be construed as the opposite of a projection: If you can project R into P1 and P2, then you may be able to join P1 and P2 to yield R.

Given this definition, we need to address the following questions:

How do we find non-loss decompositions?

When should we replace a relation by a non-loss decomposition?

What are the advantages?

Heath’s theorem (of [Heath, 1971]) addresses questions (1) and (2). The answer to the third question is stated in section 4.3 above. Heath’s theorem is stated below:

Example 2: Proof of Heath’s Theorem:

Corollary of Heath’s Theorem

An important corollary from Heath’s theorem is as follows:

Example 3: The following example illustrates the importance of the above-mentioned corollary:

Conclusion

Based on Heath’s theorem and its corollary, we can assert with confidence, the following advice:

Decompose only when there is a non-loss decomposition such that the resulting relations do not share a candidate key.

Do not decompose if each resulting relation does not have a primary key.

4.5 The First Normal Form

By definition, all relations are in 1NF. This is by no means coincidental, but by design: we defined a relation to consist of atomic attributes, and subject to the entity integrity constraint and the referential integrity constraint. However, as you will soon see, having relations in 1NF only is often not good enough.

Example 4:

Problems with Relations in 1NF Only

Relation R0 of the previous section is in 1NF only. However it is undesirable to store it as is due to a number of problems. In the interest of clarity, the relation is restated here:

R0{Supl#, SuplName, Item#, ItemName, Quantity, SuplStatus, Location}

Functional dependencies of R0 as illustrated in Figure 4-2 are as follows:

FD1: [Suppl#, Item#] → {Quantity, SuplName, SuplStatus, Location, ItemName}

FD2: Suppl# → {SuplName, SuplStatus, Location}

FD3: Item# → ItemName

The following data anomalies exist with R0 (and most relations in 1NF only):

Replication of data: Every time we record a supplier - item pair, we also have to record supplier name and item name.

Insertion anomaly: We cannot insert a new item until it is supplied; neither can we insert a new supplier until that supplier supplies some item.

Deletion anomaly: We cannot delete an item or a supplier without destroying an entire shipment, as well as information about a supplier’s location.

Update anomaly: If we desire to update a supplier’s location or item name, we have to update several records, in fact, an entire shipment, due to the duplication problem.

Insertion, deletion update anomalies constitute modification anomalies, caused by duplication of data due to improper database design.

4.6 The Second Normal Form

By non-key attribute, we mean that the attribute is not part of the primary key. Relation R0 (of the previous section), though in 1NF, is not in 2NF, due to FD2 and FD3. Using Heath’s theorem, we may decompose relation R0 as follows (note that the abbreviation PK is used to denote the primary key):

R1{Supl#, Sname, Location, SuplStatus} PK[Suppl#]

R2{Item#, Itemname} PK[Item#]

R3{Supl#, Item#, Qty} PK[Supl#, Item#]

We then check to ensure that the resulting relations are in 2NF (and they are).

So based on the definition of 2NF, and on the authority of Heath’s theorem, we would replace R0 with R1, R2, and R3. Please note the consequences of our treatment of R0 so far:

The problems with relations in 1NF only have been addressed.

By decomposing, we have introduced foreign keys in relation R3.

JOINing is the opposite of PROJecting. We can rebuild relation R0 by simply JOINing R3 with R1 and R3 with R2, on the respective foreign keys.

From the definition of 2NF, two observations should be obvious: Firstly, if you have a relation with a single attribute as the primary key, it is automatically in 2NF. Secondly, if you have a relation with n attributes and n-1 of them form the primary key, the relation is also in 2NF.

Problems with Relations in 2NF Only

In this example, relations R2 and R3 are in 2NF (in fact they are in 3NF), but we still have potential problems with R1: What if we have a situation where there may be several suppliers from a given location? Or what if we want to keep track of locations of interest? In either case, we would have modification anomalies as described below:

Insertion anomaly: We cannot record information about a location until we have at least one supplier from that location.

Deletion anomaly: We cannot delete a particular location without also deleting supplier(s) from that location.

Update anomaly: If we wish to update information on a location, we have to update all supplier records from that location.

These problems can be addressed if we take the necessary steps to bring R1 into the third normal form (3NF). But first, we must define what 3NF is.

4.7 The Third Normal Form

Transitive dependence refers to dependence among non-key attributes. In particular, if A → B and B → C, then C is transitively dependent on A (i.e. A→ C transitively).

In the previous section, relation R1 is problematic because it is not in 3NF. If it is desirable to store additional information about the locations as indicated in the previous section, then we must be smart enough to discern that location is to be treated as an entity with attributes such as location code, location name (and perhaps others). Using Heath’s theorem, we may therefore decompose R1 as follows:

R4{Supl#, Sname, LocationCode} PK[Supl#]

R5{LocationCode, LocationName} PK[LocationCode]

We now check to ensure that the relations are in 3NF (and they are). Again, please take careful notice of the consequences of our

actions to this point:

The problems with relations in 2NF only have been addressed.

Again, by decomposing, we have introduced a foreign key in relation R4.

We can rebuild relation R1 by simply JOINing R4 with R5 on the foreign key.

From the definition of 3NF, it should be obvious that if you have a relation with one candidate key and n mutually independent non-key attributes, or only one non-key attribute, it is in 3NF.

Problems with Relations in 3NF Only

Relations R2, R3, R4, and R5 above are all in 3NF. However, it has been found that 3NF-only relations suffer from certain inadequacies. It is well known that 3NF does not deal satisfactorily with cases where the following circumstances hold:

There are multiple composite candidate keys in a relation.

The candidate keys overlap (i.e. have at least one attribute in common).

For these situations, the Boyce-Codd normal form (BCNF) provides the perfect solution. As you shall soon see, the BCNF is really a refinement of 3NF. In fact, where the above-mentioned conditions do not hold, BCNF reduces to 3NF.

4.8 The Boyce-Codd Normal Form

Simply, Boyce-Codd normal form (BCNF) requirement states:

A determinant is an attribute (or group of attributes) on which some other attribute(s) is (are) fully functionally dependent. Examination of R2, R3, R4, and R5 above will quickly reveal that they are in BCNF (hence 3NF). We therefore need to find a different example that illustrates the importance of BCNF.

Consider the situation where it is desirous to keep track of animals in various zoos, and the assigned keepers for these animals. Let us tentatively construct the relation R6 as shown below:

R6{Zoo, Animal, Keeper}

Assume further that that a keeper works at one and only one zoo. We can therefore identify the following FDs:

[Zoo, Animal] → Keeper

Keeper → Zoo

Given the above, we conclude that [Zoo, Animal] is the primary key. Observe that R6 is in 3NF but not in BCNF, since Keeper is not a candidate key but is clearly a determinant. Using Heath’s theorem, we may decompose R6 as follows:

R7{Animal, Keeper} PK[Animal]

R8{Keeper, Zoo} PK[Keeper]

As on previous occasions, let us examine the consequences of

our action:

By achieving BCNF, we benefit from further reduction in data duplication, and modification anomalies.

A further advantage is that we can now store dangling records. In our example, a keeper can be assigned to a zoo even before he/she is assigned an animal.

One possible drawback with BCNF is that more relations have to be accessed (joined) in order to obtain useful information. Again referring to the example, R7 must be joined with R8 in order to derive Zoo-Animal pairs.

Observe: The principle of BCNF is very simple but profound. By being guided by it, you can actually bypass obtaining 1NF, 2NF and 3NF relations, and move directly into a set of BCNF relations. Adopting this approach will significantly simplify the analysis process. Moreover, in most practical situations, you will not be required to normalize beyond BCNF. This approach will be further clarified in the next chapter.

4.9 The Fourth Normal Form

The fourth normal form (4NF) relates to the situation where mutually independent, but related attributes form a relation and the inefficient arrangement causes duplication and hence modification anomalies. Consider the database file, CTT-Schedule, representing course-teacher-text combinations in an educational institution. Assume the following:

A course can be taught by several teachers.

A course can require any number of texts.

Teachers and texts are independent of each other i.e. the same texts are used irrespective of who teaches the course.

A teacher can teach several courses.

Figure 4-3 provides some sample data for the purpose of illustration.

Figure 4-3. CTT-Schedule File

Note that the theory so far, does not provide a method of treating such a situation, except flattening the structure (by making each attribute part of the primary key) as shown below:

R9{Course, Teacher, Text} PK[Course, Teacher, Text]

Since R9 is keyed on all its attributes, it is in BCNF. Yet, two potential problems are data redundancy and modification anomalies (the former leading to the latter). In our example, in order to record that Calculus II is taught by both Professor B and Professor C, four records are required. In fact, if a course is taught by p professors and requires n texts, the number of records required to represent this situation is p*n. This is extraordinary, and could prove to be very demanding on storage space.

Relation R9, though in BCNF, is not in 4NF, because it has a peculiar dependency, called a multi-valued dependency (MVD). In order to state the 4NF, we must first define MVD.

4.9.1 Multi-valued Dependency

A multi-valued dependency (MVD) is defined as follows:

Please note the following points arising from the definition of an MVD:

For MVD, at least three attributes must exist.

FDs are MVDs but MVDs are not necessarily FDs.

A -» B reads “A multi-determines B” or “B is multi-dependent on A.”

Let us get back to R9: Course -» Text/Teacher. Note that Course is the pivot of the MVD. Course -» Teacher since Teacher depends on Course, independent of Text. Course -» Text since Text depends on Course, independent of Teacher.

4.9.2 Fagin’s Theorem

Fagin’s theorem (named after Ronald Fagin who proposed it) may be stated as follows:

Note that like Heath’s theorem, which prescribes how to treat FDs, Fagin’s theorem states exactly how to treat MVDs. With this background, we can proceed to defining the 4NF:

In the current example, R9 is not in 4NF. This is so because although it is in BCNF, an MVD exists. Using Fagin’s theorem, we may decompose it as follows:

R10{Course, Text} PK[Course, Text]

R11{Course, Teacher} PK[Course, Teacher]

Note  Fagin’s theorem prescribes a method of decomposing a relation containing an MVD that is slightlydifferent from the decomposition of an FD as prescribed by Heath’s theorem: Figure 4-4 clarifies this.

Figure 4-4. Treating MVDs

4.10 The Fifth Normal Form

So far we have been treating relations that are decomposable into two other relations. In fact, there are relations which cannot be so decomposed, but can be decomposed into n other relations where n > 2. They are said to be n-decomposable relations (n > 2). The fifth normal form (5NF) is also commonly referred to as the projection-join normal form (PJNF) because it relates to these (n > 2) projections (of a relation not in 5NF) into decompositions that can be rejoined to yield the original relation.

Recall the SupplierSchedule relationship (linking suppliers, inventory items and projects) mentioned in section 3.5; it is represented here as outlined below:

SupplierSchedule{Suppl#, Item#, Proj#} PK[Suppl#, Item#, Proj#]

The relation represents a M:M relationship involving Suppliers, Items, and Projects. Observe the following features about the relation:

SupplierSchedule is keyed on all attributes and therefore by definition, is in BCNF. By inspection, it is also in 4NF.

It is not possible to decompose this relation into two other relations.

If there are S suppliers, N items and J projects, then theoretically, there may be up to S*N*J records. Not all of these may be valid.

If we consider S suppliers, each supplying N items to J projects, then it does not take much imagination to see that a fair amount of duplication will take place, despite the fact that the relation is in 4NF.

Let us examine a possible decomposition of SupplierSchedule as shown in Figure 4-5. If we employ the first two decompositions only, this will not result in a situation that will guarantee us the original SupplierSchedule. In fact, if we were to join these two decompositions (SI and IP), we would obtain a false representation of the original relation. The third projection (PS) is absolutely necessary, if we are to have any guarantee of obtaining the original relation after joining the projections.

Figure 4-5. Illustrating Possible Decompositions of Supplier-Schedule

Note  The first join produces SupplierSchedule plus additional spurious tuples. The effect of the second join is to eliminate the spurious tuples. To put it into perspective, SupplierSchedule is subject to a (time independent) 3-decomposable (3D) constraint, namely:

This is an example of a join dependency (JD) constraint.

4.10.1 Definition of Join Dependency

A join dependency (JD) constraint may be defined as follows:

Relations that are in 4NF, but not in 5NF (such as SupplierSchedule) suffer from duplication, which in turn leads to modification anomalies. These problems are directly related to the presence of the JD constraint(s) in such relations. Fagin’s theorem for 5NF relations provides the solution.

4.10.2 Fagin’s Theorem

Fagin’s theorem for the fifth normal form (5NF) states:

Now examine relation SupplierSchedule. SupplierSchedule is not in 5NF because it has a JD (i.e. the JD constraint) that is not a consequence of its candidate key. In other words, SupplierSchedule can be decomposed, but this is not implied by its candidate key [Supl#, Item#, Proj#].

Note  For most practical purposes, you only have to worry about 5NF if you are trying to implement an M:M relationship involving more than two relations. Once in 5NF, further decompositions would share candidate keys and are therefore to no avail (recall corollary of Heath’s theorem). Notwithstanding this, other normal forms have been proposed, as will be discussed in the upcoming section.

4.11 Other Normal Forms

The field of Database Systems is potentially a contemptuous one. Indeed, there are accounts of former friends or colleagues becoming foes over database quibbles (in Figure 4-7 of section 4.12, the current author relates a personal experience he had as a young software engineer on a project of national importance). Various individuals have proposed several database theorems and methodologies, but they have not all gained universal acceptance as have the normal forms of the previous sections. Two additional normal forms that have been, and will no doubt continue to be the subject of debate are the domain-key normal form (DKNF) and the sixth normal form (6NF). Without picking sides of the debate on these two normal forms, this section will summarize each.

4.11.1 The Domain-Key Normal Form

The domain-key normal form (DKNF) was proposed by Ronald Fagin in 1981. Unlike the other normal forms which all relate to FDs, MVDs and JDs, this normal form is defined in terms of domains and keys (hence its name). In his paper, Fagin showed that a relation DKNF has no modification anomalies, and that a relation without modification anomalies must be in DKNF. He therefore argued that a relation in DKNF needed no further normalization (at least, not for the purpose of reducing modification anomalies). The definition of DKNF is as follows:

This definition contains three important terms that need clarification:

A constraint is used to mean any rule relating to static values of attributes. Constraints therefore include integrity rules, editing rules, foreign keys, intra-relation references, FDs and MVDs, but exclude time-dependent constraints, cardinality constraints and constraints relating to changes in data values.

A key is a unique identifier of a row (as defined in Chapter 3).

A domain is a pool of legal attribute values (as defined in Chapter 3).

The implication of the DKNF is clear: If we have a relation that contains constraint(s) that is (are) not a logical consequence of its (candidate) key and domains, then that relation is not in DKNF, and should therefore be further normalized. The DKNF as proposed by Fagin, therefore represents an ideal situation to strive for.

Unfortunately, a number of problems arise from consideration of DKNF:

Any constraint that restricts the cardinality of a relation (i.e. the number of tuples in the relation) will render it in violation of DKNF. (It was perhaps for this reason that Fagin excluded from his definition of constraints, time-dependent constraints or constraints relating to data values.) However, there are many relations for which such constraints are required.

There is no known algorithm for converting a relation to DKNF. The conversion is intuitive and for this reason described as artistic rather than scientific.

Not all relations can be reduced to DKNF (relations with cardinality constraints fall in this category).

It is not precisely clear as to when a relation can be reduced to DKNF.

For these reasons, the DKNF has been compared by Date (see [Date, 2006]) to a “straw man... of some considerable theoretical interest but not yet of much practical ditto.”

4.11.2 The Sixth Normal Form

A sixth normal form (6NF) has been proposed by C. J. Date in [Date, 2003], after several years of exploitation, expounding, and research in the field of database systems. It relates to so-called temporal databases. Date wrote a whole book on the subject; a summary of the essence is presented in this sub-section. Date defines a temporal database as a database that contains historical data as well as, or instead of current data. Temporal databases are often read-only databases, or update-once databases, but they could be used otherwise. In this sense, a temporal database may be considered as a precursor to a data warehouse (discussed in Chapter 24).

For the purpose of illustration, assume that we are in a college or university setting and desire to store the relation Course as defined below:

Course {CourseNo, CourseName, CourseCred}

Suppose further that we desire to show different courses at the time they existed in the database. To make our analysis more realistic, let us also make the following additional assumptions:

The primary key is CourseNo; for any given course, the attribute CourseNo cannot be changed.

For any given course, the attribute CourseName may be changed any point in time.

For any given course, the attribute CourseCred may be changed any point in time.

We may be tempted to introduce a timestamp on each tuple, and therefore modify the definition of Course as follows:

Course {CourseNo, CourseName, CourseCred, EffectiveDate}

Figure 4-6 provides some sample data for the Course relation. By introducing the attribute EffectiveDate, we have actually introduced a new set of concerns as summarized below:

If we assume that the FD CourseNo → {CourseName, CourseCred, EffectiveDate} is (still) in vogue, then Course is in 5NF. However, in this case, if the CourseName or CourseCred of a given Course tuple changes at a given effective date, there is no way of showing what it was before, unless we create a new course and assign a new CourseNo. In either case, this clearly, is undesirable.

Suppose we assume the FDs CourseNo → CourseName and [CourseNo, EffectiveDate] → CourseCred.Then, the relation is not in 2NF, and therefore needs to be decomposed into two decompositions:

CourseDef {CourseNo, CourseName} and

CourseTimeStamp {CourseNo, EffectiveDate, CourseCred}

Both of these relations would now be in 5NF. However, if we now desire to change the CourseName of a course for a given effective date, we cannot represent this in the current schema.

We could introduce a surrogate (say CourseRef) into relation Course, and key on the surrogate, while ignoring the FDs stated in (1) and (2) above. In this case, Course would be in violation of 3NF, and if we attempt to decompose, we would revert to the situation in case (2) above.

Figure 4-6. Sample Data for the Course Relation

The reason for these problems can be explained as follows: The relation Course as described, defines the following predicate:

Each course is to be accounted for (we say Course is under contract).

Each course has a CourseName which is under contract.

Each course has a CourseCred which is under contract.

The predicate involves three distinct propositions. We are attempting to use the timestamp attribute (EffectiveDate) to represent more than one proposition about the attribute values. This, according to Date, is undesirable and violates the sixth normal form.

We now state Date’s theorem for the sixth normal form (6NF):

Note  6NF as defined, essentially refines 5NF. It is therefore obvious from the definition that a relation in 6NF is necessarily in 5NF also.

Let us now revisit the Course relation: With the introduction of the timestamp attribute (EffectiveDate), and given the requirements of the relation, there is a non-trivial JD that leads to the following projections:

CourseInTime {CourseNo, EffectiveDate} PK [CourseNo, EffectiveDate]

CourseNameInTime {CourseNo, CourseName, EffectiveDate} PK [CourseNo, EffectiveDate]

CourseCredInTime {CourseNo, CourseCred, EffectiveDate} PK [CourseNo, EffectiveDate]

Observe that the projection CourseInTime is strictly speaking, redundant, since it can be obtained by a projection from either CourseNameInTime or CourseCredInTime. However, in the interest of clarity and completeness, it has been included.

This work by C. J. Date represents a significant contribution to the field of database systems, and will no doubt be a topical point of discussion in the future.

4.12 Summary and Concluding Remarks

This concludes one of the most important topics in your database systems course. Take some time to go over the concepts. Figure 4-7 should help you to remember the salient points.

Figure 4-7. Summary of Normalization

Traditionally, it has been widely accepted that for most databases, attainment of 3NF is acceptable. This course recommends a minimum attainment of BCNF. Recall that as stated earlier (in section 4.8), BCNF is really a refinement of 3NF, and the normalization process can bypass 2NF and 3NF, and go straight to BCNF. In rare circumstances, it may be required to proceed to 5NF or 6NF, which is the ultimate.

Normalization is a technique that must be mastered by database designers. It improves with practice and experience, and ultimately becomes almost intuitive. As your mastery of normalization improves, you will find that there is a corresponding improvement in your ability to design and/or propose database systems for various software systems. However, be aware that the converse is also true: failure to master fundamental principles of database design will significantly impair one’s ability to design quality software systems. Notwithstanding this, be careful not to be antagonistic about your views, as informed as they may be. In this regard, Figure 4-8 summarizes a practical experience of the author.

Figure 4-8. Database Quibbles almost got me in Trouble

4.13 Review Questions

State the two fundamental integrity rules and explain their significance.

What is a foreign key? Illustrate using an appropriate example. How should foreign keys be treated?

Clarify the following: functional dependence; non-loss decomposition; Heath’s theorem. Provide examples that will illustrate the significance of these terms.

State the normal forms covered in this Chapter. For each normal form, state it, explain in your own words what it means, provide an example of a relation that conforms to it and one that does not.

4.14 References and/or Recommended Readings

[Codd, 1972] Codd, Edgar F. “Further Normalization of the Database Relational Model.” Database Systems, Courant Computer Science Symposia Series 6. Eaglewood Cliffs, NJ: Prentice Hall, 1972.

[Codd, 1974] Codd. Edgar F. “Recent Investigations into Relational Data Base Systems.” Proc. IFIP Congress, Stockholm, Sweden, 1974.

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York: Addison-Wesley, 2002. See Chapter 13.

[Date, 2003] Date, Christopher J., Hugh Darwen and Nikos A. Lorentzos. Temporal Databases and the Relational Model. San Francisco, CA: Morgan-Kaufmann, 2003.

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8th ed. Menlo Park, California: Addison-Wesley, 2004. See Chapters 11-13, 23.

[Date, 2006] Date, Christopher J. Database Debunking. http://www.dbdebunk.com/page/page/621935.htm (accessed June 2006)

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapters 10 and 11.

[Fagin, 1977] Fagin, Ronald. “Multi-valued Dependencies and a New Normal Form for Relational Databases.” ACM TODS 2, No. 3, September 1977.

[Fagin, 1979] Fagin, Ronald. “Normal Forms and Relational Database Operations.” Proc. 1979 ACM SIGMOD International Conference on Management of Data. Boston, MA, May-June 1979.

[Fagin, 1981] Fagin, Ronald. “A Normal Form for Relational Databases that is Based on Domains and Keys.” ACM TODS 6, No. 3, September 1981.

[Garcia-Molina, 2002] Garcia-Molina, Hector, Jeffrey Ullman and Jennifer Widom. Database Systems: The Complete Book. Upper Saddle River, NJ: Prentice Hall, 2002. See Chapter 3.

[Heath, 1971] Heath. I. J. “Unacceptable File Operations in a Relational Database.” Proc. 1971 ACM SIGFIDET Workshop on Data Description, Access, and Control. San Diego, CA., November 1971.

[Hoffer, 2007] Hoffer, Jeffrey A., Mary B. Prescott and Fred R. McFadden. Modern Database Management 6th ed. Upper Saddle River, NJ: Prentice Hall, 2002. See Chapters 5.

[Kifer, 2005] Kifer, Michael, Arthur Bernstein and Philip M. Lewis. Database Systems: An Application-Oriented Approach 2nd ed. New York, NY: Addison-Wesley, 2005. See Chapter 6.

[Kroenke, 2005] Kroenke, David M. Database Processing: Fundamentals, Design and Implementation 9th ed. Upper Saddle River, NJ: Prentice Hall, 2004. See Chapter 4.

[Lewis, 2002] Lewis. Phillip M., Arthur Bernstein and Michael Kifer. Databases and Transaction Processing: An Application Oriented Approach. New York, NY: Addison-Wesley, 2002. See Chapter 8.

[Martin, 1995] Martin, James, and Joe Leben. Client/Server Databases: Enterprise Computing. Upper Saddle River, NJ: Prentice Hall, 1995. See Chapters 10, 13, 14.

[Ozsu, 1991] Ozsu, M. Tamer, and Patrick Valduriez. Principles of Distributed Database Systems. Eaglewood Cliffs, NJ: Prentice Hall, 1991. See Chapter 2.

[Pratt, 2002] Pratt, Phillip J. and Joseph J. Adamski. Concepts of Database Management 4th ed. Boston, Massachusetts: Course Technology, 2002. See Chapter 5.

[Riccardi, 2002] Riccardi, Greg. Database Management With Web Site Development Applications. Boston, MA: Addison-Wesley, 2003. See Chapter 7.

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, MA: Course Technology, 2007. See Chapter 5.

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, NJ: Prentice Hall, 1997. See Chapter 3.

CHAPTER 5

Database Modeling and Design

Some database systems textbooks make much ado about having separate chapters for database modeling and database design. In this course, we will not, and the reason will become clear shortly. All discussions in chapters 3 and 4 contribute to database modeling and design. In particular, ERDs and normalization are paramount to a logical database model and design. In this chapter, we will clarify the roles of database modeling and database design, and then take a more detailed look at some approaches in these areas. The chapter proceeds under the following captions:

Database Model and Database Design

The E-R Model Revisited

Database Design With the E-R Model

The Extended Relational Model

Database Design With the Extended Relational Model

The UML Model

Database Design With the UML Model

Innovation: The Object/Entity Specification Grid

Database Design Using Normalization Theory

Database Model and Design Tools

Summary and Concluding Remarks

5.1 Database Model and Database Design

Database modeling and database design are closely related; in fact, the former leads to the latter. However, it is incorrect to assume that the path from database modeling to database design is an irreversible one. To the contrary, changes in your database model will affect your database design and vice versa. This course therefore purports that you can work on your database model and your database design in parallel, and with experience, you can merge both into one phase. For the purpose of discussion, let us look at each phase.

5.1.1 Database Model

The database model is the blueprint for the database design. Database modeling is therefore the preparation of that blueprint. In database modeling, we construct a representation of the database that will be useful towards the design and construction of the database. Various approaches to database modeling have been proposed by different authors; the prominent ones are:

The Entity-Relationship (E-R) Model

The Object-Relationship (O-R) Model

The Extended Relational Model

The E-R and O-R models were introduced in chapter 3 (sections 3.5.1 and 3.5.2). Chapter 3 also introduced the Relation-Attributes List (RAL) and the Relationship List (RL section 3.6) as an alternative to the E-R model in situations where E-R modeling is impractical. We will revisit the E-R model later in this chapter, and then introduce the extended relational model.

5.1.2 Database Design

The database design is the (final) specification that will be used to construct the actual database. Database designing is therefore the preparation of this specification. In preparing the database specification, the database model is used as input. As such, the guidelines given in chapter 3 (section 3.5.4) on implementing relationships are applicable. Five approaches to database design that will be discussed later in this chapter are:

Database Design via the E-R Model

Database Design via the Extended Relational Model

Database Design via the UML Model

Database Design via the Entity/Object Specification Grid

Database Design via Normalization Theory

5.2 The E-R Model Revisited

Recall that in chapter 3, the similarity and differences between an entity and a relation were noted. If we assume the similarity, then the E-R model can be construed as merely a specific interpretation of the relational model.

In order to unify the informal E-R model with the formal relational model, Codd introduced a number of conventions specific to the E-R model. These are summarized in Figure 5-1 and illustrated in Figure 5-2. Note that the model displayed in the figure represents only a section of the (partial) database model introduced in chapter 3 (review Figure 3-4b). Figure 5-2a employs the Chen notation, while Figure 5-2b employs the Crow’s Foot notation. In both cases, the attributes of entities have been omitted, except for primary key attributes (as mentioned in chapter 3 and emphasized later in this chapter, there are more creative ways to represent attributes). Figure 5-3 illustrates the representation of super-type and subtype relationships.

Figure 5-1. E-R Model Conventions

Figure 5-2a. Partial E-R Diagram for Manufacturing Firm (Chen’s Notation)

Figure 5-2b. Alternate Partial E-R Diagram for Manufacturing Firm (Crow’s Feet Notation)

Figure 5-3. Example of Type Hierarchy

5.3 Database Design via the E-R Model

Database design with the E-R model simply involves following the rules established in chapter 3 on implementing relationships (section 3.5.4). These rules tell you exactly how to treat the various kinds of relationships; take some time to review them. We may therefore construct a procedure for database design via the E-R model as follows:

Figure 5-4. Database Design Procedure Using the E-R Model

5.4 The Extended Relational Model

Even with the conventions above, the E-R model was found lacking in its treatment of certain scenarios. Recognizing this, Codd and Date introduced an alternate extended relational model (which for convenience will be abbreviated as the XR model), the essence of which is described here (for more details, see [Date, 1990] and [Date, 2004]).

The XR model makes no distinction between entities and relations; an entity is a special kind of relation. Structural and integrity aspects are more extensive and precisely defined than those of the E-R model. The XR model introduces its own special operators apart from those of the basic relational model. Additionally, entities (and relationships) are represented as a set of E-relations and P-relations.

The model includes a formal catalog structure by which relationships can be made known to the system, thus facilitating the enforcement of integrity constraints implied by such relationships.

As you will see later in the course, it turns out that the XR model forms the basis of how the system catalog is handled in most contemporary DBMS suites. We shall therefore spend a few moments to look at the main features of the model.

5.4.1 Entity Classifications

Under the XR model, the following entity classifications hold:

Kernel entities

Characteristic entities

Designative entities

Associative entities

Sub-type/super-type entities

Kernel Entities: Kernel entities are those entities that have independent existence. They are what the database is really about. For example, in an inventory system, kernel entities might be Purchase Order, Receipts of Goods (Invoice), Inventory Item, Department and Issuance of Goods (to various departments). Referring to the example used in the previous section, kernel entities would be Suppliers, InventoryItems, Projects, Employees, and Departments.

Characteristic Entities: Characteristic entities describe other entities. For instance (referring to Figure 5-2b), Dependent is a characteristic of Employee. Characteristics are existence-dependent on the entity they describe.

Designative Entities: An entity, regardless of its classification, can have a property (attribute) whosefunction is to designate (reference) some other entity, thus implementing a 1:M relationship. For instance (referring to Figure 5-2b), Employee is designative of Department, and Project is designative of Employee (due to relationship ProjMgr). Put another way, a designation is the implementation of an M:1 relationship. The designating entity is the entity on the “many-side” of a 1:M relationship. Note that a characteristic entity is necessarily designative since it designates the entity on which it is existence-dependent. Note however, that a designative entity is not necessarily characteristic. Entities Project, Employee, and Dependent amplify these points (see Figure 5-2).

Associative Entities: Associative entities represent M:M relationships among two or more entities. For example, from Figure 5-2b, the relationships ProjWork, SuppSched, SuppItems, and ItemStruct would be implemented as associative entities. In a college database with kernel entities Program and Course (among others of course), and a M:M relationship between them, the associative entity representing the relationship could be ProgramStructure, which would include foreign keys referencing Program and Course respectively. Note that the associative entity is the intersecting relation in the implementation of a M:M relationship (review chapter 3, section 3.5.4).

Subtype/Super-type Entities: If we have two entities E1 and E2, such that a record of E1 is always a record of E2, and a record of E2 is sometimes a record of E1, then E1 is said to be a subtype of E2. The existence of a subtype implies the existence of a super-type: To say that E1 is a subtype of E2, is equivalent to saying that E2 is the super-type of E1. For illustration, review Figure 5-3.

5.4.2 Surrogates

Recall that the concept of a surrogate was first introduced in chapter 3 (section 3.5.4). In understanding the X-R model, the role of surrogates is very important; we therefore revisit the concept here. Surrogates are system controlled primary keys, defined to avoid identification of records (tuples) by user-controlled primary keys. They are also often used to replace composite primary keys. Two consequences of surrogates arise (both of which can be relaxed with a slight deviation from the XR model which does not enforce surrogates, E-relations and P-relations as mandatory):

Primary and foreign keys can be made to always be non-composite.

Foreign keys always reference corresponding E-relations (more on this shortly).

Surrogates provide two significant benefits:

In some traditional DBMS suites, composite primary keys are not allowed; surrogates are therefore imperative.

Even if allowed by the DBMS, composite primary keys are sometimes cumbersome; surrogates are useful replacements in these circumstances.

To demonstrate the usefulness of surrogates in simplifying database model and ultimate design (with respect to avoiding cumbersome composite primary keys), let us suppose that we want to track purchase orders and their related invoices. The related entities that we would need to track are Supplier, InventoryItem, PurchaseOrder and PurchaseInvoice. These are not all included in Figure 5-2; however they are represented in figure 3-4b of chapter 3 (please review). By following through on the E-R model, or by applying normalization principles of chapter 4, we may construct a tentative set of normalized relations as illustrated in Figure 5-5a. Notice how potentially cumbersome the composite keys would be, particularly on relations PurchaseOrdDetail and PurchaseInvDetail. However, by introducing surrogates as illustrated in Figure 5-5b, we minimize the need to use complex composite keys.

Figure 5-5a. Model to Track Purchase Orders and Invoices (without Surrogates)

Figure 5-5b. Alternate Model to Track Purchase Orders and Invoices (with Surrogates)

5.4.3 E-Relations and P-Relations

The original XR model specification prescribes the use of E-relations and P-relations. The database would contain one E-relation for each entity type — a unary relation that lists surrogates for all tuples of that entity (type). To illustrate, let us revisit the manufacturing firm’s partial database (Figure 5-2) of previous discussions: Suppose for a moment that the Supplier relation contains two tuples, the InventoryItem relation contains three tuples, and the Department relation contains three tuples. A possible internal representation of the E-relations for this scenario is illustrated in Figure 5-6a. An “E” is inserted in front of the original relation name (for example E-Supplier) to denote the fact that this is an E-relation being represented. The percent sign (%) next to the attributes (for example Suppl%) are used to denote the fact that these attributes are really surrogates.

In addition to the E-relations concerned with tuples, a special binary E-relation would be required to link so-called E-relations to the original relations. We will call this the host E-relation. It is illustrated in Figure 5-6b. This would allow users to relate to the database using relation names that they are familiar with; translation would be transparent to them.

Properties (attributes) for a given entity type are represented by a set of P-relations. The P-relation stores all property characteristics and values of all tuples listed in the corresponding E-relation. Properties can be grouped together in a single n-ary relation, or each property can be represented by P-relation, or there can be a convenient number of P-relations used; the choice depends on the designer. In the interest of simplicity, let us assume the third approach; let us assume further, that there is a P-relation for each E-relation. A convenient possible representation for the E-relations of Figure 5-6a is illustrated in Figure 5-6c. A “P” is inserted in front of the original relation name (for example P-Supplier) to denote the fact that this is a P-relation being represented. Notice also that each P-relation contains a foreign key that ensures that each tuple is referenced back to its correspondent in the associated E-relation.

Carrying on with the assumption that there is a P-relation for each E-relation: In addition to the basic P-relations, a special P-relation would be required to store the characteristics of each property (to be) defined in the database. Let us call this the host P-relation. It is represented in Figure 5-6d. By including this relation, we allow users the flexibility of adding new properties to a relation, modifying existing properties in a relation, or deleting pre-existent properties from a relation. These changes are referred to as structural changes to a relation; they will be amplified later in the course (chapter 11).

As you will later see (in chapter 14), it is application of this methodology that assists in the implementation of sophisticated system catalogs that characterize contemporary DBMS suites. However, with this knowledge, you can actually model and design databases to mirror E-relations and P-relations as described. One obvious advantage is that if you used one P-relation instead of one for each E-relation, then in accessing the database for actual data, you would be accessing fewer relations (in fact just one relation) than if you had used another approach (such as the E-R model). The flip side to this advantage is that this relation would be extremely large for a medium sized or large database; this could potentially offset at least some of the efficiency gained from just having to access one relation for data values.

Figure 5-6a. Illustrating E-relations

Figure 5-6b. Illustrating the Host E-relation

Figure 5-6c. Illustrating The P-relations

Figure 5-6d. Illustrating the Host P-relation

5.4.4 Integrity Rules

With this set-up, accessing and manipulating data in the database is accomplished by the DBMS through the E and P relations. For this reason, additional integrity rules must be imposed. The complete list of integrity rules follows:

Entity integrity rule (review section 4.1)

Referential rule (review section 4.1)

XR Model Entity Integrity: E-relations accept insertions and deletions but no updates (surrogates don’t change)

Property Integrity: A property cannot exist in the database unless the tuple (entity) it describes exists

Characteristic Integrity: A characteristic entity (tuple) cannot exist unless the entity (tuple) it describes exists

Association Integrity: An association entity (tuple) cannot exist unless each participating entity (tuple) also exists

Designation Integrity: A designative entity (tuple) cannot exist unless the entity (tuple) it designates also exists

Subtype Integrity: A subtype entity (tuple) cannot exist except there be a corresponding super-type entity (tuple)

The following rules apply to subtypes and super-types:

All characteristics of a super-type are automatically characteristics of the corresponding subtype(s), but the converse does not hold.

All associations in which a super-type participates are automatically associations in which the corresponding subtype(s) participate, but the converse does not hold.

All properties of a given super-type apply to the corresponding subtype(s), but the converse does not hold.

A subtype of a kernel is still a kernel; a subtype of a characteristic is still a characteristic; a subtype of an association is still an association.

Exercise

5.5 Database Design via the XR Model

The following approach, developed by Date and outlined in [Date, 1990], employs the basic XR method,but relaxes the requirement that surrogates, E-relations and P-relations are mandatory. The (partial)database model represented in Figure 5-2 will be used for the purpose of illustration.

Before proceeding, we now introduce a notation that has become necessary: The notation R.A will be usedto mean attribute A in relation (or entity) R. For instance, Department.Dept# denotes attribute Dept# in the relation Department. The database design approach involves seven steps as summarized in Figure 5-7 and clarified in the upcoming subsections.

Figure 5-7. Database Design Procedure Using the XR Model

5.5.1 Determining the Kernel Entities

The first step involves determining the kernel entities. As mentioned, earlier, kernels are the core relations. In the example, the kernels are Department, Employee, Supplier, Project, and InventoryItem. Each kernel translates to a base relation. The primary key of each could be the user controlled ones, or surrogates may be introduced.

5.5.2 Determining the Characteristic Entities

The second step involves determining and properly structuring the characteristic entities. As mentioned above, a characteristic entity is existence-dependent on the entity it describes. One characteristic exists in the example, namely Dependent. Characteristics also translate to base relations. The foreign key in Dependent would be DepnEmp#, which would reference Employee.Emp#. Notice that we did not use the attribute name Emp# as the foreign key in Dependent, but DepnEmp#. This decision is based on the following principle:

It should be noted that not all DBMS products support this principle (some require that a foreign key must have the same name as the attribute it references). You should therefore check to ascertain whether the product you are using supports it (Oracle and DB2 both do). Two strong arguments can be given in defense of this principle:

Simply, it makes good sense and leads to a cleaner, more elegant database design.

It avoids confusion when queries involving the joining of multiple relations are constructed and executed on the database. This will become clearer in division C (particularly chapter 12) of the text.

Moving on to data integrity, we would require the following integrity constraints on the Dependent relation:

Null FKs not allowed

Deletion is cascaded from the referenced to the referencing records

Update cascaded from the referenced to the referencing records

Two alternatives exist for choice of primary key:

The foreign key combined with the attribute that distinguishes different characteristics within the target entity (e.g. [Emp#, DepnName]);

Introduce a surrogate (e.g. DepnRef). The surrogate could be defined in such as way as to allow you to key solely on it; or it could be defined to allow you to key on the foreign key, combined with it (e.g. [Emp#, DepnRef]).

5.5.3 Determining the Designative Entities

This third step involves identifying and properly structuring the designative entities. As mentioned earlier, a designation is a 1:M or 1:1 relationship between two entities. In the example, designations are ProjMgr, DeptEmp, and EmpDep. However, EmpDep is a characteristic (that has already been identified above).

From the theory established in chapter 3 (section 3.5.4), a designation is implemented by the introduction of a foreign key in the relation for the designating entity. Following this principle, we would introduce foreign key, EmpDept# in relation Employee (where EmpDept# references Department.Dept#), and foreign key, ProjManagerEmp# in relation Project (where ProjManagerEmp# references Employee.Emp#).

Integrity constraints for designations would typically be:

Null FKs allowed in the designating entity if the participation is partial

Null FKs not allowed in the designating entity if the participation is full

Deletion of referenced records restricted

Update of referenced records restricted (although for some practical purposes, update could be cascaded)

Typically, the foreign key in a designative entity is a non-key attribute. Consequently, there are normally no keying issues. However, there could be exceptions to this observation (for instance in the case where an intersecting relation is introduced to implement a M:M relationship).

5.5.4 Determining the Associations

Step 4 involves identification and implementation of all associations. As mentioned earlier, associations are the implementation of M:M relationships. They translate to base relations. In the example, associations are ProjWork, SuppItems, ItemStruct, and SuppSched. Again relying on the theory established in chapter 3 (section 3.5.4) on the implementation of M:M relationships, we would introduce four base relations for the four associations — ProjWork, SuppItems, ItemStruct, and SuppSched. However, as established in chapter 4 (section 4.10), SuppSched should be replaced with three relations, namely SuppItems{Suppl#, Item#}, ItemProj{Item#, Proj#}, and ProjSupp{Proj#, Suppl#}. Additionally, and in keeping with the principle of having unique attribute names for the entire database, we will change the foreign key attribute names to names that are unique but easily traceable to the attributes they reference.

Integrity constraints for associations would typically be:

Null FKs not allowed

Deletion of referenced records restricted

Update of referenced records restricted

Two alternatives exist for choice of primary key:

Key on the aggregation of the foreign keys.

Introduce a surrogate and key on it.

5.5.5 Determining Entity Subtypes and Super-types

The fifth step relates to identifying and properly implementing subtype-super-type relationships among the entities. Care should be taken here, in not introducing subtype-super-type relationships where traditional relationships would suffice (review section 3.5). Each entity type translates to a base relation. Each base relation will contain attributes corresponding to properties of the entities that apply within the type hierarchy. Again being guided by principles established in chapter 3 (section 3.5.4), each subtype will share the primary key of its super-type. Further, the primary key of a subtype is also the foreign key of the said subtype. The illustrations provided in chapter 3 (Figures 3-11 and 3.12) are also applicable here.

No subtype-super-type relationship appears in the model of Figure 5-2. However, in Figure 5-3, there are a few: Programmer, Engineer and Accountant are subtypes of Employee; SystemProgrammer and ApplicationProgrammer are subtypes of Programmer. Note also that in a subtype, except for the primary key (which is also a foreign key), no additional attributes of the super-type need to be repeated, since they are inherited. However, additional attributes may be specified (in the subtype). For example (still referring to Figure 5-3), the Programmer entity may contain the attribute ProgLanguage to store the programmer’s main programming language; this would not apply to Employee.

For subtypes, integrity constraints on foreign (primary) keys may be:

Nulls not allowed

Deletion of referenced records restricted (in super-type)

Deletion of referencing records allowed (in subtype but not in super-type)

Update cascaded from the super-type

5.5.6 Determining Component Entities

Component entities were not discussed in Date’s original work on the database design via the XR model. However, in the interest of comprehensive coverage, this sub-section has been added. As mentioned in the previous sub-section, care should be taken in not introducing component relationships where traditional relationships would suffice (review section 3.5). Each entity type translates to a base relation. Each base relation will contain attributes corresponding to properties of the entities that apply within the type hierarchy. Again being guided by principles established in chapter 3 (section 3.5.4), each component will include a foreign key, which is the primary key in the summary relation. Further, this foreign key will form part of the primary key (or a candidate key) in the component relation. For examples, please refer to figures 3-11 and 3-12 of chapter 3.

For components, integrity constraints on foreign (primary) keys may be:

Null FKs not allowed

Deletion of referenced records restricted (in summary)

Deletion of referencing records allowed (in component but not in summary)

Update cascaded from the summary

5.5.7 Determining the Properties

The final step in this (modified) XR approach is to carefully determine the properties in each relation (entity). This is actually easy, but in order to avoid mistakes, you must be diligent:

Except for associations, in your initial system investigation (which would be part of the required software engineering or systems analysis), you would have identified the basic properties for each identified entity. That is your starting point.

Next, go through steps 1-3, 5, and 6 above, and observe the guidelines for dealing with characteristics, designations, subtypes and components. These steps tell you when and where to introduce foreign keys.

Next, observe step 4 above for treating associations.

By following this procedure, you will be able to confidently determine the properties for each relation in the database; in fact, you will end up with a list that is identical or very similar to the one provided in Figure 5-5 above. This finalized list is illustrated in Figure 5-8. As you examine the figure, please note the following:

All primary key and foreign key attributes are italicized.

The principle of having each attribute with a unique attribute name (including foreign keys) has been followed.

The database specification is presented via a Relation-Attributes List (RAL) — a technique introduced in chapter 3 (section 3.6).

Figure 5-8. Partial Database Specification for Section of Manufacturing Firm’s Database

With this additional information, you can now revisit the database specification of chapter 3 (Figures 3-4b, 3-13 and 3-14) and revise it accordingly (left as an exercise for you). In so doing, please observe that while the ERD of Figure 5-2 is similar to that of Figure 3-4b, they are not identical; they highlight different aspects of a manufacturing environment, with various areas of overlap.  By examining both figures, you should come away with a better sense of what a database model and specification for such an environment would likely entail. The key is to apply sound information gathering techniques (as learned in your software engineering course and summarized in appendix 3), coupled with your database knowledge.

5.6 The UML Model

An alternate methodology for database modeling is the Unified Modeling Language (UML) notation. UML was developed by three contemporary software engineering paragons — Grady Booch, James Rumbaugh and Ivar Jacobson. These three professionals founded Rational Software during the 1990s, and among several other outstanding achievements, developed UML for the expressed purpose of being a universal modeling language. Although UML was developed, primarily for object-oriented software engineering (OOSE), it is quite suitable for database modeling. Figure 5-9 provides a description of the main symbols used in UML. Note that with UML comes a slight change in the database jargon (“entity” is replaced with “object type”), consistent with the fact that UML is primarily for OOSE.

Figure 5-9. Symbols Used in UML Notation for Database Modeling

Note that the UML notation makes a distinction between a component relationship and an aggregation relationship. In the case of the component relationship, the constituent object types are existence dependent on a main object type. In the case of an aggregation relationship, the constituent object types are existence independent of the aggregation object type. Figure 5-10 illustrates the UML diagram for the (partial) college database model that was introduced in chapter 3 (Figure 3-11). Notice that except for the StudentEmployee object type, which has been omitted, the information represented is essentially similar to what was represented in Figure 3-11. The only difference here is that the appropriate UML symbols have been used.

Figure 5-10. UML Diagram for a Partial College Database Model

Let us examine another example: Suppose that you were hired at a large marketing company that needs to keep track of its sales of various products and product lines over time. Suppose further that the company operates out of various offices strategically located across the country and/or around the world. How would you construct a database conceptual schema that would allow the company to effectively track its sales? One way to solve this problem is to employ what is called a star schema — a central relation (or object type) is connected to two or more relations (or object types) by forming a M:1 relationship with each. Figure 5-11 illustrates such a schema for the marketing company. The central relation (often referred to as the fact table) is SalesSummary. The surrounding relations (often referred to as dimensional tables) are Location, TimePeriod, ProductLine, and Product. Each forms a 1:M relationship with SaleSummary, the central relation. Notice that consistent with the theory, SalesSummary has a foreign key that references each of the referenced relations (object types). Finally, observe also that in this illustration, the attributes for each relation (object type) have been included in the diagram.

Figure 5-11. UML Diagram for Tracking Sales Summary for a Large Marketing Company

5.7 Database Design via the UML Model

Database design with the UML model is somewhat similar to database design with the E-R model. The points of divergence relate to the differences in notation between the two approaches as well as the semantic jargon used. The rules that prescribe bow to treat various types of relationships (section 3.5.4) are still applicable. However, in order to be consistent with object-oriented (OO) terminology, you would replace the term relation (or entity) with the term object type. With this in mind, we may construct a procedure for database design via the UML model as follows:

Figure 5-12. Database Design Procedure Using the UML Model

Note  Due to the inherent behavior of typical OO software products, introducing primary keys and foreign keys into object types (implemented as classes) as we have prescribed, may be unnecessary. In a purely OO environment, these links are automatically introduced by the OO software and implemented as pointers — a feature called encapsulation; however, they are internal and cannot be tracked by the user. For this reason, many OO pundits argue that normalization and data independence run counter to inheritance and encapsulation. The debate as to when to use an OO database versus a relational database and vice versa, is likely to be ongoing for some time into the foreseeable future. It will be revisited in chapter 23.

5.8 Innovation: The Object/Entity Specification Grid

This section introduces an innovative approach to database design specification that has been successfully used by the author on a number of major projects. The approach may be construed as an extension of the UML model, but is applicable to any database model.

As mentioned in chapter 3 (section 3.6), for large complex projects (involving huge databases with tens of entities or object types), unless a CASE or RAD tool which automatically generates the ORD or ERD is readily available, manually drawing and maintaining this important aspect of the project becomes virtually futile. Even with a CASE tool, perusing several pages of O-R (E-R) diagram may not be much fun. In such cases, an object/entity specification grid (O/ESG) is particularly useful. In a relational database environment, the term entity specification grid (ESG) is recommended; in an object-oriented environment, the term object specification grid (OSG) is recommended.

The O/ESG presents the specification for each object type (or entity) as it will be implemented in a database consisting of normalized relations. The conventions used for the O/ESG are shown in Figure 5-13; a summary of these conventions follows:

Each object type (or entity) is identified by a reference code, a descriptive name, and an implementation name (indicated in square brackets).

For each object type (or entity), the attributes (data elements) to be stored are identified.

Each attribute is specified by its descriptive name, the implementation name indicated in square brackets, a physical description of the attribute (as described below), and whether the attribute is a foreign key.

For physical description, the following letters will be used to denote the type of data that will be stored in that attribute, followed by a number which indicates the maximum length of the field: (A) alphanumeric, (N) numeric, or (M) memo. This is specified within square braces. For instance, the notation [Dept#] [N4] denotes a numeric attribute of maximum length 4 bytes. In the case of an attribute that stores a memo (M), no length is indicated because a memo field can store as much information as needed. If a real number value is being stored with a decimal value, two numbers will be used: the first number will indicate the length for the whole number part and the second number will indicate the field length of the decimal part (e.g. [N(9,2)]).

An attribute that is a foreign key is identified by a comment specifying what object type (or entity) is being referenced. The comment appears in curly braces.

For each object type (or entity) a comment describing the data to be stored is provided.

An itemized specification of indexes to be defined (starting with the primary key) is provided for each object type (or entity).

Each operation to be defined on an object type (or entity) will be given a descriptive name and an implementation name, indicated in square brackets.

Figure 5-13. O/ESG Conventions

Figure 5-14 provides an illustration of O/ESG for four of the object types (or entities) that would comprise the manufacturing firm’s database of earlier discussion (review Figure 5-7). In actuality, there would be one for each object type (or entity) comprising the system.

Figure 5-14. Partial O/ESG for Manufacturing Environment

5.9 Database Design via Normalization Theory

Although this is seldom done, you can actually use the normalization theory as discussed in chapter 4 to design the basic conceptual schema (involving the structure of the base relations) of a relational database. In practice, normalization is used as a check-and-balance mechanism to acceptability of a database’s conceptual schema. As such, normalization can (and should) be applied to each of the database design approaches discussed.

This section looks at two sample database design problems, and shows how the normalization theory can be used to solve them.We will advance the discussion by using two problem scenarios that will hopefully identify with.

5.9.1 Example: Mountaineering Problem

Suppose that we wish to record information about the activities of mountaineers in a relational database. Let us make the assumption that a climber can only begin one climb per day. Figure 5-15 illustrates an initial set attributes (with suggested implementation names in square brackets) for the database.

Figure 5-15. Attributes for the Mountaineering Problem

How may we obtain an appropriate conceptual schema for the mountaineering problem? We may approach this problem in one of two ways:

The Pragmatic Approach: Identify related attributes that form data entities, normalize these entities, then identify and rationalize relationships among the entities.

The Classical (theoretical) Approach: Start out by creating one large 1NF relation involving all attributes, progressively decompose into relations of higher normal forms until the given requirements are met.

In the interest of illustrating application of the theory of normalization, we shall pursue the second approach. However, please bear in mind that in most real life situations, you will be advised to employ the pragmatic approach. Several of the cases in chapter 26 (for instance, assignments 1 and 2) provide an excellent opportunity for you to do this.

Step 1 — Create a large 1NF Relation

We introduce three new attributes: Climber Identification [CID#], Mountain Identification [MTN#], and Country Code [CTRYCD]; store all attributes as relation M as shown in Figure 5-16. The figure also states the observed functional dependencies.

Figure 5-16. Revised Initial 1NF Relation for the Mountaineering Problem

Step 2 — Obtain 2NF Relations

The second step is to obtain a set of 2NFrelations. Because of this FD2, the relation can be non-loss decomposed via Heath’s theorem to obtain:

Relation M1: {CID#, CNAME, CADDR} PK [CID#]

Relation M2: {BDA.TE, CID#, EDATE, MTN#, MNAME, MHGHT, CTRYCD, CTRYNM, DIST} PK [CID#, BDATE]

Step 3 — Obtain 3NF Relations

Next, we seek to obtain 3NF relations. Based on FD3 and FD4, relation M2 is not in 3NF. Again applying Heath’s theorem for non-loss decomposition, we obtain the following relations:

Relation M3: {CTRYCD, CTRYNM} PK [CTRYCD]

Relation M4: {MTN#, MNAME, MHGHT, CTRYCD, DIST}   PK [MTN#]

Relation M5: {BDATE, CID#, MTN#, EDATE}    PK [CID#, BDATE]

Step 4 — Obtain BCNF (and higher order) Relations

Next, we seek to obtain relations of higher order normal forms. Observe that relations M1, M3, M4, and M5 are in BCNF, 4NF and 5NF.

Note  We could have forgone steps 1-3 and gone straight for BCNF relations by simply observing the FDs shown in Figure 5-16, and decomposing via Heath’s theorem. As your confidence in database design grows, you will (hopefully) be able to do this.

5.9.2 Determining Candidate Keys and then Normalizing

In many cases, the database designer may be faced with the problem where basic knowledge of data to be stored is available, but it is not immediately clear how this partial knowledge will translate into a set of normalized relations. For instance, you may be able to identify an entity (or set of entities), but are not sure what the primary key(s) to this (these) entity (entities) will be. With experience, you will be able to resolve these challenges intuitively. However, what do you do in the absence of that invaluable experience? The relational model provides a theoretical approach for dealing with this problem, as explained in the following example.

Suppose that it is desirable to record the information about the performances of students in certain courses in an educational institution environment. Assume further, that a set of functional dependencies have been identified, but it is not sure what the final set of normalized relations will be and how they will be keyed. Figure 5-17 illustrates a summary of the information (assumed to be) known in the case. As usual, we start off by assuming that the relation shown (StudPerfDraft) is in 1NF.

Figure 5-17. Initial 1NF Relation for the Student Performance Problem

Step 1 — Determine the Candidate Key

Having assumed that StudPerfDraft is in 1NF, our next step is to determine a candidate key of the relation. We do this by chasing the explicit and implicit dependencies. Any FD that ends up determining all attributes (directly or indirectly) constitutes a candidate key. The technique is referred to as computing closures.

We shall examine each explicit FD in turn and determine all the attributes that it determines (explicitly or implicitly), bearing in mind that any attribute or combination of attributes that is a determinant, necessarily determines itself. Hence, we conclude the following:

HR →CHR → CTHR

HT→ HTR → HTRC

CS → CSG → CSGT

HS → HSR→ CHSR → CHSRG → CHSRGT

C → CT

From this exercise, observe that [H,S] is the only candidate key; it is therefore the primary key (PK).

Step 2 — Obtain 2NF Relations

The next step is to obtain 2NF relations. We may rewrite the initial relation as follows:
StudPerfDraft {H, S, C, T, R, G} with PK [H,S]

Observe that StudPerfDraft is in 2NF.

Step 3 — Obtain 3NF Relations

Step 3 is to obtain 3NF relations. StudPerfDraft is not in 3NF due to FD5 (C →T). To resolve this, decompose via Heath’s Theorem to obtain:

R1 {H, S, C, R, G} with PK [H,S] and R2 {C ,T} with PK [C].

Step 4 — Obtain BCNF Relations

Our fourth step is to obtain a set of BCNF relations. Observe that R2 is in BCNF but R1 is not, due to FD3 ([C,S] → G). To resolve this, decompose via Heath’s theorem to obtain:

R3 {C, S,G} with PK [C,S] and R4 {H, S, C, R} with PK [H,S].

R3 is now in BCNF but R4 is not, due to FD1 ([H,R] → C). Decompose via Heath’s theorem to obtain:

R5 {H, R, C} with PK [H,R] and R6 {H, S, R} with PK [H,S].

We now have R2, R3, R5, and R6 all in BCNF. Additionally, note that all the FDs have been resolved, except for FD2 ([H,T] → R). This may be resolved by introducing the relation R7 as follows:

R7 {H, T, R} with PK [H, T].

Step 5 - Obtain Higher Order Normalized Relations

There are no MVDs or JDs, therefore relations R2, R3, R5, R6 and R7 are in 4NF and 5NF.

Note:

This is a rather trivial example; a college database is a much more complex system than the representation presented here. Moreover, questions may be raised as to the veracity of some of the FDs stated (for instance FD5). However, the representation succeeds in providing the application of the normalization theory, and that was the sole intent.

Relation R6 fulfills FD4 ([H,S] → R), and R7 fulfills FD2 ([H, T] → R). Strictly speaking, R7 may be considered redundant, since we can determine what teacher is in a room at a given time by accessing R5 and R2.

5.10 Database Model and Design Tools

At this point you must be wondering, how in the world are you supposed to model and design a complex database, and keep track of all the entities and relationships? The good news is, there are various tools that are readily available, so there’s no need to panic. The standard general purpose word processors (such as MS Office, Word Perfect, Open Office, etc.) are all fortified with graphics capabilities so that if you spend a little time with any of these products, you will figure out how to design fairly impressive database model/design diagrams. Better yet, there is a wide range of CASE tools and/or modeling tools that you can use. Figure 5-18 provides an alphabetic list of some of the commonly used products. While some of the products in the list are quite impressive, the list is by no means comprehensive, so you do not have to be constrained by it. Some of the products are available free of charge; for others, the parent company offers free evaluation copies.

Figure 5-18. Some Commonly Used Database Planning Tools

5.11 Summary and Concluding Remarks

It is now time to summarize what we have covered in this chapter:

A database model is a blueprint for subsequent database design. It is a representation of the database. We have looked at three database models: the E-R model, the XR model and the UML model.

Database design involves preparation of a database specification, which will be used to construct the database. We have discussed database design via the E-R model, the XR model, the UML model, the O/ESG, and normalization theory.

The E-R model is the oldest model for relational databases that has been discussed. It involves the use of certain predefined symbols to construct a graphical representation of the database.

Database design via the E-R model involves following eight steps that lead to a normalized database specification.

The XR model is an alternate model that compensates for the weaknesses in the E-R model. It involves grouping information entities into different predefined categories that will assist in the design phase.

Database design via the XR model involves following nine steps that lead to a normalized database specification.

The UML model is similar to the E-R model. However, it requires taking an object-oriented approach, and employs different notations from the E-R model.

Database design via the UML model involves following seven steps that lead to a normalized database specification.

The O/ESG methodology describes an efficient way of developing a comprehensive, normalized database specification.

Database design via normalization theory describes a rudimentary approach to database design that relies on mastery of the principles of normalization.

You are now armed with all the requisite knowledge needed to design quality databases. However, you will likely find that a review of this and the previous two chapters, along with practice, may be necessary until you have gained mastery and confidence with the concepts, principles and methodologies. The next chapter discusses the design of the user interface for a database system.

5.12 Review Questions

Describe in your own words (with appropriate illustrations) the approach to database design based on the following (in each case, describe approach, then outline the advantages and disadvantages from your perspective):

The E-R Model

The X-R Model

The UML Model

The O/ESG Methodology

Normalization Theory

The following are three software systems for which a database is to be designed:

An Inventory Management System

A Lab Scheduling System for users of a computer lab

A Bookshop Manager System for an educational institution

For each system, develop a database specification based on at least three of the five approaches above. Compare the database specifications derived. Compare the methodologies. Which one(s) are you most comfortable with and why?

How important are surrogates in the design of a database? By considering an appropriate example, demonstrate how surrogates may be used in database design.

Compare the UML database model with the E-R model. What are the similarities? What are the differences? You may use appropriate illustrations in your response.

5.13 References and/or Recommended Readings

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York, NY: Addison-Wesley, 2002. See chapters 13-15.

[Date, 1986] Date, Christopher J. Relational Database: Selected Writings. Reading, MA: Addison-Wesley, 1986.

[Date, 1990] Date, Christopher J. Introduction to Database Systems 5h ed. Menlo Park, CA: Addison-Wesley, 1990. See chapter 22.

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8h ed. Menlo Park, CA: Addison-Wesley, 2004. See chapters 12-14.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See chapters 7, 10 and 11.

[Garcia-Molina, 2002] Garcia-Molina, Hector, Jeffrey Ullman and Jennifer Widom. Database Systems: The Complete Book. Upper Saddle River, NJ: Prentice Hall, 2002. See chapter 3.

[Hoffer, 2007] Hoffer, Jeffrey A., Mary B. Prescott and Fred R. McFadden. Modern Database Management 8th ed. Upper Saddle River, NJ: Prentice Hall, 2007. See chapters 5.

[Jensen, 1992] Jensen, C. S. and L. Mark. “Queries on Change in Extended Relational Model.” IEEE Transactions on Knowledge and Data Engineering Vol. 4 Issue 2, April 1992. pp. 192-200.

[Kifer, 2005] Kifer, Michael, Arthur Bernstein and Philip M. Lewis. Database Systems: An Application-Oriented Approach 2nd ed. New York: Addison-Wesley, 2005. See chapter 4.

[Kroenke, 2006] Kroenke, David M. Database Processing: Fundamentals, Design and Implementation 10h ed. Upper Saddle River, NJ: Prentice Hall, 2006. See chapters 3-6.

[Lee, 2002] Lee, Richard C. and William M. Tepfenhart. Practical Object-Oriented Development With UML and Java. Upper Saddle River, NJ: Prentice Hall, 2002. See chapter 8.

[Lewis, 2002] Lewis, Phillip M., Arthur Bernstein and Michael Kifer. Databases and Transaction Processing: An Application Oriented Approach. New York, NY: Addison-Wesley, 2002. See chapter 8.

[Martin, 1993] Martin, James and James Odell. Principles of Object Oriented Analysis and Design. Eaglewood Cliffs, NJ: Pretence Hall, 1993. See chapters 6 and 7.

[Pratt, 2002] Pratt, Phillip J. and Joseph J. Adamski. Concepts of Database Management 4th ed. Boston, MA: Course Technology, 2002. See chapters 5 and 6.

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, MA: Course Technology, 2007. See chapters 4, 6 and 9.

[Rumbaugh, 1991] Rumbaugh, James, et. al. Object Oriented Modeling And Design. Eaglewood Cliffs, NJ: Pretence Hall, 1991. See chapter 4.

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, NJ: Prentice Hall, 1997. See chapter 3.

CHAPTER 6

Database User Interface Design

This chapter covers the essentials of good database user interface design. A properly designed database system typically includes a user interface that facilitates end users accessing the system. The chapter presumes that you are familiar with basic software engineering principles as well as user interface design principles as covered in your undergraduate degree. The chapter is therefore a summary that includes:

Introduction

Deciding on the User Interface

Steps in the User Interface Design

User Interface Development and Implementation

Summary and Concluding Remarks

6.1  Introduction

At this stage we have settled on all relations (and attributes). Remember, we implement relations, tuples and attributes as files, records and fields.

Designing the user interface to facilitate user access is the next step. The user interface should facilitate at least the following basic functions: data insertion, data update, data deletion, and information retrieval (query and print). This is important, as it is not acceptable to give end users direct, unfettered access to the database; were this to be done, the integrity of the system would be compromised in very short order. What is more desirable is to provide the end users with a user friendly, controlled environment that gives users all the privileges and functionalities that they need and nothing more.

The user interface will consist of menus from which user operations can be accessed. Depending on the software development tool used, it will be constructed from various building blocks, and there will be various categories of user interface objects (review your software engineering notes).

The system must also facilitate various user (external) views through logical interpretation of objects. This must be developed using the DBMS and/or whatever software development tool is being used. Note that if the O/ESG methodology (discussed in section 5.8) is employed, you will be well on your way with the user interface specification.

Example: By the way of illustration, let us revisit the O/ESG for the partial database specification of the manufacturing firm, discussed in the previous chapter (section 5.8). Figure 6-1 shows a repeat of the O/ESG for the Employee entity. According to the figure (adopting the conventions from section 5.8), this entity could be implemented as a relational table named RMEmployee_BR. The user interface should anticipate and support various logical views on this relational table. Following are some examples:

Employees arranged by Name

Employees arranged by Telephone Numbers

Employees arranged by Departments

Employees arranged by Social Security Number

Figure 6-1. Excerpt from the Partial O/ESG for Manufacturing Environment

6.2  Deciding on User Interface

User interfaces can be put into three broad categories — menu-driven interface, command interface and graphical user interface (GUI). Figure 6-2 compares the approaches in terms of relative complexity of design (COD), response time (RT), and ease of use (EOU).

Command entry interfaces are the oldest type; they typify traditional operating systems, compilers and other software development tools. Up until the mid-1990s, menu driven interfaces were the most frequently used, dominating the arena of business information and application system. Since the late 1980s, graphical interfaces have become very popular, and clearly dominate user interfaces of the current era. Of course, the approaches can be combined.

Figure 6-2. Comparison of User Interface Categories

6.3  Steps in User Interface Design

How you design the user interface will depend to a large extent on the type of user interface your software requires, It will also depend on the intended users of the software (experts, knowledgeable intermittent, or novices).

6.3.1 Menu or Graphical User Interface

If the user interface is to be menu driven or graphical, the following steps are recommended (assuming object-oriented design):

Put system objects (structures and operations) into logical groups. At the highest level, the menu will contain options pointing to summarized logical groups.

For each summarized logical group, determine the component sub-groups where applicable, until all logical groups have been identified.

Let each logical group represent a component menu.

For each menu, determine options using an object-oriented strategy to structure the menu hierarchy (object first, operation last).

Design the menus to link the various options. Develop a menu hierarchy tree or a user interface topology chart (UITC).

Program the implementation.

Figure 6-3 illustrates a partial user interface topology chart (UITC) for a college/university administrative information system (CUAIS). The UITC displays the main user interface structure for the system; it is fully discussed in [Foster, 2010]; however, it is intuitive enough for you to understand it. The CUAIS project is also described in [Foster, 2010]; like the UITC, a full discussion is not necessary here.

Figure 6-3. Partial UITC for a CUAIS Project

6.3.2 Command-Based User Interface

If the user interface is command-driven, the following steps are recommended:

Develop an operations-set i.e. a list of operations that will be required.

Categorize the operations — user operations as opposed to system operations.

Develop a mapping of operations with underlying database objects.

Determine required parameters for each operation.

Develop a list of commands (may be identical to operations set). If this is different from the operations set, each command must link to its corresponding system operations.

Define a syntax for the command.

Develop a user interface support for each command (and by extension each operation). This interface support must be consistent with the defined command syntax.

Program the implementation of each operation.

6.4  User Interface Development and Implementation

Designing, constructing and implementing the user interface really belongs to the realm of software engineering, not database systems. However, as you are aware (and as has been emphasized in this course), the two fields are closely related. In order to construct the user interface, you will need to have an appropriate set of software development tools. Of course, development and testing of the user interface must proceed according to established software development standards.

The software development tool used to develop the user interface will depend to a large extent on the user requirements (another software engineering matter). Figure 6-4 provides some possible scenarios.

Figure 6-4. Possible Scenarios for User Interface

It must be emphasized that user interface design and development can and often occur independent of database design and development. This is one of the potent results of data independence (review sections 1.2 and 2.9): the user interface applications are immune to structural and/or physical changes in the database. In fact, as mentioned in section 2.9, the user interface (which is part of the front-end system) may reside on a different machine (with a different operating system) from the actual database (which is part of the back-end system). So in addition to data independence, we can have platform independence.

Two prominent protocols that facilitate platform independence between database and user interface are open database connectivity (ODBC) and Java database connectivity (JDBC).

Open Database Connectivity: ODBC is an open standard application programming interface (API) for accessing a database. A software product that desires to access an external database must include in its suite, an ODBC driver for that database. The ODBC driver converts the database objects into a generic format that is understood by the software. The target DBMS must also support ODBC. This facilitates communication and transfer of data among heterogeneous databases, irrespective of the platform that they reside on. Microsoft is a strong proponent of ODBC; in fact, the ODBC software is typically bundled with the Windows operating system (under Control Panel &#x224; Administrative Tools). ODBC is also supported by other leading operating systems (Unix, Linux, Windows, System i, etc.).

Java Database Connectivity: JDBC is a Sun Microsystems product that allows Java programs to access heterogeneous databases, irrespective of their platforms. This API is included in J2SE and J2EE releases. JDBC cooperates with the ODBC protocol; as such, a program running JDBC can reach ODBC-accessible databases.

ODBC and JDBC may be considered as subsets of the wider set of protocols described as Common Object Request Broker Architecture (CORBA). CORBA will be further discussed in chapter 22 (section 22.6). For the purpose of illustration, Figure 6-5 provides a summary of the steps you would take in order to configure an Oracle database server to be accessed from Delphi 7.0 (or some later version) through ODBC (assuming a Windows environment).

Figure 6-5. Accessing an Oracle Database from Delphi via ODBC

6.5  Summary and Concluding Remarks

Here is a summary of what we have covered in this chapter:

The user interface for a database system should provide the end users with a user friendly, controlled environment that gives users all the privileges and functionalities that they need and nothing more.

In planning the user interface, you must first decide what type of interface will be provided. The interface may be command-based, menu-driven, or GUI-based.

Next, you should design the user interface using established principles of user interface design.

The final step is the development and implementation of the user interface. Features that the user interface will provide will influence the tools used in developing the user interface. For instance, if the database is to be accessible from the WWW, then the tools used must facilitate such capability.

So now you know how to design a database and a user interface for that database. Later in the course, you will learn Structured Query Language (SQL), the universal database language. In preparation for this, the next two chapters discuss the relational algebra and relational calculus respectively — two subjects areas that will enhance your appreciation of SQL.

6.6  Review Questions

What are the types of user interfaces which may be constructed for end users of a database system? How do they compare in terms of efficiency and convenience?

Outline the steps to be taken in the construction of a user interface for end users of a database system.

Referring to revision question 2 of the previous chapter, propose a user interface model for any (or all) of the following:

An Inventory Management System

A Lab Scheduling System for users of a computer lab

A Bookshop Manager System for an educational institution

Identify four different scenarios for development of a user interface for a database system. Recommend an appropriate software development tool for each scenario.

6.7  References and/or Recommend Readings

[Carroll, 2002] Carroll, John. Human-Computer Interaction in the New Millennium. Reading, MA: Addison-Wesley, 2002.

[Foster, 2010] Foster, Elvis. Software Engineering — A Methodical Approach. Bloomington, IN: Xlibris Publishing, 2010. See chapters 6, 11, and 12.

[Raskin, 2000] Raskin, Jef. The Human Interface: New Directions for Designing Interactive Systems. Reading, MA: Addison-Wesley, 2000.

[Shneiderman, 2005] Shneiderman, Ben. Designing the User Interface: Strategies for Effective Human-Computer Interaction 4th ed. Reading, MA: Addison-Wesley, 2005.

[Sommerville, 2001] Sommerville, Ian. Software Engineering, 6th ed. Reading, Massachusetts, Addison-Wesley, 2001. See chapter 15.

CHAPTER 7

Relational Algebra

One of the reasons for the success and longevity of the relational model is that it is firmly grounded on mathematical principles (of linear algebra and set theory), which are well known and well documented. The (normalized) relational tables are by design, organized in such a way as to promote and facilitate manipulation of data to yield meaningful information. As mentioned earlier in the course (chapter 2), data manipulation relates to the addition, update, deletion, retrieval, reorganization, and aggregation of data. It turns out that the first three aspects are far more straightforward than the latter three.

In this and the next chapter, the focus is on the latter three aspects of data manipulation — retrieval, reorganization and aggregation. The intent is to expose you to the underlying database theory that had to be developed in order to have DBMS suites that support these data manipulation requirements. As you will soon come to appreciate, mastery of SQL (which is mandatory, if you intend to do well in this field) or any database language is greatly enhanced by an understanding of the fundamental underlying theory. The ecture proceeds under the following sub-topics:

Introduction

Basic Operations of Relational Algebra

Syntax of Relational Algebra

Aliases, Renaming and the Relational Assignment

Other Operations

Summary and Concluding Remarks

7.1 Introduction

Relational algebra consists of a collection of operations on relations. Each operation produces new relation(s) from one or more already existing relation(s).

Through relational algebra, we can achieve the following objectives:

Defining a scope for retrieval

Defining a scope for update

Defining virtual information

Defining snapshot data for snapshot relations

Defining access rights i.e. data for which authorization of some kind is to be granted

Defining integrity constraints apart from those that are part of the relational model

Earlier in the course, it was mentioned that every DBMS has a data sublanguage (DSL). Typically, the DSL is based on relational algebra, relational calculus (to be discussed in the next chapter), or a combination of both. It is therefore imperative for you to have a good grasp of this topic.

The relational algebra is said to be prescriptive — you specify precisely how an activity is to be carried out. In so doing, there are certain operations that you will learn to use.

7.2 Basic Operations of Relational Algebra

There are eight basic relational algebra operations with which you need to become familiar; they are informally described below:

Union (UNION): The union of relations R1 and R2 builds a relation R3 consisting of tuples in either R1 or R2 inclusive. Corresponding attributes of R1 and R2 must be defined on the same domain.

Difference (MINUS): The difference of relation R1 and relation R2 is a new relation R3, with tuples appearing in R1, but not in R2. Corresponding attributes from R1 and R2 must be defined on the same domain.

Restriction (RESTRICT): A restriction extracts specified tuples from a specified relation R1, by imposing some condition on the relation. The resulting relation R2, has only the specified tuples. RESTRICT is replaced by the SELECT in modern systems, the latter usually being more powerful than the original.

Product (PRODUCT): The Cartesian product of two relations R1 and R2 is a third relation R3, consisting of the concatenation of every tuple in R1 with every tuple R2.

Projection (PROJECT): A projection extracts specified attributes from a specified relation R1 into a new relation R2.

Join (JOIN): The join of two relations R1 and R2 builds a new relation R3, such that the tuples from R1 and R2 satisfy some specified condition. However, in the unqualified form, it is generally used to mean natural join. A more generic form of the JOIN operation is discussed in section 7.3.5.

The natural join (implied by the notation R1 JOIN R2) assumes that there is an attribute or combination of attributes that is common to both R1 and R2 (same attribute name and characteristics). Let us denote the common attribute(s) in R1 and R2 as Z. The natural join creates a relation R3, where every tuple from R1 is concatenated with every tuple from R2, provided that R1.Z = R2.Z. Finally, a projection is performed on the result in the previous step to yield a single copy of each attribute (thus removing either R1.Z or R2.Z) in the final result.

Intersection (INTERSECT): The intersection builds a relation R3, from two specified relations R1 and R2, where R3 has tuple that exist in both R1 and R2. Corresponding attributes of R1 and R2 must be defined on the same domain.

Division (DIVIDE BY): Division takes a relation R1 of degree m+n and a second relation R2 of degree n and produces a third relation R3 of degree m, where all the n attribute values appearing in R2 also appear in R1, concatenated with the other m attribute values.

7.2.1 Primary and Secondary Operations

The operations may be classified into primary operations and secondary operations: Primary operations include UNION, DIFFERENCE, SELECT, PROJECT, and PRODUCT. Secondary operations include JOIN, INTERSECT, and DIVIDE BY. Secondary operations are derivable from the primary operations.

7.2.2 Codd’s Original Classification of Operations

Codd’s original classifications placed the operations in two broad categories as follows: Traditional operations included UNION, INTERSECT, DIFFERENCE, and PRODUCT. Special operations included RESTRICT, PROJECT, JOIN, and DIVIDEBY.

7.2.3 Nested Operations

The output of each operation is another relation; likewise, the input to each operation is a set of one or more relations. We say that the relational operation is closed on relational algebra. Since each operation produces a new relation as output, we can nest operations so that the output of one forms the input to another. This will become clear in the ensuing examples.

7.3 Syntax of Relational Algebra

We shall use as frame of reference for this and the next few chapters, a section of a college database, as defined in Figure 7-1. Some sample data is provided in Figure 7-2.

Figure 7-1. Cross Section of a College Database

Figure 7-2. Sample Data for a Cross-section of a College/University Database[Showing only attributes relevant to the examples discussed]

In the interest of clarity, two commonly used notations have been adopted, both with slight modifications. They are the Ullman notation [Garcia-Molina, 2002] and the Date notation [Date, 2004]. Bear in mind however, that the syntactical implementation of relational algebra will vary from one product to another. What is important therefore, is that you understand the concepts discussed. The BNF (Backus-Naur Form) conventions shown in Figure 7-3 will be assumed. Additionally, several of the examples will contain clarifying comments that conform to how comments are made in the C-based programming languages (e.g. // this is a comment, /* and so is this */). Use of the BNF notation and C-based comments will continue through chapter 8 as well.

Figure 7-3. BNF Notations

7.3.1 Select Statement

Based on the Ullman notation, the SELECT statement is of the form

Note:

Theta represents a valid Boolean operator i.e. any of the following: =, >, <, <>, <=, >=

A relational expression is either a relation or an expression that results in the creation of a relation.

A condition is an expression that evaluates to true or false. Complex conditions may be built by using brackets and connectors AND, OR, NOT as in normal Boolean logic.

Example 1: To select all students enrolled in program B.Sc. in MIS, the following statement (based on the Ullman notation) would apply:

An alternate syntax of the SELECT statement as described in [Date, 2004] follows:

As you will soon see, this syntax is much closer to the actual implementation in SQL.

Example1b: To select all students enrolled in program B.Sc. in MIS, the following statement (based on the Date notation) would apply:

7.3.2 Projection Statement

Based on the Ullman notation, the PROJECT statement is of the form

Example 2a: The following are two projections on the relation Student

Based on the Date notation, the alternate syntax of the PROJECT statement that is much closer to the SQL implementation is as follows:

Example 2b: With this syntax, the above two projections would be written as follows:

7.3.3 Natural Join Statement

The Ullman notation and Date notation essentially agree on the format of the natural join. The natural join statement is of the form

Example 3: To join Student with Hall on the foreign key Hall#:

Please note:

The natural join is realized by the join logical file in an IBM System i environment (supported by the operating system), as well as query and open query. However, generally, the RDBMS provides the facility to create complex logical views involving joins of several relations.

On modern systems, the designer is allowed to clearly define the foreign key constraint at (or subsequent to) the point of creation of the table Student. In more sophisticated environments (for example, DB2 and Oracle), the foreign key name could be different from the referenced attribute, so long as its characteristic features are the same (as mentioned in previous chapters, it is good design practice to use distinct attribute names in different relations). In such cases, an equijoin (to be clarified shortly) would normally take the place of a natural join.

The operating system or DBMs traverses both relations and joins on matching keys i.e. the foreign key in the first relation is used to reference the primary key in the second relation.

The natural join is associative. Moreover, in specifying it, it is not necessary to identify the participating attributes, since they would have been identified when the foreign key constraint is defined.

Example 4: To obtain a list showing Sname and Hallname combinations:

7.3.4 Cartesian Product

Both the Ullman notation and the Date notation essentially agree on the format of the Cartesian (or cross) product. The PRODUCT statement is of the form

The resulting virtual relation has all tuples of the first relation concatenated with all tuples of the second relation.

Example 5a: We desire to have a list showing combinations of department name and name of the related department head: Two solutions are possible. Firstly we can take a cartesian product of Dept and Staff, followed by a selection, followed by a projection. Secondly, we could take an equijoin, followed by a projection. The second solution is clarified in the next sub-section (example 5b).

Please note:

This example illustrates that a natural join is equivalent to a Cartesian product followed by a selection. Proof of this principle is not necessary for this course. However, the principle can be easily illustrated as in Figure 7-4.

Figure 7-4. Illustrating That a Join is Obtained by a Cross Product Followed by a Selection

If there is at least one pair of matching keys in both relations, a natural join (if matching attribute names are identical) or an equijoin (if matching attribute names have different names) is preferred to a cross product. The match must be between a foreign key (in the primary relation) and a primary key (in the referenced relation).

The Cartesian product, as defined in the relational model, is associative. However its mathematical counterpart (from which it is drawn) is not. The reason for this is partly due to the fact that in the definition of a relation, no emphasis is placed on the order of the attributes (review section 3.3.1).

7.3.5 Theta-Join

The Theta-join is presented in some texts as the general case of the JOIN operation. The BNF form for the theta-join based on Ullman’s notation is as follows:

The BNF form for the theta join based on Date’s notation is as follows:

An alternate notation for representing this has been described by [Russell, 2006], and is particularly clear; it is paraphrased below:

Incidentally, this latter notation is really a simplification of the Ullman notation, and is fully reconcilable with either the Date or Ulman notation, bringing additional clarity to the specification of the theta-join. This is illustrated in example 5b. Moreover, as you will see in chapter 12, the ANSI version of the SQL join is comparable to this notation; we will therefore refer to it as the ANSI join notation.

Observe that both the Ullman notation and the Date notation convey the point that a join is really a cross product followed by a selection; the Russell notation implies it. In each of the three cases, the result is a set of tuples from the cross product of the first relation with the second relation that satisfy the given condition. Moreover, with this definition, we can bring further clarity to the equijoin and the natural join as follows:

The equijoin is simply the specific case where theta (the operator) is the equal operation. It is applicable in situations where matching attributes do not have identical names; typically, one attribute (or combination of attributes) constitutes a foreign key in one relation, while the other attribute (or combination of attributes) constitutes a candidate key in the other relation.

The natural join is a special case of the equijoin, where the attributes compared from both relations are not just defined on the same domain, but have the same name. Further, the natural join will eliminate one of the two (sets of) identical attributes from the final result.

Example 5b: Let us revisit the problem in example 5a — constructing a list showing combinations of department name and name of the related department head. This second solution involves an equijoin.

7.3.6 Union, Intersection, Difference Statements

The UNION, INTERSECT and MINUS statements are similarly represented in Ullman’s notation as well as Date’s notation. The respective syntax forms are as follows:

Note:

These are binary operations and in each case, both relations must have corresponding attributes that are defined on the same domain.

Union and intersect are associative, but minus is not.

7.3.7 Division Statement

The DIVIDEBY statement is of the form

In this format, relation R1 contains [m + n] attributes and relation R2 has n attributes.

The operation of division is as follows:

Every attribute of R2 must be an attribute of R1.

The resulting relation will have remaining attributes of R1.

A tuple occurs in R1 DIVIDEBY R2 if it occurs in R1, concatenated with every tuple in R2.

Example 6: Figure 7-5 provides two illustrations that should help you gain insight into the division operation. The first demonstrates how the division operation works; the second demonstrates that the division operation is the opposite of the Cartesian product operation.

Figure 7-5. Illustrating the Division Operation

A formal definition of division is as follows:

This definition has been recently refined in (the latest editions of) Date’s classic text on database system [Date, 2004]. If you need additional information on the subject matter, Date’s text is recommended. However, since mastery of this is scarcely applicable (if at all) at this level (in any event, division is a secondary operation), no further discussion will take place.

7.4 Aliases, Renaming and the Relational Assignment

In addition to the basic operations, three additional (advanced) operations defined on relations are

Alias Operation

Rename Operation

Assignment Operation

7.4.1 The Alias Operation

Aliasing is giving an alternate name to a relation. The format is:

Example 7:

Aliasing is used when it is desirable to refer to different tuples in the same relation, or when it is desirable to use a more convenient relation name in an application. Later in the course, as you learn SQL (in an Oracle environment), you will see that aliases are implemented as synonyms in Oracle.

Example 8: Suppose that we need to find all students who share birthdays with at least one other student. The solution is obtained by taking the cross product of Student and itself, and then searching for matches on DoB:

7.4.2 The Assignment Operation

The assignment operator is a colon followed by an equal sign (:=), and is used to have the system remember the results of other operation(s), which may be required for later use. The format of the assignment is:

The example in the upcoming subsection illustrates how relational assignments are done.

7.4.3 The Rename Operation

The RENAME operation renames specified attributes of a specified relation R1, into a new relation R2. We have established that it is desirable to have unique attribute names in each relation. Further, in the case of nested operations, it is a good habit to have an unambiguous way of referring to attributes of the results of an inner expression, from an outer expression. The RENAME operation is ideal for these situations. The format of the RENAME operation (based on Date’s notation) is:

As with selection, brackets may be used to avoid ambiguity. Ullman’s notation has been omitted here; though slightly different in syntax, its result is similar.

Examples 9: The following examples illustrate the use of renaming and relational assignment:

7.5 Other Operators

Other operations to facilitate computation usually exist. A brief summary of some of the common additional operators is provided here (based on Date’s notation). These will be further clarified, once the student is exposed to a DSL such as SQL (which we will discuss later in the course).

Extend: The EXTEND operator takes a relation as input, and returns a replica of it with one additional column (attribute), as defined by the user. The BNF notation for the syntax is

Example 10: List courses in program “BSC1”, with an additional descriptive column

Aggregate: The aggregate operations are used to summarize (aggregate) the values of a column of a relation. The standard operators are COUNT, COUNTD, SUM, SUMD, AVGD, MAX, MIN. COUNT, SUM and AVG return numeric values; MAX and MIN return alphanumeric or numeric values, depending on the column in question. COUNTD, SUMD, and AVGD avoid duplicate values for that column. Each operator is specified with a column (attribute) as follows:

The aggregate operations are typically used in a scenario where data is grouped.

Grouping: By grouping data from a relation, we can derive summarized information for the purpose of analysis. Grouping and aggregation usually go together. For the purpose of discussion, let us assume that the relational algebra syntax for defining a group is as follows:

where <Attribute List> is simply a list of attributes, with the comma as the separator. Grouping is normally embedded as part of some larger relational algebra expression. With the above notation, we can extract summary information from relations as in the following example.

Example 11: Produce a list showing the academic program and corresponding number of courses for each program:

7.6 Summary and Concluding Remarks

Let us summarize what we have covered in this chapter:

The relational algebra is said to be prescriptive — you specify precisely how an activity is to be carried out.

The eight basic operations of relational algebra are UNION, MINUS, RESTRICT, PRODUCT, PROJECT, JOIN, INTERSECT and DIVIDE BY. Each operation follows a specific syntax.

In addition to the basic operations, other relational operations include ALIASES, RENAME, EXTEND and GROUP BY.

The next chapter discusses relational calculus as an equivalent alternative to relational algebra.

7.7 Review Questions

Why is relational algebra important?

Briefly describe the basic operations of relational algebra.

Using the college database described in this chapter, practice writing relational algebra statements that will yield certain desirable results.

The following is an abridged specification of tables comprising a music database:

Write relational algebra statements to realize the following:

Registered musicians from CUB or CAN (where “CUB” and “CAN” are abbreviated codes for Cuba and Canada respectively).

Give the ENO of every ensemble that includes a VIOLIN or GUITAR player.

Give the ENO of every ensemble that includes a VIOLIN player but not a GUITAR player.

List all compositions (CNO and TITLE) by DAVID FOSTER.

List all performances (PNO, CNO, MNO, & PCOUNTRY) of compositions that have been performed in the country of origin.

7.8 References and/or Recommended Readings

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York, NY: Addison-Wesley, 2002. See Chapter 4.

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8h ed. Menlo Park, CA: Addison-Wesley, 2004. See Chapter 7.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapter 6.

[Garcia-Molina, 2002] Garcia-Molina, Hector, Jeffrey Ullman and Jennifer Widom. Database Systems: The Complete Book. Upper Saddle River, NJ: Prentice Hall, 2002. See Chapter 5.

[Kifer, 2005] Kifer, Michael, Arthur Bernstein and Philip M. Lewis. Database Systems: An Application-Oriented Approach 2nd ed. New York, NY: Addison-Wesley, 2005. See Chapter 5.

[Martin, 1995] Martin, James, and Joe Leben. Client/Server Databases: Enterprise Computing. Upper Saddle River, NJ: Prentice Hall, 1995. See Chapter 11.

[Russell, 2006] Russell, Gordon. Database eLearning. http://www.grussell.org/ (accessed July 2006).

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, NJ: Prentice Hall, 1997. See Chapter 4.

CHAPTER 8

Relational Calculus

This chapter discusses relational calculus as an alternate way of manipulating relations in a database. The partial college database, introduced in the previous chapter (Figures 7-1 and 7-2) will be used as a frame or reference. It must be constantly borne in mind that relational algebra and relational calculus are mutual equivalents; most DBMS suites will implement one or the other, or aspects of both, depending on what is convenient to the developers. The chapter proceeds under the following subtopics:

Introduction

Calculus Notations and Illustrations

Quantifiers, Free and Bound Variables

Substitution Rule and Standardization Rules

Query Optimization

Domain Related Calculus

Summary and Concluding Remarks

8.1 Introduction

In relational calculus, we simply specify what is required, not how to obtain the required relation(s). Relational algebra, on the other hand, provides a collection of explicit operations in SELECT, JOIN, PROJECT etc., which can be used to tell the system how to derive desired relation(s).

Example 1: Suppose that we are interested in obtaining a list of program names and associated course names for the Bachelor of Science in MIS.

In relational calculus, the user describes what is required; the system is left to decide how to service the user's request. Relational calculus is therefore said to be descriptive, while relational algebra is said to be prescriptive. As you shall see, this distinction is only superficial. More fundamentally, relational calculus and relational algebra are mutual equivalents: any algebra formation has a calculus equivalent and vice versa. Relational calculus is derived from a branch of mathematics called predicate calculus. It uses the idea of tuple variable (range variable). A tuple variable refers to a row of a relation at any given time. The relation is called the range. The Ingres language QUEL is similar to the original relational calculus language ALPHA, introduced by Codd.

Referencing an attribute is done by the following notation:

Example 2a:

Tuple variables may be implicit or explicit in SQL, as illustrated below:

Example 2b:

Tuple variables may also be implicit or explicit in QUEL, as illustrated below:

Example 2c:

Relational calculus of this sort is sometimes referred to as tuple calculus. An alternate domain calculus, based on domains, has been developed by Lacroix and Pirotte [Lacroix, 1977]. Query by example (QBE) is a language developed on this (domain calculus). This course focuses on the former, and not the latter.

8.2 Calculus Notations and Illustrations

As in the previous chapter, the BNF notation for expressing syntactical components of a language will be employed. In the examples that will follow to the end of the chapter, the semicolon is used to punctuate the calculus statements in the interest of clarity. However, bear in mind that the original QUEL language did not require such punctuations. The salient syntactical components of relational calculus are as follows:

Relational Operators include =, <, >, <=, >=, =>, <> while connectives include AND, NOT, OR

Comparisons are done variable-with-variable, attribute-with-literal or attribute-with-attribute, separated by operators.

Note  A comparison evaluates to true or false (similar to a relational algebra condition). The implication operator (=>) works as follows: P => Q means if condition P holds, then so does condition Q. Two alternate interpretation are as follows:

If P is true, so is Q; otherwise P is false.

Either Q is true or P is false, i.e. P => Q means Q or P'.

De Morgan’s Law for logical expressions is applicable:

Boolean Expressions can be derived by combination of comparisons using connectors AND, OR, NOT:

Note  Boolean expressions may be bracketed in the interest of clarity. This is necessary in situations where there are nested expressions.

Range Definition is as follows:

Relational Expression: A tuple calculus relational expression is of the following form:

Note:

Where the tuple variable alone is specified, all attributes associated with the variable are implied (remember that a tuple variable is defined on a relation which has attributes).

The relational expression defines a projection on attributes in the target list from the Cartesian product of all referenced relations (from the target list). The WFF (well formed formula) dictates a selection from the Cartesian product.

Well-Formed Formula (WFF): A well-formed formula (WFF) is of the following form:

Example 3: List all students, (names and ID#) who share surname with other students.

Note  The QUEL verb RETRIEVE describes retrieval. The pure calculus follows the RETRIEVE verb. Also note that the original QUEL did not use a semicolon to punctuate statements. It is done here simply to improve readability.

Example 4a: List all programs (showing Pgm#) that include the course M100.

Example 4b: List all programs (showing Pgm# and PgmName) that include the course M100.

Example 5: List all program codes and related course codes.

Example 6: List all program names and related course names.

Example 7: We could rewrite the solution to example 4 as follows:

8.3 Quantifiers, Free and Bound Variables

Carrying on, we need to introduce two quantifiers, and clarify what is meant by free and bound variables. The mathematical notations will be introduced, followed by the relational calculus notations. The mathematical notations are indicated in Figure 8-1:

Figure 8-1. Quantifiers

Additionally, if P(x) is a condition in tuple variable x, then

($x) (P(x)) means $x satisfying P(x)

("x) (P(x)) means all variables in range of x satisfies the condition P(x)

As expressed in the previous section, the relational calculus notations for the universal and existential quantifiers as follows:

8.3.1 Well-Formed Formula

Based on the notation given in the previous section, a WFF may be clarified as follows:

A simple comparison (condition) is a WFF

If F is a WFF, so are NOT (F) and (F)

If F1, F2 are WFF then so are (F1 AND F2), (F1 OR F2) and (If F1 THEN F2)

If the tuple variable x occurs freely in a WFF F then $x (F) and "x (F) are WFFs

Nothing else

In layman’s terms, a WFF is a simple or complex comparison involving attributes and scalar values. As can be confirmed from the forgoing sections and examples, WFFs are necessary for constructing appropriate data retrieval statements from the database.

8.3.2 Free and Bound Variables

A tuple variable is bounded if it occurs with either an existential or universal quantifier. Consider the scenario below:

This is an example of a bound occurrence of tuple variable p. The condition on p is either true or false, even if a particular value of p is not substituted. Note also that p can be replaced by q. Now consider the following:

This is an example of a free occurrence of variable p. The variable occurs in a simple condition (comparison) — when a particular value of p is substituted we can have a difference. Note that p cannot be replaced by q.

The following are some rules for free and bound variables:

All occurrences of variables in a simple condition (comparison) are free.

Any free/bound occurrence in WFF F is also free/bound in NOT F, (F).

Any free/bound occurrence in WFF F1, F2 is free/bound in (F1) AND (F2) as well as (F1) OR (F2).

Every free occurrence of x in F is bound in $x (F) or "x (F); the occurrence of other variables are not affected.

A tuple variable cannot be both free and bound in the same statement.

So to paraphrase and summarize, a tuple variable is bound in an expression if it is associated with a quantifier; once bound, it remains in this state for the expression. Here is another illustration:

Example 8: The example below should further clarify the correct usage of bound and free variables.

Example 9: List surname of students with DoB after 1975 (assuming the date format of YYYYMMDD).

Example 10: Find programs (names) that do not include course M100.

Example 11: List courses (code and name) that occur in all disciplines programs).

Example 12: Give the name of all programs with course M100.

Example 13: Give the names of department heads that are not division heads.

8.4 Substitution Rule and Standardization Rules

It is sometimes necessary to introduce “dummy” (apparently redundant) tuple variables, in conformance with an old rule that forbids retrieval of a quantified (bounded) variable(s). Further, for many data sub-languages (including SQL), if x and y are tuple variables and W is a WFF involving x and y, then

This rule often simplifies the database query statement by avoiding the use of the existential quantifier, and will become clear when we discuss SQL later in the course.

Additionally, due to the limitation of some DBMS suites in their support of relational calculus notations, it is often useful to apply certain substitution and standardization rules. These are mentioned below (Figure 8-2):

Figure 8-2. Substitution and Standardization Rules

8.5 Query Optimization

We have stated that every relational calculus expression has a relational algebra equivalent and vice versa. Query optimization involves the transformation from calculus to algebra to optimized algebra. The rules for conversion are outlined in Figure 8-3:

Figure 8-3. Query Optimization Rules

Example 14: Replace the calculus specification in example 12 with an algebra specification.

The two solutions are illustrated in Figure 8-4. Note that existential quantifiers are ignored in the translation process as long as the quantified variables are not implicated as part of retrieved attribute-list (hence the rule towards the beginning of the previous section). Also recall that existential qualifiers can effectively replace universal quantifiers.

Figure 8-4a. Graphical Representation of Non-optimized Solution to Example 12

Figure 8-4b. Graphical Representation of Optimized Solution to Example 12

8.6 Domain Oriented Relational Calculus

Domain related relational calculus involves the manipulation of domain variables instead of tuple variables (these must be defined similar to tuple variables). It supports an additional form of comparison called the membership condition. A member condition takes the form

R(A:V {,A:V}) where

R is a relation, A is an attribute, V is a domain variable or a scalar value (also called a literal). The condition evaluates true iff $ a tuple in R having specified values for the specified attribute.

Example 15:

Example 16:

Query-by-Example (QBE) is an attractive implementation of domain calculus by Microsoft Access. The user simply completes a table on the screen in order to make a query request. Further discussion of domain calculus is omitted from this course. Suffice it to say that domain calculus has its advantages in more complicated comparisons and is particularly useful when one is willing to concentrate on domain variables rather then tuple variables.

8.7 Summary and Concluding Remarks

It’s now time to summarize what we have covered in this chapter:

Relational calculus is said to be descriptive, meaning, you describe precisely the activity required. The chapter concentrated on tuple calculus, i.e., calculus related to the manipulation of tuple variables.

There are a number of standard relational calculus notations that allow you to succinctly describe data to be retrieved from relational tables. Retrieval statements essentially involve manipulating tuple variables using well-formed formulas (WFFs). A WFF is essentially a simple or complex condition.

WFFs often involve the use of free and bound variables. The rules for these must be strictly followed.

Query optimization is the process of obtaining the most efficient relational algebra equivalent for a relational calculus statement. In so doing, the basic rules for query optimization must be observed.

Domain oriented calculus is calculus related to the manipulation of domain variables. It is an alternative to tuple calculus.

The language QUEL very closely resembled the relational calculus. The more contemporary SQL represents exhibits both relational calculus and relational algebra features.

8.8 Review Questions

Why is relational calculus important? Explain why relational algebra is said to be prescriptive, and relational calculus is said to be descriptive. Provide an example to illustrate.

Describe the salient components of relational calculus.

Using the college database described in this chapter, practice writing relational calculus statements that will yield certain desirable results.

Explain the concept and process of query optimization. Use an appropriate example to illustrate.

8.9 References and/or Recommended Readings

[Codd, 1971] Codd, Edgar F. “A Data Base Sub-language Founded on the Relational Calculus.” Proc. 1971 ACM SIGFIDET Workshop on Data Description, Access and Control. San Diego, CA, November 1971.

[Codd, 1972] Codd, Edgar F. “Relational Completeness of Database Sub-languages.” In Data Base Systems: Courant Computer Science Symposia Series 6. Eaglewood Cliffs, NJ: Prentice Hall, 1972.

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York, NY: Addison-Wesley, 2002. See Chapter 4.

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8h ed. Menlo Park, CA: Addison-Wesley, 2004. See Chapter 8.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapter 6.

[Kuhns, 1967] Kuhns, J.L. “Answering Questions by Computer: A Logical Study.” Report RM-5428-PR. Santa Monica, CA: Rand Corporation, 1967.

[Lacroix, 1977] Lacroix M. and A. Pirotte. “Domain-Oriented Relational Languages.” Proc. 3rd International Conference on very Large Data Bases. October 1977.

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, New Jersey: Prentice Hall, 1997. See Chapter 4.

CHAPTER 9

Relational System — a Closer Look

We have covered much ground in our study of database systems. We have also established the importance of the relational model and its significant contribution to the field of database systems. We now pause to conduct a more enlightened discussion of this contribution and its effect on the field. This rather short chapter proceeds under the following subtopics:

The Relational Model Summarized

Ramifications of the Relational Model

Summary and Concluding Remarks

9.1 The Relational Model Summarized

Figure 9-1 summarizes the salient features of the relational model that have been established so far.

Figure 9-1. Salient Features of Relational Model

The above features define the minimum requirements of a relational system. Other desirable features of a database system discussed in chapters 1 and 2 are still important. As it has turned out, meeting those standards was (and is) not easy. For years, they have eluded, and continue elude software engineering firms aspiring to construct and market DBMS suites. As you will soon see, the standards bar has been raised even higher for contemporary products.

9.2 Ramifications of the Relational Model

The relational model has very far-reaching implications, and makes very stringent demands on the software engineering industry, to deliver DBMS products that meet a minimum set of standards. Let us briefly look at some of these ramifications.

9.2.1 Codd’s Early Benchmark

In 1982, Edgar F. Codd (referred to by many as the father of database systems), in a paper entitled “Relational Database: A Practical Foundation for Productivity” (see [Codd, 1982]), proposed that a system could be regarded as relational if it supported at least, the following:

Relational databases

The operations RESTRICT, PROJECT and (natural) JOIN, without requiring any prior definition of access paths to support these operations

Codd further asserted the following:

The operations may be supported explicitly or implicitly.

The system must internally optimize user requests for desirable performance.

Based on those minimum requirements, DBMS suites were classified in one of four categories: tabular, minimally relational, relationally complete, and fully relational. These classifications are clarified in Figure 9-2.

Figure 9-2. Categorization of DBMS Suites

Not many products were able to survive the rigors of that benchmark. Interestingly, three of the products that survived the test are still doing well in the industry today; they are Oracle, DB2 and INGRES.

9.2.2 Revised Definition of a Relational System

In 1985, Codd, in a paper captioned “Is Your DBMS Really Relational?” (see [Codd, 1985]), revised the definition of a relational system. Christopher Date later made some important recommendations about the revised model. The revised model with its recommendations is presented here (Figure 9-3).

Figure 9-3. Codd’s Revised Definition of a Relational System

Additionally, Codd proposed 12 rules for determining how relational a DBMS product is. With these 12 rules added to the redefined model, a DBMS is considered to be fully relational if it satisfies all structural, integrity and manipulative features, and fulfills the 12 rules. Christopher Date subsequently proposed a zero-rule that essentially summarized Codd’s 12 rules. The zero-rule and Codd’s 12 rules are presented here.

Date’s Zero-Rule

According to [Date, 1990], a system qualifies as a relational, database, and management system (highlights are deliberate) iff it uses its relational facilities exclusively to manage its database.

This is a loaded statement that will become clearer as you progress through the course. For instance, after studying chapters 12 – 14, you will have a better appreciation of what it means to use the relation facilities of a database to manage the said database. Moreover, it is not sufficient to take a system that is fundamentally not relational, add an interface that facilitates PROJECT, JOIN, SELECT (only), and then claim that the system is relational. Instead, everything must be relational — the view mechanism, the catalog, the structure, the integrity features, and the operations supported.

Codd’s Twelve Rules

Rule 1: The Information Rule: All information in the database must be represented as relational tables, subject to established integrity constraints, structural and manipulation features (mentioned earlier).

Rule 2: The Guaranteed Access Rule: All data stored in the database must be logically addressable by specifying the relation, the related attribute(s), and the primary-key-value.

Rule 3: Systematic Treatment of Null Values: The DBMS is required to have a consistent way of representing and treating so-called “missing information” that is different from regular data values, and independent of the data types supported.

Rule 4: The Active Online Catalog Rule: The system should host a comprehensive relational catalog that is accessible to authorized users via the regular query language. Chapter 14 will provide more clarification on the importance of the system catalog.

Rule 5: Comprehensive Data Sub-Language: The system must support at least one relational language that meets the following criteria:

has a linear syntax;

can be used both interactively and within application programs;

provides adequate support of DDL, DML, and DCL operations.

This requirement is adequately fulfilled in SQL, the universal standard database language that will be covered in chapters 10 – 14.

Rule 6: The View Updating Rule: All views that are theoretically updateable must be updateable by the system (this will be clarified in chapter 13).

Rule 7: High-level Insert, Update, and Deletion: The system must support set-wise DML operations such as INSERT, UPDATE, and DELETE operations. Chapter 11 will demonstrate that SQL fulfills this requirement.

Rule 8: Physical Data Independence: The system should isolate all application programs (and end user accesses) from the physical structure of the database. Changes in one should not affect the other.

Rule 9: Logical Data Independence: The system should isolate all application programs (and end user accesses) from the logical structure of the database. Changes in one should not affect the other.

Rule 10: Integrity Independence: Integrity constraints must be specified separately from application programs and stored in the system catalog. It must be possible to change such constraints as required, without effect on the applications that access the database.

Rule 11: Distribution Independence: Existing applications should continue to operate successfully when distributed versions of the DBMS are first introduced or upgraded.

Rule 12: Nun-subversion: If the system provides a low-level (record-at-a-time) interface, then it should not be possible to use this interface to undermine or bypass relational security or integrity constraints of the system.

9.2.3 Far Reaching Consequences

These constraints set a very high standard for relational DBMS (RDBMS) suites to attain. In fact, for a considerable period of time, the industry did not see a product that irrefutably met all of these requirements. On the other hand, many proposed products have fallen by the wayside, due to failure to come close enough to the established standards. With incremental improvements to products such as (but not only) DB2, Oracle, Informix, and Sybase over several years, the industry can now boast of products meeting these standards (but not without room for improvement). Chapters 16 – 20 take a look at some of these products.

Is the benchmark too high for RDBMS products? Not at all. It defines an ideal that software engineering firms can strive to attain. It also establishes a firm mathematical basis for the relational model. In this regard, the work of Codd, Date, Fagin, and others cannot be over applauded. To a certain extent, the standards have protected the consuming public from rogue companies that might have tried to exploit us by marketing inferior database products under false claim of them being relational. We have seen many such attempts, but for the most part, they have not gone very far.

In the next two divisions of the text, you will discover a rather interesting phenomenon: Many of the standards described in the revised benchmark for a RDBMS have been implemented in SQL, the universal database language, and leading DBMS products such as DB2, Oracle, Sybase, Informix, MS SQL Server, and MySQL. This is comforting information.

9.3 Summary and Concluding Remarks

Here is a summary of what has been covered in this chapter:

The benchmarks for a relational database system may be defined in terms of the data structure requirements, the data integrity requirements, and the data manipulation requirements.

Over the years, the database systems industry has embraced Codd’s benchmarks (first introduced in 1982 and subsequently revised in 1985) as the standards for relational DBMS suites. The revised standards also include Codd’s twelve rules and Date’s zero-rule.

Even though these standards are more than two decades old, they still remain binding on the industry. In fact, many of the smaller DBMS products still struggle to meet them.

Although Codd died in 2003, his legacy will no doubt continue to live and guide the field of database systems well into the foreseeable future. Date, a colleague of Codd, continues to be a renowned author and consultant in the field. So thankfully, we are in good hands.

9.4 Review Questions

Describe in your own words, what is meant by a relational DBMS.

Conduct a critical evaluation and comparative analysis of three leading DBMS products that you are familiar with. Use Codd’s revised definition of a relational database system as your benchmark.

Conduct a critical evaluation of E.F. Codd’s contribution to the field of database systems.

9.5 References

[Codd, 1982] Codd, Edgar F. “Relational Database: A Practical Foundation for Productivity,” Communications of the ACM, Vol. 25, Issue 2, February 1982. pp. 109-117.

[Codd, 1985] Codd, Edgar F. “Is Your DBMS Really Relational?” and “Does Your DBMS Run By the Rules?” ComputerWorld, October 14 and October 21, 1985.

[Date, 1990] Date, Christopher J. Introduction to Database Systems Vol. I 5th ed. Menlo Park, California: Addison-Wesley, 1990. See Chapter 15.

PART C

The Structured Query Language

The next six chapters will focus on the Structured Query Language (SQL). This language has become the universal standard database language. It is therefore imperative that as a student of computer science, or a practicing IT professional, you are not just familiar, but have a working knowledge of the language. The objectives of this division are:

to provide a solid overview of SQL as a database language;

to help you gain a good working knowledge of the main SQL data definition statements;

to help you gain a good working knowledge of the main SQL data manipulation statements;

to help you gain insights and a good working knowledge of logical views and system database security;

to discuss and illustrate the importance and usefulness of the system catalog;

to discuss some limitations of SQL.

Chapters to be covered include:

Chapter 10 — Overview of SQL

Chapter 11 — SQL Definition Statements

Chapter 12 — SQL Data Manipulation Statements

Chapter 13 — Logical Views & Security

Chapter 14 — The System Catalog

Chapter 15 — Some Limitations of SQL

CHAPTER 10

Overview of SQL

The Structured Query Language (SQL) has become the universal language of choice for DBMS products. A study of this language is therefore imperative for the student of computer science or computer information systems. This and the next few chapters will help you acquire a working knowledge of the language, as implemented in the Oracle environment (Oracle 10G or 11G). One fact you need to be immediately cognizant of is that there are different implementations of SQL. However, the implementations usually have more in common than differences; therefore once you have mastered the language in one DBMS environment, adjusting to another environment is a trivial matter. The chapter proceeds under the following subheadings:

Important Facts

Advantages of SQL

Summary and Concluding Remarks

10.1 Important Facts

Structure Query Language (SQL) is an example of a DSL — consisting of DDL, DCL, and DML as defined in chapter 2. First developed by IBM in the 1970s, SQL is the universal language of databases.

SQL may be described as an interactive query language as well as a database programming language. Commands can be entered directly at the command prompt, or embedded in application programs, written in some other language. Examples of languages that support embedded SQL include COBOL, BASIC, RPG IV, Java, Oracle PLSQL, Pascal, C++, Visual Basic, FoxPro, etc. The latest set of standards for SQL that is available for public access is SQL-2007. SQL-2007 is an enhancement of SQL-2003 to allow for better compatibility with the Extensible Markup Language (XML), and other refinements. In this text, we essentially concentrate on core SQL statements; these statements are also consistent with SQL-2003. The standards are updated roughly every four years. For more information, please see the references listed in section 10.5.

SQL is a non-procedural (descriptive) language that closely mirrors the relational calculus as discussed in chapter 8; there are also features that mirror the relational algebra of chapter 7. Although the language was originally introduced by IBM, no organization has a monopoly on it. Different implementations of SQL have their own idiosyncrasies and flavors. Some of the major DBMS suites are Oracle, DB2, Informix, Sybase, Ingres, Delphi, MS SQL Server, and MySQL.

10.1.1 Commonly Used DDL Statements

Figure 10-1 provides a list of commonly used DDL statements of SQL. You will observe that the statements are self-explanatory. This makes learning of the language very easy, particularly when compared to more cryptic traditional programming languages. Another feature that you will notice about SQL is that it is symmetrical and follows very consistent standards (language experts favor the term orthogonal to describe this feature). Thus, for most database objects, you can create the object, alter (i.e. modify) it, or drop (i.e. remove) it from the system. And the main database objects that you will be working with are tables, indexes, constraints and views; in the case of Oracle, there are also databases, tablespaces, sequences and synonyms. Of course, there are other database objects that will be covered later in the course.

Figure 10-1. Commonly Used DDL Statements of SQL

10.1.2 Commonly Used DML and DCL Statements

The commonly used DML and DCL statements of SQL are indicated in Figure 10-2. The DML statements are often referred to as SUDI (select, update, delete and insert) statements. The DCL statements fall into three categories: those that affect how DML operations take place (mainly COMMIT and ROLLBACK); those that relate to system privileges; and those that affect the environmental settings of the end user.

Figure 10-2. Commonly Used DML & DCL Statements of SQL

10.1.3 Syntax Convention

In the next four chapters (chapters 11 – 14), the core DDL DML and DCL statements of SQL will be discussed. The convention that will be employed for each statement is to present the full (or close-to-full) syntax in BNF format, then discuss the most common abridged format(s) of the statement; all the examples provided will be based on abridged formats of the statements discussed. In most cases, it is unlikely that you will need to use the full syntax of a statement; however, the full syntax is given to provide you with an accurate perspective of the possibilities, and in the interest of completeness.

10.2 Advantages of SQL

SQL brings a number of significant advantages to the software engineering industry. Some of these advantages are as follows:

Rapid Software Development: SQL enhances rapid development of business application systems by its powerful and easy-to-learn statements. In fact, most RAD tools, DBMS suites, and CASE tools support the language.

Higher software Quality: SQL brings higher software quality to the software engineering arena. By using more powerful SQL statements, than would be possible in traditional high level languages, the software engineer is likely to produce shorter code with fewer errors. The descriptive nature of the language is useful in this regard.

Higher Productivity: SQL brings higher productivity to businesses by providing superior database management features than possible in systems developed with traditional high-level languages alone. These superior database management features include the following:

Faster access to data

Larger files

Set-at-a-time access instead of record-by-record access

Powerful data aggregation facilities to provide meaningful end-user information

Facilitation of logical views to provide meaningful end-user information

Facilities for the enforcement of data integrity constraints, independent of application programs

Facilities for logically reorganizing data to provide useful information to end users

Facilities for the enforcement of database security constraints

Data Independence: SQL helps DBMS suites to meet the objective of data independence, with its associated benefits (review chapter 1).

Standardization: SQL facilitates standardization among competing software development tools, since they all are forced to support the language.

10.3 Summary and Concluding Remarks

Let us summarize what we have covered in this chapter:

SQL is the universe database language. It consists of various statements for creating and administering a database. These statements can be classified as DDL statements, DML statement, and DCL statements.

SQL can be used at the command prompt, or embedded in other high-level language programs.

SQL brings a number of significant advantages to the software engineering industry.

SQL is not without a fair amount of limitations. In order to fully understand the limitations, you need to have a working knowledge of the language first. With this in mind, a discussion of these limitations is left for chapter 15.

The next chapter discusses the main DDL statements of SQL. As you will see, they are easy to learn and use.

10.4 Review Questions

What are the most common DDL statements of SQL? Briefly explain the purpose of each.

What are the main DML statements of SQL? Briefly explain the purpose of each.

Explain the three categories of DCL statements in SQL.

Identify some of the advantages of SQL.

10.5 Recommended Readings

[Eisenberg, 2000] Eisenberg, Andrew and Jim Melton. “SQL Standardization: The Next Step.” SIGMOD Record Vol.29, no.1, March 2000. pp. 63 – 67.

[Kulkarni, 2003] Kulkarni, Krishna. Overview of SQL:2003. San Jose, CA: Silicon Valley Laboratory, IBM Corp., 2003. http://www.wiscorp.com/SQL2003Features.pdf (accessed August 2008).

[WISC, 2008] Whitemarsh Information Systems Corp. SQL. http://www.wiscorp.com/SQLStandards.html (accessed August 2008).

CHAPTER 11

SQL Data Definition Statements

The main SQL definition statements in an Oracle 10G environment are shown in Figure 11-1. These statements relate to the six basic types of database objects in Oracle — tables, indexes, views, constraints, synonyms, and sequences. Other more advanced types of database objects include databases, table-spaces, data-files, users, user profiles, functions, and procedures.

Figure 11-1. Commonly Used SQL Definition Statements

This chapter will focus on tables, synonyms, sequences and indexes; a discussion of the more advanced objects will follow in subsequent chapters. The chapter will proceed under the following subtopics:

Overview of Oracle’s SQL Environment

Database Creation

Database Management

Tablespace Creation

Tablespace Management

Table Creation Statement

Dropping or Modifying a Table

Working With Indexes

Creating and Managing Sequences

Altering and Dropping Sequences

Creating and Managing Synonyms

Summary and Concluding Remarks

11.1 Overview of Oracle’s SQL Environment

The most basic working environment provided in Oracle is the Oracle SQL *Plus environment. It is a command entry interface, and gives the new SQL user an excellent opportunity to learn the language. This default editor is a line editor: you are allowed to enter your SQL command on a line-by-line basis, terminated by a semicolon or forward slash (/). Some line editing commands and file editing commands are included in Figure 11-2.

Figure 11-2. Oracle SQL *Plus Editing Commands

Alternately, you can use Notepad or any other text editor to key in your SQL statements, save it as a text file, then read it into the SQL *Plus environment via the GET or START command for text (.txt) files, or the @ command for SQL (.sql) files. In such case, the path and filename must be specified within single quotes.

In addition to SQL *Plus, Oracle provides other more user-friendly GUI-based components for learning and using SQL. These include Oracle Enterprise Manager (OEM), Oracle SQL Developer (OSQLD), and Oracle iSQL *Plus. You may read more about these components in Chapters 16 and 25. Finally, Oracle implements its own host language, called PL/SQL. It is a simple language with a predominantly Pascal-like syntax. Although coverage of the syntax of this language is beyond the scope of this course, a few examples will contain Pl/SQL code (particularly in Chapter 12). Because of the simplicity of the syntax, you should be able to read the code and understand it, so there is no need to panic.

Throughout the remainder of the course, a slight modification of the BNF notation will be used for specifying the syntax of SQL statements. The symbols used are shown in Figure 11-3. Additionally, from time to time, you will observe the inclusion of clarifying comments that conform to how comments are made in the C-based programming languages (e.g. // this is a comment, /* and so is this */). These comments serve to clarify the syntactic representations or examples that they appear in.

Figure 11-3. BNF Notation Symbols

11.2 Database Creation

In the Oracle environment, database creation is quite involved, requiring you to be aware of several intricate details about the Oracle DBMS itself. A database may be created in one of three ways: manually, using the Database Configuration Assistant (DBCA), or during installation of the Oracle Database server software. In each case, the Create-Database statement is employed (manually or automatically). However, there are certain procedures and intricate details which have to be carefully observed. In the interest of getting you started, these intricacies will be de-emphasized for now. Below (Figure 11-4) is the syntax of the Create-Database statement, followed by a simple example.

Figure 11-4. Syntax of the Create-Database Statement

Observation: In some systems, database creation is as simple as creating a directory (library or folder depending on the operating system used). Unfortunately, this is not the case in Oracle. For more details on this matter, see the Oracle Product Documentation [Oracle, 2008].

11.3 Database Management

Once the database has been created, it must be populated with database objects. Database objects include tablespaces (specific to Oracle), tables, indexes, views, synonyms, procedures, triggers, packages, sequences, users, roles, etc. As a good DBA and/or software engineer, you will also need to carry out performance tuning on your database. This may involve reorganizing database tables and indexes, deleting unnecessary indexes or moving other objects. You will also be required to periodically perform backup and recovery procedures, or make alterations to the database itself. These issues will be more thoroughly discussed in Chapter 21. However, to enhance your overall appreciation, the syntax for the Alter-Database statement is provided (Figure 11-5). This statement is used extensively to effect structural as well as status changes to the database.

Figure 11-5. Syntax for Alter-Database Statement

There is also a Drop-Database statement that allows you to delete the database. In order to use it you must have administrator privilege. Also, be aware that not all component files are removed from the operating system when this command is issued. As an alternative, you can use the DBCA to delete a database.

11.4 Tablespace Creation

In the Oracle environment, another database object that must be created as part of the database preparation is a tablespace. A tablespace is the holding area for all database objects (actually, the objects are stored in datafile(s) contained within the tablespace). A database must have at least one tablespace, but typically has several. The following are tablespaces which are typically found in a database:

System tablespace (prefixed SYSTEM): This is a default tablespace used for system resources.

Temporary tablespace (prefixed TEMP): Used for intermediate results such as internal sorts for user queries.

Tools tablespace (prefixed TOOLS): Used for Oracle administrative tools.

Index tablespace (prefixed INDX): For storing and maintaining indexes.

Undo tablespace (prefixed UNDOTBS): For automatic logs due to data changes (this is particularly useful for data rollbacks and/or data recoveries after a catastrophe).

Users tablespace (prefixed USER): For storing user created tables.

A full discussion of Oracle tablespaces is beyond the scope of this course. However, in the interest of a credible introduction, Figure 11-6 provides the syntax of the Create-Tablespace statement, along with an example.

Figure 11-6. Syntax for Create-Tablespace Statement

11.5 Tablespace Management

Once the tablespace has been created and populated with database objects, it will need to be managed. This is normally done via the Alter-Tablespace command. Through this command, you can effect structural as well as status changes to the tablespace. As with the Create-Tablespace command, a full discussion of is beyond the scope of this course. However, in the interest of credibility, its syntax, as well as an example is provided in Figure 11-7. Of course, you can also delete a tablespace via the Drop-Tablespace statement. Figure 11-8 provides the syntax along with an example. Finally, through a component called Oracle Enterprise Manager (OEM), you can manage tablespaces (and most other database objects) in a GUI environment.

Figure 11-7. Syntax for Alter-Tablespace Statement

Figure 11-8. Syntax for Drop-Tablespace Statement

11.6 Table Creation Statement

Once the database and tablespaces have been configured, the next logical step is to create database objects which (in Oracle’s case) will be stored in the tablespace(s). The first type of database objects to be created is the relational and/or object tables; this is done via the Create-Table statement. The Create-Table statement is one of the most multi-faceted statements in SQL (next to the Select statement). The complete syntax is shown in Figure 11-9, while an abridged version of the statement is provided in Figure 11-10.

Figure 11-9. The Create-Table Statement

Figure 11-10. Abridged Version of the Create-Table Statement

The Data-Type-Spec defines a set of valid data types are indicated in Figure 11-11. This list is not exhaustive, but includes the more commonly used data types.

Figure 11-11. Valid Data Types

Please note:

The semicolon (;) signals the end of the SQL statement and each clause is separated by a comma (,).

In some systems (e.g. DB2), primary keys must be declared NOT NULL.

Most DBMS suites provide a utility that allows the user to key data into the table, once it has been created. Earlier versions of Oracle do not. However, through Enterprise Manager, later versions (starting with Oracle 9i) do.

Data can be entered in the table via the SQL insert statement (which you will learn in Chapter 12) or an application program, written for that purpose.

The matter of missing values is of much concern to any creditable DBMS. In DB2 and Oracle, nulls are used to represent missing or irrelevant information. Note that null is not the same as zero or blank. If nulls are allowed for an attribute and no data is provided, the system provides a null value for that attribute.

In many modern DBMS, a GUI sits on top of SQL definition statements so user may not use them.

Some DBMS, (e.g. Oracle) provide the user with the flexibility of specifying which attribute of a referenced table is to be used in the foreign key characteristic.

The operator specified in the Check-Clause may be any valid Boolean operator as defined in Chapters 7 and 8. When the third form of the Check-Clause is used, the column specified must be one of the values specified.

Value, Value1 and Value2 in the Check-Clause represent literal values.

Example 1: To create base relations Program, Course and Pgm-Struct as described in Chapter 7.

Example 2: The following example illustrates how the check constraints can be specified during table creation.

Once your table has been created, you may view the structure of the table via the Describe statement, which has the following format:

Example 3: The following statement will cause the structure of the table Student to be listed on the screen.

You can also create a table from an existing table by feeding the output of a Select statement as the input to a Create-Table statement. The format of the Create-Table statement for this is:

The sub-query component of the statement is specified by a Select statement. An example of this is provided below; this will be much clearer after you have been introduced to the Select statement (next chapter).

Example 4: The following statements create two snapshot relations: the first one stores Computer Science courses only; the second stores Computer Science and Information Systems majors only.

11.7 Dropping or Modifying a Table

A table may be dropped (deleted from the system) via the Drop-Table statement. The syntax is:

To drop all referential integrity constraints that refer to primary and unique keys in the dropped table, specify the Cascade-Constraints-Clause. If you omit this clause, and such referential integrity constraints exist, Oracle will return an error and will not drop the table.

Example 5: The following statement removes table CSCourses from the system:

When a table is removed, all references to it are removed from the system catalog. Any attempt to access a non-existent table will result in an execution error.

The structure of a table can be modified via the Alter-Table statement. Figure 11-12 provides the detailed syntax for the Alter-Table statement, while Figure 11-13 shows an abridged version.

Figure 11-12. The Alter-Table Statement

Figure 11-13. Abridged Version of the Alter-Table Statement

Example 6: We could add an additional attribute for course-abbreviation to the Course table:

Example 7: The following statement appends an additional attribute (and foreign key) to the Student table.

Oracle also allows you to rename a table (or any valid database object) via the Rename command. You can also remove all rows from a table via the Truncate-Table command. The abridged syntax for each command is as follows:

Example 8: The following statements a) renames the Hall table to Dormitory and b) deletes all tuples of the Student table.

11.8 Working with Indexes

An index is a database object that is used to speed up the retrieval of tuples from base relations. The index is independent of the table it indexes. It stores address pointers to tuples of the base relation. Once created, indexes are automatically maintained by the DBMS, and used to service certain user access requests as required. The software engineer’s (or DBA’s) only responsibility is to create them.

An index may be created whenever any of the following circumstances holds:

A column or combination of columns is frequently used in system query conditions (e.g. join conditions).

A column contains a wide range of values.

A column contains a large number of null values.

The table is large and most queries on it are expected to retrieve less than 2 % – 4% of the tuples.

A column or combination of columns is frequently used in the Order-By clause of queries.

An index is not necessary when any of the following circumstances holds:

The table is small.

Most queries are expected to retrieve more than 2% - 4% of the tuples.

The table is frequently updated (there are exceptions to this rule).

Columns are not often used in database queries.

An experienced software engineer or DBA can reasonably determine what indexes are to be created in a database, as this could seriously affect the performance of the database. When queries are executed, the DBMS first checks to see whether there are indexes that will facilitate efficient access of the requested data. If it finds such indexes, it uses them; otherwise, it creates temporary indexes required to service the queries.

Figure 11-14 shows the full syntax of the Create-Index statement, an abridged version of which is provided in Figure 11-16. Figure 11-15 provides the full syntax for the Alter-Index statement.

Figure 11-14. The Create-Index Statement

Figure 11-15. The Alter-Index Statement

Figure 11-16. Abridged Version of the Create-Index Statement

Note:

The default order is ASC(ending).

The (left-right) ordering of the columns in the index is significant in the usual major-minor convention.

The CLUSTER option specifies that this is a clustering index — index values are clustered.

The UNIQUE option specifies that no two rows in the indexed table will be allowed to have the same value for the index column(s).

The default algorithm for Oracle indexes is B-tree; however the user has the option of creating a bit-map index. The bit-map is like a matrix representing row-IDs and columns. It is useful in situations where many column values are identical (for instance where the primary key is composite).

Finally, you can drop an index via the Drop-Index statement. The syntax for this statement is shown in Figure 11-17.

Figure 11-17. The Drop-Index Statement

Example 9: The following statements create indexes on the tables indicated:

Example 10: The following statement drops the index CrsX:

Note:  When a table is dropped, all its related indexes are automatically dropped also.

11.9 Creating and Managing Sequences

A sequence is an Oracle database object that automatically generates unique numbers. It is typically used to create primary key values, particularly if the primary key is a single attribute. For composite primary keys, or other alphanumeric codes, the sequence could still be useful in generating a unique number, which is to be concatenated with some other data to comprise a code.

A sequence provides two significant advantages:

Its use could lead to shorter application code.

When sequence values are cached, processing efficiency is enhanced.

The syntax for the Create-Sequence statement is shown in Figure 11-18. As you can see, the statement is quite straightforward, and the clauses self-explanatory.

Figure 11-18. The Create-Sequence Statement

The sequence is accessed via two pseudo columns CURRVAL and NEXTVAL. Typically, it is accessed via a Select statement to retrieve its current value, or via an Insert statement to insert data into a table (both statements will be discussed in the next chapter).

Example 11: The following examples illustrate how a sequence is created and used:

To use the sequence in inserting a row into the table, the following statement may be embedded into a PL/SQL program block:

To view the current value, do a selection on DUAL (DUAL is the general purpose pseudo table used when a scalar value which is not stored in a specific database table is to be displayed):

11.10 Altering and Dropping Sequences

The attributes of a sequence may be modified via the Alter-Sequence statement. Its syntax is similar to that of the Create-Sequence statement (see Figure 11-19): The following guidelines apply to sequence modification:

Only the owner of a sequence (or a user with alter privilege to the sequence) can modify it.

Only future sequence numbers are affected by the modification.

To restart a sequence, it has to be deleted and then recreated.

Figure 11-19. The Alter-Sequence Statement

A sequence may be removed via the Drop-Sequence command, which has the following syntax:

11.11 Creating and Managing Synonyms

A synonym is a virtual object (i.e. the alias of an object), used to fulfill any or both of the following purposes:

Shortening the length of an object’s name

Referring to an object (typically a table) owned by another schema (user)

The Create-Synonym command has the following syntax:

Note:  If the synonym is public (created with keyword PUBLIC), other users have access to it; otherwise, it is private to the user who created it. Synonym, as used here, corresponds to the term “alias”, as used in Chapter 7.

Example 12: The following example creates a synonym called ValidDate for an object of the same name in another schema:

Here, the object ValidDate (which could be any valid Oracle database object, including a function) resides in a schema called Jones, and synonym is created in the current schema.

The syntax for deleting a synonym is:

11.12 Summary and Concluding Remarks

It’s time to summarize what was covered in this chapter:

The Oracle SQL*Plus environment provides a line editor that allows you to enter, edit and execute SQL statements. However, you may use any text editor of your choice and import the SQL statement into the SQL*Plus environment. Other alter GUI-based environments are iSQL *Plus, Oracle Enterprise Manager (OEM), and Oracle SQL Developer (OSQLD).

The Create-Database statement allows you to create an Oracle database. Alternately, you may use the Oracle DBCA.

The Alter-Database statement allows you to change features of a database, and the Drop-Database statement allows you to delete the current database. Alternately, you may use the Oracle DBCA.

The Create-Tablespace statement allows you to create a tablespace. The Alter-Tablespace statement allows you to change features of a tablespace, and the Drop-Tablespace statement allows you to delete a tablespace.

The Create-Table statement allows you to create a table. The Alter-Table statement allows you to change features of a table, and the Drop-Table statement allows you to delete a table.

The Create-Index statement allows you to create an index. The Alter-Index statement allows you to change features of an index, and the Drop-Index statement allows you to delete an index.

The Create-Sequence statement allows you to create a sequence. The Alter-Sequence statement allows you to change features of a sequence, and the Drop-Sequence statement allows you to delete a sequence.

The Create-Synonym statement allows you to create a synonym. The Drop-Synonym statement allows you to delete a synonym.

Each of the database objects discussed in this chapter can be created and fully managed using the OEM, OSQLD, or Oracle iSQL *Plus. However, in the interest of learning SQL, it is recommended that you stick with SQL *Plus. The upcoming chapter discusses the common DML and DCL statements of SQL.

11.13 Review Questions

What are the commonly used SQL definition statements?

Use the Oracle DBCA component to create a database. Practice writing SQL statements to create simple databases.

Practice writing SQL statements to modify different aspects of a database.

Write SQL statement to add a tablespace to your database. Create a second tablespace to hold the tables described in your sample college database.

Practice writing SQL statements to modify different aspects of a tablespace.

Write SQL statements to create the tables described in the sample college database described in Chapter 7 (Figure 7-1).

Write appropriate SQL statements to do the following:

Add a constraint to a table.

Modify a constraint.

Enable or disable a constraint.

Write SQL statements to define indexes on your database tables.

Practice writing SQL statements to modify or drop indexes.

By considering the Student table in your sample college database, write SQL statement to create a sequence for this table. Describe and demonstrate how this sequence could be used.

What is a synonym and how is it used in a database? Practice writing SQL statements to create and drop a synonym.

11.14 References and/or Recommended Readings

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York, NY: Addison-Wesley, 2002. See Chapter 6.

[Couchman, 1999] Couchman, Jason and Christopher Allen. Oracle Certified Professional: Application Developer Guide. New York, NY: Osborne/McGraw-Hill, 1999. See Chapters 3 and 4.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapter 8.

[Martin, 1995] Martin, James, and Joe Leben. Client/Server Databases: Enterprise Computing. Upper Saddle River, NJ: Prentice Hall, 1995. See Chapters 15 and 19.

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, MA: Course Technology, 2007. See Chapter 7.

[Oracle, 2008] Oracle Corporation. SQL Reference. http://www.oracle.com/technology/index.html (accessed October 2008).

[Shah, 2002] Shah, Nilesh. Database Systems Using Oracle: A Simplified Guide to SQL and PL/SQL. Upper Saddle River, NJ: Prentice Hall, 2002. See Chapters 3 and 4.

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, NJ: Prentice Hall, 1997. See Chapter 5.

CHAPTER 12

SQL Data Manipulation Statements

There are four core DML statements in SQL, namely INSERT, UPDATE, DELETE, and SELECT. These statements apply to base tables and views (views will be discussed in the next chapter). The chapter proceeds under the following subheadings:

Insertion of Data

Update Operations

Deletion of Data

Commit and Rollback Operations

Basic Syntax for Queries

Simple Queries

Queries Involving Multiple Tables

Queries Involving the Use of Functions

Queries Using LIKE, BETWEEN and IN Operators

Nested Queries

Queries Involving Set Operators

Queries With Runtime Variables

Queries Involving SQL Plus Format Commands

Embedded SQL

Dynamic Queries

Summary and Concluding Remarks

The chapter will continue to use examples based on the college database described in Chapter 7, so feel free to take some time to review this.

12.1 Insertion of Data

There are three general formats of the Insert statement; the abridged form of the syntax for each format is provided in Figure 12-1:

Note:

The first format is used when the attribute values to be inserted are explicitly provided with the statement (typically by an SQL user).

The second format is used when the attribute values to be inserted are implicitly provided from program variables (useful in embedded SQL). Oracle’s host language is PL/SQL (a Pascal-like language). The optional ampersand (&) that precedes the program variable denotes an execution time variable; the user will be prompted to specify a value when the SQL statement is executed. If no ampersand precedes the variable, then the SQL statement must appear as an embedded SQL statement within a PL/SQL program block; the values for the insertion are obtained from the specified program variables; some languages (for example C++) require a colon (:) to precede source program variables.

The third format is used when tuples to be inserted are to come from the result of some query.

If all the column names are omitted, this is equivalent to specifying them all in the same order as they were specified in the Create-Table statement.

Figure 12-1. Abridged Form of the Insert Statement

Example 1: Add a new department to the database with the following details: Code = D500; Name = Engineering; Head = S20:

Example 2: Save all students enrolled in the BSc in MIS program:

From the above example, the following should be noted:

The result of the SELECT is placed into table MajorsMIS. When used in this way, the Insert statement can insert a set of several rows at once; it illustrates what was referred to as set-at-a-time insertion in Chapter 9.

Table MajorsMIS is an example of a temporary relation or snapshot as discussed in Chapter 3.

Example 3: The following example illustrates a PL/SQL procedure to insert data into the Course table, where the values come from program variables:

Note:  This procedure can be called from the SQL prompt, or from within an application program by issuing the call statement as in the following example:

 Call InsertCourse ('CS101', 'Computer Applications');

An alternate way of specifying the above procedure is to remove the parameters and write the Insert-Statement with execution-time variables. An execution-time variable is a variable with an ampersand (&) preceding it. This indicates to the SQL parser that the user will be prompted to supply a value for the variable. You will see examples of the use of such variables in upcoming examples.

12.2 Update Operations

The Update statement is used for updating rows of a table. The update may be on a single row or a set of rows, depending on the condition specified in the Where-Clause. The abridged general formats of the Update statement are shown below (Figure 12-2):

Note:

A scalar expression is any combination of columns, arithmetic operators and SQL functions that evaluates to a scalar value (i.e. a number or an alphanumeric value). More will be said about SQL functions later.

The first form of the update statement is applicable when the update values are to come from literals that the user (programmer) will supply, or other columns of the table, or a combination of columns and literals.

The second form of the update statement is used in an embedded-SQL scenario, or where execution time prompt is desired.

Whenever a sub-query is specified within an update statement (the third form), it must provide values for the attributes to be updated.

Figure 12-2. Abridged Form of the Update Statement

Example 4: Change the Computer Science department name to ‘Department of Computer Science and Mathematics’; change the department head to Professor Hans Gaur (staff code of “S20”):

Note:  This is an example of a simple update (assuming that “CSC” is the unique code for the department). The record that satisfies the condition specified in the Where-Clause is fetched and updated.

Example 5: Assign all male students in the BSC in MIS to Urvine Hall:

Note:  This is an example of a multiple update. All records satisfying the condition specified are fetched and updated. This also illustrates what was referred to as set-at-a-time update in Chapter 9.

Example 6: Change the course code CS100 to CS105:

Note:

It is not a good practice to change codes that are primary key since it increases the possibility of violation of the referential integrity rule.

The Oracle DBMS will allow such updates providing that no violation of the referential integrity rule would result; if a violation would result, the instruction would be disallowed (also true for DB2). An alternative to this approach is to restrict the update of primary keys (the system would allow insertion and deletion only), except in the case where the DBMS introduces and maintains surrogates, transparent to the end user. Still another alternative is to allow cascaded update, but this would be very expensive.

Example 7: The following statement uses execution time variable in an Update statement:

Example 8: The following statement uses a query to update a row:

12.3 Deletion of Data

To remove rows from a table, the Delete statement is used. Alternately, you may use the Truncate statement to delete all rows from a table. Figure 12-3 shows the syntax of both statements.

Figure 12-3. The Delete Statement and the Truncate Statement

Example 9: Delete course ‘C9999’ from the database:

Note: The DBMS will not allow referenced tuples to be deleted unless DELETE CASCADES was specified when the related foreign key(s) was (were) specified. The default referential integrity for most DBMS suites is DELETE RESTRICTED.

Example 10: Delete all students that belong to Mary Seacole Hall:

Example 11: Delete all staff members:

or

Note: Examples 10 and 11 are illustrations of set-at-a-time deletions as mentioned in Chapter 9. In the case of Example 10, all records satisfying the condition are deleted; in the case of Example 11, all records are deleted from the table.

12.4 Commit and Rollback Operations

The Commit statement and the Rollback statement are both DCL statements that have relevance to data manipulation. They are useful in transaction management: the Commit forces a write to the database table while the Rollback causes recovery to a safe point. This is particularly important when multiple related tuples are written to different tables (for instance, an invoice may consist of a summary tuple, written to one table, and several detail tuples, written to another table). The syntax for both statements are shown:

Two related statements are the Set-Transaction statement and the Save-Point statement. The former is used to define the beginning of a transaction block; the latter is used to create a recovery point in case of catastrophes such as power loss or system failure. The syntax of each is in Figure 12-4:

Note:

These statements are typically used in a programming block, not at the SQL prompt.

The default transaction mode for Oracle is READ WRITE. However, for application programs that are required to read data only, this should be changed to READ ONLY to prevent record locking. This is particularly important in a multi-user environment.

These transaction management statements work the same way for insertions, updates, and deletions.

Figure 12-4. Syntax For Set-Transaction and Save-Point Statements

Example 12: A PL/SQL program block for inserting an invoice might look as follows:

12.5 Basic Syntax for Queries

From an end user perspective, one of the most important services of the database system is the facility to retrieve information and put it to use. This is made possible via the SQL Select statement. The Select statement is by far, one of the most powerful and complex statements in SQL. It is used to retrieve data from database tables, but as you will soon see, it has many optional clauses. Figure 12-5 provides a general form of the Select statement, showing the more commonly used clauses.

Figure 12-5. Syntax for the Select Statement

Observe: Except for a few omissions, the query condition closely mirrors the definition of a well-formed formula, given in Chapter 8. This is not an accident; it is deliberate.

Note:

The Function-Spec-Clause may involve any combination of a vast list of SQL functions, some of which will be mentioned in subsequent sections. The following are some categories of functions that are applicable:

Row functions

Date functions

Data conversion functions

Aggregation (group) functions

Programmer defined functions

Condition is as discussed in Chapter 8 (WFFs) except for the following:

In SQL, the construct IF <Comparison> THEN WFF is not allowed.

The syntax for using the existential quantifier s slightly different — the tuple variable is implicit rather than explicit.

The universal quantifier is not widely supported in SQL implementations. However, in light of available substitution rules (review section 8.4) we will not pursue its use any further.

DISTINCT stipulates that unique tuples will be provided as output where duplicate may occur.

A scalar expression is an appropriate combination of columns/literals and arithmetic operators, which results in a scalar value (typically numeric or alphanumeric). The valid scalar operators are multiplication (*), division (/), addition (+), subtraction (-) and concatenation (||).

Relational operators used in the Where-Clause or Having-Clause include the following:

We will start by looking as simple examples, then pull in additional features as we proceed.

12.6 Simple Queries

We will use the following format for a simple query:

Example 13: List name and ID of students enrolled in “BSC1” program:

Example 14: List all courses offered:

Example 15: Refer to the supplier-items database of earlier Chapters (3 and 4). Suppose that we have relations InventoryItem {Item#, ItemName, ItemPrice, Weight, …}, Supplier {Sup#, Sname, Scity, …} and Schedule {Sup#, Item#, Qty …}. We wish to show shipment amount on item received. Suppose each shipment is a box of 12 items. We could have the following query:

The result would look like this:

Example 16: Produce a list of students (showing the student’s full name and StudentID) from Chancellor Hall sorted on first name within surname:

Note the use of the concatenation operator in the scalar expression (Sname || ‘ ‘ || Fname) and the introduction of the alias (FullName). Please be careful not to confuse “alias” as used here, with the term described in Chapter 7. What we are talking about here is a column alias.

12.7 Queries Involving Multiple Tables

Quite often, it will be required to pull data from multiple tables in order to service a particular query. The situation may warrant a natural join, a theta join, an outer join, or a Cartesian product. Oracle supports two different approaches to dealing with queries from multiple tables — the traditional approach, and the newly introduced ANSI (American National Standards Institute) approach.

12.7.1 The Traditional Method

The traditional method for treating queries involving multiple tables is to specify the join condition in the Where-Clause. Let us take a few examples:

Example 17: Referring to supplier-items database (of Chapters 3 and 4), suppose we need to multiply quantity by item price to obtain the value of each shipped box. We will display supplier-name and item-name also. The query is shown below:

Note: This is an example of an equijoin. It also includes a scalar expression (SH.Qty * I.ItemPrice) and a column alias (Value). Note the introduction of explicit tuple variables SH, S and I, and the new column, Value. It is a good habit to introduce tuple variables when the information queried is to come from multiple tables, and/or comparisons are to be made in order to service the query. The result of the above query would look like this:

Example 18: Show all program-name and course-name combinations.

Example 19: Find department heads that are not division heads:

Note: This is an example of a theta-join. Here, we make use of the fact that the table Division stores an attribute (DvHead#) which indicates the staff member who is head of the division. We also make use of the fact that the table Dept stores a similar attribute (Dhead#) to indicate the staff member who is head of the department. Further, every department belongs to a division.

Example 20: Get all students who share a birthday with another student.

Note: This is an example of a Cartesian product of a relation with itself. Compare this with Example 8 in section 7.4 and Example 3 in section 8.2.

Caution: If you wish to join relation R1 with relation R2, make sure the R1’s foreign key does not allow for null values, since this will produce a spurious result. If in doubt, use an outer join, as clarified in the next example.

Example 21: Provide a list of all students and their assigned hall. Assume that it is possible for Hall# in the table Student to have null values. The following solution would be incorrect, since it could produce a spurious result:

Note: The results will exclude all students who have not been assigned to a hall (incidentally, this is an example of a natural join). To avoid this situation, use the outer join by simply including a parenthesized plus sign (+) on the attribute of the table where there may not be a corresponding row (in this case H.Hall#), when specifying the join condition. The Query should therefore be written as:

Note:

The plus sign indicates that rows from the implied relation may have null values (or non-matching values) for the attribute of interest; these rows should be included. If the plus sign is on the right, we have a left outer join; if the plus sign is on the left, we have a right outer join.

Using this approach, you can have the parenthesized plus sign (+)on either but not both sides of the join condition. This means that you have a left outer join, a right outer join, but not a full outer join. To obtain a full outer join, you have to take the union of a left outer join with a right outer join.

Left outer joins means take all rows from the table on the left; right outer join means take all rows from the table on the right.

12.7.2 The ANSI Method

Oracle 10g supports the more newly introduced ANSI syntax for handling queries from multiple tables. With this new syntax, the From-Clause is modified so that you are forced to explicitly specify the join. The syntax for the modified From-Clause is shown in Figure 12-6.

Figure 12-6. Modified From-Clause for ANSI Join

Please Note:

The first format describes the natural join. For this, the join column(s) in each table must be identical. Moreover, you are not allowed to qualify column names.

The second format is useful where more than one column in each table have the same name; you then need to specify which columns should be used for the joining. If you have a properly designed database with unique attribute-names in each table, there will not be many occasions to use this format.

The third format is the most widely used, because it is the most flexible. You specify the join condition after the keyword ON.

The fourth format is for taking Cartesian (cross) products; no join condition is required.

The fifth format is for outer joins; there are three types:

The left outer join is a join between two tables that returns rows based on the join condition, and unmatched rows from the table on the left.

The right outer join is a join between two tables that returns rows based on the join condition, and unmatched rows from the table on the right.

The full outer join is a join between two tables that returns rows based on the join condition, and unmatched rows from the table on the left, as well as the table on the right.

When the keyword LEFT | RIGHT | FULL is used, the keyword OUTER is implied and is therefore optional. Conversely, when the keyword LEFT | RIGHT | FULL is omitted, the keyword INNER is implied.

Left outer join means that all rows from the table on the left are kept; right outer join means that all rows from the table on the right are kept; full outer join means that all rows are kept from both tables (the one on the left and the one on the right).

One significant advantage of the ANSI syntax over the traditional is that you can separate join conditions from other conditions that can still be specified in the Where-Clause. Another advantage is a much easier achievement of a full outer join (in the traditional approach, you have to take a union of a left outer with a right outer).

Example 22: Example 18 is repeated here using ANSI syntax (show all program-name and course-name combinations).

Example 23: Example 21 is repeated here using ANSI syntax (student-name and hall-name combinations, including students not assigned to halls):

12.8 Queries Involving the use of Functions

SQL allows the use of several functions in order to provide the user (programmer) with flexibility in specifying queries. We will briefly discuss the following categories of SQL functions:

Row functions

Date functions

Data conversion functions

Aggregation (group) functions

Programmer defined functions

12.8.1 Row Functions

Row functions are functions that act on rows (tuples) of a query result, typically affecting the value of specific columns of a given row. Figure 12-7 provides a list of commonly row functions that can be used within a query specification. The list is by no means exhaustive.

Example 24: The following examples illustrate how the row functions may be used (note the use of the pseudo table DUAL for non-database data):

Figure 12-7. Commonly Used SQL Row Functions

Note: You may be surprised by a negative precision used with the rounding function. It simply means that rounding takes place to the left (instead of right) of the decimal point. The following examples should provide clarification:

ROUND (134.4, -1) = 130

ROUND (134.4, -2) = 100

ROUND (134.4, -3) = 0

ROUND (158.6, -1) = 160

ROUND (158.6, -2) = 200

ROUND (158.6, -3) = 0

12.8.2 Date Functions

Date functions constitute a special group of row functions in Oracle. Oracle has an internal representation of date, which stores the day, month, year (including century), hour, minute and second. The default date format is DD-MON-YY. Additionally, Oracle provides a number of date manipulation functions (Figure 12-8):

Figure 12-8. SQL Date Manipulation Functions

Example 25: The following examples illustrate how the date functions may be used:

To change the default format for system, you can use the Alter-Session statement, and change the pseudo column nls_date_format as in the following example:

12.8.3 Data Conversion Functions

Data conversion functions are used to convert data in the following ways:

Character to number

Number to character

Character to date

Date to character

Figure 12-9 indicates the three common data conversion functions (other less common functions are not discussed in this course). Figure 12-10 shows date formats and numeric formats which are often used with these functions.

Figure 12-9. SQL Commonly Used Data Conversion Functions

Figure 12-10. Valid Date and Numeric Formats

Example 26: The following examples illustrate how the conversion functions may be used:

12.8.4 Programmer-Defined Functions

Oracle supports programmer-defined functions; these are used in same manner as SQL system functions. Programmer defined functions are written in Oracle’s PL/SQL and stored on the server. For more information on how to define them, refer to your Oracle documentation.

Example 27: Below is a programmer defined PL/SQL function to determine y raised to the power x:

12.8.5 Aggregation Functions

SQL provides several aggregate functions to provide summarized view of data. Some commonly used aggregate (group) functions are described in Figure 12-11:

Note:

SUM, AVG, STDDEV and VARIANCE work with numeric values.

For MAX and MIN, the DISTINCT option is irrelevant.

COUNT works with the DISTINCT option always, except for the case where COUNT (*) or COUNT(1) is used.

Figure 12-11. Commonly Used SQL Aggregate Functions

Example 28: How many courses are offered?

Note: The second statement is preferred to the first, since it is more efficient.

Example 29: How many programs offer course M100?

Example 30: Referring to the supplier-items database, how many boxes of item ‘I100’ is supplied? What is the maximum quantity and the minimum quantity?

Aggregate functions are also often used in situations where data is to be grouped, in order to set up control breaks. This is illustrated in the following examples.

Example 31: Referring to the supplier-items database, produce a list showing item number and quantity of items shipped:

The aggregation functions are often used when grouping data (hence the alternate name, group functions) in order to set up control (summary) lines in reports. The next two examples will illustrate how this is done.

For the next two examples (as well as some of the upcoming ones), consider the section of the sample Oracle database (which is shipped with the Oracle product) consisting of the following tables (table names and column names may vary with different implementations):

Dept {Deptno, Dname, Loc} with primary key [Deptno]
Emp {Empno, Ename, Job, Mgr, Hiredate, Sal, Comm, Deptno} with primary key [Empno]

Assume further that Emp.Deptno is a foreign key, which references Dept.Deptno.

Example 32: Develop a list from the employee table, showing for each department, the total salary, average salary, minimum salary, maximum salary and standard deviation:

Note: In typical implementations of SQL, it is not permissible to use the GROUP BY clause in a join query. The solution below would therefore be rejected by the SQL compiler. However, there are ways to get around this hurdle (for instance creating a view or snapshot, then writing a query on the view or snapshot). This will be further discussed in Chapter 15).

The Having-Clause may be used to restrict groups. HAVING… works after the groups have been selected; WHERE… works on rows before group selection.

Example 33: Develop a list from the employee table, showing for each department, the total salary, average salary, minimum salary, maximum salary and standard deviation; show only departments with a total salary of at least $60,000:

Note: The question of when a column should be included in a query that involves grouping of data has often troubled inexperienced users of SQL. Here is a simple guide:

12.9 Queries Using LIKE, BETWEEN and IN Operators

The LIKE operator is used to test for the existence of string patterns in a column. The BETWEEN … AND operator is used to test for column values within a range of values. The IN operator is used to test for column values within a set of values; the set of values may be expressed explicitly, or implied from the result of a sub query (sub-queries will be discussed in the next section).

Example 34: Get names of all courses with code beginning with the acronym “CS” (all computer science courses, for instance):

Note: The wildcard % is used to denote that the string being searched for must begin with the character(s) specified; the characters that follow do not matter.

Example 35: Get names of all second year courses (assuming that a three-digit sequence number after the course acronym, indicates the level of the course):

Example 36: Referring to the Oracle employee-department database (of the previous sub-section), list employees within a salary range of 50,000 and 120,000:

Example 37: List all students enrolled in MIS, Computer Science or Mathematics (assuming sample data of Figure 7-2):

Note: In the next section, you will see that the IN operator works quite nicely with sub-queries.

12.10 Nested Queries

A sub-query is a nested query, i.e. a query within another query. A sub-query is particularly useful when a query on a given table depends on data in the table itself. Additionally, nested queries are alternatives to queries involving natural joins (not recommended for complex join conditions). There are two types of nested queries:

Single-row sub-query: only one row is retrieved

Multiple-row sub-query: multiple rows are retrieved

The typical format for a sub-query is:

In this format, the following rules apply:

The sub-query is another Select statement, which cannot include an Order-By-Clause.

The inner query is executed first, and its result is passed to the outer query.

Within the Where-Clause, a scalar expression involving at least one column may be specified (immediately following WHERE) instead of a column.

An alternate format for a sub-query is:

In this format, the following rules apply:

The relation-list includes at least one sub-query.

It is possible to have this format combined with the first.

The inner query is executed first, and its result is passed to the outer query.

Example 38: Produce a list showing student name and associated major (program) name (assuming a student’s program code is never null):

Example 39: Referring to the Oracle employee-department database, produce a list of employees who earn the maximum salary in their respective departments:

Note: In queries of this form, do not use the equal operator (=) as the connecting operator unless you are sure that the inner sub-query produces only one row. In any event, the IN operator is safer.

The ANY (or SOME) operator and ALL operator both work nicely with nested queries. The expression on the left is compared to any row or all rows from the sub-query on the right. In using these operators, the following convention must be noted:

< ANY(…) means less than the maximum value

= ANY(…) is equivalent to IN (…)

> ANY(…) means greater than the minimum value

> ALL(…) means greater than the maximum value

< ALL(…) means less than the minimum value

Example 40: Referring to the Oracle employee-department database, produce a list of employees who earn less than the maximum salary of a secretary:

Example 41: Referring to the Oracle employee-department database, produce a list of employees who earn less than the minimum salary of a secretary:

Example 42: Referring to the Oracle employee-department database (described in section 12.8.5), produce a list of employees who earn more than the average earning for his/her department (show both the employee’s salary and his/her department’s average salary):

Note:

Several layers of nesting may be constructed; however, this is not recommended.

Within nested queries, a scalar variable is confined (known only) to its select block at each level. However, (scalar) variables within a higher block, A, are known to the inner blocks of A. For instance, if a query solution consists of blocks A, B, C, where A consists of B and C, then:

Variables of block C are not known to block B and vice versa.

Variables of blocks B and C are not known to block A.

Variables of block A are known to blocks B and C.

Query Using Exists and NOT Exists

SQL also allows the specification of nested queries that involve the use of the existential quantifiers (EXISTS and NOT EXISTS). As mentioned earlier, the universal quantifier is not supported in some implementations of the language (it is supported in Oracle 10G and Oracle 11G). However, considering the standardization rules of Chapter 8 (section 8.4), this is not a serious deterrent. Moreover, in most cases, the rule of not retrieving a bounded tuple variable, along with its consequences, ensure that SQL statements can be specified without the use of quantifiers. That said, SQL nonetheless supports the use of the existential quantifiers, albeit in a manner that is slightly different from what was discussed in Chapter 8: the tuple variable is usually implicit.

Example 43: Let us revisit the problem of determining program names of programs that include the course M100:

12.11 Queries Involving Set Operators

SQL supports the binary set operators UNION, UNION ALL, INTERSECT and MINUS. Recall from Chapter 7, that a basic requirement for these operations is that (in each case) the two participating relations must have corresponding attributes defined on the same domain. Since from a design perspective, it is highly unlikely that a database will have many base relations that meet this criterion, these set operators are usually used in queries where two sub-queries produce results that meet the compatibility criterion.

The general syntax, therefore, is:

By way of review (except for UNION ALL), the set operators are explained below:

UNION: Returns all rows from both queries, but duplicate rows are not displayed.

UNION ALL: Returns all rows from both queries, including duplicate rows.

INTERSECT: Returns rows that appear in both query results.

MINUS: Returns rows that appear in the first query result, but not in the second.

Example 44: Produce a list all students enrolled in MIS, Computer Science or Mathematics (assuming sample data of Figure 7-2), as in Example 37, but this time using the union of sets:

Example 45: Find all students from Lenheim Hall who are mathematics majors:

Example 46: Find all male students from Lenheim Hall who are not mathematics majors:

12.12 Queries with Runtime Variables

It has already been established that runtime variables may be specified in an SQL data manipulation statement. This applies to data insertion, update, and deletion statements; it also applies to data retrieval statements (i.e. queries). The rules are unchanged.

Example 47: Produce a list of courses offered, starting from a particular course which the user will specify:

Note: The ampersand (&) preceding the variable InputCode indicates that user will be prompted to specify a value for the variable. This value will then be used to complete the query. A double ampersand indicates (to Oracle) that the input is to be obtained from the previous input value.

Example 48: Search for a course based on name or description, which the user will specify:

The Oracle PL/SQL Accept statement may also be used but it must be specified within a program block. The format is

Example 49: The following is a program block which includes use of the Accept statement:

12.13 Queries Involving SQL Plus Format Commands

The Oracle SQL Plus environment provides certain format commands that can be used to affect the appearance of outputs from queries. Figure 12-12 shows some of the commonly used format commands, while Figure 12-13 provides a list of valid format codes that may be used in a format specification:

Figure 12-12. Commonly Used Format Commands

Figure 12-13. Valid Format Codes

Example 50: The following are examples of the use of such commands:

To clear the format command settings, you use the format clearing statements. The syntax of two such commands are as follows:

12.14 Embedded SQL

When developing applications for industrial or commercial purposes, SQL is typically used as a database language that is used by the software engineer when it is convenient to do so. What this often means is that an application program may be written in a high level language (such as Pascal, C++, Java, etc.), but containings embedded SQL statements to take care of database access issues.

The general convention for embedded SQL is as follows:

Wherever a column, literal or scalar expression is applicable in an SQL statement, a program variable, or scalar expression involving programming variables, may be applied.

In many languages, the convention for specifying program variable is to precede the variable with a colon (:). One exception to this rule is Oracle’s PL/SQL, where a program variable is preceded either by the ampersand (&) in the case of execution-time variables, or nothing at all.

In the case where data is to be retrieved into program variables, the Select statement is specified with an Into-Clause. The required syntax is shown below.

In the case where a query will produce multiple rows, a cursor is set up. Although cursors are implemented differently from one host language to the other, the general principle is somewhat similar: A full discussion of cursors is beyond the scope of this course; suffice it to say that a cursor is a holding area for rows retrieved by a query. Cursors are normally used in stored procedures (in the host language), where multiple rows are retrieved for subsequent usage.

Embedded SQL was mentioned earlier in this chapter (see sections 12.1 and 12.2); in fact, Examples 3 and 12 illustrate embedded SQL relating to data insertion and transaction control respectively. The host language used here is Oracle’s PL/SQL. The general syntax for such queries is shown in Figure 12-14, followed by illustrations in Examples 51-53.

Figure 12-14. Syntax for Embedded Query

Example 51: The following examples illustrate two PL/SQL program block that each retrieves a single row into program variables (the second more elegantly than the first):

Example 52: The following example illustrates a PL/SQL program block that retrieves multiple rows via a cursor:

Example 53: The following example performs the same activity as the previous, but here a PL/SQL procedure is used:

12.15 Dynamic Queries

Dynamic queries are used a lot in complex programming environments, where it is either difficult or cumbersome to write a single SQL statement to correspond to each request that the end user is likely to make. Instead of attempting this feat, the application programmer or software engineer writes a sophisticated program (typically in a high level language) which constructs or generates SQL statements that are pertinent to the user request, and passes them on to an SQL parser. Three scenarios for this kind of programming come readily to mind:

A sophisticated user needs to conduct ad-hoc queries on an underlying database. The user is shielded from implementation details of the database, but is aware of underlying base tables or logical views, and the corresponding data fields (physical and/or logical) that they contain. The user is allowed to determine what details are to be included in his/her query.

A front-end system (such as Delphi, C++ Builder, Visual Basic, etc.) is being used to construct a user interface for a software system that accesses one or more underlying databases. The software developer incorporates embedded SQL statements in various application queries, based on user input. This request is then sent to the relevant database server to be serviced. The database server processes the request and sends the response back to the client.

A database administrator needs to periodically back up several components of an Oracle database. Rather than repeatedly issuing backup statements for the different items (for instances tablespaces) he/she may write an SQL script to dynamically generate the required SQL statements needed to perform backup of each component.

Figure 12-15 illustrates an inquiry screen for an application developed in Delphi. The end user is accessing a database for a list of publishers that the institution conducts business with. On the screen, you will notice a push button labeled Search, a radio group (with entries ByCode and ByName), and an input field. This application program works as follows:

The end user can specify any search argument in the input field, to indicate the starting publisher code or name of interest (blank means start at the beginning of the list of publishers).

The user will use the radio group to select whether information will be displayed sorted by publisher code or publisher name.

When the user clicks the Search push button, the program will examine the entries made, build an appropriate SQL statement to fetch the information requested, invoke the SQL parser to request this information from the underlying database (which could be in a different back-end system such as Oracle, DB2, etc.). The code for building the SQL statement is shown in the figure.

The database server that receives this request will process it and return the information to the program. The program will then load the grid shown with the information, display it on screen, and then await the next request from the end user.

Figure 12-15. Dynamic SQL For End User Access From Front-end System

Figure 12-16 provides another illustration of dynamic SQL. A detailed explanation is beyond the scope of this course; however a cursory clarification is in order, and will suffice:

The SPOOL statement (sixth line down) causes the output of the subsequent query to be redirected to the file specified.

The SELECT statement causes a series of rows to be written to the spool file. These rows actually contain SQL statements to backup tablespaces found in the current database.

The spool file can therefore be subsequently run as an SQL script to backup these tablespaces.

Figure 12-16. Automatic Backup of Tablespaces in a Database via Dynamic SQL Statements

12.16 Summary and Concluding Remarks

Relatively speaking, this has been a rather lengthy chapter. In recognition, the following paragraphs provide a summary of the various related topics covered, followed by some concluding remarks.

The Insert statement facilitates insertion of data into a specific table. The statement allows data insertion in one of three ways:

Insertion by specifying literal column-values for a row of a table

Insertion via execution-time variables for a row of a table

Insertion by redirecting the result of a query into a table (multiple records insertion is supported by this strategy)

The Update statement facilitates update of rows of a table. Depending on the condition specified (in the Where-Clause), the update might affect a single row, multiple rows, or all the rows (if no condition is specified, or all the rows meet the condition specified). The data to be used for the update can be specified in one of three ways:

By specifying literal column-values for the row(s) of the table that meet(s) the condition specified

By specifying execution-time variables corresponding to stated columns, for the row(s) of the specified table that meet(s) the condition specified

By specifying a sub-query that includes columns corresponding to the stated columns, for the row(s) of the specified table that meet(s) the condition specified

The Delete statement facilitates deletion of rows from a specified table. Depending on the condition specified (in the Where-Clause), the deletion might be for a single row, multiple rows, or all the rows (if no condition is specified, or all the rows meet the condition specified).

The Commit and Rollback statements facilitate transaction management, and are useful in situations where it is desirable to have a transaction wholly committed, or not recorded, in order to preserve the integrity of the database.

The Select statement is one of the most powerful and widely used statements in SQL. It facilitates retrieval of data from one or more tables and presenting the information to the end-user. This process is called querying. It is very flexible and can therefore be used in several different ways:

Simple queries involve data retrieval (via the Select statement) from one table only.

Queries involving multiple tables can be constructed in one of two ways:

The traditional method requires you to specify the join condition in the Where-Clause of the Select statement.

The American National Standard Institute (ANSI) method requires you to specify the join using join keywords. The ANSI method is more verbose but is also more flexible than the traditional method.

Queries involving the use of functions provide additional functionality and flexibility in data retrieval. The functions can be classified into five categories: row functions, date functions, data conversion functions, aggregation functions, and programmer-defined functions.

Queries can be constructed using the special operators LIKE, BETWEEN and IN.

You can have queries containing other queries – nested queries. These queries may involve the use of keywords such as ANY, ALL, EXISTS and NOT EXISTS.

You can define queries involving the use of set operators — UNION, UNION ALL, INTERSECT, and MINUS.

You can define queries that make use of execution-time variables.

You can define queries that make use of various SQL*Plus format commands.

Typically, software applications do not use SQL only, but embed SQL statements in programs written in another HLL.

A dynamic query is a query that uses data fed to it to construct a Select statement that is then executed. It avoids hard-coding values, thereby providing more flexibility to the end uses that use it.

Complete discussion of all aspects of the Select statement could easily take up a considerable portion of a book. The chapter captures and discusses the salient features, which if mastered, will place you on a solid footing in any environment that requires expertise in SQL.

One thing that should be clear is the remarkable power of the language. This can be appreciated if you attempt to write a Java or C++ program to replace any of the examples provided in the chapter. No wonder it is the universal database language. But go on to the next chapter, as there is much more to learn about the language.

12.17 Review Questions

Describe three ways in which data may be inserted into base table.

Write PL/SQL blocks to allow you to quickly populate the sample college database (created in Chapter 11) with data. Test these out in Oracle.

Write PL/SQL blocks to allow you to quickly modify data in your database. Test these out in Oracle.

Write PL/SQL blocks to allow you to easily delete data from your database. Test these out in Oracle.

Explain the purpose of the Commit and Rollback statements. Demonstrate how they may be used.

Practice writing the following:

Simple queries

Queries Involving Multiple Tables

Queries Involving the Use of Functions

Queries Using LIKE, BETWEEN and IN Operators

Nested Queries

Queries Involving Set Operators

Queries With Runtime Variables

Queries Involving SQL Plus Format Commands

Differentiate between embedded SQL code and dynamic SQL code. Describe a scenario that would warrant the use of each.

12.18 References and/or Recommended Readings

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York, NY: Addison-Wesley, 2002. See Chapters 20 and 21.

[Couchman, 1999] Couchman, Jason and Christopher Allen. Oracle Certified Professional: Application Developer Guide. New York, NY: Osborne/McGraw-Hill, 1999. See Chapters 1, 2, 5 and 6.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapters 8 and 9.

[Garcia-Molina, 2002] Garcia-Molina, Hector, Jeffrey Ullman and Jennifer Widom. Database Systems: The Complete Book. Upper Saddle River, NJ: Prentice Hall, 2002. See Chapter 6.

[Martin, 1995] Martin, James, and Joe Leben. Client/Server Databases: Enterprise Computing. Upper Saddle River, NJ: Prentice Hall, 1995. See Chapters 16-18.

[Oracle, 2008] Oracle Corporation. SQL Reference. http://www.oracle.com/technology/index.html (accessed October 2008).

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, MA: Course Technology, 2007. See Chapter 8.

[Shah, 2002] Shah, Nilesh. Database Systems Using Oracle: A Simplified Guide to SQL and PL/SQL. Upper Saddle River, NJ: Prentice Hall, 2002. See Chapters 5 - 12.

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, NJ: Prentice Hall, 1997. See Chapter 5.

CHAPTER 13

SQL Views and System Security

Two very powerful and important features of SQL are the facility to create and manage logical views, and the capability to manage security issues of a database. This chapter discusses these two related issues. The chapter proceeds under the following subtopics:

Traditional Logical Views

System Security

Materialized Views

Summary and Concluding Remarks

13.1 Traditional Logical Views

As pointed out in Chapter 3, a logical view is a virtual relation that allows end users to access information in a manner that is consistent with their requirements. Any creditable DBMS will allow the creation and manipulation of logical views. Following are a few important points about views:

SQL views allow for logical interpretation of information in the database.

Views include virtual, named (but not base), and derived relations; these significantly help to comprise the external schema of the database (review chapter 2).

Like named relations, views are created, and dropped using the Create and Drop statements. However, views cannot be structurally altered; they are virtual relations that are created and dropped as the situation dictates.

Views are stored in the system catalog.

Data modification (insert, update, and delete operations) can be applied through views in the normal (SQL) manner, providing that the view is updateable (more on this later). The view statements are compiled or translated (depending on the DBMS), and the implied base relations are updated at execution time.

Views can also be queried in a manner that is identical to how base relations are queried.

13.1.1 View Creation

An abridged version of the Create-View statement appears in Figure 13-1; the full syntax is shown in Figure 13-2.

Figure 13-1. Abridged Form of Create-View Statement

Figure 13-2. The Create-View Statement

Note:

Any derivable table (which can be obtained via Select statement) can be defined as a view. The sub-query is not allowed to include an Order-By-Clause, but this is not a serious omission, as there are alternatives around this constraint.

The view is in a way, similar to an insertion with a Select-Clause. The difference is that the insertion inserts actual data into a base relation; the view on the other hand, stores virtual data, i.e. it stores the access path to actual data that resides in the underlying base relation(s).

The OR REPLACE option replaces an existing view with the same name.

The FORCE option creates the view even if the underlying relation does not exist; the default is NOFORCE.

The WITH CHECK OPTION applies to the condition specified in the sub-query (in the Where-Clause): It allows insertion and update of rows based on the condition. This CHECK OPTION may be given an optional constraint name.

The READ ONLY option ensures that the view cannot be used for update of the underlying base relation.

Example 1: Create a logical view CSCourses that stores Computer Science Courses only (assume that the first two characters of the Course Code for Computer Science is “CS”):

Example 2: Create a logical view that stores for each academic program, a full breakout of all courses included in that program, as they would appear in a college bulletin:

13.1.2 View Modification and Removal

To modify a logical view, use the Alter-View statement. To remove it from the system catalog, use the Drop-View statement. The syntax for each statement is shown in Figure 13-3.

Figure 13-3. The Alter-View and Drop-View Statements

Note:

You are not allowed to alter the logical structure of a view, only constraints defined on it. In practice, this statement is seldom used.

When a view is dropped, it is removed from the system catalog.

The Cascade-Constraints-Clause is used to drop all referential integrity constraints that refer to primary and/or unique keys in the view to be dropped. If omitted, and such constraints exist, the Drop-View statement will fail.

Dropping a base relation automatically drops all associated views on that base relation.

Example 3: Remove the view CSCourses from the system:

13.1.3 Usefulness and Manipulation of Logical Views

Logical views are very useful, particularly during development of software systems that access related databases. The following are some advantages of views:

Provision of some amount of logical data independence in the face of restructuring of the database

Facilitation of assorted external views of the data stored

Simplification of the perception of end users — the users concentrate only on data that is of concern to them

Aiding system security — users have access only to data that concerns them, and cannot access or manipulate in any way, data to which they are not authorized

Data manipulation operations on updateable views are converted to equivalent operations on the underlying base table(s). In the case of data retrieval (via the Select statement), the conversion is straightforward and traces directly to operations on the underlying base relation(s). In the case of data changes (insertion, update or deletion), more care is required, as explained below.

A logical view is updateable if it meets all of the following criteria:

The view is defined to include attributes that constitute a candidate key of the underlying base relation

The view does not involve a JOIN*, UNION or INTERSECT operation

The Select statement does not contain the keyword DISTINCT

The Select statement does not include use of any aggregate function, a Group-By-Clause or a Having-Clause

The view does not contain derived columns

The view does not include the READ ONLY option

13.2 System Security

In a database system (and many software applications), there are three possible levels of system security:

Access to the system

Access to the system resources

Access to system data

Let us briefly examine each level, and see how they are facilitated in SQL. For the most part, this is examined in the context of an Oracle environment. However, the principles covered are also applicable in non-Oracle database environments as well.

13.2.1 Access to the System

Oracle allows access to the system through user profiles and user accounts. A profile is a working environment for a group of user accounts. When you create a database, Oracle creates a default profile called DEFAULT. When you create a user account, if you do not specify a profile, Oracle assigns the DEFAULT profile to the user account. A user account consists of a user name, a password and other optional parameters. Each user account has an associated profile.

Figure 13-4 shows the syntax for the Create-Profile statement, and example 4 illustrates how it is used. As can be seen from the syntax, the command is used to help create a working environment for the user.

Figure 13-4. The Create-Profile Statement

Example 4: The following statement creates a user profile called InventoryProfile. The parameters not specified will have default values.

As expected, Oracle allows you to modify or drop a profile via the Alter-Profile statement, or the Drop-Profile statement respectively. The syntax for each is shown below (Figure 13-5).

Figure 13-5. The Alter-Profile and Drop-Profile Statements

Note: The CASCADE option on the Drop-Profile statement instructs Oracle to reassign all user accounts formerly assigned to the profile, to the DEFAULT profile.

In addition to profiles, Oracle allows the DBA (or some user with system admin privileges) to create and remove users accounts. Each user account is assigned to a profile. If the profile is not specified at account creation, Oracle assigns the account to the DEFAULT profile. The required syntax for the Create-User statement follows:

Figure 13-6. The Create-User Statement

Note:

Oracle supports a local database user, a global user (authenticated by Enterprise Directory Service), or an external operating system user.

To activate the password expiration (defined in the user’s assigned profile), specify PASSWORD EXPIRE.

Use the ACCOUNT LOCK / UNLOCK option to lock the account (make it inaccessible by the user) or unlock the account.

When you create a user, Oracle creates a schema by the same name as the user. All objects created by that user will be owned by the user’s schema.

Example 5: The following statement creates a user called Bremar.

Of course, user accounts may be modified or dropped from the system. The Alter-User statement allows you to change all the parameters on the Create-User statement, but gives the added flexibility of changing some additional settings for the account. A condensed version of the syntax is shown in Figure 13-7, which also includes the syntax for the Drop-User statement.

Figure 13-7. The Alter-User Statement

Note:

You can use this command to assign roles to the user account. We will discuss roles shortly.

You can also use the Proxy-Clause to grant other privileges to the user (for more on this, see the Oracle documentation for the Alter-User statement).

The CASCADE option on the Drop-User statement drops all objects owned by the user’s schema before dropping the user.

Example 6: The following statement removes the user called Bremar:

13.2.2 Access to the System Resources

In the context of a database, we are concerned with access to system objects such as tables, views, indexes, sequences, synonyms, procedures, and other objects that the DBMS may support. Three possible actions (considered as privileges) may apply to each of these objects: creation, alteration, and dropping. Additionally, any valid SQL statement is considered a privilege. Finally, there are other system-wide privileges and roles (as you will see shortly).

The Grant statement is used for granting access of resources to user(s) and/or role(s), while the Revoke statement is used for revoking access of resources from user(s) and/or role(s). The basic syntax of each command is shown in Figure 13-8.

Figure 13-8. The Grant and Revoke Statements

Note:

The ON Clause is used if the privilege relates to a database object.

The privilege is typically a command name or a role; the recipient of the privilege may be a user or a role (roles will be clarified shortly).

The WITH ADMIN OPTION enables the recipient (user) to be able to grant/revoke this system privilege to/from other users or roles.

The WITH GRANT OPTION enables the recipient (user) to be able to grant/revoke this object privilege to/from other users or roles.

Example 7: Create user BruceJones and grant certain privileges to him; also allow user Scott to be able to run queries on the Program table:

Example 8: Remove query access of Program from Bruce Jones and all privileges from Stalker:

Development Privileges

An application developer should have the following system privileges:

The Create-Session privilege is required for logging onto Oracle and starting a work session. The Analyze-Any privilege is useful for database performance analysis and tuning (discussed in chapter 21). The Select-Any-Dictionary privilege is required in order to query the system catalog (to be discussed in the upcoming chapter). The other privileges are all self-explanatory, and need no further clarification at this point.

Roles

In managing multiple users in different user groups, roles are particularly useful. You can define a role to consist of several privileges, and then grant the role to users. Additionally, roles can be granted to other roles. A role is created via the Create-Role statement, modified by the Alter-Role statement, and removed via the Drop-Role statement. Figure 13-9 shows abridged formats of these statements.

Figure 13-9. Create-Role, Alter-Role and Drop-Role Statements

Note: To create a role, you need to have the Create-Any-Role system privilege (the Any is optional; if supplied, you can create a role in any schema; if omitted, you can create roles in your schema only). Only role owners and users with Alter-Any-Role and Drop-Any-Role system privileges can modify or delete a role.

Example 9: Create a role called Developer with appropriate privileges and grant the role to Bruce Jones:

You can use terms such as ANY, ALL, NONE and EXCEPT in assigning privileges.

Example 10: The following examples illustrate the use of these keywords:

13.2.3 Access to the System Data

Access to system data can be managed in one of three ways:

Through object privileges

Through logical views

Through intricate database design

Security via Object Privileges

Object privileges apply to specific database objects and are sometimes referred to as SUDI (select, update, delete, and insert) privileges. Below (Figure 13-10) is a list of possible object privileges and the relevant types of database object to which they apply:

Figure 13-10. Object Privileges

The Grant statement is used to issue object privileges. When used for this purpose, the following rules apply:

The ON-Clause is required.

The privilege specified must be an object privilege, or the keyword ALL.

Example 11: The following examples assume the prior creation of the user accounts specified:

Example 12: The following statement revokes UPDATE privilege on table Student, for user BruceJones.

Security via Views

As stated in the previous section, views can be used to enhance system security by allowing users to access only what is relevant to them. Conversely, views can be used to prevent users from accessing data for which they have no access privilege.

Example 13: Referring to the college database of earlier discussions, develop a set of logical views that allow department heads to have access to student information if and only if the student is enrolled in a major offered by that department:

Note:You can block direct access to data in a base table (for example Student), and force users to access the data through logical views on the table. These logical views can in turn have restricted access to specific users as illustrated.

Security via Database Design

In addition to views, a database designer may design database tables with security attributes that will subsequently be used to control user access. Only authorized users will have access to these security attributes, but they can be used to block other users from accessing sensitive data. However, you would be required to create and maintain some additional tables (an example of this approach appears in [Foster, 1999]).

13.3 Materialized Views

Oracle supports database objects called materialized views. A materialized view is a database object that stores the results of a query. It differs from the traditional logical view in that whereas the logical view stores the definition of the query, the materialized view stores the result of the query. A materialized view would therefore qualify as a snapshot relation (review chapter 3). A full discussion of materialized views is beyond the scope of this course; however, a brief introduction is worthwhile.

The From-Clause of the sub-query that feeds a materialized view can name tables, views, and other materialized views. Collectively these are called master tables (a replication term) or detail tables (a data warehouse term). Databases that contain the master tables are called the master databases.

Materialized views are used in replication environments, as well as in data warehousing (to be discussed in chapter 24). In replication environments, the materialized views commonly created are primary key views, rowed views, object views, and sub-query views. For data warehousing purposes, the materialized views commonly created are materialized aggregate views, single-table materialized aggregate views, and materialized join views.

To create a materialized view in your own schema, you need the following system privileges:

Create Materialized View

Create Table or Create Any Table

Select Any Table

To create a materialized view in another user’s schema, you need the following system privileges:

Create Any Materialized View

Create Table or Create Any Table

Select Any Table

If you desire to create a materialized view with QUERY REWRITE enabled (see syntax below), then in addition to the above-mentioned privileges, the following must hold:

The owner of the master tables must have the QUERY REWRITE system privilege.

If you are not the owner of the master tables, you must have the GLOBAL QUERY REWRITE system privilege or the QUERY REWRITE object privilege on each table outside your schema.

If the schema owner does not own the master tables, then the schema owner must have the GLOBAL QUERY REWRITE privilege or the QUERY REWRITE object privilege on each table outside the schema.

If you are defining the materialized view on a pre-built container (see syntax below), you must have the Select privilege WITH GRANT OPTION on the container table.

The user whose schema contains the materialized view must have sufficient quota in the target tablespace to store the materialized view’s master table and index, or must have the UNLIMITED TABLESPACE system privilege.

13.3.1 Creating a Materialized View

The Create-Materialized-View statement is used for creating materialized views. The syntax is shown in Figure 13-11.

Figure 13-11. Create-Materialized-View Statement

Example 14: The views of example 13 are replaced by materialized views below:

13.3.2 Altering or Dropping a Materialized View

As you no doubt expect, there is an Alter-Materialized-View statement and a Drop-Materialized-View statement. The respective syntactical structures are shown in Figure 13-12.

Figure 13-12. Alter-Materialized-View and Drop-Materialized-View Statements

Example 15: The following statements change methods used for materialized view refresh.

13.4 Summary and Concluding Remarks

Let us summarize what we have covered in this chapter:

A logical view is a virtual relation that allows end users to access information in a manner that is consistent with their requirements. The view is created by the Create-View statement, which simply allows few required keywords to be inserted ahead of a query (see Figure 13-2).

The view is treated just like a normal base table. It can be designed to be updateable or read-only.

The Alter-View statement facilitates modification of the view, and the Drop-View statement facilitates deletion of the view.

Views are very beneficial in enhancing logical data independence, facilitation of assorted external views of the database, simplification of the perception of end users, and enhancing system security.

SQL facilitates the enforcement of a stringent security mechanism at three levels: access to the system, access to system resources, and access to data.

Access to the system is controlled by profiles and user accounts. SQL provides statements for creating, altering and dropping of profiles as well as user accounts.

Access to system resources is controlled by privileges and roles. You can grant privileges and/or roles to users via the Grant statement, and revoke them via the Revoke statement. You can lump privileges together by creating a role via the Create-Role statement, and then granting the privileges to the role. The role can then be granted to or revoked from other users. Of course, a role can be altered (via the Alter-Role statement) and dropped (via the Drop-Role statement).

Access to system data can be controlled through object privileges to base tables, logical views and object privileges to them, or intricate database design.

A materialized view is a database view that stores both definition and the result of the related sub-query. It therefore qualifies as a snapshot relation. You can create, alter and drop materialized views via the Create-Materialized-View statement, the Alter-Materialized-View statement and the Drop-Materialized-View statement respectively.

Logical views constitute a very important part of a database, providing a number of conveniences that translate to improved efficiency, flexibility and productivity. As you will see in the upcoming chapter, they also form an integral part of the database system catalog.

13.5 Review Questions

What is a logical view? Discuss the importance and usefulness of logical views in a database.

When are views updateable and when are they not? Discuss.

Practice writing SQL statements to define logical views for various scenarios. Use the sample college database as your model.

Briefly explain the three levels of security in a typical database system. Explain how Oracle’s implementation of SQL facilitates these three levels.

Improve on your college database (from chapter 11) by doing the following:

Create a user profile called EndUser, for your database. Create a second profile called Developer.

Create two roles: one for each profile (you may call them EndUserR and DeveloperR respectively). Grant appropriate system privileges to these roles.

Create two users: BruceEnd and BruceDev. BruceEnd must belong to profile EndUser, with default role EndUserR; BruceDev must belong to profile Developer, with default role DeveloperR. Both users must have your tablespace (created from chapter 11) as their default tablespace, with QUOTA UNLIMITED.

Grant full access of all your database objects created so far to user BruceDev. Grant limited access of your database objects to user BruceEnd.

Alternately log on to the system as BruceDev and then BruceEnd, and check to see whether the privilege restrictions you have set are taking effect.

Practice writing SQL statements to manipulate system and object privileges. Test with your account (assumed to be System or SysDBA), BruceDev and BruceEnd.

What are materialized views? Discuss their relevance usefulness. When would you use a materialized view versus a traditional logical view, and vice versa?

Describe a situation that would warrant the use of a materialized view in your college database. Write an SQL statement to define that materialized view.

Practice writing SQL statements to modify materialized views.

13.6 References and/or Recommended Readings

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York, NY: Addison-Wesley, 2002. See Chapter 18.

[Couchman, 1999] Couchman, Jason and Christopher Allen. Oracle Certified Professional: Application Developer Guide. New York, NY: Osborne/McGraw-Hill, 1999. See Chapter 4.

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8th ed. Menlo Park, California: Addison-Wesley, 2004. See Chapters 9 and 17.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapter 23.

[Foster, 1999] Foster, Elvis C. Labour Market Information System: Thesis. Kingston, Jamaica: Department of Mathematics and Computer Science, University of the West Indies, 1999. See Chapter 4 and appendix 8.

[Oracle, 2008] Oracle Corporation. SQL Reference. http://www.oracle.com/technology/index.html (accessed August 2008).

[Shah, 2002] Shah, Nilesh. Database Systems Using Oracle: A Simplified Guide to SQL and PL/SQL. Upper Saddle River, NJ: Prentice Hall, 2002. See Chapter 13.

CHAPTER 14

The System Catalog

Every reputable DBMS contains a system catalog (also called the data dictionary) of some form. This has been alluded to several times earlier in the course. This chapter discusses this very important component of the database system. The chapter proceeds under the following subtopics:

Introduction

Three Important Catalog Tables

Other Catalog Tables

Querying the System Catalog

Updating the System Catalog

Summary and Concluding Remarks

14.1 Introduction

The system catalog (data dictionary) is perhaps the most important database object in a database system. This is so because it facilitates and supports all or most of the other database objects. The system catalog typically contains metadata about the database. By metadata, we mean data about other data. This catalog itself consists of relational tables, which can be manipulated using SQL statements. The system catalog provides the following benefits:

The system catalog, by maintaining metadata in the form of other relational tables, facilitates most (if not all) of the other database objects. This fulfills the requirements of the Zero Rule of Chapter 9.

Through the system catalog, the DBMS is able to deliver on the requirements of physical and logical data independence (Chapters 1 and 9) in a sleek manner.

Through the system catalog, the DBMS is able to deliver on the requirement of integrity independence (Chapters 4, 5 and 9).

System and object privileges (discussed in the previous chapter) are stored in special catalog tables, thus facilitating the management of the security mechanisms of the database.

As you will see later in the course (Chapter 22), the system catalog also facilitates the successful implementation of distributed database systems.

The rest of this chapter will focus on aspects of the system catalog, as implemented in Oracle. Note however, that all features of the Oracle catalog may not apply to catalogs of other DBMS suites; also there may be features of others, not included in the Oracle catalog. Note also that a comprehensive discussion of the Oracle data dictionary is beyond the scope and intent of this course; the discussion here is necessarily cursory, but detailed enough to give you a good appreciation of the subject matter.

The Oracle system catalog contains system tables for various database objects: Figure 14-1 provides some of the most commonly referenced objects that are facilitated by the system catalog.

Figure 14-1. Commonly Referenced Catalog Tables

These tables are automatically maintained by the DBMS in a manner that is transparent to the user. Oracle allows manipulation of catalog tables only through views. For each table, three views are often available: the view that has the prefix DBA (for all database objects), a view that has the prefix USER (for the objects owned by the current user), and a view that has the prefix ALL (for all objects that are accessible to the current user). However, there are exceptions to this rule. Additionally, views prefixed by V$ are dynamic performance views which can be queried, irrespective of your schema, provided that you have the appropriate privilege. Finally, views prefixed by GV$ are global dynamic views.

14.2 Three Important Catalog Tables

To illustrate the importance of the system catalog, let us focus our attention on three important catalog tables: Tables, Tab_Columns and Indexes. We will focus on three views on these tables: User_Tables, User_Tab_Columns and User_Indexes.

14.2.1 The User_Tables View

This catalog view is based on the underlying table Tables (the DB2 equivalent being Systables). It contains a row for every base table in the user’s schema. When a user account is created, Oracle creates a schema with the same name as the user name. All objects created by that user are linked to his/her schema, and are eventually stored in a tablespace, the default tablespace carrying the name System (for a full discussion of tablespaces, see the Oracle production documentation).

To observe the structure of User_Tables, you may invoke the describe statement thus:

You will observe that the table contains a number of columns. Among the ones of immediate concern are those indicated in Figure 14-2.

Figure 14-2. Columns of User_Tables

14.2.2 The User_Tab_Columns View

This catalog view is based on the underlying table Tab_Columns (DB2 equivalent being Syscolumns). It contains a row for every column of every table mentioned in User_Tables. Figure 14-3 shows the columns of immediate concern.

Figure 14-3. Columns of Tab_Columns

14.2.3 The User_Indexes View

This catalog view is based on the underlying table Indexes (DB2 equivalent being Sysindexes). It contains a row for every index in the user’s schema. Some columns of interest are mentioned in Figure 14-4.

Figure 14-4. Columns of User_Indexes

Figure 14-5 provides a simplified illustration of what the catalog data would be, assuming that the database consists of the tables mentioned in the sample college database that we have been referencing (since Chapter 7).

Figure 14-5. Simplified Illustration of System Catalog for College Database

14.3 Other Important Catalog Tables

There are several other catalog tables all of which are managed in a manner transparent to the database user. Figure 14-6 provides a .list of commonly used catalog views. This list is by no means exhaustive. However, studying and probing this list will give you a good insight of the usefulness of the catalog views, as you manage the database. Do not feel disconcerted or overwhelmed if you are not familiar with all of the views listed in the figure. These you would normally cover in a course on Oracle database administration. The intent here is to give you a reasonable overview of the role and complexity of the system catalog.

Figure 14-6. Commonly Used Catalog Views

14.4 Querying the System Catalog

The system catalog can be queried using SQL Select statements in a manner similar to any created relational table. This is one of the many remarkable features of the relational DBMS, and a powerful witness to the potency of Date’s Zero Rule (see Chapter 9).

Example 1: What columns does the table Student have?

Example 2: What relation(s) contain(s) the attribute Pgm#?

Example 3: What relation(s) contain(s) an attribute that is CHAR(7)?

Example 4: The catalog may be used to keep track of certain related tables, defined for a particular system. For instance, if the tables of interest are prefixed ‘CMP’ then the following statement will yield a list of tables of interest:

Example 5: List all logical views in the system, and all valid users:

14.5 Updating the System Catalog

Direct update (via INSERT, UPDATE, DELETE) of system catalog record, is not allowed as this would be an avenue for compromising the integrity of the database.

The system catalog is automatically updated by the system DBMS when statement such as CREATE TABLE, ALTER TABLE, DROP TABLE, CREATE VIEW, DROP VIEW, CRETE INDEX, DROP INDEX, CREATE SYNONYM, DROP SYNONYM, etc. are issued. In short, whenever a database object is created or modified, the system catalog is automatically updated by the DBMS.

The Comment statement is the only SQL statement that allows direct update of the system catalog by a user. It allows update of the Comments column in either the Tab_Comments table or the Col_Comments table. The format of the command is:

Note The column-name specified must be qualified by its table-name. The string supplied is enclosed in single quotes.

In order to understand how this statement works, you need to familiarize yourself with the structure and purpose of the catalog tables Tab_Comments and Col_Comments. The Describe statement will allow you to view the structure (left as an exercise). You will observe the following columns in respective tables:

The Tab_Comments table is used to store comments on tables of the database; the Col_Comments table is used to store comments on table columns of the database. The Comment statement allows modification of these comments.

Example 6: Store comments on tables and columns of the database:

To access these comments, you may query any of the following the views:

DBA_Tab_Comments: All table comments for the database.

USER_Tab_Comments: All table comments owned by the current user.

ALL_Tab_Comments: All table comments to which the current user has access.

DBA_Col_Comments: All column comments for the database.

USER_Col_Comments: All column comments owned by the current user.

ALL_Col_Comments: All column comments to which the current user has access.

Example 7: The following statement lists table comments for tables owned by schemas beginning with the name Bruce:

14.6 Summary and Concluding Remarks

It is now time to summarize what has been discussed in this chapter:

The system catalog is the most important database object in a database system. This is so because it facilitates and supports all or most of the other database objects. The system catalog typically contains metadata about the database.

The Oracle system catalog contains system tables for various database objects including (but not confined to) tablespaces, datafiles, tables, views, table columns, constraints, indexes, users, roles, privileges, etc. Oracle does not allow direct access of its catalog tables; rather, it provides views prefixed by DBA, ALL, USER, and GC$.

Three commonly used Oracle catalog views are User_Tables, User_Tab_Columns, and User_Indexes. In actuality, the Oracle catalog contains scores of catalog views (review section 14.3).

You can query catalog views just as you would any other table. This often provides useful information to the DBA or software engineer.

As a rule, Oracle does not allow direct update of its catalog tables. However, note that the catalog is automatically updated every time the physical or logical structure of the database is modified. The only exception to this rule is the Comment statement. This statement allows the specification of comments for database tables and columns.

If you consider what has been said about the system catalog in light of Date’s Zero Rule and Codd’s twelve rules for relational DBMS suites, you will soon realize the catalog is an absolute necessity if the DBMS is to stand up to the lofty industry expectations. To be more direct, if you are evaluating a DBMS suite and discover that it does not host a comprehensive system catalog, you are pursuing a product that is not going to stand up to much rigor; you’d be well advised to save your effort for some more meaningful project.

As powerful as useful as SQL is, the language is not without limitations. The next chapter discusses some of these limitations.

14.7 Review Questions

Discuss the importance of the system catalog in a database system.

State some commonly referenced catalog tables.

Discuss three important catalog tables, and explain how their use can help in the management of a database.

The PL/SQL Describe statement allows you to view the structure of a database table. Write an equivalent Select statement on the system catalog, that provides the same information. Use one of the tables in the sample college database as your frame of reference.

Review the as many of the catalog views listed in Figure 14-6 as possible, and practice writing Select statements on them.

Explain clearly, how the system catalog is maintained. Provide useful examples.

14.8 References and/or Recommended Readings

[Couchman, 1999] Couchman, Jason and Christopher Allen. Oracle Certified Professional: Application Developer Guide. New York, NY: Osborne/McGraw-Hill, 1999. See Chapter 3.

[Martin, 1995] Martin, James, and Joe Leben. Client/Server Databases: Enterprise Computing. Upper Saddle River, NJ: Prentice Hall, 1995. See Chapter 5.

[Oracle, 2008] Oracle Corporation. SQL Reference. http://www.oracle.com/technology/index.html (accessed August 2008).

[Ullman, 1997] Ullman, Jeffrey D., and Jennifer Widom. A First Course in Database Systems. Upper Saddle River, NJ: Prentice Hall, 1997. See Chapter 7.

CHAPTER 15

Some Limitations of SQL

As can be seen from Chapters 11 – 14, SQL is a very powerful programming language, ideally suited for the management of databases. However, like all languages, SQL has limitations. This chapter briefly examines some of these limitations. The chapter proceeds as follows:

Programming Limitations

Limitations on Views

Foreign Key Constraint Specification

Superfluous Enforcement of Referential Integrity

Limitations on Calculated Columns

If-Then Limitation

Summary and Concluding Remarks

15.1 Programming Limitations

While SQL is a very sophisticated fourth generation language (4GL) for database management, it does not have many facilities that are normally present in a traditional high-level language (HLL). These include user interface programming and traditional internal processing facilities for basic data structures. The truth is, SQL was never intended for these facilities. SQL is therefore most effective in an environment where it is embedded in HLL code. The HLL may be the host language of a DBMS, or some other language that the DBMS supports.

Quite often, in complex software development projects, the developer will encounter situations where a single SQL statement is inadequate to service the needs of the user. What is required is a series of SQL statements (and possibly non-SQL statements). The limitations on logical views (discussed in the following section) provide a case in point. Again, an HLL support is often a perfect antidote for these scenarios.

15.2 Limitations on Views

The limitations on logical views were mentioned in Chapter 13 (sections 13.1 and 13.2), without much elaboration. Let us revisit this matter here.

15.2.1 Restriction on use of the Order-By-Clause

When a view is created, the Order-By-Clause is not supported in the sub-query. This means that you cannot create a view that orders data. The rationale for this limitation is that a view is a virtual relation, and therefore ordering of the data would only increase system overheads. The obvious response to this argument is this: since a view merely stores a definition, why not include data ordering in the definition?

The work-around for this limitation is to create the view (obviously without ordering data), then when the view is being accessed via a query, employ the Order-By-Clause on the query.

Example 1: Creation of the College Bulletin from the college database of earlier discussions:

15.2.2 Restriction on Data Manipulation for Views involving UNION, INTERSECT or JOIN

As pointed out in Chapter 13, a view is not updateable if it involves a JOIN, a UNION or an INTERSECT operation. A little thought will reveal that while this is an understandable constraint, it is not always a prudent one, as there are situations that could warrant updateable views involving these operations (for more elaboration, see [Date, 2004]).

The example above is a useful illustration: it should be possible for a user to modify Pgmname and/or Crsnme for a logical row in Bulletin as follows:

If Pgmname is modified, the DBMS should use Pgm# to access the correct row in the table Program and modify its corresponding column for Pgmname.

If Crsname is modified, the DBMS should use Crs# to access the correct row in the table Course and modify its corresponding column for Crsname.

In either case, the search for a corresponding column must not be made merely on the column name (since queries can rename columns), but the name as well as characteristics of the column (which can be obtained from the system catalog).

Note: Administering changes (insertion, update or deletion of rows) to join logical views of this sort is by no means a trivial matter, hence SQL does not support it. The point to note here is that it is thinkable and indeed doable, though complex. A similar argument applies to logical views involving UNION and INTERSECT operations.

15.3 Foreign Key Constraint Specification

Some implementations of SQL (example early versions of Gupta SQL-Windows) do not provide the designer the flexibility of specifying which attribute(s) of a referenced table are to be considered when a foreign key constraint is defined.

Example 2: Referring to the college database of Chapter 7, the following is what a Gupta SQL-Windows specification for the foreign keys of the Pgm_Struct table would look like:

Note: Gupta SQL-Windows has since been upgraded to Team Developer; therefore, this example is not a commentary on the current product. The [previous SQL] syntax did not allow for the designer to specify which attribute in the referenced relation is to be used. Rather, the DBMS chose the attribute with similar characteristics to the referencing attribute (i.e. the foreign key). This idea is good for basic scenarios; however, if there are more than one attribute in the referenced relation with similar characteristics to the foreign key, confusion could arise.

Work-around: The obvious work-around for a scenario like this is to allow the designer to explicitly specify which attribute of the referenced relation is to be used in the foreign key constraint. Oracle does this quite nicely; the specification for this scenario might be as follows:

15.4 Superfluous Enforcement of Referential Integrity

Some implementations of SQL exhibit a superfluous enforcement of the referential integrity rule with respect to update of non-key attributes of tuples in a referenced relation.

Example 3: Suppose that we have a course — M100 College Algebra — that occurs in all academic programs. We wish to change the description of the course to “Elementary Algebra”, thus:

Note: Some DBMSs (e.g. Oracle) will forbid this update, since M100 would be a referenced tuple. This might be considered imprudent. What might be required is a restriction on deletion, not update of this sort.

Work-around: Oracle avoids this situation quite elegantly, by allowing the designer to temporarily disable a constraint, and enable it at a subsequent time. If your DBMS does not have this kind of facility, what you may have to do is drop the foreign key constraint from the referencing table(s), make the update, and then reintroduce the foreign key constraint.

Example 4: Here is an illustration of how you could circumvent the problem in Oracle:

15.5 Limitations on Calculated Columns

In most implementations of SQL (that this writer has worked with), you are forbidden to specify a virtual (calculated) column in a Group-By-Cause, Having-Clause, or a Where-Clause in the same Select statement (this excludes accessing virtual columns in nested queries as demonstrated in section 12.10 of Chapter 12). Neither can you define a virtual (calculated) column in terms of another calculated column in the same Select statement.

Example 5: Consider a table that is keyed on a numeric attribute, Trans_Date, that stores the date (YYYYMMDD format). One might want to create a virtual attribute, Trans_Year, that stores the year only (first four bytes of Trans_Date), and cumulate another column of the table, based on Trans_Year. In the writer’s experience, this cannot be done in a straightforward manner.

Work-around: One obvious work-around would be as follows:

Create a logical view with the virtual column;

Write a query on the view, grouping on the virtual column.

Alternate Work-around: An alternate work-around is to simply restate the calculated expression wherever it is needed within the query.

Example 6: Example 33 from Chapter 12 is repeated here to illustrate the point: We desire a list from the employee table (of the Oracle default database), showing for each department, the total salary, average salary, minimum salary, maximum salary and standard deviation; we want to show only departments with a total salary of at least $600,000:

Example 7: Referring to the default Oracle database, suppose we want to list employees beyond a certain hire year. Following is an incorrect SQL statement followed by a correct SQL statement for the problem:

15.6 If-Then Limitation

In Chapter 8 (section 8.2), it was mentioned that one form of a WFF is as follows:

It appears that this format is seldom implemented in typical SQL implementations (though there are deviations of it). The work-around is to apply the appropriate standardization rule (review section 8.4 of Chapter 8), which is shown below:

This is by no means a significant setback. One can get by without ever using an if-then construct, and simply using its equivalent.

15.7 Summary and Concluding Remarks

Here is a summary of what was covered in this chapter:

SQL is a powerful language, but not without limitations. The first is that SQL is a database language by design and intent. It does not include features for building complex user interfaces, because it was not intended to.

You are not allowed to use the Order-By-Clause on SQL views. Additionally, a view is not updateable if it includes the use of the set operator UNION, INTERSECT, or JOIN.

Some implementations of SQL do not provide the designer the flexibility of specifying which attribute(s) of a referenced table are to be considered when a foreign key constraint is defined.

Some implementations of SQL exhibit a superfluous enforcement of the referential integrity rule with respect to update of non-key attributes of rows in a referenced relation.

You are not allowed to define a calculated column in terms of another calculated column in the same query, or to reference a calculated column in the same Select statement.

It appears that the if-then construct is not widely supported in SQL.

Fortunately, there is a work-around for each limitation. This is perhaps why SQL is so popular: its benefits far outweigh its limitations. We can therefore expect that SQL will continue to be the universal database language a long time yet.

This brings us to the end of our study of SQL. The next five chapters provide an overview of five commonly used DBMS suites.

15.8 Review Questions

What are the programming limitations of SQL? As a software engineer, how do you make up for these limitations?

Discuss the limitations of SQL with respect to logical views. What are the implications of these limitations?

What problems relating to referential integrity are often present in implementations of SQL? How are these problems addressed in Oracle 10g?

Describe the SQL limitation on calculated columns. Carefully explain how it can be circumvented.

15.9 Recommended Readings

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8th ed. Menlo Park, CA: Addison-Wesley, 2004. See Chapters 4 and10.

[Oracle, 2008] Oracle Corporation. SQL Reference. http://www.oracle.com/technology/index.html (accessed August 2008).

PART D

Some Commonly Used DBMS Suites

This division allows you to peruse through a professional summary of six commonly used DBMS suites. Two of the products are regarded as comprehensive (front-end and back-end) products; two are regarded as a back-end products; one is regarded as a front-end product. While the products chosen are widely regarded as leaders in their respective categories, these chapters are not to be regarded as promotion of the products. In fact, it should be noted that the software engineering industry is rich with excellent alternatives to the products discussed in the division.

The primary purpose of the division is to expose you to six excellent DBMS alternatives. The chapters to be covered include:

Chapter 16 — Overview of Oracle

Chapter 17 — Overview of DB2

Chapter 18 — Overview of Microsoft SQL Server

Chapter 19 — Overview of MySQL

Chapter 20 — Overview of Delphi

There are several other products that time will not allow for us to explore. Included in this list are the following

Ingres

Informix

Sybase

It is hoped that the ones covered will provide you with the impetus to go on and explore more about them, as well as others not covered in the text.

CHAPTER 16

Overview of Oracle

This chapter provides an overview of the Oracle DBMS. Actually, you have been exposed to Oracle throughout division three, since the implementation of SQL that we have studied is Oracle-based. The chapter proceeds under the following subheadings.

Introduction

Main Components of the Oracle Suite

Shortcomings of Oracle

Summary and Concluding Remarks

16.1 Introduction

Oracle is widely regarded as one of the world’s leading RDBMS suites. The product was developed by a company that bears the same name as the product. The product, unlike many of its competitors has benefited from a very focused corporate mission. The Oracle DBMS was first introduced in the 1980’s. By the early 1990’s Oracle Corporation was the world’s leading software engineering company. Today it consistently shares an overall second position with IBM (behind the phenomenal Microsoft), and first in the areas of database management systems and internet applications.

The Oracle suite is a comprehensive package of software development tools for developing, as well as managing an information system with an underlying Oracle database. The product has been through several stages of revision and upgrade. The latest version is Oracle11G, which supports grid computing and no upper limit on database size. By the time this volume is published, Oracle 11G will be in production in many organizations.

In terms of connectivity, Oracle communicates with all the major alternate DBMS suites in the industry (typically via ODBC) — DB2, Sybase, Informix, Ingress, MS SQL Server, and MySQL.

The Oracle suite is marketed for major modern operating systems including, Windows, Unix, and Linux. The following are the Oracle product editions currently marketed:

Enterprise Edition

Standard Edition

Standard Edition One

Express Edition

Personal Edition

These editions all come with various components and features to service different needs. Typically, when you purchase Oracle, you specify what edition you are interested in. You will be allowed to install the appropriate edition when you access the Oracle CDs.

Some advantages of the Oracle DBMS suite are:

The product is recognized as one of the leading DBMS suites in the industry.

Oracle provides a comprehensive and at times innovative implementation of SQL.

The product supports RAD (though limited).

The Oracle DBMS handles large databases very effectively.

Oracle handles distributed databases quite well.

Oracle also handles object databases quite well.

Oracle facilitates the construction and management of small, medium sized, and large data warehouses.

Oracle facilitates communication with other databases.

The Oracle DBMS hosts a comprehensive system catalog, thus allowing it to effectively handle complex databases consisting of different types of objects.

Oracle provides a user interface that encourages Oracle experts, while facilitating novices.

Oracle provides availability and scalability with grid computing, industry-leading security, lower costs with its self-managing database.

16.2 Main Components of the Oracle Suite

The Oracle 11G product family includes the several main components. The more visible ones are:

Oracle Server

Oracle PL/SQL

Oracle Developer

Oracle Database Configuration Assistant (DBCA)

Oracle Network Services

Oracle Admin Assistant

Oracle Database Upgrade Assistant

Oracle Enterprise Manager (OEM) Database Control

Oracle Enterprise Manager Grid Control

Oracle Enterprise Manager Configuration Assistant

Oracle HTTP Server

Oracle Documentation

Oracle Transparent Gateways

Oracle Development Kit

Oracle JDBC/OCI Interfaces

Oracle iSQL *Plus and SQL *Plus

Oracle Real Application Clusters

Oracle Enterprise Integration Gateways

Oracle Application Server

Oracle Integration

Oracle Internet File System

Oracle Recovery Manager

Oracle Warehouse Builder

Oracle OLAP Client

In addition, there are several other add-ons as well as less visible but significant components in the Oracle suite. A brief discussion (overview) of a few of these components follows.

16.2.1 Oracle Server

The Oracle Server is the central database engine of the Oracle Suite. It hosts a fairly comprehensive implementation of SQL, supporting all major SQL statements, and with enhancements of its own.

The developer typically writes SQL statements in an SQL *Plus environment. This environment provides:

Additional commands which can be used along with standard SQL commands

A default editor (notepad), which may also be used to key in commands

A line editor, which may be used to key in commands

The Oracle Server supports a very comprehensive system catalog. There are system tables for all the database object types supported. These include (but are not confined to):

Databases

Tablespaces

Datafiles

(Database) Tables

Sequences

Columns of tables

Constraints (Cons-tables and Cons-columns)

Synonyms

Logical views

Indexes

Privileges (Role_privs and Table_privs)

Table_Comments

Column_Comments

These tables are typically accessed indirectly via logical views prefixed by ‘DBA’ , ‘User’ or ‘All’ (e.g. we have DBA_tables, User_tables and All_tables).

16.2.2 Oracle PL/SQL and SQL *Plus

Oracle has its own host language, PL/SQL. This high level language (very Pascal-like) is what is typically used to develop an application (although Java is supported).

PL/SQL was specifically developed for Oracle applications and is portable on Oracle Servers. It exhibits all the main features of a classical HLL, but avoids ambiguities (for instance about the If-Statement). PL/SQL is a limited HLL; it focuses solely on database application development. It has limited treatment of arrays and pointers.

Oracle provides a standard SQL Editor through SQL *Plus. SQL *Plus is a basic line editor that allows you to enter and SQL statements. You were introduced to basic SQL *Plus commands in section 11.1. Due to its limitations, many Oracle users tend to use other text editors to specify SQL instructions and integrate them into SQL *Plus. This process is seamless and is handled quite well by SQL *Plus.

16.2.3 Oracle Developer Suite

Oracle Developer is a sophisticated suite component that facilitates rapid application development (RAD). Like most RAD tools, Oracle Developer provides several facilities in a GUI environment. Following is a summary of the prominent subsystems:

Oracle Business Intelligence Beans: This subsystem provides a set of standard JavaBeans components to build business intelligence applications.

Oracle Designer: This subsystem provides a complete toolset to be employed in modeling the requirements and design of enterprise applications.

Oracle Discoverer: This subsystem is an ad-hoc query and reporting tool for publishing information drawn from data marts, data warehouses, online transaction processing systems, and other Oracle back-end systems on the Web.

Oracle Forms Developer: This subsystem is a PL/SQL-based development environment for building GUI-based applications that may be designed to be Web-accessible (see chapter 25).

Oracle JDeveloper: This subsystem is a Java-based alternative to the Forms Developer, facilitating the development of a GUI-based application that may be designed to be Web-accessible. It provides an Interactive Development Environment (IDE) that mirrors that of the Sun Microsystems equivalent product called NetBeans. With the recent acquisition of Sun Microsystems, we can expect further improvement of the related services and capabilities in this area. This is a huge acquisition for the Oracle Corporation.

Oracle Reports Developer: This subsystem provides report-building tools for designing attractive reports by pulling information from an Oracle database.

Oracle Software Configuration Manager: This subsystem facilitates management of structured and unstructured data and all types of files throughout the software development life cycle.

Oracle Warehouse Builder: This subsystem provides tools required for the construction and management of a data warehouse (see chapter 24).

16.2.4 Oracle Enterprise Manager Database Control and SQL Developer

The Oracle Enterprise Manager (OEM) Database Control is a component that provides a user friendly GUI environment for the DBA to perform administrative work on the database. Through OEM, the DBA can manage a given database, or a group of databases in a cluster. Moreover, OEM is Web-accessible; through it, you can access your database from any computer that has an internet connection via the URL address http://<Machine.Domain>:5500/em (you supply the machine name and domain name for your network).

When you access a database through OEM, all its component objects (schemas, users, tablespaces, tables, procedures, triggers, indexes, constraints, etc.) are available in a hierarchical manner. Each of these database components can be accessed, and their properties changed. OEM then generates the required SQL code or database scripts, in order to effect these changes, and executes them. OEM can also be used to create database objects (user accounts, tables, views, tablespaces, indexes, constraints, etc.). The productivity of the DBA is therefore greatly enhanced. Of course, you will need a valid user account, password, and appropriate privileges.

Oracle also provides a GUI-based component called Oracle SQL Developer (OSQLD). It provides functions similar to OEM. Students as well as practitioners will find this component very effective in providing a shield from the slightly gorier rigor of SQL syntax (but the truth is, SQL syntax is by no means a tragedy; we could live with it).

Still, Oracle provides another GUI-based component called iSQL *Plus, which facilitates Web-accessibility to the database in a manner that is similar to OEM. You can also access your database server from any machine that has an internet connection, via the URL http://<Machine.Domain>:5560/isqlplus (again, you supply the machine name and domain name for your network).

16.2.5 Oracle Enterprise Manager Grid Control

Oracle Enterprise Manager Grid Control is an HTML-based interface that provides complete monitoring across the entire Oracle technology stack — business applications, application servers, databases, and the E-Business Suite — as well as non Oracle components. The components of Grid Control include:

Oracle Management Service (OMS)

Oracle Management Agents

Oracle Management Repository

These components communicate with each other through HTTP and we can achieve secure communications between tiers within firewall-protected environments by enabling the secure socket layer (SSL) protocol. Using Grid Control, an administrator can view alerts, overall system status, performance metrics, and be alerted when failure occurs.

16.2.6 Oracle Database Configuration Assistant

Earlier versions of Oracle did not have OEM, only Oracle Database Configuration Assistant (DBCA). Unfortunately, creating, configuring and managing Oracle databases can be quite complex. DBCA helps to take some complexity out of this by automating the process. The user is shielded from gory syntactical details via a user interface, which generates the requisite SQL and database scripts, based on the user responses to friendly prompts.

DBCA and OEM are complimentary products; the former is particularly useful for database creation and initial configuration; the latter is extremely helpful during database administration and monitoring.

16.2.7 Oracle Warehouse Builder

The Oracle Warehouse Builder (OWB) is comprised of a number of components that facilitate the construction and management of a data warehouse environment. You will learn more about data warehousing in chapter 24. For now, you may consider a data warehouse as a database consisting of read-only information obtained by extracting, aggregating, and possibly reformatting data from multiple source databases. With OWB, you can construct a simple data warehouse for a set of departmental databases, as well as a complex data warehouse for an entire enterprise or group of enterprises.

16.3 Shortcomings of Oracle

Oracle, despite its apparent monopoly on the industry, has a few significant shortcomings, primarily in the area of its user interface:

SQL Environment: The standard SQL environment provided through SQL *Plus is unfortunately not graphical, but command-based. This means that the developer has to memorize SQL syntax rules in order to be productive. Out of recognition of this shortcoming, Oracle provides three complimentary alternatives to SQL *Plus — DBCA, OEM, and Oracle SQL Developer (OSQLD).

Oracle Forms Developer: In earlier versions of Oracle Forms Developer (OFD), the user interface needed improvement. Direct manipulation was not always provided in an elegant manner. Case in point: when an object was created on a form:

The user had to access the Property Palette in order to change certain attributes about the object. The problem was, changes made on the Property Palette were not always obvious to the user (e.g. color scheme selection).

The user had to constantly switch among Object Navigator, Layout Editor and Property Palette. Although the three perspectives were automatically connected, the process of constant switching was counter-productive.

Form Builder was not as easy to learn as equivalent components in other products.

At the time of writing, these problems were not thoroughly checked in OFD 11G. However, it is hoped that they have been corrected in this new version. Moreover, the introduction of Oracle JDeveloper (OJD) in Oracle 10G, as an alternative to earlier versions of OFD, has effectively circumvented these problems, since such problems do not exist in OJD.

PL/SQL Support: Like SQL support, Oracle’s support of its own PL/SQL is somewhat lacking: no GUI or interactive command prompt. The developer has to memorize PL/SQL command syntax, and there are no context sensitive prompts to help.

System Integration: As stated earlier, the Oracle suite consists of several components. Two observed problems here:

There are (in the writer’s view) too many splinter components, many of which could be merged.

These components are not all gracefully integrated (one example of this is OJD of Oracle 10G; this component requires some caution in order to be properly installed and configured).

The reason for this is that Oracle shows signs of after-thought evolution, rather than purposeful planning. This problem becomes evident when one considers for example, pre-10G versions of OEM and DBCA. These two products could have been easily merged with a third — Recovery Manager (RMAN) — into one product, providing comprehensive coverage of database administration. The problem of integration becomes very glaring when full installation of Oracle (particularly so on Unix and Linux platforms) is attempted. You will observe that in Oracle10G and 11G, the OEM has been greatly improved. This was not always the case.

Code Generation: Earlier versions of Oracle provided very negligible automatic code generation. For instance, early versions of Form Builder provided a useful environment for application development, but did not generate much PL/SQL code. The developer was still required to memorize SQL and PL/SQL syntax. It must be said, however, that components such as OEM, OSQLD, and DBCA do generate a considerable amount of code. It is hoped that this trend will continue in the future.

Database Management: Most DBMS suites have a simple, straightforward way of dealing with creating a database and populating it with database objects. Oracle does not. To achieve this objective, you have to follow the following procedure:

Create an Oracle database.

Create one or more tablespaces. Each tablespace will consist of at least one datafile.

Create user(s) and grant appropriate quotas to the tablespace(s).

Populate the tablespace(s) with database objects.

The DBA or someone with DBA privileges typically does the first three steps. Each of these steps is multifaceted, involving several subservient steps. By using DBCA and OEM, these steps have been greatly simplified, but they can still be thought intensive. The final step can be done by users with appropriate privileges.

Affordability: Traditionally, Oracle solutions have been prohibitively expensive for small and medium size companies. In recognition of this, Oracle Corporation provides a special educational program for colleges and universities in North America. This program is called the Oracle Academic Initiative (OAI). Under this program, enrolled institutions pay a nominal membership fee; this entitles them to free access to Oracle products, which they are authorized to use for education and research purposes.

Combined Effect: When we combine the effect of all the flaws mentioned, the end result is that Oracle remains a product that is relatively difficult to learn and use. On the other hand, if these problems were to be corrected, the product would truly qualify as a “killer application” [Downes, 1998]. Moreover, Oracle 9I, 10G, and 11G represent significant improvements over its earlier versions of the product.

16.4 Summary and Concluding Remarks

Let us summarize what we have covered in this chapter:

Oracle is widely regarded as one of the world’s leading RDBMS suites. The product runs on all the major operating system platforms, except OS-400, is marketed under five editions — Enterprise Edition (EE), Standard Edition (SE), Standard Edition One (SE1), Express Edition (XE), and Personal Edition (PE).

The Oracle suite includes a number of components. These include (but are not confined to) Oracle Server, Oracle PL/SQL, Oracle JDeveloper, Oracle Database Configuration Assistant, Oracle Enterprise Manager, Oracle SQL Developer, Oracle SQL*Plus, and Oracle iSQL*Plus.

Oracle has some shortcomings relating to the component integration, limited code generation, complex database creation, and affordability.

Notwithstanding the shortcomings, Oracle is an excellent DBMS suite, and for this reason, it is expected to continue to dominate the database systems arena well into the foreseeable future. Oracle’s recent acquisition of Sun Microsystems represents a significant possession for the company. This means that we can expect to see greater integration of Java-based interfaces in the Oracle products (something Oracle has been working on for some time), as well as increased platform independence.

16.5 Review Questions

What are the main editions of Oracle that are marketed?

What are the main components of an Oracle suite? Briefly discuss how these components are related.

Identify some benefits and drawbacks of the Oracle suite.

16.6 References and/or Recommended Readings

[Couchman, 1999] Couchman, Jason and Christopher Allen. Oracle Certified Professional: Application Developer Guide. New York, NY: Osborne/McGraw-Hill, 1999.

[Downes, 1998] Downes, Larry and Chunka Mui. Unleashing the Killer App – Digital Strategies For Market Dominance. Boston: Harvard Business School Press, 1998.

[Oracle, 2008] Oracle Corporation. SQL Reference. http://www.oracle.com/technology/index.html (accessed August 2008).

[Shah, 2002] Shah, Nilesh. Database Systems Using Oracle: A Simplified Guide to SQL and PL/SQL. Upper Saddle River, NJ: Prentice Hall, 2002.

CHAPTER 17

Overview of DB2

This chapter provides an overview of the DB2 DBMS. The chapter proceeds under the following captions:

Introduction

Main Components of the DB2 Suite

Shortcomings of DB2

Summary and Concluding Remarks

17.1 Introduction

DB2 is another leading RDBMS suite in the software engineering industry (Oracle being its main arch rival). Developed by IBM Corporation, the DB2 RDBMS was first introduced for MVS/370 and MVS/XA in 1983 and for MVS/ESA in 1988. In 1996 The DB2 Universal Database, the industry’s first fully scalable, Web-accessible database management system was announced by IBM. It is called “universal” because of its ability to sort and query alphanumeric data as well as text documents, images, audio, video, and other complex objects. Also, DB2 supports the two dominant database models — the relational model and the object-oriented model, hence the term universal DBMS (UDBMS). Like Oracle, the DB2 suite is quite comprehensive.

DB2 Universal Database (UDB)offers many database and information management enhancements. The most recent version of the product is DB2 UDB version 9.7. It is the database of choice for the development and deployment of critical solutions in areas such as:

E-business

Business intelligence

Content management

Enterprise Resource Planning

Customer Relationship Management

Data Warehousing

Being a leading DBMS product, DB2 provides a wide range advantages. The following are some of the more widely acclaimed advantages of the DB2 UDB suite:

The product is recognized as one of the leading DBMS suites in the industry.

DB2 provides a comprehensive and at times innovative implementation of SQL.

The DB2 suite includes a number of front-end RAD tools that all seamlessly support DB2.

The DB2 UDB handles large databases very effectively.

DB2 UDB handles distributed databases and object databases quite well.

DB2 UDB facilitates the construction and management of small, medium, and large data warehouses.

DB2 UDB facilitates communication with other databases.

The DB2 UDB hosts a comprehensive system catalog, thus allowing it to effectively handle complex databases consisting of different types of objects.

DB2 UDB provides a user interface that encourages experts, while facilitating novices.

DB2 facilitates seamless integration of various products.

The DB2 suite is marketed for major modern operating systems including i5/OS, Windows, AIX (IBM’s implementation of UNIX), Solaris, Linux, HP-UX, and Mac OS-X. The following are the DB2 UDB product editions currently marketed:

DB2 UDB Everyplace Edition

DB2 UDB Personal Edition

DB2 UDB Workgroup Server Edition

DB2 UDB Express Edition

DB2 UDB Enterprise Server Edition

These editions all come with various components and features to service different needs. IBM uses the mantra “DB2 is DB2 is DB2” to mean, all DB2 database applications are platform independent across the above-mentioned editions and operating systems [IBM, 2009a].

17.2 Main Components of the DB2 Suite

The DB2 product family includes the following main components:

DB2 Universal Database Core

IBM InfoSphere Information Server

IBM Data Studio

IBM InfoSphere Warehouse

Depending on the edition of the DB2 UDB, various other components may be included. Additionally, there are several other components comprising these main components. Some of these DB2 UDB components are summarized in Figure 17-1. This list is not comprehensive. However, it should convey the reality that the DB2 UDB suite is quite comprehensive.

Figure 17-1. DB2 Components

Following is a summary of the main components mentioned above. As the DB2 product documentation is a much more comprehensive source than this chapter, you are advised to check that source (see [IBM, 2008], [IBM, 2009a], and [IBM, 2009b]) for additional information.

17.2.1 DB2 Universal Database Core

The DB2 Universal Database Core (also called the DB2 Common Code) contains the core database services. Superimposed on top of this is a layer of code that is specific to the operating system (OS) platform that the DBS suite runs on. This approach gives the product the platform independence that it enjoys. When you purchase DB2, you must specify and choose the OS platform that will be used for your implementation, as well as the desired edition. Depending on the edition chosen, you will have other features and components bundled together in a seamless whole (for more information on this, see the production documentation via [IBM, 2009a]).

Irrespective of the edition and OS platform, all DB2 applications are accessible with each other, and from IBM software development tools such as Rational and WebSphere. Additionally, through ODBC, other non-DB2 databases can be reached.

17.2.2 IBM InfoSphere Information Server

The InfoSphere Information Server (previously called the Information Integrator in earlier versions) is a collection of technologies that combines database management systems, Web services, replication, federated systems, and data warehousing functions into a common platform. It also includes a variety of programming interfaces and data models. It is used for data integration from heterogeneous data sources (hence the name). Using the Information Integration technology, you can access diverse types of data (structured, unstructured, and semi-structured). The source data may reside on different back-end systems (example Oracle, Microsoft SQL Server, Informix, etc.). You can transform that data into a format that provides easy access to information across the enterprise, while giving the end user the illusion that all the data resides on their machine.

Information integration enables the integration of data from multiple sources with the following functions:

Provision of real-time read and write access

Transformation of data for business analysis and data interchange

Management of data placement for performance, currency, and availability

The DB2 information integration strategy includes the following goals for users:

To provide users with the ability to continue to more easily manipulate legacy data

To provide users with the ability to take advantage of familiar software to use known assets and resources

To provide users with the ability to acquire and easily maintain new data

To provide users with the ability to use existing data management tools to access data wherever it is located

Figure 17-2 provides a list of some of the prominent components comprising the InfoSphere Information Server (these were earlier clarified in Figure 17-1).

Figure 17-2. Prominent Constituent Components of the InfoSphere Information Server

17.2.3 IBM Data Studio

The Data Studio spans the entire life cycle (design, development, implementation, deployment, and management) for all DB2 applications, irrespective of editions and OS platform. Among the services provided are the following:

Accelerated solution delivery

Integrated database administration

Data growth management

Optimized performance

Data privacy

Streamlined data test management

Streamlined upgrades and migration

Under these services, a number of components have been bundled. Figure 17-3 provides a list of some of the more prominent ones.

Figure 17-3. Prominent Constituent Components of the Data Studio

17.2.4 IBM InfoSphere Warehouse

The DB2 UDB suite provides four editions of a data warehousing software. The editions of InfoSphere Warehouse provided are tailored to meet the varied needs of the organization. They are as follows:

InfoSphere Warehouse Departmental Edition

InfoSphere Warehouse Departmental Base Edition

InfoSphere Warehouse Enterprise Edition

InfoSphere Warehouse Enterprise Base Edition

With these resources, you can construct simple departmental data warehouses from departmental databases, or more complex data warehouses for an entire enterprise or group of enterprises.

17.3 Shortcomings of DB2

DB2 represents a huge effort by IBM to develop and market a top quality product. To a large extent the company has succeeded. There have not been many serious complaints about the product. Nonetheless, as the saying goes, there is no perfect software. A few complaints have been made against the product, as summarized below:

Backup and Recovery: DB2 offers a very basic set of backup and recovery capabilities, but lacks the completeness and depth of high availability functionality required by most e-businesses today. One is therefore forced to rely on cold backups at the operation system level. More improvement is needed in this area.

Support of Domains: Some users would like to see DB2 support domain-based calculus. It is felt that by adding domains to the DBMS, stronger data integrity constraints can be achieved.

Affordability: Like Oracle, DB2 is prohibitively expensive for small and medium size companies. In recognition of this, IBM provides a special educational program for colleges and universities in North America. Under this program, enrolled institutions purchase the products for a nominal fee, and are able to use them for education and research purposes. Additionally, IBM provides the public with the flexibility of choosing from five different editions according to prevailing needs and financial constraints.

Documentation: IBM provides a Web-accessible information center for the DB2 product family (see [IBM, 2008]). It must be stated that compared to other similar resources, this site could benefit from some improvements. In the experience of this author, one has to spend considerable time in finding the information sought.

Combined Effect: DB2 is on the verge of becoming a truly superb product. It is hoped that IBM can address the few problem areas for the product.

17.4 Summary and Concluding Remarks

It is now time to summarize what we have covered in this chapter:

DB2 is widely regarded as one of the world’s leading UDBMS suites. The product runs on all the major operating system platforms, and is marketed under five editions: DB2 UDB Everyplace Edition, DB2 UDB Personal Edition, DB2 UDB Workgroup Server Edition, DB2 UDB Express Edition, and DB2 UDB Enterprise Server Edition.

The DB2 suite includes a number of components. These include (but are not confined to) DB2 Universal Database Core, IBM InfoSphere Information Server, IBM Data Studio, and IBM InfoSphere Warehouse.

DB2 has some shortcomings relating to backup and recovery, and affordability.

Not withstanding the shortcomings, DB2 is an excellent DBMS suite, and is arguably the one with the most comprehensive and coherent design. For these and other reasons, it is expected that the product will continue to dominate the database systems arena well into the foreseeable future.

17.5 Review Questions

What are the main editions of DB2 that are marketed?

What are the main components of the DB2 suite? Briefly discuss how these components are related.

Identify some benefits and drawbacks of the DB2 suite.

17.6 References and/or Recommended Readings

[Downes, 1998] Downes, Larry and Chunka Mui. Unleashing the Killer App — Digital Strategies For Market Dominance. Boston: Harvard Business School Press, 1998.

[IBM, 2008] IBM Corporation. IBM DB2 Database for Linux, UNIX, and Windows Information Center. http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp (accessed August 2008).

[IBM, 2009a] IBM Corporation. Compare the Distributed DB2 9.5 Data Servers. http://www.ibm.com/developerworks/db2/library/techarticle/0301zikopoulos/0301zikopoulos1.html (accessed August 2009).

[IBM, 2009b] IBM Corporation. IBM Data Management. http://www-01.ibm.com/software/data/management/ (accessed August 2009).

CHAPTER 18

Overview of MS SQL Server

This chapter provides an overview of the Microsoft SQL Server DBMS. Relatively speaking, SQL Server represents the most recent introduction of a major DBMS suite in the marketplace. It is also somewhat different from other products. For these two reasons, we will deviate slightly from the structure of the previous two chapters, as we cover additional but relevant information about the product. The chapter therefore proceeds under the following captions:

Introduction

Main Features of SQL Server

Editions of SQL Server

Main Components of the SQL Server Suite

SQL Server Default databases

SQL Server Default Logins

Named versus Default Instances

Removing SQL Server

Shortcomings of SQL Server

Summary and Concluding Remarks

By the time this volume is available to the public, many of components mentioned will likely be outdated. The focus therefore is to describe the general product environment that an MS SQL Server user is likely to see. The discussion will be broadly based on products MS SQL Server 2005 and MS SQL Server 2008.

18.1 Introduction

MS SQL Server (version 8.0) is Microsoft’s flagship relational database engine product. The product was developed by Microsoft for its Windows operation system platform, and has been through several stages of revision.

18.1.1 Brief History

Microsoft SQL Server uses a version called Transact-SQL (T-SQL). Microsoft initially developed SQL Server (a database product that implements the SQL language) with Sybase Corporation for use on the OS/2 platform. After the IBM-Microsoft collaboration broke down, Microsoft abandoned OS/2 in favor of its then new network operating system, Windows NT Advanced Server. At that point, Microsoft decided to further develop the SQL Server engine for Windows NT by itself. The resulting product was Microsoft SQL Server 4.2, which was updated to version 4.2.1.

After Microsoft and Sybase parted ways, Sybase further developed its database engine to run on Windows NT (currently known as Sybase Adaptive Server Enterprise), and Microsoft developed SQL Server 6.0, then SQL Server 6.5, which also ran on top of Windows NT. SQL Server 7.0 introduced the capability to run on Windows NT as well as on Windows 95 and Windows 98. With SQL Server 7.0, Microsoft dramatically rewrote and modified the Sybase code. The company redesigned the core database engine and introduced a sophisticated query optimizer and an advanced database storage engine. SQL Server 2005 enhanced this new code line, adding significant new features. The most recent version of the DBMS is MS SQL Server 2008.

18.1.2 Operating Environment

As mentioned earlier, MS SQL Server has been specifically designed, developed, and tailored to operate in, and maximize the use of the features of the Windows operating system. The DBMS can run as a Windows service. As you are no doubt aware, a service is an application that Windows can start either automatically when booting up, or manually on demand. Services on Windows have a generic application programming interface (API) that can be controlled programmatically. Services facilitate the running of applications such as MS SQL Server without requiring that a user be logged in to the server computer.

Technically speaking, MS SQL Server is a back-end system. However, being developed and marketed by Microsoft, all the front-end Microsoft RAD tools are designed to integrate with SQL server. The effect is that when SQL Server is implemented in a Microsoft Windows environment (as it must be), one has a choice from several front-end tools that will seamlessly integrate with the SQL Server database.

18.1.3 MS SQL Server and the Client-Server Model

Like most modern DBMS suites, Microsoft’s SQL Server is a client-server software system. The server side of the application provides database security, fault tolerance, database performance, concurrency, and reliable backups. The client side provides the user interface that facilitates access to the database via reports, queries, and forms. Various clients can connect to MS SQL Server, including the utilities that come with SQL Server, such as SQL Server Query Analyzer. Non-Microsoft applications can also access the backend via ODBC.

18.2 Main Features of MS SQL Server

According to Microsoft, MS SQL Server provides a number of significant features, some of which are mentioned below:

Database Support: The SQL Server database engine provides facilities for supporting relational databases as well as unstructured data (via XML). It also includes security features and other related features required to create and support complex Web-accessible databases.

Replication Services: The SQL Server relational database engine supports the features required to support demanding data processing environments. The database engine protects data integrity while minimizing the overhead of managing thousands of users who may be concurrently modifying the database. SQL Server distributed queries facilitate referencing of data from multiple sources as if the data all resided in the local SQL Server database. At the same time, the distributed transaction support protects the integrity of any updates of the distributed data. Replication facilitates maintenance of multiple copies of data, while ensuring that the separate copies remain synchronized.

Ease of Installation and Usage: SQL Server includes a set of administrative and development tools that improve upon the process of installing, deploying, managing, and using SQL Server across several sites. SQL Server also supports a standards-based programming model integrated with the Microsoft Distributed interNet Applications (DNA) architecture, allowing for easy integration with the World Wide Web (WWW). These features allow software engineers to rapidly deliver SQL Server applications that customers can implement with a minimum of installation and administrative overhead.

Interoperability: SQL Server includes facilities for communicating with heterogeneous databases.

Notification Services: These services come in the form of capabilities for the development and deployment of applications that can deliver personalized, timely information updates to a variety of connected and mobile devices.

Integration Services: These services include data extraction, transformation, and loading (ETL) capabilities for data warehousing and enterprise-wide data integration.

Analysis Services: These services include online analytical processing (OLAP) capabilities for the rapid and sophisticated analysis of large and complex datasets using multidimensional storage.

Reporting Services: These services provide a comprehensive solution for creating, managing, and delivering both traditional, paper-oriented reports and interactive, Web-based reports from pre-existing databases.

Management Tools: SQL Server includes integrated management tools for administering and tuning the database. Two significant products provided are Microsoft Operations Manager (MOM) and Microsoft Systems Management Server (SMS).

Development Tools: SQL Server ships with various integrated development tools for the database engine, data extraction, transformation, and loading, data mining, OLAP, and application development through Microsoft Visual Studio.

18.3 Editions of MS SQL Server

Microsoft markets two server editions (Enterprise and Standard, both in 32 bit and 64 bit flavors) of SQL Server and several specialized editions of SQL Server. After examining their requirements and specifications, it should be obvious which one to use. However, regardless of the edition of SQL Server that one chooses, they are all built on a common code base (except for the version for Windows CE), so the same rules, conditions, and administration apply. A brief summary of each edition follows.

SQL Server Standard Edition: The Standard Edition is what most people mean when they refer to SQL Server. This version of the product supplies the essential functionality needed for e-commerce, data warehousing, and line-of-business solutions. It is intended to run on Windows platforms, starting at Windows Server 2003 (SP2) to more contemporary versions of the operating system including Windows Server 2008. Among the prominent features are the following:

Support of 16 multiple instances

Support of 4 concurrent processors

RAM of 3 GB

Compliant with 64 bit architecture

Unlimited database size

Notification services

SQL Server Enterprise Edition: Enterprise Edition (64-bit) provides the most scalable data platform. Addressing more memory than any other edition of SQL Server, it scales to the performance levels required to support the largest data warehousing and analysis applications, ecommerce websites and enterprise business systems. This edition runs on various Windows platforms ranging from Microsoft Windows 2003 Server with Service Pack (SP) 2 or later to Windows Server 2008. This edition includes the full range of features in the SQL Server suite. Among the prominent features are the following:

Support of 50 multiple instances

No limit on concurrent processors

RAM of 512 GB, or based on the underlying operating system

Compliant with 64 bit architecture

Unlimited database size

Table and index partitioning

Indexed views

Parallel index operations

Database snapshots

Integration services

Notification services

SQL Server Workgroup Edition: The Workgroup Edition is the data management solution for small organizations or workgroups within larger entities. It includes the entire core database features needed for data management in an affordable and simple-to-manage package. The platform requirements are similar to that of the Enterprise Edition. Among the prominent features are the following:

Support of 50 multiple instances

Can support 2 concurrent processors

RAM of 3 GB

Unlimited database size

SQL Server Express Edition: The Express Edition helps developers to use a lightweight version of the product to build robust and reliable applications. The product is free and can be downloaded from the Microsoft Web site. Among the prominent features are the following:

Support for 1 processor only

RAM of 1 GB

Database size of 4 GB

SQL Server Compact Edition: The Microsoft SQL Server 2008 Compact Edition is also a free lightweight version of SQL Server that facilitates a low maintenance, compact embedded database for single-user client applications for all Windows platforms including tablet PCs, pocket PCs, smart phones and desktops.

SQL Server Developer Edition: The Developer Edition is designed to enable developers to build any type of application on top of SQL Server 2008. It includes all the functionality of the Enterprise Edition but with a special development and test license agreement that prohibits production deployment.

18.4 Main Components of MS SQL Server Suite

In this section, we shall briefly examine the main components of the SQL Server suite. We shall do so under the following categorizations:

Server Components

Management Tools

Client Connectivity

Development Tools

Code Samples

Optional Components

18.4.1 Server Components

The main server components are as follows:

The SQL Server (also known as SQL Server Agent) is the core database engine and the management support service. This option is enabled by default.

The Upgrade Tools is the database upgrade support, so one can run the Version Upgrade Wizard to upgrade a SQL Server 6.5 installation. This option is enabled by default.

Replication Support should be left enabled (as it is by default) if you plan to use replication.

Full-Text Search uses technology from Microsoft Index Server to build indexes on textual data. This option is installed by default and supports full-text search in a fail-over cluster configuration.

Debug Symbols, enabled by default, provides debug files should one ever need to troubleshoot SQL Server with Microsoft Product Support. This option should be left enabled.

Performance Counters should be left enabled (as it is by default) if one wants the ability to monitor SQL Server’s performance with the performance monitor utility that Windows provides.

18.4.2 Management Tools

The management tools are enabled by default; they consist of the following:

Enterprise Manager is the graphical management interface for both development and administration of SQL Server.

The SQL Server Profiler is a great tool for monitoring SQL Server activities, including queries and how these queries are run. It also provides performance measurements.

Query Analyzer is the component used to enter and run Transact-SQL statements. One can also view how SQL Server will optimize and run queries. Experienced T-SQL developers tend to use this tool.

DTC Client Support allows you to use the Distributed Transaction Coordinator to run data modification queries across multiple systems.

Conflict Viewer is the conflict resolution wizard for merge replication.

18.4.3 Client Connectivity

Client Connectivity is a set of components that facilitates communication with the SQL Server. This set includes components for Open Database Connectivity (ODBC), Object Linking and Embedding Database (OLE DB), and DB-Library. Each library allows you to write or use programs that connect to SQL Server. Collectively, they are known as the MDAC (Microsoft Data Access Components).

18.4.4 Development Tools

The main development tools are described below:

Headers and Libraries are the C++ files needed for development of SQL Server programs.

MDAC system development kits (SDKs) are the Software Development Kits for XML and the Microsoft Data Access Components. These SDKs allow enable and support the development of programs using XML and MDAC.

Backup/Restore API includes a sample program, necessary C/C++ files, and documentation on how to build backup and restore programs.

Debugger Interface installs the components necessary to allow Microsoft Visual Studio components and the SQL Server Query Analyzer utility the capability to debug stored procedures. This option is selected by default.

18.4.5 Code Samples

None of the code samples is installed by default. However, depending on your choice, their inclusion could serve to enrich your SQL Server environment. The following are the main options:

ADO includes programming examples for ActiveX Data Objects (ADO).

DBLIB includes programming examples for the DB-Library API. DB-Library was the native Database Application Programming Interface (API) of SQL Server in earlier releases, and is supported in SQL Server for backward compatibility.

Desktop includes code samples on setting up unattended install operations for the Microsoft SQL Server Desktop Engine (MSDE).

DTS includes programming examples for data transformation services (DTS). DTS provides a way to move data from one data source to another.

ESQLC includes the programming examples for Embedded SQL for the C programming language.

MSDTC includes the programming examples for the Microsoft Distributed Transaction Coordinator.

ODBC includes the programming examples for the open database connectivity programming API in SQL Server.

ODS includes the programming examples for the open data services (ODS) API for SQL Server.

OLE Automation includes the programming examples to support OLE Automation for SQL Server.

Replication includes the programming examples for SQL Server replication.

SQLDMO includes programming examples for the SQL-Distributed Management Objects administrative programming interface.

SQLNS includes programming examples for the SQL Name Space administrative programming interface.

18.4.6 SQL Server Optional Components

After installing SQL Server, you can install three additional services: the Microsoft Search Service, the Microsoft SQL Server Analysis Services, and Microsoft English Query. Although you can the Microsoft Search Service (full-text indexing) during the default setup of SQL Server, you also have the option of installing the other two services after the initial setup is complete. This can be done independent of SQL Server if so desired.

Microsoft SQL Server Analysis Services: Microsoft SQL Server Analysis Services comprise a set of technologies to extend data warehousing into SQL Server. The Server Analysis Services help us build OLAP (On Line Analytical Processing) data to perform detailed trend analysis in many ways, as well as support data mining. The services provide the capability to build and control these cubes, and a user interface to build, administer, and query these cubes is also installed. The server side installs only on Windows 2003 (or later). The client components and user interface are also available on Windows clients.

Microsoft English Query: Microsoft English Query allows an administrator to configure a database schema and allows end users to run their database queries in English instead of Transact-SQL. This capability is particularly beneficial for Internet-based applications that don’t want to force users to run SQL statements. For example, we can say, “Show me the number of books sold for each author this year,” rather than use a complicated SQL statement.

18.5 MS SQL Server Default Databases

When you install SQL Server, the following databases are installed. You can add additional databases later, but these databases are guaranteed to be there. Some of them (master, model, tempdb, and MSDB) are system databases — they cannot be dropped without causing serious harm to SQL Server. The other two, pubs and Northwind, are simply samples to help we learn SQL Server. These can be safely dropped from production SQL Servers.

The master Database: The master database is the key database for running SQL Server. It contains a pointer to the primary data file for every other database installed on system, as well as key server wide information. This server wide information includes such items as system wide error messages, login information, system stored procedures, and connected or linked servers. The master database can be recovered only in the event of a disaster with special techniques.

The model Database: The model database is best thought of as a template database. Each time a new database is created, the model database is actually copied, and then the size and other changes requested for new database are applied. Therefore, any object that exists in the model database is copied to the new database at the time of creation. For example, you can place a table, a username, and other essential objects in this database right after installation of SQL Server. Each time a new database is created after that, these essential objects are automatically included. The model database is about 768KB after installation. Because the model is copied to create each new database, no database can be smaller than the model.

The tempdb Database: The tempdb database is the place where sorts, joins, and other activities that require temporary space are performed. It’s approximately 2MB after installation, but as is the case with all databases in SQL Server by default, it can grow as more space is required. The tempdb database is reinitialized each time SQL Server (the SQL Server service) is started.

The MSDB Database: The MSDB database supports the SQL Server Agent service, including storing information about jobs, alerts, events, and replication. A history of all backup and restore activity is also kept in this database. The MSDB database is about 12MB by default.

The pubs Database: The pubs database is meant to be a learning tool. It contains a sample database about a publisher, including information about authors, books, and sales. Most of the examples in the SQL Server Books Online are based on the pubs database. Most database features are highlighted via their implementation in the pubs database. The pubs database is just under 2MB in size after installation.

The Northwind Database: The Northwind database is an alternative learning database to the pubs database. Northwind has been the sample database supplied with Microsoft Access for some time now. Because more and more Microsoft Access users are migrating to SQL Server, the Northwind database was brought over to assist them in learning the features of the product with a familiar database. Northwind is about 3.5MB by default.

18.6 MS SQL Server Default Logins

One of the first things you should do after installing SQL Server is to log in. If you went with the default and selected Windows Authentication Mode, you simply need to select the Windows NT Authentication option on any dialog that asks you to connect to SQL Server. If you choose to use Mixed Mode, you could either do that or use the default login for SQL Server, sa (lowercase on case-sensitive sort-order servers). The letter sa stands for system administrator.

sa: The sa user account is a member of the sysadmin fixed server role. As a member of this role, sa can do anything in SQL Server. The sa account always exists and cannot be dropped. However, you cannot use it when we are in Windows Authentication Mode. If you are in Mixed Mode, you can select to login using this account. Still, it is more likely you will use the next option, logging in via membership in local administrators group.

Windows Local Administrators Group: If you are on a Windows computer and are a member of the local administrators group, you do not have to use the SQL Server authentication. During setup, SQL Server adds the local Windows Administrators group to the sysadmin role, just as sa is added. As a result, all local administrators are made SQL Server administrators by default. On earlier versions of Windows platforms (prior to 2003), Windows authentication is not available, so you must use sa. Password control is not necessary in SQL Server when we are using Windows authentication; SQL Server simply uses Windows login credentials.

18.7 Named versus Default Instances

SQL Server has introduced the capability to run multiple, independent copies of the database server on a single Windows-based computer. Microsoft supports up to 16 copies of SQL Server installed on a single computer.

Default Instance: One can connect to the default instance of SQL Server (there is only one per machine) by specifying only the server name when connecting via an application program. In earlier versions, connection to the SQL Server was done this way — the SQL Server instance was specified as the machine name on which SQL Server was running. For instance, if the computer is named BDServ, then to connect to the default instance, you would specify DBServ when prompted for the SQL Server name. The program files install to the default path of \MSSQL. The service names for the actual SQL Server services are as follows:

SQL Server: MSSQLServer

SQL Server Agent: SQLServerAgent

Named Instance: A named instance of SQL Server is one that is named during setup. When you want to connect to a named instance, you specify both the server name and the instance name that was entered during setup. The services are created with unique names as well:

SQL Server: MSSQL$Instancename

SQL Server Agent: SQLAgent$Instancename

Common Components: Some common components are shared between installations of SQL Server; they do not really belong to either the default or any of the named instances that might have been installed on the computer. They include the SQL Server tools, as well as system components such as MDAC 2.6. For more information on these, please refer to the MS SQL Server product documentation.

18.8 Removing MS SQL Server

If, for some reason, you need to remove SQL Server, simply use Control Panel ’s Add/Remove Programs — just like any other application on your computer. Selecting this option removes all files and Registry keys related to SQL Server, but does not remove shared components that were installed, such as the MDAC components. Also, if this is not the last instance of SQL Server to be removed, the tools will remain. When you remove the last instance of SQL Server, the tools are also removed.

We need to address one important issue regarding upgrade from to SQL Server from an earlier version of the software. If for instance, you desire to upgrade from SQL Server 6.5, do not run the 6.5 setup program to remove the previous release of SQL Server; this can damage your SQL Server installation. Microsoft provides a special uninstall program for SQL Server 6.5; it is accessible via a shortcut in the Microsoft SQL Server Switch menu; it is called Uninstall SQL Server 6.x.

18.9 Shortcomings of MS SQL Server

SQL Server represents a huge effort by Microsoft to develop and market a top quality product. Whether the company will succeed in this venture is a bit too early to tell; we will have to wait and see. There are however, a few areas of concern as summarized below:

Support Only for Microsoft Windows: Almost all Windows operating systems provide support for SQL Server. If you have anything other than Microsoft Windows as your operating system platform, however, you will have to forego using this product, and instead look to another RDBMS vendor. It is obvious why Microsoft has taken this course of action, but it would be nice to have the ability to install the product on other operating systems. Platform independence or some semblance of it would certainly add credence to MS SQL Server, as it competes in the DBMS industry.

No Native Load Balancing Capabilities: SQL Server provides any built-in support for load balancing.While it is possible to achieve a "load-balanced" solution with horizontal data portioning, this is not true load balancing. The DBMS provides no logic that analyses the load on one server and then passes the request to another based on the results of the analysis. This does not mean that load balancing cannot be achieved with SQL Server; it can be achieved via a third party solution such as Cisco’s Content Services Switch.

No Version Control on Database Objects: SQL Server provides no versioning support of stored procedures, views, or even any Data Definition Language (DDL). You have to script your database and objects, and then use a product such as Visual SourceSafe (VSS) to version the scripts. This is not good, particularly if you have developers who come along and change the structure slightly, without updating the external source control repository.

Poor Performance and Configuration Out of the Box: Most DBAs know that to improve performance of your RDBMS, you should separate your program, data, and transaction log files onto physically separate disk drives. With SQL Server, you can split the data and program files from each other at installation time, but the transaction log files are installed by default with the data partition. This results in poor performance.

Cannot Read Transaction Log Files: There is no capability with SQL Server’s own tools to read the transaction logs. This means that you need to either buy a third-party tool, or take a wild guess and restore the database to a point in time. Transaction logs are used only during recovery of database, if the database is corrupt. The ability to read the transaction logs is valuable for debugging purposes. This therefore represents a serious omission.

No Support for Java: Microsoft’s non-support stance on Java is well known. Unfortunately, this has meant that your Java code is not portable from other DBMS platforms. Code that has been written for your DB2 or Oracle installation will need to be translated into T-SQL to be used within SQL Server. This creates a challenge when migrating from other RDBMS platforms. In fact, SQL Server doesn’t offer any support for any programming language, except of course SQL. To use VBScript in SQL Server, you need to develop DTS packages that interface with the RDBMS. These related limitations constitute a serious handicap to the product.

Combined Effect: MS SQL Server has made a reasonably impressive entrance to the DBMS market place. The product is scalable to some degree; its robustness and stability is yet to be tested; however, there are a few success stories that Microsoft will hasten to point out.Given the fact that for better or for worse, the Windows operating system has become a household name, and that MS SQL Server seamlessly integrates with that operating system (both products produced and marketed by Microsoft), MS SQL Server is guaranteed a promising future.

18.10 Summary and Concluding Remarks

Here is a summary of what we have covered in this chapter:

MS SQL Sever is an emerging DBMS suite that is seeking to increase market share in the database systems market. The product runs on Windows platforms. The product is marketed under six editions: Standard Edition, Enterprise Edition, Workgroup Edition, Express Edition, Compact Edition, and Developer Edition.

The MS SQL Sever suite includes a number of components. These components have been classified under Server Components, Database Management Tools, Client Connectivity Tools, Development Tools, Code Samples, and Optional Components.

MS SQL Sever is shipped with the following default databases: master, model, tempdb, MSDB, pubs, and Northwind.

MS SQL Sever has a number of shortcomings relating to platform dependence (running only on Windows platform), no load-balancing, poor performance, difficult reading transaction logs, security, lack of support for Java.

MS SQL Sever has made a fair entry into a very competitive market. If not for any other reason, we expect it to be around for the foreseeable future, due to its marriage to the Windows platform.

18.11 Review Questions

What are the main editions of MS SQL Server that are marketed?

What are the main components of the MS SQL Server suite? Briefly discuss how these components are related.

Describe the default databases and logins that are provided by MS SQL Server.

Identify some benefits and drawbacks of the MS SQL Server suite.

18.12 References and/or Recommended Readings

[Beauchemin, 2005] Beauchemin, Bob, Niels Berglund, and Dan Sullivan. A First Look at Microsoft SQL Server 2005 for Developers. Microsoft Press, 2005.

[DeBetta, 2005] DeBetta, Peter. Introducing Microsoft SQL Server (TM) 2005 for Developers (Pro – Developer). Microsoft Press, 2005.

[Microsoft, 2008] Microsoft. SQL Server 2005. http://www.microsoft.com/sql/2005/default.mspx (accessed October 2008).

[Microsoft, 2008] Microsoft, SQL Server 2008. http://msdn.microsoft.com/en-us/library/bb418432(SQL.10) aspx (accessed October 2008).

CHAPTER 19

Overview of MySQL

This chapter provides an overview of MySQL. The chapter proceeds under the following captions:

Introduction to MySQL

Main Features of MySQL

Main Components of MySQL

Shortcomings of MySQL

Summary and Concluding Remarks

19.1 Introduction to MySQL

MySQL has become the most popular open source DBMS and the fastest growing DBMS in the industry. The product is an attractive alternative to higher-cost, more complex DBMS suites. Its award-winning speed, scalability and reliability, combined with the fact that it is free, are some of the reasons for the product’s increasing popularity. The current production release series is MySQL 5.0, which was declared stable for production use in October 2005.

MySQL is currently marketed in three editions:

MySQL Standard Edition

MySQL Enterprise Edition

MySQL Cluster Edition

Figure 19-1 provides the salient differences between these editions, in terms of their characteristic services and features. The editions and features are clarified on the MySQL website (see [MySQL, 2010]).

Figure 19-1. Prominent Services and Features of the MySQL Editions

The original creator of MySQL is a Swedish company called MySQL AB. The company had been in operation for over ten years (1995 – 2008), before being acquired by Sun Microsystems, which has been recently acquired by Oracle. In the early stages, MySQL was used primarily for internal purposes. Over the past five years, the product’s ascendancy to international acclaim has been quite noticeable. MySQL runs on multiple platforms, including Unix, Linux, Windows, Solaris, and MacOS. The maximum tablespace size supported is 64 TB. Database tables can be of any size up to this limit.

MySQL brings a number of advantages to the database arena. These are summarized in Figure 19-2.

Figure 19-2. Main Benefits of MySQL

19.2 Main Features of MySQL

In order to fully MySQL, it is important to take note of the underlying features of the product. These are summarized in Figure 19-3. These features were adopted from the official MySQL Web site [MySQL AB, 2008]; minor modifications have been added to improve readability. From the list of features provided, it is clear that the intent of MySQL AB is to develop and market a comprehensive DBMS.

Figure 19-3. Main Underlying Features of MySQL

19.3 Main Components of MySQL

Compared to other DBMS suites on the market, MySQL is relatively simple, and therefore does not include a sophisticated (or convoluted) list of components. Rather, there is a list of important programs that make up the MySQL suite. The most important programs are summarized in Figure 19-4. The information provided here was adopted from the MySQL AB Web site (with minor changes to improve readability and clarity). For a more detailed discussion, the reader is referred to that site. Each program provides a help option that allows you to access useful documentation on the program in question.

Figure 19-4. Important MySQL Component Programs

19.4 Shortcomings of MySQL

Like all software products, MySQL does have some limitations in its current production release (version 5.1). What is most impressive about the MySQL venture is that the developers readily admit the limitations and document them. Following is a summary of these limitations:

19.4.1 Limitation on Joins and Views

Limitation on Joins: The maximum number of tables that can be referenced in a single join is 61. This also applies to the number of tables that can be referenced in the definition of a view. This is a reasonable threshold that should not cause anyone to lose sleep.

Limitations on Views: View processing is not optimized in MySQL. Three restrictions have been documented by MySQL AB:

You are not allowed to create indexes on views. This is significant, so MySQL AB would do well to remove it in the near future.

Indexes can be used for views processed using MySQL’s merge algorithm. However, a view that is processed with the temptable algorithm is unable to take advantage of indexes on its underlying tables (although indexes can be used during generation of the temporary tables). This is also a significant restriction that should be lifted in the future.

Subqueries cannot be used in the From-Clause of a view. MySQL AB promises to lift this limitation in the future.

19.4.2 Limitations on Sub-queries

MySQL AB lists a number of restrictions on sub-queries, and promises to address them in the near future. Some of the more prominent ones are mentioned below (for a full list, go to the MySQL AB site [MySQL AB, 2008]):

If you compare a null value to a sub-query using operators ALL, ANY, or SOME, and the subquery returns an empty result, the comparison might evaluate to the non-standard result of NULL rather than to TRUE or FALSE.

A subquery’s outer statement can be any one of the following: SELECT, INSERT, UPDATE, DELETE, SET, or DO.

Sub-query optimization for the IN (<sub-query>) construct is not as effective as for the equal (=) operator or for IN (<value-list>) constructs. A typical case for poor performance of the IN (<sub-query>) construct is when the sub-query returns a small number of rows but the outer query returns a large number of rows to be compared to the sub-query result. The problem is that, for a statement that uses an IN (<sub-query>) construct, the optimizer rewrites it as a correlated sub-query.

Example 1: The following example illustrates:

If the inner and outer queries return M and N rows, respectively, the execution time becomes on the order of f(M×N), rather than f(M+N) as it would be for an uncorrelated subquery. An implication is that query construct with IN (<sub-query>) can be much slower than a query written using an IN(<value-list>) construct that lists the same values that the sub-query would return. Obviously, this is a very undesirable situation.

The optimizer is more mature for joins than for subqueries, so in many cases a statement that uses a subquery can be executed more efficiently if you rewrite it as a join. An exception to this occurs for the case where an IN (<sub-query>) construct can be rewritten as a SELECT DISTINCT join construct.

Example 2: The following example illustrates:

Note however, that in this case, the join requires an extra DISTINCT operation and is not more efficient than the subquery. If you are not looking for distinct values in the result set, then the Select statement with the join is preferred to the Select statement with a subquery.

19.4.3 Limitations on server-side Cursors

Server-side cursors are implemented in the C programming language via the mysql_stmt_attr_set() function. The same implementation is used for cursors in stored routines. A server-side cursor allows a result set to be generated on the server side only; the entire result set is not necessarily transferred to the client, but only for the rows requested by the client. For example, if a client executes a query but is only interested in a few rows, the remaining rows are not transferred. In MySQL, a server-side cursor is materialized into a temporary table. Initially, this is a memory table, but is converted to disk table if its size reaches the value of the max_heap_table_size system variable. One limitation of the implementation is that for a large result set, retrieving its rows through a cursor might be slow.

19.4.4 Other Limitations

There are other restrictions relating to stored procedures, functions, triggers and XA transactions. (The acronym XA means extended architecture. The term XA transactions is often used to describe transactions involving different systems or databases. For instance, a client application program acting as a transaction manager in one system, may request information from a database server acting as a resource manager in another system.) These will not be discussed here; the interested reader is referred to the MySQL AB Web site [MySQL AB, 2008].

19.5 Summary and Concluding Remarks

Let us summarize what has been discussed in this chapter:

MySQL is the most popular open source DBMS. It runs on Windows and Unix platforms, and is marketed under three editions: Standard Edition, Enterprise Edition, and Cluster Edition.

The MySQL suite consists of a number of important component programs, including but not confined to: mysqld, mysqld_safe, mysql.server, mysqlmanager, mysql_multi, etc.

The official MySQL Web site reports a number of shortcomings that are being addressed. These include but are not confined to limitations on joins, limitations on views, limitations on sub-queries, limitations on server-side cursors, etc.

MySQL has made an impressive entry into the software engineering industry, and for this reason, it is expected that the product will be around for the foreseeable future.

19.6 Review Questions

Briefly account for the history of MySQL.

Outline the characteristic features of MySQL.

What are the main components of the MySQL suite? Briefly discuss the functional responsibilities of each component.

Discuss the benefits and shortcomings of MySQL.

19.7 References and/or Recommended Readings

[Downes, 1998] Downes, Larry and Chunka Mui. Unleashing the Killer App – Digital Strategies For Market Dominance. Boston, MA: Harvard Business School Press, 1998.

[Dubois, 2005] Dubois, Paul. MySQL 3rd ed. Indianapolis, IN: Sams, 2005.

[Kofler, 2005] Kofler, Michael. The Definitive Guide to MySQL 5 3rd ed. Berkely, CA: Apress, 2005.

[MySQL AB, 2008] MySQL AB. Documentation: MySQL. http://dev.mysql.com/doc/ (accessed October 2008).

[MySQL, 2010] MySQL. MySQL Editions: http://www.mysql.com/products/ (accessed August 2010).

CHAPTER 20

Overview of Delphi

This chapter looks at the second front-end system that we will consider — Delphi. The chapter provides an overview of the product under the following subheadings:

Introduction

Main Components of the Delphi Suite

Shortcomings of Delphi

Summary and Concluding Remarks

20.1 Introduction

Delphi is an object-oriented RAD tool, which allows the software engineer to develop applications for Windows platforms. Delphi also provides a simple cross-platform solution when used in conjunction with Kylix, the equivalent RAD tool for the Linux platform. Kylix understands most of the code developed under a Delphi environment, making cross-platform software development more accessible to developers.

Delphi was developed by Borland in the early 1990s. The product evolved from Borland Turbo Pascal after the company decided to make an object-oriented Pascal version of the popular product and simplify greatly the software development process. Since 2003, the market has seen several releases including Delphi V7, Delphi V8, Delphi 2005, Delphi 2007, and Delphi 2009. The rest of the chapter will be based on a review of Delphi 2005 and Delphi 2007. However, since the chapter deliberately avoids implementation details, you will find that the information provided also applies to earlier versions all the way back to Delphi V6, as well as more recent versions. Since 2008, the Borland Software Development Division (responsible for the Delphi product line) has been acquired by software engineering firm called Embarcadero Technologies. The contemporary version of the product is called Delphi Delphi XE2, and is marketed by Embarcadero.

Delphi is best described as a front-end RAD tool that interfaces with various databases through at least one of the following strategies:

Database drivers for Paradox, Interbase, MS SQL Server, Oracle, DB2, Informix, MySQL, Sybase, Blackfish SQL, and SQL Anywhere

Borland Database Engine (BDE) for local database drivers

Borland Data Provider (BDP) for .NET (this includes Interbase 7.5, MS SQL Server, Oracle, DB2, Sybase, and MS Access)

Delphi allows you to wrap an object-oriented GUI around a relational database. Figure 20-1 summarizes some of the different types of projects that Delphi can be used for, and Figure 20-2 summarizes the main strengths of the product.

Figure 20-1. Some Common Delphi Project Alternatives

Figure 20-2. Areas of Strength of Delphi

20.2 Major Components of the Delphi Suite

Delphi has the following major components:

Database Development Environment

Integrated Development Environment (IDE) and Visual Component Library (VCL)

Borland Database Engine (BDE)

Component Library for Cross-Reference (CLX)

Enterprise Core Object (ECO) Subsystem

Documentation Subsystem

20.2.1 The Database Development Environment

The available tools for manipulating databases are as follows:

Database Desktop: A simple interface for creating and managing local databases.

Database Explorer: An intuitive interface to the different databases, allowing the user to view all the different settings, as well as performing SQL queries and data manipulation.

Borland Database Engine (BDE): A subsystem that allows programmatic access and modification of databases and the data contained in them. It is the engine between the application and the data, and facilitates access to various databases. It also provides access to non-native databases using ODBC, and other vendor products using their respective drivers.

Through the database development environment, the user can carry out the following functions:

Create a database alias

Create tables

Modify table structures

Create and modify indexes

Create and modify logical views

Create and modify queries, which include graphic interpretation of data

Create and modify reports which include graphic interpretation of data

Additionally, the developer has the option of writing original SQL statements and storing these in files for later use. In summary, the main database object types facilitated are database aliases, tables, indexes, views and query files.

20.2.2 Interactive Development Environment

The Interactive Development Environment (IDE) is the Delphi component that is used for application development for Delphi, Delphi for .NET, or C++. It provides a number of useful features:

A GUI with a number of predefined application building blocks, located on over 30 tool palettes. More commonly used tool palettes include Standard, Additional, Data Access, Data Controls, dbExpress, BDE, ActiveX, COM+, and ADO (ActiveX Data Objects).

Context sensitive prompts to guide the application development process.

Interactive command prompts so the developer does not have to memorize the syntax for using certain built-in procedures.

Automatic code generation: A fairly sophisticated application can be developed with negligible coding from the developer. Code is generated in Object Pascal or C++.

Code accessibility: For more complex/sophisticated applications, the developer can easily access the generated code and modify it.

Each Delphi project may include several types of files. The more common file types are: project files (extensions BDSPROJ and DPR), a Pascal unit file (extension PAS) with Object Pascal code for each operation with a corresponding form, and a form file (extension DFM) for each operation with a corresponding Pascal unit. By using the tool palettes and applying Object Pascal (or C++) code, other components can be easily added to the project. Some commonly used application building blocks are mentioned in Figure 20-3.

Figure 20-3. Commonly Used Building Blocks in Delphi

Theses tools provide software engineers and application developers with much flexibility; the tools can be used to build a wide range of software applications including:

Traditional database applications

Client-server applications

E-Business applications

Web applications and services using W3C compliant technologies such as SOAP, XML, WSDL

Web applications frameworks compatible with Apache, IIS and Netscape Web servers

Text editors and other GUI-based applications

Delphi generates all code in Object Pascal. The code can be easily accessed and modified by the application developer. In practice, this is normally done, in order to create an acceptable user interface.

20.2.3 Database Engine

The Borland Database Engine (BDE) was the original component that allowed programmatic access and modification of databases and data. It was the engine between the application and the physical database. In application mode, the developer would typically superimpose BDE building blocks (via the BDE Tool Palette) onto forms to facilitate database access. The BDE also provided access to non-native databases using ODBC and other vendor products using their drivers. In more contemporary versions of Delphi, the DBE has been replaced by three alternatives that provide identical functionality: one is called Absolute Database; the other is called Accuracer Database system; the third component is called dbExpress.

20.2.4 Component Library for Cross Reference

Through its Component Library for Cross-Referencing (CLX), Delphi allows developing applications that can later be recompiled and used in the Linux platform (using Kylix). These libraries include:

Traditional database applications

Client-server applications

E-Business applications

Web applications

20.2.5 Enterprise Core Object Subsystem

The Enterprise Core Object (ECO) subsystem facilitates code visualization and the development and maintenance of class diagrams for an application. This component is fairly new and therefore still needs refinement.

20.2.6 Documentation

Delphi is marketed with a comprehensive help system, which covers all aspects of the software. The help system is in hypermedia format. Additionally, context sensitive help can be obtained by pressing the F1 key from anywhere during application or database development. Borland also maintains on-line documentation of the product.

20.3 Shortcomings of Delphi

Figure 20-4 provides a summary of some observed flaws about Delphi. The flaws are mainly superficial, relating to the product’s user interface.

Figure 20-4. Some Observed Flaws of Delphi

Delphi provides a RAD tool that once learned, can cut down development greatly, it simplifies Windows application development, and it gives the freedom to use assembler instructions, or object-oriented constructs and components that promise modularity and reduced maintenance in the long run. With its good performance at compile and run time, scalability and good database support, Delphi competes with the best software developments tools in the market.

20.4 Summary and Concluding Remarks

It is now time to summarize what has been discussed in this chapter:

Delphi is best described as a front-end RAD tool that connects to any back-end database via ODBC; but it also provides limited back-end database services. The product runs on Windows, Linux and Unix platforms.

The Delphi suite includes the following components: Database Development Environment, Integrated Development Environment (IDE), Visual Component Library (VCL), Borland database Engine (BDE), Component Library for Cross-Reference (CLX), and the Delphi Documentation.

A few shortcomings have been observed about Delphi; they relate to support for dBase and Paradox databases, the documentation, and product registration.

Delphi is an excellent product that has been tested and refined over time. It is anticipated that this product will be around for the foreseeable future.

20.5 Review Questions

Identify some benefits of running a Delphi database (front-end) environment.

Identify and briefly discuss the main components of the Delphi suite.

Discuss some of the observed shortcomings of Delphi.

20.6 References and/or Recommended Readings

[Embarcadero, 2008] Embarcadero Technologies. Delphi. http://www.embarcadero.com/products/delphi/ (accessed October 2008).

[Cantu, 2005] Cantu, Marco. Mastering Delphi 2005. Sybex Publishing, 2005.

[Intersimone, 2007a] Intersimone, David. Antique Software: Turbo Pascal 1.0 http://bdn.borland.com/article/0,1410,20693,00.html (accessed October 2008).

[Intersimone, 2007b] Intersimone, David. Antique Software: Turbo Pascal 5.5 http://bdn.borland.com/article/0,1410,20803,00.html (accessed October 2008).

PART E

Advanced Topics

This division of the text covers some advanced topics in database system that you should be familiar with.

The objectives of this division are

to introduce you to the salient issues related to database administration;

to introduce you to the theory, advantages, and challenges of distributed databases;

to provide an overview of object databases, and point out the challenges faced by the approach;

to provide an overview of data warehousing;

to provide an overview of Web-accessible databases, and the supporting technologies.

The division includes five chapters, each providing an overview of an area of database systems that could be further explored.

The chapters to be covered include the following:

Chapter 21 — Database Management

Chapter 22 — Distributed Database Systems

Chapter 23 — Object Databases

Chapter 24 — Data Warehousing

Chapter 25 — Web-Accessible Databases

Please note that for each of these topics, several texts have been written. It will therefore not be possible to cover them in detail. Rather, in each case, an overview is provided, outlining the salient issues.

CHAPTER 21

Database Administration

We have established the importance of a database as a valuable information resource in the organization. This resource must be carefully administered (managed) in order to ensure the continued operation and success of the organization. In this regard, the database administrator is extremely important (database administrators are among the highest paid IT professionals).

This chapter provides an overview of database administration. It discusses (from an administrative view point) the following issues:

Database Installation, Creation and Configuration

Database Security

Database Management

Database Backup and Recovery

Database Tuning

Database Removal

Summary and Concluding Remarks

Please note that a solitary chapter which overviews database administration will not make you a good database administrator (DBA). To achieve that objective, you will need to apply the knowledge in this course, combined with additional knowledge gained from a special course in database administration. This chapter helps to prepare you for that vocation by providing an overview of database administration issues.

21.1 Database Installation, Creation, and Configuration

Before any work can be done, the database software must be installed. This is usually a straightforward, but time consuming process. For large, sophisticated products such as Oracle and DB2, installation could get complicated, since decisions have to be made about what components to install, where (in terms of directories or folders) to store certain resources, and what environmental or configuration settings to choose. For simpler products like MySQL and Delphi, the installation process is correspondingly simpler.

Next, the database must be created and configured. Depending on the DBMS suite that is in use, database creation and configuration may be very simple or quite complicated. In some systems (for example Delphi), database creation is a simple act of creating a directory (in the Linux environment) or folder (in the Windows environment), and then creating database aliases that point to that directory or folder. In others such as MySQL, there is a highly simplified Create-Database statement that allows you to create a database within seconds. On the other end of the spectrum, there are systems such as Oracle, for which you must first complete a course before you know how to properly create a database (review Oracle’s version of the Create-Database statement in Chapter 11). And there are those products in between the two extremes.

Once you have created the database, there are issues that must be determined as part of the database configuration. These issues include the following:

Location of the database and its related files

Communication issues for server and client (for client-server databases) in a multi-user environment

Physical structure of the database

Logical structure of the database

Users of the database and their access rights

Like database creation, the complexity of these issues, and the flexibility with which they can be addressed, will depend to a large extent on the products involved, as well as the complexity of the database itself.

21.2 Database Security

Database security is a very important aspect of database administration. Ideally, the security mechanism must be multi-tiered, controlling access to the system, the system resources, and the system data. The DBA must ensure the following:

Access to the system is controlled.

Authorized users must be able to access (insert, modify, retrieve or delete) data that they are authorized to access.

Authorized users must be restricted to the data and resources that they are duly authorized to access and nothing more.

Unauthorized users must have absolutely no access.

In order to achieve this, the DBA must be fully conversant with SQL facilities (commands) for managing database users and objects. The information covered in Chapter 13 is particularly relevant here. Some DBMS suites are marketed with a GUI, which is superimposed on the basic SQL interface, and provides a more user-friendly environment for working, by generating SQL code (which can be subsequently accessed and modified) to GUI-based instructions. Examples of this in the Oracle suite include Oracle Enterprise Manager (OEM), iSQL Plus, and Oracle SQL Developer (OSQLD).

To further reinforce the security mechanism of the database, more sophisticated products provide the facility to encrypt data stored in database files. In other products, the conventional wisdom is to rely on the encryption feature provided by the underlying operating system.

21.3 Database Management

As mentioned in Chapter 11, once the database has been created, it must be populated with database objects. Database objects include tablespaces (specific to Oracle), tables, indexes, views, synonyms, procedures, triggers, packages, sequences, users, roles, etc. Most of these database objects are dynamic and will grow or shrink in size, with the passing of time. The database and its objects must therefore be managed.

Database management is a complex matter that covers a wide range of activities; among these are the following:

Reorganizing existing database tables and indexes

Deleting unnecessary indexes or moving other objects

Making alterations to the database itself

Making alterations to database components (tablespaces, datafiles, tables, procedures, etc.)

Creating additional database objects (tablespaces, datafiles, tables, users, indexes, procedures, etc.)

Training users

Backup and recovery of database objects

Database performance tuning

Most of these issues have been discussed in previous chapters. The last two issues — backup and recovery, and database performance tuning — deserve some attention. They will be addressed in the next two sections.

21.4 Database Backup and Recovery

In general, the term database backup and recovery refers to the various strategies and procedures involved in protecting a database against data loss, and reconstructing the data should that loss occur. The reconstructing of data is achieved through media recovery, which refers to the various operations involved in restoring, rolling forward, and rolling back a backup of database files. Like database installation, creation and configuration, backup and recovery can be quite simple or very complex, depending on the database environment, and the desired objectives. For the remainder of this section, we shall consider, as case study, backup and recovery in the Oracle environment (it does not get any more complicated than this).

21.4.1 Oracle Backups: Basic Concept

A backup is a copy of data. This backup may include important parts of the database such as the control file, datafile(s), or tablespace(s); alternately, it may involve the entire database. A backup is a safeguard against unexpected data loss and application errors. If you lose the original data, then you can reconstruct it by using a backup.

Backups are divided into physical backups and logical backups. Physical backups, which are the primary concern in a backup and recovery strategy, are copies of physical database files. You can make physical backups with either the Oracle Recovery Manager (RMAN) utility or operating system utilities. In contrast, logical backups contain logical data (for example, tables and stored procedures) extracted with the Oracle Export utility and stored in a binary file. You can use logical backups to supplement physical backups.

21.4.2 Oracle Recovery: Basic Concept

Recovery is the opposite of backup. A database recovery is effected from a database backup, so if the backup was not done, the recovery is not an option. Like backup, recovery may involve a component or section of the database (from a control file, datafile(s) or tablespace(s)), or it may involve an entire database.

To restore a physical backup of a datafile or control file is to reconstruct it and make it available to the Oracle database server. To recover a restored datafile is to update it by applying archived redo logs, and online redo logs, that is, records of changes made to the database after the backup was taken. If you use RMAN, then you can also recover restored datafiles with incremental backups, which are backups of a datafile that contain only blocks that changed after a previous incremental backup.

After the necessary files are restored, media recovery must be initiated by the user. Media recovery can use both archived redo logs and online redo logs to recover the datafiles. If you use SQL*Plus, then you can run the RECOVER command to perform recovery. If you use RMAN, then you run the RMAN RECOVER command to perform recovery.

21.4.3 Types of Failures

Several circumstances can halt the operation of an Oracle database. The most common types of failure are described in Figure 21-1. Oracle provides for complete recovery from all possible types of hardware failures, including disk crashes. Options are provided so that a database can be completely recovered or partially recovered to a specific point in time.

Figure 21-1. Types of Database Failures

If some datafiles are damaged during a disk failure, but most of the database is intact and operational, the database can remain open while the required tablespaces are individually recovered. Therefore, undamaged portions of a database are available for normal use while damaged portions are being recovered. This is a very desirable feature, especially for large corporate databases that must be up and running “twenty four by seven.”

21.4.4 Database Backups

Because one or more files can be physically damaged as the result of a disk failure, media recovery requires the restoration of the damaged files from the most recent operating system backup of a database. There are several ways to back up the files of a database.

Whole Database Backups

A whole database backup is an operating system backup of all datafiles, online redo log files, and the control file of an Oracle database. A whole database backup is performed when the database is closed and unavailable for use.

Partial Backups

A partial backup is an operating system backup of part of a database. The backup of an individual tablespace’s datafiles and the backup of a control file are examples of partial backups. Partial backups are useful only when the database’s redo log is operated in ARCHIVELOG mode.

A variety of partial backups can be taken to accommodate any backup strategy. For example, you can back up datafiles and control files when the database is open or closed, or when a specific tablespace is online or offline. Because the redo log is operated in ARCHIVELOG mode, additional backups of the redo log are not necessary. The archived redo log is a backup of filled online redo log files.

21.4.5 Basic Recovery Steps

Because of the way the Oracle Database Writer (DBWn) writes database buffers to datafiles, at any given time, a datafile might contain some tentative modifications by uncommitted transactions and might not contain some modifications by committed transactions. Therefore, two potential situations can result after a failure:

Data blocks containing committed modifications were not written to the datafiles, so the changes appear only in the redo log. Therefore, the redo log contains committed data that must be applied to the datafiles.

Because the redo log can contain data that was not committed, uncommitted transaction changes applied by the redo log during recovery must be erased from the datafiles.

To solve this situation, two separate steps are used by Oracle during recovery from an instance or media failure: rolling forward and rolling back.

Rolling Forward

The first step of recovery is to roll forward, which is to reapply to the datafiles all of the changes recorded in the redo log. Rolling forward proceeds through as many redo log files as necessary to bring the datafiles forward to the required time.

If all necessary redo information is online, Oracle rolls forward automatically when the database starts. After roll forward, the datafiles contain all committed changes as well as any uncommitted changes that were recorded in the redo log.

Rolling Back

The roll forward is only half of recovery. After the roll forward, any changes that were not committed must be undone. After the redo log files have been applied, then the undo records are used to identify and undo transactions that were never committed yet were recorded in the redo log. This process is called rolling back. Oracle completes this step automatically.

21.4.6 Oracle’s Backup and Recovery Solutions

There are two methods for performing Oracle backup and recovery: Recovery Manager (RMAN) and user-managed backup and recovery. RMAN is a utility automatically installed with the database that can back up any Oracle8 or later database. RMAN uses server sessions on the database to perform the work of backup and recovery. RMAN has its own syntax and is accessible either through a command-line interface or though the Oracle Enterprise Manager GUI. RMAN comes with an API that allows it to function with a third-party media manager.

One of the principal advantages of RMAN is that it obtains and stores metadata about its operations in the control file of the production database. You can also set up an independent recovery catalog, which is a schema that contains metadata imported from the control file, in a separate recovery catalog database. RMAN performs the necessary record keeping of backups, archived logs, and so forth using the metadata, so restore and recovery is greatly simplified.

An alternative method of performing recovery is to use operating system commands for backups and SQL*Plus for recovery. This method, also called user-managed backup and recovery, is fully supported by Oracle Corporation, although use of RMAN is highly recommended because it is more robust and greatly simplifies administration.

Whether RMAN is used, or user-managed methods, physical backups can be supplemented with logical backups of schema objects made using the Oracle Export utility. The utility writes data from an Oracle database to binary operating system files. You can later use Oracle Import to restore this data into a database.

21.5 Database Tuning

To maintain acceptable database performance, it is necessary to carry out periodic performance tuning. As the database is being used, it will exhibit a natural tendency to degraded performance. The current database system may not be performing acceptably, based on user-defined criteria, due to any of the following:

Poor database design

Database growth

Changing application requirements (possibly including a redefinition of what acceptable performance is)

Note however, that there might be occasions when database tuning efforts are not fully effective. When components that are external to the database, yet vital to the entire client-server application performance, fail to perform acceptably, database tuning might not help without the corresponding tuning of these other application infrastructure pieces. The main components external to the backend database are the backend operating system, the network, and the client operating system. The major examples are the following:

Very weak clients (PCs)

Network saturation

Very weak, saturated, or poorly tuned operating system

21.5.1 Tuning Goals

There are different ways of determining the goals of a performance tuning effort. A DBA should consider them all. Database systems can be sampled on various quantitative measures; the most important of these are the following:

Throughput: This is the accomplished work per unit time, as measured by transactions per second (tps); higher is better.

Response time: This is the time it takes for an application to respond, as measured in milliseconds or seconds, lower is better.

Wait time: This is the elapsed time a program takes to run; lower is better.

In any system, throughput and response time usually run counter to one another as tuning goals. If response time is high (bad), throughput might be high (good). If throughput is low (bad), response time might be low (good).

Common sense helps when sorting out these two conflicting measures. The more users that are concurrently using a system within a certain amount of time, the more likely it is that each user will experience longer delays than normal, but the number of transactions going through the system will be greater. On the other hand, if you decrease the number of concurrent users accessing the system within a certain time window, each user will enjoy faster response time at the expense of fewer overall transactions completing in that duration.

Typically, on-line transaction processing (OLTP) systems (also called operational databases) want low response time or high throughput, in terms of transactions per second, depending on the application needs. A decision support system (DSS) also wants low response time. However, a DSS might also want high throughput in terms of data blocks read or written per unit time. This type of throughput is not necessarily counterproductive to high concurrency and low response times. A batch (production) system typically wants lower wait times. For example, everyone likes for the payroll application to complete on time!

It is important to always consider the following two central tuning goals:

Maximize your return on investment (ROI). Invest your time and effort wisely by working on the problems most likely to yield the optimum improvement.

Minimize contention. Bottlenecks are characterized by delays and waits; eliminate or reduce these whenever possible.

Finally, the following general-purpose database tuning goals should be considered:

Minimize the number of data blocks that need to be accessed; review and rewrite database access code as necessary.

Use caching, buffering, and queuing whenever possible to compensate for the electro-mechanical disadvantage (memory is faster than disk).

Minimize the data transfer rates (the time it takes to read or write data); fast disks, RAID, and parallel operations help do this.

Schedule concurrent programs that complement instead of compete with each other.

21.5.2 Tuning Methodology

It is best to approach tuning with a structured methodology. After ensuring that the operating system is performing at its peak and that sufficient operating system resources have been allocated to your database system, you should tune following in this order:

Database design

Database application

Memory management

I/O management

Database contention

Database Design Tuning: In database design tuning, you are concerned with the physical and logical structure of the database. During this exercise, the DBA examines and takes decisions about refining the logical and physical structure of the database. Among the issues that may be addressed are the following:

Determining whether critical database components (tablespaces, datafiles, log-files, etc) need to be redefined and/or relocated (to different directories/folders).

Determining whether critical database objects (primarily tablespaces and tables) need to be partitioned (i.e. fragmented into different partitions).

Determining whether critical database tables need to be restructured.

Determining whether additional database objects (tables, logical views, etc.) are required, and if so, where they should be placed.

Database Application Tuning: In database application tuning, the concern is on end-user access to the database. During this exercise, the DBA determines ways to better facilitate access to the database by the various applications that need to use it. Among the issues that may be addressed are the following:

Ascertaining whether existing database application objects (procedures, triggers, etc.) are performing according to expectations.

Determining whether additional database application objects (procedures, triggers, etc.) are required and where to place them.

Determining whether adequate database access points (including ODBC connections, database service connections, etc.) are in place and are working acceptably.

Memory and I/O Management Tuning: Memory management tuning in closely related to database design tuning. This is critical because poor database design could lead to poor memory performance which in turn leads to poor database performance. Among the issues that may be addressed are the following:

Storage allocations for database objects (primarily tablespaces, datafiles, and tables).

Storage allocations for the database itself (these parameters are set at database creation or database alteration).

Oracle provides a number of utilities for managing memory performance of database tables. However, a full discussion of these is beyond the scope of this course. Suffice it to say that the database fault rate on each table can be monitored. If the fault rate is high, the table needs to be reorganized.

Database Contention: Database contention relates to how the database is handling multi-user access as well as concurrent access. Like memory management, there are specific utilities for managing this issue; these utilities are typically provided by the DBMS suite.

21.6 Database Removal

Sometimes, it becomes necessary to remove a database. Often, there is no specific command for this, for obvious reasons: The database exists under the auspices of the host operating system. Database removal is therefore an operating system command.

Depending on the DBMS being used, database removal may be a trivial matter, or one requiring a few steps. For instance, removal of a Delphi or MySQL database involves a single step. On the other hand, removing an Oracle database involves several steps of deleting related folders/directories managed by the DBMS in collaboration with the underlying operating system. These folders/directories were created when the database was created (or altered).

Once a database has been deleted, it is completely gone, and can only be reintroduced via a recovery operation.

21.7 Summary and Concluding Remarks

Here is a summary of what has been discussed in this chapter:

Depending on the DBMS being used, database creation may be complex or simple. Delphi and Oracle are at the two extreme ends of the spectrum — database creation is very simple in Delphi, and very complex in Oracle.

Database security must ideally be multi-tiered. It must address access to the system, access to the system resources, and access to data.

Database management must continue after database creation. It must address issues relating to the performance of the database system in the face of growing data collection and changing user needs. Database tuning is and integral part of this.

Backup must be carefully planned and methodically implemented, in order to minimize or eliminate data loss due to system failures. The recovery procedures must also be reviewed as required.

Like database creation, depending on the DBMS used, database removal may be trivial or complex.

21.8 Review Questions

What are the main issues to be considered when creating a database?

What are the critical issues to be addressed when configuring the security mechanism of a database?

What are the main issues to be addressed during the management of a database?

Why are backups important? Discuss how backup and recovery are handled in Oracle.

Why is performance tuning of a database important? Identify basic tuning guidelines to be observed.

21.9 References and/or Recommended Readings

[Hoffer, 2007] Hoffer, Jeffrey A., Mary B. Prescott and Fred R. McFadden. Modern Database Management 8th ed. Upper Saddle River, NJ: Prentice Hall, 2007. See Chapter 12.

[Mullins, 2002] Mullins, Craig S. Database Administration: The Complete Guide to Practices and Procedures. Reading, MA: Addison-Wesley, 2002.

[Oracle, 2008] Oracle Corporation. Documentation: Oracle10G Database Release 2 Documentation. http://technet.Oracle.com (accessed October 2008).

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, MA: Course Technology, 2007. See Chapters 11 and 15.

CHAPTER 22

Distributed Database Systems

As wonderful as database systems are, they would not be delivering on their true potential, if they could not be networked in a distributed environment. This chapter discusses distributed database systems under the following subheadings:

Introduction

Advantages of Distributed Database Systems

Twelve Rules for Distributed Database Systems

Challenges to Distributed Database Systems

Database Gateways

The Future of Distributed Database Systems

Summary and Concluding Remarks

22.1 Introduction

The concept of a distributed system was introduced in Chapter 2. A distributive database system consists of a collection of sites, connected via a communication network in which:

Each site is a database system in its own right.

The sites work together (if necessary) so that a user at any given site can access data at any other site as if the data resides of the host (user’s) site.

Section 2.7 of Chapter 2 mentioned some connectivity possibilities. Here are a few noteworthy points to remember:

From the definition, the user is given the notion of virtual database systems consisting of data that may reside anywhere in the network.

The sites may be distributed over a wide geographical area, or in a local area/building. A distributed database system can therefore be a LAN (local area network), a MAN (metropolitan area network), or a WAN (wide area network).

Distributed database systems are not to be confused with remote access systems, sometimes called distributive processing systems. The latter has been around for some time. In such systems, the user accesses data at remote sites but the operation is not seamless; the user is aware and the consequences may be obvious. In a distributed database system, access across sites is seamless.

The literature [on electronic communications and computer networks] documents several alternate approaches to setting up a distributed database. Three prevalent ones are as follows:

Using client-server technology to set up a federated database

Setting up a virtual private network (VPN)

Setting up a data warehouse

There is no shortage of information on client-server technology and VPNs. Exploration of this is beyond the scope of this course; however, references [Martin, 1995] and [Ozsu, 1999] should provide you with a useful start. Suffice it to say that Oracle as described earlier (Chapters 10-16) qualifies as a distributed DBMS. When you install Oracle Server on a node in a network, that node acts as a database server. If you then install Oracle Client on other nodes in the network, your database server can be accessed from anywhere in the network (as well as from other network systems with Web accessibility) in a seamless manner. Also, Chapters 24 and 25 provide overview of data warehousing and Web-accessible databases respectively.

22.2 Advantages of Distributed Database Systems

Figure 22-1 provides some benefits that distributed database systems provide. The benefits may be summarized in three categories:

Efficiency and Productivity

Convenience

Reliability

Figure 22-1. Benefits of Distributed Database Systems

When weighed against the challenges posed by distributed database (discussed later), they outweigh them by a huge margin; we therefore expect the continued proliferation of distributed databases.

22.3 Twelve Rules for Distributed Database Systems

In his classic text, Introduction to Database Systems, Christopher Date discusses twelve rules (objectives) for distributed database systems [Date, 2004]. Let us take a brief look at these rules.

Rule 1: Local Autonomy

The sites should be autonomous to the maximum possible extent. All operations at a site are governed by that site alone. This is not always entirely possible, but is an objective to strive for.

Rule 2: No Reliance on Central Site

This is a consequence of objective 1. The sites must be treated as equals. There is no reliance on a central site. Reliance on a central site would make the system vulnerable to the central site (bottleneck could occur or the central site could go down). This is undesirable.

Rule 3: Continuous Operation

The system must be able to run continuously. There should be no need for a planned shut down in order to carry out any function (for instance backup or tuning).

Rule 4: Location Independence (Transparency)

Users should not need to know where data is physically stored in order to access it; the system should operate as if all the data resided at the local site. The distributed nature of the database must be transparent to the end end users.

Rule 5: Fragmentation Independence

The system should support data fragmentation — it should be possible to partition a given relation into fragments that are stored at different sites. Thus, data can be stored where it is most frequently used. Network trafficking is therefore reduced. Fragmentation should be transparent to the end users.

Example 1: Suppose that a large organization has employee records in an Employee relation. The departments are at different localities. Records for each department are stored at different sites in those respective departments. This can be easily facilitated in Oracle or DB2 by partitioning the Employee table. A full discussion of table partitioning is beyond the scope of this course. However, suffice it to say that that this is the technique used by several leading DBMS suites to facilitate fragmentation independence.

Rule 6: Replication Independence

A given relation (or fragment of a relation) can be replicated at different sites. Replication may improve access time and hence performance. The drawback however, is that at update, all copies have to be updated simultaneously. Replication should be transparent to the end users.

Rule 7: Distributed Query Processing

Distributed query processing must be facilitated among different sites. Records are transmitted set (relation) at a time instead of record at a time.

Example 2: Suppose we have an international company, IBM, say, where employees are stored in the relation Employee, fragmented in various countries, where there are IBM offices. IBM Canada, issues the request: “Find all Jamaican male employees.” Then:

Suppose there are n records that satisfy this request. If the system is relational, the query will involve tw o messages one from Canada to Jamaica and one from Jamaica to Canada. If the system is not relational, but record-at-a-time, the query will involve 2n messages — n from Canada to Jamaica and n from Jamaica to Canada.

Query optimization of the request occurs before execution (record-at-a -time requests cannot be optimized).

Due to these two points, distributive database systems are usually relational.

Rule 8: Distributed Transaction Management

Each transaction must be atomic — fully committed or fully rolled back. This objective must be met irrespective of the agents (constituent processes) of the transaction. Concurrency control must be ensured (usually by data locking).

Rule 9: Hardware Independence

It should be possible to integrate the system across different hardware platforms. It should therefore be possible to run the DBMS on different hardware systems.

Rule 10: Operating System Independence

It should be possible to integrate the system across different operating system platforms. It should therefore be possible to run the DBMS on different operating systems.

Rule 11: Network Independence

The system should be able to support different sites with different hardware and different operating systems and networking protocols.

Rule 12: DBMS Independence

The DBMS suites used may be different. For instance DB2 and Oracle both support SQL and open database connectivity (ODBC); it should therefore be possible to link databases running on the two DBMS suites. The same argument should apply for other DBMS suites.

22.4 Challenges to Distributed Database Systems

Distributed databases did not come easily; neither were they easy to maintain. Fortunately, the software engineering industry has figured out how to address these challenges. However, improving the algorithms used, and finding new ones are always topical research issues. There are five well known challenges to distributed database systems. These are:

Query Optimization

Catalog Management

Update Propagation

Concurrency Control

Transaction Management

Query Optimization

Query optimizing processing must be distributed in order to minimize network trafficking. To illustrate, consider a query Qa of site A, accessing to relations in a natural join: Rb of site B and Rc of site C. The optimizer must decide on one of the following strategies:

Move copies of Rb and Rc to site A

Move copy of Rb to site C and process the join there

Move copy of Rc to site B and process the join there

The optimizer must be able to calculate what would be most economical alternative (given the structure and configuration of the underlying network) and choose that alternative. For example, Oracle implements two query optimization strategies — a rule-based optimization and a cost-based optimization. Before executing a query, the query optimizer optimizes the query by converting it to an internal format (based on an Oracle algorithm) that will ensure the most efficient execution.

Catalog Management

Catalog management is one of the most complex issues that a distributed database must resolve. This is so since additional information must be stored for the database objects (e.g. fragmentation, replication, location etc.). Where and how the catalog should be stored is a complicated issue. Below are some alternatives:

Centralized: The catalog is stored at a centralized location, and is accessible to the other participating sites.

Fully Replicated: The catalog is replicated at each participating site.

Partitioned: Each site maintains its own catalog. The total catalog is the union of each site catalog.

Hybrid: Each site maintains its own catalog; additionally, a central site maintains the global catalog.

Each of these approaches has its related advantages and challenges. Resolving this issue is often done with the use of simulation software, and much research into the matter.

Update Propagation

In the case where data is replicated at different sites, it may not be possible to effect update to all replicas at the desired time. How is this resolved? The primary copy approach is a common method of resolution:

One replica is deemed the primary copy. As soon as that copy is updated, the update process is deemed completed.

The site with the primary copy is responsible for updating the other sites as son as possible.

This somewhat contradicts the objectives of transaction independence and redundancy control. As was emphasized in chapter 4, once data replication is introduced in a database, with it come various other data integrity problems. Resolution is therefore a matter of tradeoff.

Concurrency

Concurrency is another issue that must be resolved. To illustrate, consider what might happen if user X tries to retrieve a particular data set for update purposes, but that data set is being updated by another user Y. Consider that in a distributed system, there might be thousands of users, so that this kind of contention could easily develop among several users. Typically, the DBMS handles this problem by record locking: a record or data set that is retrieved for update is locked to that transaction until the update is completed; it is then released (unlocked) for other users. Requests to release (unlock) objects in a distributed database system must be managed. This is a serious overhead.

In application development where distributed databases are accessed, or there is multi-user access of a single database, the application programmer must check for record lock on a data set before attempting to lock that data set. The application programmer must program a graceful recovery from a record lock situation (normally done by issuing an appropriate message to the end user and allowing them to defer that particular request until some subsequent time).

Transaction Management

Issues such as when to lock records, and when to commit or rollback transactions are critical in a distributed database. The application developer must be familiar with the facilities provided by the DBMS and SQL (COMMIT and ROLLBACK) for managing transactions.

22.5 Database Gateways

Traditionally, a database gateway is a software component that links two different DBMS suites. It could run on either of the two systems running the dissimilar DBMS suites, or on a separate machine for that purpose. The simplest configuration is to run the software on either system as a driver for the other DBMS. Another alternative is to use the software called Open Database Connectivity (ODBC). ODBC is marketed with the Windows operating system, and is readily available for other operating systems.

Suppose for instance, that Oracle and DB2 both support SQL (as in fact, they do). It should therefore be possible to link the two DBMS suites, as illustrated in Figure 22-2. In reality, each product includes an ODBC driver that in effect acts as the gateway. Following are some functions of the gateway:

Mapping between the two different protocols (formats)

Mapping between the two different dialects of SQL (e.g. Oracle’s dialect versus DB2’s dialect)

Mapping between the two different system catalogs so that users of one DBMS (Oracle) can access all files from the other DBMS (DB2)

Mapping between the two different sets of data used

Resolution of semantic mismatches across the two systems (e.g. attribute Employee.Empno in a DB2 table may map to Empl.Emp# in an Oracle table)

Resolution of transaction management issues such as data locking, updates, commit and rollback

Resolution of security and accessibility issues across the two different systems

Figure 22-2. Illustrating the Function of a Database Gateway

By way of example, Oracle operates a very sophisticated gateway system that allows connectivity to other non-Oracle databases either directly through its gateway, or indirectly through ODBC (see [Oracle, 2008]). Among the many features of the Oracle gateway system are the following:

Location independence and transparency

Data-type translations between Oracle and non-Oracle systems

Data dictionary between Oracle and non-Oracle systems

Read/write access

Support for large objects (LOBs)

Support for non-Oracle stored procedures

Transmission of pass-through SQL between Oracle and non-Oracle system

Data encryption services

22.6 The Future of Distributed Database Systems

Since the mid-1990’s, two significant technologies have significantly influenced the development and direction of distributed databases. These are:

Object Technology (OT)

Electronic Communication Systems (ECS)

22.6.1 Object Technology

In a few short years, OT has come to dominate the contemporary software engineering industry. New products are forced by industry demands to support OT in some form. Much work has been done by the Object Management Group (OMG) in establishing the Common Object Request Broker Architecture (CORBA) standards for distributed database systems. The Microsoft equivalents of CORBA are Component Object Model (COM), Distributed COM (DCOM), and more recently, the .NET framework.

CORBA standards span a wide range of specifications from user interface to object-object communication. They are supported by some of the leading software engineering firms in the industry. With the emergence of Java, the software industry has made significant progress in the area of platform independent software components than ever before. As mentioned in chapter 6, Java (through JDBC) supports both CORBA and ODBC.

22.6.2 Electronic Communication Systems

Complimentary to the advances in OT, the past decade has seen much achievement in the arena of electronic communication systems (ECS). Contemporary operating systems are more sophisticated, supporting a wider range of communication protocols. The protocols and their underlying technology have been refined to provide much higher transmission rates. Also, communication protocols provide much more services than previously.

With a refinement of, and emphasis on standards, interoperability is now a much more attainable goal than in the previous decade. These advances, when combined with those in OT, will contribute to the proliferation of heterogeneous information models.

22.7 Summary and Concluding Remarks

Let us summarize what we have covered in this chapter:

A distributed database systems is a conglomeration of database systems in which each system operates as an autonomous system on its own, or collaborates with other systems as required.

Distributed databases systems provide a number of benefits in the areas of efficiency and productivity, convenience, and reliability.

Distributed database systems should strive to conform to the standards outlined in Date’s twelve rules for such systems.

Distributed databases face challenges in the areas of query optimization, catalog management, update propagation, concurrency control and transaction management.

A database gateway is a software component that links two different DBMS suites.

Distributed databases have been significantly affected by developments in object technology and electronic communications technology. This is expected to continue in the foreseeable future.

Distributed databases have helped to transform our world in a significant way. To fully appreciate the power of distributed databases, just consider for a moment, a world without them: no World Wide Web; minimized reliability on critical company databases; reduced capabilities on operating systems; limited remote access to databases. The next chapter takes a closer look at object databases.

22.8 Review Questions

Define a distributed database system. Discuss the advantages of such systems.

Outline and clarify the twelve rules for distributed database systems.

Discuss the challenges to distributed database system.

As an IT professional (perhaps in training), what are your future expectations for distributed database systems?

22.9 References and/or Recommended Readings

[Chung, 1997] Chung, P. Emerald, Yennun Huang, Shalini Yajnik, Deron Liang, Joanne C. Shih, Chung-Yih Wang, and Yi-Min Wang. DCOM and CORBA Side by Side, Step by Step and Layer by Layer. http://research.microsoft.com/~ymwang/papers/HTML/DCOMnCORBA/S.html (accessed October 2008).

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8th ed. Menlo Park, CA: Addison-Wesley, 2004. See Chapter 21.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapter 25.

[Kifer, 2005] Kifer, Michael, Arthur Bernstein, and Philip M. Lewis. Database Systems: An Application-Oriented Approach 2nd ed. New York, NY: Addison-Wesley, 2005. See Chapter 16.

[Martin, 1995] Martin, James, and Joe Leben. Client/Server Databases: Enterprise Computing. Upper Saddle River, NJ: Prentice Hall, 1995. See Chapters 7-9.

[Oracle, 2008] Oracle Corporation. Oracle Database Gateways. http://www.oracle.com/technology/products/gateways/index.html (accessed October 2008).

[Ozsu, 1999] Ozsu, M. Tamer, and Patrick Velduriez. Principles of Distributed Database Systems. 2nd ed. Eaglewood, NJ: Prentice Hall, 1999.

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, MA: Course Technology, 2007. See Chapter 12.

[Silberschatz, 2006] Silberschatz, Abraham, Henry Korth, and S. Sudarshan. Database System Concepts 5th ed. Boston, MA: McGraw-Hill, 2006. See Chapter 22.

CHAPTER 23

Object Databases

The previous decade has witnessed the advancement of several so-called object database management systems (ODBMS) and universal database management systems (UDBMS). This Chapter discusses such systems under the following subheadings:

Introduction

Overview of Object Oriented Database Management Systems

Challenges to Object Oriented Database Management Systems

Hybrid Approaches

Summary and Concluding Remarks

23.1 Introduction

Object Technology (OT) has been dominating the software engineering industry in recent times. For better or worse, there has been a heightened interest and indulgence in object-oriented methodologies (OOM). Full treatment of OT and OOM is better handled in another course on the subject. However, for completeness, a brief introduction is made here.

OT provides obvious advantages to application programming, with benefits of encapsulation, polymorphism and complexity (information) hiding, code reusability, etc (Figure 23-1 summarizes the commonly mentioned advantages). By contrast, an OO approach to database design may or may not bring significant benefits, depending on the situation. Data structure encapsulation may or may not be desirable; besides, the principle of encapsulation often contradicts the principle of data independence in database design.

Figure 23-1. Benefits of Object Technology

It is widely believed that an OO approach to database design and implementation is preferable for complex applications such as:

CAD/CAM systems

CASE tools

Geographic information systems (GIS)

Document Storage and retrieval systems

Artificial intelligence (AI) systems and expert systems

Which approach is the better? This is a difficult question to answer. It depends on the situation at hand, but the following is a summary of the alternatives:

Truly function oriented systems with relational database and procedural application development

Truly OO systems involving (encapsulation of) both data structure and operation

Hybrid approach A with relational database and OO user interface

Hybrid approach B with object/relational database and OO user interface

The first approach is a well-known traditional approach, and will not be discussed further. The latter three approaches are more aligned to current practices in the software engineering industry. They will be discussed in the upcoming sections.

23.2 Overview of Object-Oriented Database Management Systems

In a truly object-oriented DBMS (OO DBMS), the following concepts hold (for more details, see the references):

An object type is a concept or thing about which data is to be stored, and a set of operations is to be defined.

An object is an instance of an object type.

An operation is a set of actions to be performed on an object.

A method specifies the way an operation is to be performed.

Encapsulation is the packaging of data structure and operations, typically into a class. The internal structure of the class is hidden from the outside, and only members (member functions) of the class have access to it.

A subclass may inherit properties (structure or operations from a subclass. Also a class may be comprised of several component classes).

Polymorphism is the phenomenon where a given object or operation may take on a different form, depending on the context of usage.

Objects communicate by sending messages to each other. These messages are managed via events; an object therefore responds to events.

23.3 Challenges for Object-Oriented Database Management Systems

A number of challenges stand in the way of achieving purely object-oriented DBMSs, some of which have been articulated in [Date, 2004]. Figure 23-2 provides a brief summary of these challenges.

Figure 23-2. Challenges to Object-Oriented Database Management Systems

23.4 Hybrid Approaches

Due to the above-mentioned challenges, it is unlikely that we will see a proliferation of purely object- oriented DBMS suites involving only encapsulated database objects in the immediate future. The benefits of OT are therefore likely to continue to be more significant in the area of user interface than database (of course with a few exceptions as mentioned earlier). The relational model on the other hand, has long proven its worth. The best we can therefore expect is a peaceful coexistence of both OO systems and relational databases — a kind of hybrid. The next two subsections describe two hybrid approaches.

23.4.1 Hybrid Approach A

In the Hybrid Approach A, a relational database is accessed by an OO user interface. This means that the application development is done via an OO Programming Language (OOPL), CASE tool, or RAD tool.

There is no shortage of OOPL (some of them pure OOPLs, others are hybrid OOPLs). The more popular ones include C++, Java, C#, and Object Pascal. There are many object-oriented CASE (OO-CASE) tools and RAD tools that support this approach. They include (but are not confined to) products such as Team Developer, Delphi, Oracle JDeveloper, WebSphere, NetBeans etc.

Three advantages of this approach are as follows:

It can facilitate legacy systems (software systems based on old technology), which are prevalent in large organizations.

It facilitates peaceful coexistence of traditional and more contemporary system approaches.

It reaps the benefits of a relational database and OO application development.

The main disadvantage of the approach is that it does not address the earlier mentioned situations that warrant OO database design. However, the skillful database designer can use techniques mentioned in Chapter 3 for implementing relationships such as subtype, component and aggregation.

23.4.2 Hybrid Approach B

In this approach, a relational/object database (often referred to as a universal database) is accessed via an OO user interface. Like the Hybrid Approach A, application development is done via an OOPL, OO-CASE tool, or RAD tool.

The universal database supports both the relational database as well as the OO database. The designer can therefore make critical decisions as to which approach is preferred, given the scenario. The skillful database designer also has the flexibility of employing techniques mentioned in Chapter 3 for implementing relationships such as sub-type, component and aggregation.

Several of the leading software engineering companies have in recent times, introduced products that support object databases. Examples include:

IBM’s DB2 Universal

Oracle Universal Database Server

Informix’s Universal Data Option for its Dynamic Server

Advantages of the approach are as follows:

The strengths of the relational model and the OO model can be emphasized, and the respective weak points avoided.

Legacy systems can be facilitated.

It facilitates peaceful coexistence of traditional and more contemporary system approaches.

One possible disadvantage is that the database designer could be sometimes forced to make difficult decisions on database design.

23.5 Summary and Concluding Remarks

It is time to summarize what we have discussed in this Chapter:

OT provides several huge benefits to the software engineering arena.

An OO approach to database design may or may not bring significant benefits, depending on the situation. In some situations the OO approach is ideal; in others it is not.

There are a number of challenges to object databases that do not exist in a relational database.

The most pragmatic approach for merging the benefits of the relational model and the OO model is to superimpose an OO user interface on a relational or universal database.

The strength of the relational model lies in its firm mathematical foundations, its huge benefits, and the immense financial outlay that have been placed in relational systems. This third fact is significant: Several large corporations have invested millions of dollars hugely in relational databases. They are not likely to discard these investments. The strength of the OO model lies in its huge benefits in certain situations, and its intuitiveness. Given the strength of both models, we can expect them to continue to peacefully coexist in the foreseeable future, complementing instead of rivaling each other.

23.6 Review Questions

Identify the main benefits that object technology brings to the arena of database systems.

Describe the main features of the OO DBMS model.

Discuss the main challenges to the object-oriented database management systems.

Discuss the hybrid approach to database systems.

23.7 References and/or Recommended Readings

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8th ed. Menlo Park, CA: Addison-Wesley, 2004. See Chapters 25–26.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapters 20–22.

[Kifer, 2005] Kifer, Michael, Arthur Bernstein and Philip M. Lewis. Database Systems: An Application-Oriented Approach 2nd ed. New York, NY: Addison-Wesley, 2005. See Chapter 14.

[Lee, 2002] Lee, Richard C. and William M. Tepfenhart. Practical Object-Oriented Development With UML and Java. Upper Saddle River, NJ: Prentice Hall, 2002.

[Martin, 1993] Martin, James and James Odell. Principles of Object Oriented Analysis and Design. Englewood Cliffs, NJ: Prentice Hall, 1993.

[Rumbaugh, 1991] Rumbaugh, James, et. al. Object Oriented Modeling And Design. Eaglewood Cliffs, NJ: Pretence Hall, 1991.

[Silberschatz, 2006] Silberschatz, Abraham, Henry Korth, and S. Sudarshan. Database System Concepts 5th ed. Boston, MA: McGraw-Hill, 2006. See Chapter 9.

CHAPTER 24

Data Warehousing

Since the mid1990s, database technology has expanded into a new area of interest — the development and management of data warehouses. It is a fascinating field of study that deserves some attention. This chapter provides an overview of data warehousing and information extraction. The chapter proceeds under the following subheadings:

Introduction

Rationale For Data Warehousing

Characteristics of a Data Warehouse

Data Warehouse Architectures

Extraction, Transformation and Loading

Summary and Concluding Remarks

24.1 Introduction

The concept of a data warehouse springs from the combination of two sets of needs:

The business requirement for a global view of information, independent of and despite its source or underlying structure

The need of information systems (IS) professionals to manage large volumes of company data in a more effective manner

The data warehouse has been approached many times and from many directions in the last decade; many implementations exist today. In order to proceed, we must make a distinction between data and information:

Data is the computerized representation of business information.

Information is the assimilation of data to convey meaning as understood and used by end users.

A data warehouse is an integrated, subject-oriented, time-variant, nonvolatile, consistent database, constructed from multiple sources, and made available (in the form of read-only access) to support decision making in a business context. As you will soon see (in section 24.3), the highlighted terms in this definition are deliberate because of their significance.

Here is an alternate definition: A data warehouse is a relational database that is typically constructed from multiple transactional databases (called source databases), and designed for query and analysis rather than transaction processing. The data warehouse usually contains historical data that is derived from transaction data from multiple sources. It separates analysis workload from transaction workload, and enables a business to consolidate data from several sources.

In addition to a relational database, a data warehouse environment often consists of an ETL (extract, transformation, and load) solution, an OLAP (on-line analytical processing) engine, client analysis tools, and other applications that manage the process of gathering data and delivering it to business users. End users typically require some kind of catalog that describes the data in its business context, and acts as a guide to the location and use of this information. Finally, end users require a set of tools to analyze and manipulate the information thus made available.

As you are no doubt aware by now, achieving comprehensiveness and consistency of data in today’s business environment is often a complex and challenging undertaking. This is also true for a data warehouse. The following steps are necessary for the construction of a data warehouse:

Conduct an information infrastructure analysis to determine the required structure of the data warehouse.

Identify the source databases that will feed the data warehouse.

Design the integrated logical data model and determine the architecture of the data warehouse.

Develop and implement a comprehensive meta-data methodology.

Determine, and then implement the physical structure of the data warehouse.

Design and implement an integrated staging area for the data warehouse.

Extract, transform and load the data (from various sources) into the data warehouse. This involves first cleansing the source data of various structural and content errors.

Conduct comprehensive post-implementation review(s) to ensure that the data warehouse is performing acceptably.

Maintain the data warehouse.

Data mining is the act of extracting data/information from assorted sources and presenting information in a manner that is consistent with user requirement. Data mining often implies the existence of data warehouses, so the two terms are closely related. Another related term is information extraction (IE) —the extraction of structured information from unstructured text. IE sometimes involve access of data warehouse(s) either as the source or destination of information.

24.2 Rationale for Data Warehousing

Data warehousing is a technology that is fast enhancing more traditional decision support systems (DSS), because of the added flexibility and benefits that the new technology brings. Let us briefly examine the problems that DSS end users tend to have from two perspectives — user constraints and information system (IS) constraints.

User Constraints: In the absence of data warehousing, users of traditional decision support systems developed using the traditional application-driven approach, commonly complained of the following difficulties:

Difficulty in finding and accessing information needed

Difficulty in understanding information found

Information obtained is not as useful as expected

IS Constraints: In the absence of data warehousing, IS personnel also complained of a variety of problems:

Developing copy programs is often very challenging

Maintaining copy programs and copy databases presents serious integrity and work scheduling problems

Data storage volumes tend to grow rapidly

Database administration also tends to become quite complex

The solution to the above-mentioned problems is the implementation of a data warehouse. A data warehouse provides the decision support benefits that a traditional DSS provides, while providing more flexibility for expansion beyond the confines of the company. This is so for two reasons:

The data warehouse has the capacity to attract interest in the salient facts about the organization, without providing unnecessary details.

While companies may be hesitant about putting their transaction database(s) into the public domain (due to security and confidentiality concerns), they are more likely to be willing to put their data warehouse into the public domain (via the World Wide Web).

24.3 Characteristics of a Data Warehouse

In the definition of a data warehouse, a number of terms were deliberately highlighted. These terms convey important characteristics about a data warehouse. These will be clarified in the next subsection. Next, we will examine what kind of data that is typically stored in a data warehouse. We also examine the processing requirements of a data warehouse. Finally, we will review twelve rules that govern data warehouses.

24.3.1 Definitive Features

In the introduction, it was established that a data warehouse is an integrated, subject-oriented, time-variant, nonvolatile database. Let us briefly examine what these adjectives mean.

Subject-Oriented: Data warehouses are designed to aid the analysis of data in order to make decisions. For example, to learn more about your company’s sales data, you can build a warehouse that concentrates on sales. Using this warehouse, you can answer questions like “Who was our best customer for this item last year?” This ability to define a data warehouse by subject matter, sales in this case, makes the data warehouse subject-oriented.

Integrated: Integration is closely related to subject orientation. Data warehouses typically contain data from disparate sources into a consistent format. They must resolve such problems as naming conflicts and inconsistencies among units of measure. When a data warehouse achieves this, it is said to be integrated.

Nonvolatile:  Nonvolatile means that, once entered into the warehouse, data should not change. This is logical because the purpose of a warehouse is to enable you to analyze historical data.

Time-Variant: A data warehouse’s focus may change over time; also, it could grow (in terms of data volume, data structure and complexity).

24.3.2 Nature of Data Stored

Of importance also, is the nature of data stored in a data warehouse. A data warehouse differs from an operational database in the nature of data stored. An operational database consists of a set of normalized relational tables that store atomic data. A data warehouse on the other hand, stores decision support data, often in non-normalized, aggregated formats. Three distinctions can be made between operational data and decision support data:

Time Span: Operational data represent atomic transactions at specific points in time. Decision support data represent (aggregated) data over a period of time.

Granularity: Operational data is atomic; decision support data is often aggregated. Data warehouses contain decision support data that have been aggregated from various sources and transformed to its intended format.

Dimension: Operational data is instamatic; decision support data is multi-dimensional, typically involving the dimension of time as well as other factors of concern.

24.3.3 Processing Requirements

Data warehouses have very different processing requirements from OLTP systems and operational databases, as explained below.

Workload: Data warehouses are designed to accommodate ad hoc queries. The workload of the data warehouse might not be known in advance, so a data warehouse should be optimized to perform well for a wide variety of possible query operations. OLTP systems and operational databases support only predefined operations. Your applications might be specifically tuned or designed to support only these operations.

Data Modifications: A data warehouse is updated on a regular basis by the ETL process (run nightly or weekly) using bulk data modification techniques. The end users of a data warehouse do not directly update the data warehouse. In OLTP systems and operational databases, end users routinely issue individual data modification statements to the database. The OLTP database is always up to date, and reflects the current state of each business transaction.

Schema Design: Data warehouses often use non-normalized or partially normalized schemas (such as a star schema) to optimize query performance. OLTP systems and operational databases often use fully normalized schemas to optimize update/insert/delete performance, and to guarantee data consistency.

The star schema was introduced in Chapter 5 (section 5.6); it is so widely mentioned in database literature, it deserves a bit of attention: A star schema describes a mechanism where there is a “central” table referred to as a fact table, and other so-called dimensional tables that relate to the fact table via 1:M relationships. The dimensional tables contain dimensional data about details stored in the fact table. While the star schema is widely used in data warehouses, it is also applicable in operational databases as well. Figure 24-1 provides an illustration of a star schema of five relational tables for tracking the graduation statistics from a regional university that operates multiple schools and programs in multiple locations. Tables Location, AcademicProgram, TimePeriod, and School qualify as dimensional tables, while table GraduationSummary qualifies as the fact table.

Figure 24-1. Illustration of a Star Schema for Graduations from a Regional University

Typical Operations: A typical data warehouse query may scan thousands or millions of rows of data. For example, a data warehouse query may involve determining the total sales for all customers in a specific time period. Except for sophisticated queries, a typical OLTP or operational database operation accesses only a small percentage of records, relative to the amount stored. For example, an operational query may involve retrieving the current purchase order for a particular customer, from a table storing hundreds of thousands of purchase orders.

Figure 24-2 provides a summary of what we have established so far: that a data warehouse is typically constructed for various operational databases (sources), and possesses certain characteristics.

Figure 24-2. Constructing a Data Warehouse

24.3.4 Twelve Rules That Govern a Data Warehousing

Data warehouse was first introduced by William Inmon in the 1990s; he still operates as an expert in the field (see [Inmon, 2002] and [Inmon, 2007]). Like E.F. Codd and C. J. Date of the relational model, Inmon introduced twelve rules for governing data warehouses. These rules aptly summarize the previously mentioned characteristics of the data warehouse. Many of these rules have been subsumed in the foregoing discussions. Nonetheless, for emphasis, they are paraphrased below:

Separation: The data warehouse and operational database environments should be separated.

Integration: The data warehouse data are integrated from various operational sources.

Time Horizon: The data warehouse typically contains historical data with an extended time horizon. This is in contrast to a significantly shorter time horizon for the operational databases that may be used as sources for the data warehouse.

Nature of Data: The data in a data warehouse represent snapshot captures (from operational data sources) at specific points in time.

Orientation: The data contained in the warehouse are subject-oriented.

Accessibility: The data warehouse is a predominantly read-only database, with periodic batch updates from the operational databases that are connected to it. It does not support on-line interactive updates.

Life Cycle: The data warehouse development life cycle differs from classical approach to database development in that whereas the data warehouse development is data-driven, the classical database approach tends to be process-driven.

Levels of Detail: In a typical data warehouse, there may be several levels of detail. These include current detail, old detail, lightly summarized data, and highly summarized data.

Data Set: The data warehouse is characterized by read-only transactions on very large data sets. This is in contrast to the operational database, which is characterized by numerous update transactions on a more narrowly defined data set.

Relevance: The data warehouse environment has a system that keeps track of all data sources, transformations and storage. This is essential if the data warehouse is to maintain its relevance.

Metadata: The data warehouse’s metadata forms a critical component of its environment. The metadata provides the following functions: definitions of data elements in the warehouse; identification of data source, transformation, integration, storage, usage, relationships, and history of each data element.

Resource Usage: The data warehouse typically enforces optimal usage of the data by enforcing some form of chargeback mechanism for resource usage.

24.4 Data Warehouse Architecture

The architecture of a data warehouse varies depending upon the specifics of an organization’s situation. Three common architectures have been identified:

Basic Data Warehouse

Data Warehouse With a Staging Area

Data Warehouse With Staging Area and Data Marts

24.4.1 Basic Data Warehouse Architecture

Figure 24-3 shows a simple architecture for a data warehouse. End users directly access data derived from several source systems through the data warehouse.

Figure 24-3. Basic Data Warehouse

The data warehouse consists of raw data (from operational databases), summary data and metadata. Summaries are very valuable in data warehouses because they pre-compute long operations in advance. For example, a typical data warehouse query may be to retrieve something like aggregate sales for a specific period. In Oracle, this may be implemented as a materialized view or snapshot relation set up for that purpose.

24.4.2 Data Warehouse Architecture with a Staging Area

In the basic data warehouse, you need to clean and process your operational data before putting it into the warehouse. You can do this programmatically, although most data warehouses use a staging area instead. A staging area simplifies building summaries and general warehouse management. Figure 24-4 illustrates this typical architecture.

Figure 24-4. Data Warehouse with Staging Area

24.4.3 Data Warehouse Architecture with a Staging Area and Data Marts

Although the data warehouse with staging area is quite common, you may want to customize your warehouse’s architecture for different groups within your organization, or across different related organizations. You can do this by adding data marts. A data mart is a small, single-subject data warehouse that provides decision support for a particular aspect (line) of the business. Figure 24-5 summarizes the approach for three data marts. For example, the data marts may respectively represent information relating to purchasing, sales, and inventory for an organization. A financial analyst would then be able to conduct separate analysis on purchasing, sales and inventory, and then a global analysis on the three aspects combined.

Figure 24-5. Data Warehouse with Staging Area and Data Marts

24.5 Extraction, Transformation, and Loading

The data warehouse must be loaded regularly so that it can serve its purpose of facilitating business analysis. To do this, data from one or more operational databases needs to be extracted, transformed (where necessary), and copied into the warehouse. The acronym ETL is often used to refer to this extraction, transformation and loading of data. The acronym is perhaps too simplistic, since it omits the transportation phase and implies that each of the other phases of the process is distinct. We refer to the entire process, including data loading, as ETL. However, you should understand that ETL refers to a broad process, and not three well-defined steps.

The methodologies and tasks of ETL have been well known for many years, and are not necessarily unique to data warehouse environments. A wide variety of proprietary applications and database systems are used as the IT backbone business enterprises. Data are shared between applications or systems, trying to integrate them, giving at least two applications the same picture of the world. This data sharing was mostly addressed by mechanisms similar to what we now call ETL. Data warehouse environments face the same challenge with the additional burden that they not only have to exchange, but to integrate, rearrange and consolidate data over many operational systems, thereby providing a new unified information base for business intelligence. Additionally, the data volume in data warehouse environments tends to be very large.

24.5.1 What Happens During the ETL Process

During extraction, the desired data is identified and extracted from many different sources, including database systems and applications. Very often, it is not possible to identify the specific subset of interest, therefore more data than necessary has to be extracted, so the identification of the relevant data will be done at a later point in time.

Depending on the source system’s capabilities (for example, operating system resources), some transformations may take place during this extraction process. The size of the extracted data varies from hundreds of kilobytes up to gigabytes, depending on the source system and the business situation. The same is true for the time difference between two (logically) identical extractions: the time span may vary between days/hours and minutes to near real-time. Web server log files for example can easily become hundreds of megabytes in a very short period of time. After extracting data, it has to be physically transported to the target system or an intermediate system for further processing. Depending on the chosen mode of transportation, some transformations can be done during this process, too. For example, an SQL statement that directly accesses a remote target through a gateway can concatenate two or more columns as part of the Select statement.

After transportation to the target system, the data may undergo transformation into the desired formats for the target system. Once this process is completed, the data is loaded into the data warehouse.

24.5.2 ETL Tools

Designing and maintaining the ETL process is often considered one of the most difficult and resource-intensive portions of a data warehouse project. Many data warehousing projects use ETL tools to manage this process.

Oracle Tools

Oracle Warehouse Builder (OWB) provides ETL capabilities and takes advantage of inherent database abilities. Other data warehouse builders create their own ETL tools and processes, either inside or outside the database.

Besides the support of extraction, transformation, and loading, there are some other tasks that are important for a successful ETL implementation as part of the daily operations of the data warehouse and its support for further enhancements. The OWB is quite a sophisticated component that facilitates the construction simple as well as complex data warehouses, the population of them via the ETL process, and the management of them.

DB2 Tools

As mentioned in Chapter 17, DB2 provides four main products for managing data warehouses:

InfoSphere Warehouse Departmental Edition

InfoSphere Warehouse Departmental Base Edition

InfoSphere Warehouse Enterprise Edition

InfoSphere Warehouse Enterprise Base Edition

Each of these products (the latter being an upgrade of the former) consists of related components that allow for the creation, population (via ETL transactions), and management of data warehouses, according to organizational requirements.

24.5.3 Daily Operations and Expansion of the Data Warehouse

Successive ETL transactions to the data warehouse must be scheduled and processed in a specific order. Depending on the success or failure of the operation or parts of it, the result must be tracked and subsequent, alternative processes might be started. The control of the progress and the definition of a business workflow of the operations are typically addressed by special ETL tools provided by the DBMS suite.

As the data warehouse is an active information system, data sources and targets are not beyond the prospect of change. These changes must be maintained and tracked through the lifespan of the system without overwriting or deleting the old ETL process flow information. To build and keep a level of trust about the information in the warehouse, the process flow of each individual record in the warehouse can be reconstructed at any point in time in the future. With time, the data warehouse could therefore expand into something larger and different.

24.6  Summary and Concluding Remarks

Here is a summary of what we have discussed in this chapter:

A data warehouse is an integrated, subject-oriented, time-variant, nonvolatile, consistent database, obtained from a variety of sources and made available to support decision making in a business context.

A data warehouse is a relational database that is designed for query and analysis rather than transaction processing.

The data warehouse is updated via the ETL process.

A data warehouse provides the decision support benefits that a traditional DSS provides, while providing more flexibility for expansion beyond the confines of the company.

Data warehouses often use non-normalized or partially normalized schemas to optimize query performance.

Data warehouses should conform to Inmon’s twelve rules for data warehouses.

Three common data warehouse architectures are the basic data warehouse, data warehouse with a staging area, and data warehouse with a staging area and data marts.

The field of data warehousing is a fascinating breakthrough and is the subject of many contemporary researches. Since it is relatively new, vast opportunities for data warehouse architecture and ETL transaction optimization still abound. Data warehousing is studied as an advanced course in several undergraduate degree programs, as well as graduate programs. The supporting technologies are provided via products from the three leading software engineering companies — IBM’s DB2, Oracle (from Oracle Corporation), and Microsoft’s SQL Server. Additionally, these companies provide readily available documentation on the topic.

With the advancement of the WWW and Web-accessible databases, it is anticipated that the fascination in data warehousing will continue into the foreseeable future. And speaking of Web-accessible databases, the next chapter discusses this topic.

24.7 Review Questions

Give the definition of a data warehouse. Clearly outline what data warehousing entails.

Provide a rationale for data warehousing.

Describe a scenario that would warrant the use of a data warehouse.

Discuss the main characteristics of a data warehouse in terms of:

Definitive features

Nature of data stored

Processing requirements

Rules that govern the data warehouse

State the three architectural approaches to data warehouse design. For each approach, describe the basic architecture, and provide a scenario that would warrant such an approach.

Clearly explain the ETL process. Give examples of ETL tools.

24.8 References and/or Recommended Readings

[Adelman, 2000] Adelman, Sid and Larissa Terpeluk Moss. Data Warehouse Project Management. Boston, MA: Addison-Wesley, 2000.

[Connolly, 2002] Connolly, Thomas and Carolyn Begg. Database Systems: A Practical Approach to Design, Implementation and Management 3rd ed. New York, NY: Addison-Wesley, 2002. See Chapters 30-32.

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8h ed. Menlo Park, CA: Addison-Wesley, 2004. See Chapter 22.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapters 28 and 29.

[Hoffer, 2007] Hoffer, Jeffrey A., Mary B. Prescott and Fred R. McFadden. Modern Database Management 8th ed. Upper Saddle River, NJ: Prentice Hall, 2007. See chapter 11.

[IBM, 2008] IBM Corporation. Data Warehouse. http://www-01.ibm.com/software/data/infosphere/warehouse/ (accessed October 2008).

[Inmon, 2002] Inmon, William. Building the Data Warehouse 3 rd ed. New York, NY: John Wiley, 2002.

[Inmon, 2007] Inmon Associates Inc. http://www.billinmon.com/ (accessed October 2008).

[Oracle, 2008] Oracle Corporation. Oracle10g Database Release 2. http://technetOracle.com (accessed October 2008).

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, MA: Course Technology, 2007. See Chapter 13.

[Silberschatz, 2006] Silberschatz, Abraham, Henry Korth, and S. Sudarshan. Database System Concepts 5th ed. Boston, MA: McGraw-Hill, 2006. See Chapters 18 – 19.

CHAPTER 25

Web-Accessible Databases

Another new area of database systems that has become widespread since the 1990s is Web-accessible databases. The proliferation of these databases is strongly correlated to the growth of the World Wide Web (WWW or W3); both phenomena have become part and parcel of twenty-first century lifestyle, and both promise to be an integral part of life in the foreseeable future. This Chapter provides an overview of Web-accessible databases. The Chapter proceeds under the following subheadings:

Introduction

Web-Accessible Database Architecture

Supporting Technologies

Implementation with Oracle

Implementation with DB2

Generic Implementation via Front-end and Back-end Tools

Summary and Concluding Remarks

25.1 Introduction

A Web-accessible database is simply a database that is accessible via the WWW. The rationale for web-accessible databases can be easily appreciated when one considers the huge benefits that they bring to the business community. Figure 25-1 provides a brief discussion of some of the significant benefits.

Figure 25-1. Significant Benefits of Web-Accessible Databases

25.2 Web-Accessible Database Architecture

In sections 2.6 and 2.7, we discussed the idea of separating a database system into front-end and back-end. This principle is commonly used in implementing Web-accessible databases. Two approaches are common: the two-tiered approach as represented in Figure 25-2, and the three-tiered approach as represented in Figure 25-3. In the two-tiered approach, client applications send requests to the DBMS, which is running on a database server. These requests are processed according to some scheduling algorithm. In the three-tiered approach, additional sophistication is provided by an intersecting application server, which services client requests from various (heterogeneous) applications and filters them to the DBMS for processing. This provides additional flexibility and functionality to the system.

Figure 25-2. Two-Tiered Approach to Web-Accessible Database

Figure 25-3. Three-Tiered Approach to Web-Accessible Database

As you view these two figures, bear in mind that users (in the figures) include end-users as well as businesses making electronic requests from other Web-accessible databases. This is so because there are two types of Web-accessible database systems that are prevalent:

Consumer-to-business (C2B) systems facilitate individual users accessing a company database.

Business-to-business (B2B) systems facilitate businesses accessing other Web-accessible databases of other businesses in a manner that is often transparent to the end-user. This relatively new market provides huge opportunities for companies to improve their efficiency and productivity by concentrating on what they do best, and outsourcing non-essential functions that other businesses provide more efficiently. This gives them the opportunity to also forge powerful alliances for more effective operation.

25.3 Supporting Technologies

The usefulness of separating front-end and back-end subsystems was illustrated in the previous section. This strategy allows us to design and secure the database as a separate activity from developing applications that access that database. Web applications could be one of the many different applications that access the database. Following are some supporting technologies for Web-accessible databases.

Web Servers: A Web server is a sophisticated software system that allows a computer to provide various services to multiple client requests made via the WWW. The server runs on an operating system (which runs on a computer). However, loosely speaking, we usually refer to the entire package of machine and software as the Web server. Popular Web server software products include Apache, Microsoft’s Internet Information Service (IIS), CERN server, NCSA (National Center for Supercomputing Applications) server, Spinner server, Plexus server, Perl server, Tomcat server, etc.

Server-side Extensions: A server-side extension is a software that communicates with a Web server to handle assorted client requests. Often, the server-side extension program acts as an intermediary between the Web server and the database, farming out all SQL requests to the DBMS. Both the DBMS and the server-side extension program must be ODBC-compliant. Products such as ColdFusion, Delphi, Java Studio Enterprise, etc. qualify as server-side extension programs.

Web Server Interfaces: Web server interfaces facilitate the display of information on dynamic Web pages. There are two popular categories:

Common Gateway Interface (CGI) is a set of rules that specify how parameters are passed between client programs and Web servers. A client program that can run on a Web server is called a script, hence the term CGI script. Common scripting languages include JavaScript, Active Server Pages (ASP), and PHP. However, high-level languages such as C++, Java, Perl, etc. also qualify.

Application Program Interface (API) is a set of routines, protocols and tools that facilitate easy software construction. Since APIs are typically shared code that is resident in memory (in the case of Web technology, they reside on the Web server), they tend to be more efficient than CGI scripts.

Extensible Markup Language (XML): XML is a meta-language that was designed specifically to facilitate the representation, manipulation and transmission of structured data over the WWW. It was first published by the WWW Consortium (W3C) in 1998, and to no surprise, has become the de facto data exchange standard for e-commerce, thus circumventing the pre-existing problem of interoperability among different Web servers.

XML was developed from an earlier standard called the Standard Generalized Markup Language (SGML). And as expected, other XML-based languages are emerging. Three examples are Extensible Business Reporting Language (XBRL), Structured Product Labeling (SPL), and Extensible Style Language (XSL).

Simple Object Access Protocol (SOAP): The original emphasis of SOAP was to support remote procedure calls (RPC). However, the norm is for SOAP messages to be sent by web servers as XML documents (synchronous as well as asynchronous). SOAP messages are frequently transmitted as the data portion of HTTP (hyper-text transport protocol) messages.

Hypertext Transport Protocol (HTTP): HTTP is the protocol used to transfer of information over the WWW. It is characterized by a simply request-response structure that represents interactions between a client (Web browser) and a Web server. It is assumed that you are familiar with the Internet, basic Web page construction, Uniform Resource Locators (URLs), domain names and other related issues.

Web Services Description Language (WSDL): The original authors of WSDL define it as “an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information” [Ariba, 2001] WSDL can be construed as an interface definition language (IDL) with bindings that clearly specify how the various components of a message are mapped. WSDL messages are frequently transmitted as SOAP messages over HTTP.

Client-side Extensions: These add functionality to the Web browsers. Client-side extensions are available in various forms. Following is a summary of the most commonly encountered ones:

Plug-ins: A plug-in is an external application program that is automatically invoked by the Web browser when required. For example, a Web browser, upon receiving a PDF (portable document format) document will invoke the available PDF-reader on the host operating system.

Java: As you are no doubt aware, Java is a platform independent programming language. Most operating systems have a Java Virtual Machine (JVM), which allows Java code to execute on the local machine (if there isn’t a JVM in the operating system, this is readily available from the Sun Microsystems Web site). Calls to Java routines are often embedded in HTML pages. When a Web browser encounters this, it invokes the local JVM to execute the code.

JavaScript: JavaScript is a Java-like scripting language (developed by Netscape), but is much simpler. JavaScript code is often embedded in Web pages. It is downloaded whenever the Web page is activated, as well as on certain specific events (such as loading of a specific page from the server, or a mouse-click, etc.) Whenever the browser encounters this, it invokes the JavaScript plug-in to execute this code.

ActiveX: ActiveX is Microsoft’s alternative to Java. It works perfectly in a Windows environment. Although possible in other languages, C++ and Visual Basic are well known for their facilitation of ActiveX controls.

VBScript: VBScript is another Microsoft product that is often used to add functionality to the Web browser. Like JavaScript, VBScript code is often embedded in Web pages. Invocation and execution are also similar to JavaScript.

Cookies: Cookies are used to expedite user requests when the user visits a site more than once. When the site is first visited, the Web server creates a cookie with basic user information (such as e-mail) and sends it to the browser. When the browser is subsequently used to visit that site, the cookie, upon recognition by the Web server, is used to expedite the user request.

25.4 Implementation with Oracle

Oracle implements a database that is by definition, Web-accessible (this applies since Oracle 10G). This is achieved through the products Oracle Enterprise Manager, (OEM), iSQL*Plus, Oracle JDeveloper, and Oracle SQL Developer in the following ways:

When you install the Oracle Server suite (including OEM) on a node in your company or home network, the machine is automatically configures as a database server.

Oracle automatically installs and configures a database listener on database server to respond to incoming requests from client nodes (running Oracle 10G or 11G Client) in the network, on the internet, or an extranet.

Typically, your database server should have at least one database. This can be created during installation, or subsequently via the Oracle Database Configuration Assistant (DBCA) component.

You can access your database server from any machine that has an internet connection, through OEM via the URL http://<Machine.Domain>:5500/em (you supply the machine name and domain name for your network). Of course, you will need a valid user account, password, and appropriate privileges.

You can also access your database server from any machine that has an internet connection, through Oracle’s iSQL*Plus via the URL http://<Machine.Domain>:5560/isqlplus (again, you supply the machine name and domain name for your network). Again, you will need a valid user account, password, and appropriate privileges.

Through Oracle JDeveloper (OJD), Oracle allows Web-accessible Java-based applications to be constructed. These applications may access local or remote Oracle (or heterogeneous) databases.

Through Oracle Forms Developer (OFD), Oracle also allows Web-accessible PL/SQL-based applications to be constructed. These applications may access local or remote Oracle (or heterogeneous) databases.

This development represents a huge step forward for Oracle, and has no doubt fuelled its invigorated claim of being the leading software engineering firm for Web-accessible databases.

25.5 Implementation with DB2

Like Oracle, DB2 implements a Web-accessible database (this applies to DB2 8 and subsequent versions). This is achieved through the products DB2 Connect, DB2 Everyplace and WebSphere in the following ways:

If you configure a mainframe, or mini computer to be a DB2 server, and create your DB2 database on it, you can access the database over an intranet, the Internet, or an extranet by running DB2 Connect from your client machine. As in the case of the Oracle database, you will need to know the machine name, the domain name and port number that the database server listens on, in order to connect to it.

DB2 Everyplace is a miniscule version of DB2 that runs on mobile devices such as personal digital assistants (PDAs), cell phones, etc. DB2 Everyplace can be used as a local independent DBMS, or to query information residing at a remote database server, via the WWW.

Through WebSphere, IBM allows Web-accessible Java-based applications to be constructed. These applications may access local or remote DB2 (or heterogeneous) databases. There is also a WebSphere Everyplace version for PDAs, cell phones and other pervasive devices.

Like Oracle, IBM also claims to be the leading software engineering firm for Web-accessible databases. The truth is, both companies are archrivals at the top of the database systems market.

25.6 Generic Implementation via a Front-end and a Back-end Tool

This section describes a generic approach to constructing a Web-accessible database system, based on front-end and back-end tools. The operation can be summarized in two steps:

Create the database using an appropriate back-end tool. The tool used must support ODBC, JDBC, or both. Of course, it is assumed that appropriate planning and design as discussed in earlier Chapters, have taken place.

Create the Web-application using an appropriate front-end tool that incorporates the requisite Web-supporting technologies as discussed in section 25.3. The tool must support ODBC and/or JDBC, and must facilitate code in at least one of the accepted scripting languages (JavaScript, PHP, ASP, etc.). It must also support XML. Again, the basic assumption is that sound principles of software engineering will be used to design the user interface as a pre-requisite to this activity.

Figure 25-4 provides a list of commonly used tools. They have been listed in three categories: front-end tools, back-end tools, and programming languages. In each, it is recommended that you use the most current version of the stated software product (for the RAD tools and DBMS suites, the versions stated are simply safe starting points).

Figure 25-4. Commonly Used Tools for Web-accessible Databases

25.7 Summary and Concluding Remarks

Let us summarize what we have covered in this Chapter:

A Web-accessible database is simply a database that can be accessed via the WWW.

Web-accessible databases provide a number of benefits to companies and individuals that implement and/or use them. Among the benefits are facilitation of e-commerce, broadening of the company’s scope of operation and market reach, a wide range of conveniences, improved productivity, and improved competitive advantage.

A Web-accessible database may be implemented as a two-tiered system or a three-tiered system. Additionally, they may be C2B or B2B.

The supporting technologies for Wed-accessible databases include Web servers, server-side extensions, server interfaces, XML, SOAP, WSDL, and client-extensions.

Both Oracle and DB2 implement Web-accessible databases as a matter of policy.

Implementation of a Web-accessible database can be summarized into two simple but profound steps: creating the database and creating the Web-accessible user interface. The tools used must meet minimum industry standards.

Web-accessible databases are among one of the technology-related phenomena that have transformed life in the twenty-first century. They are expected to continue to be an integral part of life in the foreseeable future.

This takes us to the end of the chapters for the course. If you understand most of the issues discussed, and now have a desire to delve more deeply into some aspects of the field, then the course has succeeded in its intent. If you find that you now have a strong desire to make database systems one of your areas of expertise, then welcome to the community! You will find it a wonderfully rewarding and progressive field.

25.8 Review Questions

Give the definition of a web-accessible database. Provide justification for their existence.

Discuss two example of the usefulness of Web-accessible databases.

Discuss the two-tiered approach to implementing Web-accessible database. When should you use such an approach?

Discuss the three-tiered approach to implementing Web-accessible database. When should you use such an approach?

Briefly describe the main supporting technologies for Web-accessible databases.

Describe how Oracle implements Web-accessible databases.

Describe how DB2 implements Web-accessible databases.

Describe a generic approach for implementing a Web-accessible database. What precautions must be taken?

25.9 References and/or Recommended Readings

[Ariba, 2001] Ariba, IBM Corporation, and Microsoft. Web Services Description Language (WSDL) 1.1. http://www.w3.org/TR/wsdl (accessed August 2009).

[Date, 2004] Date, Christopher J. Introduction to Database Systems 8th ed. Menlo Park, CA: Addison-Wesley, 2004. See Chapter 27.

[Elmasri, 2007] Elmasri, Ramez and Shamkant B. Navathe. Fundamentals of Database Systems 5th ed. Reading, MA: Addison-Wesley, 2007. See Chapters 26 and 27.

[Hoffer, 2007] Hoffer, Jeffrey A., Mary B. Prescott and Fred R. McFadden. Modern Database Management 8th ed. Upper Saddle River, NJ: Prentice Hall, 2007. See Chapter 10.

[Kifer, 2005] Kifer, Michael, Arthur Bernstein and Philip M. Lewis. Database Systems: An Application-Oriented Approach 2nd ed. New York, NY: Addison-Wesley, 2005. See Chapter 17.

[Riccardi, 2003] Riccardi, Greg. Database Management With Web Site Development Applications. Boston, MA: Addison-Wesley, 2003. See Chapters 12-15.

[Rob, 2007] Rob, Peter and Carlos Coronel. Database Systems: Design, Implementation & Management 7th ed. Boston, MA: Course Technology, 2007. See Chapter 14.

[Silberschatz, 2006] Silberschatz, Abraham, Henry Korth, and S. Sudarshan. Database System Concepts 5th ed. Boston, MA: McGraw-Hill, 2006. See Chapter 10.

[W3C, 2003] World Wide Web Consortium. Extensible Markup Language (XML). http://www.w3.org/XML/ (accessed August 2009).

PART F

Final Preparations

This penultimate division has two objectives:

to help you to review and assimilate the concepts and principles covered in the course so that you will be able to apply them to real situations;

to help you prepare for final examinations.

The division consists of a final chapter that includes sample examination questions and case studies for your usage.

CHAPTER 26

Sample Exercises and Examination Questions

This final chapter of sample exercises, examinations, and case studies, will proceed under the following captions:

Introduction

Sample Assignment 1A

Sample Assignment 2B

Sample Assignment 3A

Sample Assignment 4A

Sample Assignment 5A

Sample Assignment 6A

Sample Assignment 7A

Sample Assignment 8A

Sample Interim Examination A

Sample Interim Examination B

Sample Final Examination A

Sample Final Examination B

Sample Final Examination C

26.1 Introduction

This chapter provides you with some sample examination questions and case studies, designed to help you solidify in your mind, the concepts and principles covered in the course. The problems are arranged in assignments and examinations. The intent is to enlighten you to the type of problems they are likely to be asked to solve.

Generally speaking, assignment questions are more demanding than examination questions, and require more time. You will also observe that most of the examination questions are drawn from actual assignments. You should therefore do the assignments before attempting the examinations. The suggested weight (in points) for each question is indicated immediately following the question in curly brackets.No solution is provided for the problems posed in this chapter, for the following reasons:

The problems are intended to test your understanding of the materials covered. If you find that you are struggling with the solution to a problem, then you need to review the relevant sections before continuing.

In some cases (particularly where you are asked to design a database), there may be more than one solution to the problem(s) posed. This in typical in software engineering as well as in database systems, where you analyze alternate solutions to a problem, and choose the most prudent one.

If your are using this book as a prescribed text for a course in database systems, your professor will want to have a say in what questions you ought to focus on.

26.2 Sample Assignment 1A

Zealot Industries Inc. is a manufacturing firm with over 2000 employees. The firm has been expanding well beyond the founders’ expectations. However, it faces a severe hindrance to additional expansion — the absence of a computerized Human Resource Management System (HRMS) that could facilitate management of human resource related issues.

Precision Software Inc. is a software engineering firm contracted to develop and implement the HRMS project. After a preliminary meeting with Zealot’s senior management, the lead software engineer on the Precision Software team documents his findings as follows:

A detailed employee profile consists of the employee’s personal information, employment history, education history and beneficiary information.

Every employee belongs to a department, and is assigned a specific job description (there may be several employees with the same job description).

Employees are classified according to the salary range that they fall in.

Each employee has a compensation package which outlines basic salary and other benefits. Benefits include health insurance, retirement plan, life insurance, professional development allowance, housing allowance, traveling allowance, entertainment allowance, vacation allowance, and education allowance.

A payroll log keeps track of remuneration paid out to employees.

Each employee is assigned to at least one project; the number of employees assigned to a project depends on the size and complexity of the project.

Identify all information entities (object types) mentioned or implied in the case.     {10}

Identify all relationships among the entities (object types) and represent them on an ERD or ORD.     {20}

Use the information in the case, combined with the information conveyed by the E-R diagram and your own intuitive design skills to propose relations and their associated attributes for each of the entities represented on the E-R diagram. You may also introduce additional relation(s) to take care of M:M relationship(s) which may exist.     {60}

26.3 Sample Assignment 2B

What are the possible advantages of using distributive processing in a database system environment?     {06}

What are the problems that can be experienced from having non-normalized files in a system?     {06}

Relation R(A B C . . . K) is in 1NF only.  The following FDs hold:

Primary key is (ABC)

A → D

B  −> E

F  → G,H,I

J  →  B

K → C

(AF) −>> J/K

By repeatedly using Heath’s or Fagin’s theorems, decompose R into a set of 5NF relations. Show how the new relations will be keyed.     {15}

The following atomic data elements were taken from the data dictionary of an inventory system:

Group the attributes into related entities. Then for each entity, identify the FDs, then normalize it. Grade may be assigned as follows:

Identification of all possible FDs.     {11}

Putting the elements into normalized relations. You may introduce elements as required with appropriate explanation.     {44}

26.4 Sample Assignment 3A

Define the following terms:

Candidate key     {02}

Foreign key     {02}

Functional dependence     {03}

Transitive dependence     {03}

Give the primary key and the highest normal form of each of the following relations. State any assumptions made and give reasons for your answers.

Student {Student#, Name, Address, Gender, Age DOB}     {03}

Flight {Flight#, Date, Seats-Available, Flight-Time, From-City, To-City}     {03}

Account {Account#, Customer#, Customer-Name, Balance}     {03}

Supply {Part#, Quantity, Supplier#, Supplier-Address}     {03}

In the relation Participation, are attributes Manager, Project, Hours (spent on project per month), Salary. A manager's salary is fixed, and he/she can work on many projects.

3a.    Draw a FD diagram of the relation.     {03}

3b.    Would you store Participation as defined? Justify your answer.     {04}

3c.    Provide some sample data for the relation.     {04}

3d.    What is the highest normal form of Participation?     {03}

3e.    Show how you would design a conceptual schema to store the data mentioned for relation Participation?     {11}

A scientific research establishment organizes its work in projects, each project consisting of several experiments. For each experiment, the following is recorded:

Project number

Project manager

Project name

Experiment number

Experiment name

Lab in which experiment is to be conducted

List of scientists to work on the project

Note: A project manager may work on an experiment. A scientist may work on more than one experiment, provided that they are done in the same lab. A manager may manage more than one project.

4a.    From the information given, develop an ERD.     {10}

4b.    Using the XR model, derive a set of BCNF relations for the system. Use self- explanatory relation and attribute names (you may introduce new attributes).     {20}

Figure 26-1 provides a snapshot of a live database. Primary keys are highlighted.  Based on the data, derive an E-R diagram (state any assumptions made).     {10}

Figure 26-1. Snapshot of a Database

26.5  Sample Assignment 4A

Figure 26-2 provides basic specifications for a music database. Carefully analyze the figure, and answer the questions that follow.

Figure 26-2. Basic Specifications Music Database

Write relational calculus statements to realize the following:

1a.    Registered musicians (MNO and MNAME) from USA or JAM (where “USA” and “JAM” are abbreviated codes for United States and Jamaica respectively).      {03}

1b.    List every ensemble (ENO) that includes a SAXAPHONE or CLARINET player.         {03}

1c.    Give the ENO of every ensemble that includes a SAXAPHONE but not a CLARINET player.      {04}

1d.    List all compositions (CNO and TITLE) by MOZART.  {04}

1e.    List all performances (PNO, CNO, MNO, and PCOUNTRY) of compositions in the country of origin.    {06}

1f.    Give the ENO of every ensemble that includes a SAXAPHONE or CLARINET player, but not both.      {06}

1g.    Find CNO for compositions all of which have been performed in USA.        {03}

1h.    List countries in which MOZART's compositions have been performed.        {03}

1i.    Give ENAME of ensembles whose manager is RUSSIAN.             {03}

Write relational algebra statements corresponding to each of the relational calculus statements of question #1.           {35}

26.6  Sample Assignment 5A

The (cross-section of the) college database introduced in chapter 7 (and discussed in subsequent chapters) is repeated here (Figure 26-3).

Figure 26-3. Cross Section of College Database Requirements

Write SQL statements to create these tables in your schema of the class database. You may add additional attributes to the structure of each database table, as you deem appropriate. Store these statements in an SQL script file.    {40}

Use your SQL script file to create the tables in the class database.           {16}

Populate your tables with sample data (at least six records per table).            {16}

26.7  Sample Assignment 6A

Figure 26-2 is repeated as Figure 26-4 for ease of reference. Carefully analyze the figure, and answer the questions that follow.

Figure 26-4. Basic Specifications for Music Database

Write appropriate SQL statements to create the tables specified in the figure (you may assign appropriate column types and lengths). Also include important integrity constraints.          {30}

Write SQL statements to realize the following:

2a.    Registered musicians (MNO and MNAME) from USA or JAM (where “USA” and “JAM” are abbreviated codes for United States and Jamaica respectively).      {03}

2b.    List every ensemble (ENO) that includes a SAXAPHONE or CLARINET player.        {03}

2c.    Give the ENO of every ensemble that includes a SAXAPHONE but not a CLARINET player.       {04}

2d.    List all compositions (CNO and TITLE) by MOZART.          {04}

2e.    List all performances (PNO, CNO, MNO, and PCOUNTRY) of compositions in the country of origin.     {06}

2f.    Give the ENO of every ensemble that includes a SAXAPHONE or CLARINET player, but not both.      {06}

2g.    Find CNO for compositions all of which have been performed in USA.        {03}

2h.    List countries in which MOZART's compositions have been performed.          {04}

2i.    Give ENAME of ensembles whose manager is RUSSIAN.                 {03}

Define and specify SQL views to realize the above requirements.          {36}

Given the above database, and assuming an Oracle 10g environment, show what the contents of the system catalog tables User_Tables (attributes Table_Name and Tablespace_Name) and User_Columns (attributes Column_Name and Table_Name) would be.        {10}

The above database may be refined by introducing a Countries table, and adjusting three other tables to each have a foreign key that references this table. Show the required SQL statements to make this adjustment.    {12}

26.8  Sample Assignment 7A

For each question, show the SQL statement(s) that you have used in order to address the question.

Create a test table called JonesB_TestTable (where “JonesB” represents your user name) and populate it with some sample data.          {10}

2a.    Create a role and for its name, use the concatenation of  your user account and the word “Role.” For instance, the role may be called JobesB_Role (where “JonesB” represents your user name).      {02}

2b.    Grant to your role the system privileges that facilitate application development (see section 13.2.2 for guidelines on this).        {11}

2c.    Grant to your role, SUDI privileges on JonesB_TestTable (review section 13.2.3 for the SUDI privileges).     {04}

2d.    There is a catalog table that you can access to see all the system privileges granted to all roles in the system. Its name is Role_Sys_Privs. Study its structure, and then issue an appropriate SQL statement to display the system privileges associated with your role. Show the SQL statement used.         {06}

Create a test user account called JonesB_User (where “JonesB” represents your user name). Assign the user to your tablespace; also use a password that you will remember. Your current tablespace name is TBS_JonesB (where “JonesB” represents your user name).      {04}

3a.    Try logging on to the database as JonesB_User. Record the result of your attempt, and provide an explanation for this observation.         {04}

3b.    Log on with your normal user account and grant your role (created in 2a) to user JonesB_User.    {02}

3c.    Try logging on as JonesB_User once more and record the result of your attempt.  Provide an explanation for this observation.            {04}

3d.    Log on as JonesB_Useer and try running a query on your test table. To do this, you must either create synonym for JonesB_TestTable in the schema of JonesB_User, then run the query on the synonym, or you must qualify the table name when you run the query on JonesB_TestTable. Record the result of your attempt, and explain why you obtained that result.            {10}

3e.    Switch to your normal user account and revoke the SUDI privileges on JonesB_TestTable from your role.     {04}

3f.    Log on as JonesB again and try running a query on your JonesB_TestTable. Record the result of your attempt. Provide an explanation for this result.    {06}

26.9  Sample Assignment 8A

Identify and summarize five issues to be addressed in database administration.        {12}

2.1.    Give the definition of a distributed database system.  {03}

2.2.    What is the difference between database fragmentation and database replication? Cite a situation that would warrant each.          {04}

2.3.    State and clarify four of Date’s twelve rules for distributed databases.          {12}

2.4.    Discuss two challenges to distributed databases, and alternate approaches for addressing each challenge.    {08}

3.1.    Describe two challenges to OO database management systems.          {06}

3.2.    Describe one hybrid approach that facilitates the peaceful coexistence of relational databases and object technology.         {04}

26.10 Sample Interim Examination A

Instruction: Answer all four questions.

1a.    What are the main components of a Database System? With the aid of a diagram, show how they are related.                {10}

1b.    State six objectives of a database system.    {06}

1c.    What is meant by data independence?  Using example(s) explain its importance.        {04}

2a.    Clarify the terms external schema, conceptual schema and internal schema.        {06}

2b.    Clarify the terms data sub language (DSL), data definition language (DDL) and data manipulation language (DML). Relate SQL to these terms.      {08}

3a.    What are the main functions of the DBMS?    {06}

3b.    Give two examples of a DBMS.      {02}

3c.    What are the functional components of the DBMS? By use of a diagram, show how they are related.    {12}

4a.    Define the following terms:

Primary key          {02}

Foreign key          {02}

Attributes          {02}

Relation          {02}

Domain          {03}

4b.    Figure 26-5 is an excerpt from the database design proposal for a college administrative management system:

Figure 26-5. Excerpt from College Database Proposal

Study the figure and answer the following questions:

Identify all foreign keys.        {02}

Describe verbally how you could obtain a list of course names and chapterr names, lecturing these courses at specific times.          {02}

Explain verbally how you could obtain a list of program names and course names in appropriate order for the college bulletin.        {02}

26.11 Sample Interim Examination B

Instruction: Answer all five questions.

1a.    What is a database?                     {02}

1b.    State four primary objectives and four secondary objectives of a database system.          {08}

1c.    What is data independence? Explain its importance in the design of a database.         {06}

1d.    State four advantages that a DBS has over traditional file processing systems.          {04}

2a.    Briefly describe the three levels of architecture in a database system. You may use diagram(s) in your explanation.           {10}

2b.    With the use of diagram(s) where necessary, explain how communication is achieved among end users, the database, application programs, DBMS and the hardware.      {10}

3a.    State four functions of the DBMS.               {04}

3b.    ITC Software Inc. is developing a database system for one of its clients, X-Don Inc., based in Miami, Florida.  X-Don has experienced Windows users. They use the Oracle RDBMS. Additionally, X-Don has very experienced Oracle Programmers. However, it is strategically imprudent for ITC Software to design and develop the database in Oracle or on the Windows platform. They prefer to work on the Linux platform, using a product called Kylix and PHP. To complicate matters, ITC Software operates out of Port of Spain, Trinidad. ITC Software therefore desires to access the X-Don’s database, hosted in Miami, via telecommunication linkage. Is the project feasible? Using block diagram, explain how the project could be implemented.      {08}

3c.    Briefly explain the following: Data Definition Language (DDL); Data Manipulation Language (DML); Data sub-language (DSL); Structured Query Language (SQL)    {08}

4.    Developing an inventory management system is standard in most organizations. The main entities to be identified are Purchase Orders, Receipts (of goods), Inventory (Items), Inventory Categories, Inventory Requisitions, Departments, Inventory Issues. Next, we identify relationships among the entities:

Departments make requisitions for inventory items;

Issues of inventory are made to departments in response to requisitions;

Purchase orders are made to suppliers for inventory items;

Goods received are normally accompanied by an invoice, specifying the items and cost (of course, invoices come from suppliers);

Each inventory item is assigned a category.

4a.    From the information given, draw an ERD for the inventory management system described.    {10}

4b.    For each relationship, state whether mandatory or optional; 1:1 or 1:M or M:M or M:1.        {10}

5a.    Write brief notes on five of the terms in following list (Figure 26-6):               {10}

Figure 26-6. Basic Terms Used in the Relational Model

5b.    Complete the following triplets with the most appropriate words or expressions to be taken from the list provided below (Figure 26-7).         {10}

Figure 26-7. Database Terminology Puzzle

26.12  Sample Final Examination A

This examination consists of nine (9) questions distributed over four (4) sections. You are required to do four (4) questions, but no more than one question from a section.

Section A:  Do all questions from this section.

1a.    What are the main components of a database system?  With the aid of a diagram, show how they are related.  {10}

1b.    State six (6) objectives of a database system.    {06}

1c.    Give four (4) examples of popular DBMS suites.     {04}

2a.    Briefly clarify the terms external schema, conceptual schema and internal schema.By use of an appropriate diagram, show how the three terms are related.    {10}

2b.    Explain what is meant by data sub-language (DSL), data definition language (DDL) and data manipulation language (DML). How does SQL relate to these terms?  {04}

2c.    State four (4) significant roles/functions of the DBMS.               {04}

2d.    What is the difference between a DBMS and a compiler?              {02}

Section B:  You may do one question from this section.

The following atomic data elements were taken from the data dictionary of an inventory system:

Figure 26-8a. Excerpts from Inventory Management System

Put the elements into normalized relations, clearly stating the basis for your decision. You may introduce data elements as required. Clearly state any assumptions or observations made.        {20}

4a.    Complete the following triplets with the most appropriate words or expressions to be taken from the list provided in Figure 26-8b.                     {10}

4b.    Give a formal definition of a relation. Clarify any ambiguity that may exist between an entity and a relation.  {04}

4c.    State three (3) kinds of relations that may be found in a database; explain what each is with an example.  {06}

Figure 26-8b. Database Terminology Puzzle

5a.    State and briefly clarify two integrity rules that must govern a database.         {04}

5b.    How should deletion of referenced tuples be treated? Give an appropriate example.      {06}

5c.    Suppose that it is desirable to record the following information in a normalized database:

Course  [C]

Teacher   [T]

Hour    [H]

Room    [R]

Student  [S]

Grade    [G]

Assume that the following dependencies hold:

[H+R]→ C

[H+T] → R

[C+S] → G

[H+S] → R

[C]   → T

By completing closure, determine the candidate key in the given relation.         {03}

Derive a set of 4NF relations.       {07}

Section C:  Examine the section of a college/university database, shown in Figure 26-9 and answer one question from this section.

Write relational calculus statements that will produce the results specified below:

6a.    List all students (name, ID, and gender) who share surname with other students      {03}

6b.    List all program-names and related course-names          {04}

6c.    List the names of all programs that include the course (code) ‘CS100’            {03}

6d.    List the names of all programs that do not include course (code) ‘ENGL101’          {03}

6e.    List all Canadian (country code ‘CAN’) students (name, ID, Hall-name) enrolled in the B.S. in Computer Science (program code ‘P11SC’)        {04}

6f.    List the name of each department along with the name of its chair-person        {03}

Write relational algebra statements that will produce the results specified below:

7a.    List all students (name, ID, and gender) who share surname with other students      {03}

7b.    List all program-names and related course-names          {04}

7c.    List the names of all programs that include the course (code) ‘CS100’            {03}

7d.    List the names of all programs that do not include course (code) ‘ENGL101’          {03}

7e.    List all Canadian (country code ‘CAN’) students (name, ID, Hall-name) enrolled in the B.S. in Computer Science (program code ‘P11SC’)        {04}

7f.    List the name of each department along with the name of its chair-person        {03}

Section D:  Examine the section of a college/university database, shown in Figure 26-9 and answer one question from this section.

Figure 26-9. Section of a College Database

Write SQL statements that will produce the results specified below:

8a.    List all students (name, ID, and gender) who share surname with other students      {03}

8b.    List all program-names and related course-names   04}

8c.    List the names of all programs that include the course (code) ‘CS100’          {03}

8d.    List the names of all programs that do not include course (code) ‘ENGL101’          {03}

8e.    List all Canadian (country code ‘CAN’) students (name, ID, Hall-name) enrolled in the B.S. in Computer Science (program code ‘P11SC’)        {04}

8f.    List the name of each department along with the name of its chair-person        {03}

9a.    How important are (logical) views in a database (cite possible advantages)?        {04}

9b.    Identify four (4) scenarios that could render a logical view non-updateable.         {04}

9c.    Referring to the college database of Figure 26-9, define a view that “stores” Computer Science courses only (assume that all Computer Science course-codes begin with the acronym ‘CS’).           {03}

9d.    How important is the system catalog in a DBMS? Describe three important tables that are usually included in the system catalog (use the Oracle DBMS as a frame of reference).          {06}

9e.    Given the information provided in Figure 26-9, illustrate what might be the contents of one of the catalog tables mentioned in (9d) above. Explain how the system catalog is maintained.        {03}

26.13 Sample Final Examination B

This examination consists of four (4) sections. You are required to do all questions in section A, and three additional questions — one from each of the other three sections.

Section A:  Answer all questions in this section. For multiple-choice questions, simply highlight the responses that you deem to be correct.

State six significant objectives of a database system.  {06}

State three advantages of a database system.                       {03}

State three primary functions of a DBMS.        {03}

Which of the following is not a valid example of a DBMS?                  {01}

Oracle

DB2

Java Development Kit

Informix

A record is to a file:            {01}

What a mouse is to a computer

What a tuple is to a relation

What a procedure is to a program

What peach is to a fruit salad

Cardinality of a relation refers to:      {01}

The age (in months) of the relation

The sexual orientation of parties in the relation

The number of tuples in the relation

None of the above

A foreign key may be defined as:      {01}

An attribute or combination of attributes that uniquely identify tuples of a relation

An attribute or combination of attributes that ought not to have been included in a relation

An attribute that may contain null values

An attribute or combination of attributes of a relation R1 that forms the primary key in some other relation R2

Let R1[A,B,C] and R2[C,D,E] be two relations with R1.C being defined on the same domain as R2.C. Assume further that the primary key of R1 is A and the primary key of R2 is C. Which of the following statements are valid?      {02}

SELECT * FROM R1, R2 WHERE R1.C = R2.C

SELECT * FROM R1, R2 WHERE R1.C = R2.D

There is a M:M relationship between R1 and R2

There is a 1:M relationship between R2 and R1

Based on the principle of data independence, which of the following can be concluded?       {02}

One can decompose a relation into smaller projections in any way he/she chooses

Structural changes to a database should not affect the application programs that access the database

Different users of a database will have different external perspectives of its data

All of the above

Which of the following statements regarding the results of normalization are false?        {02}

Redundancy and data anomalies are reduced

Redundancy and data anomalies are exacerbated

Data integrity and data independence are enhanced

Data integrity and data independence are compromised

Let R1[A,B,C] and R2[C,D,E] be two relations with R1.C being defined on the same domain as R2.C. Assume further that the primary key of R1 is A and the primary key of R2 is C. R2 consists of N2 tuples each of which is referenced by at least one of the N1 tuples of R1. Which of the following assertions are plausible/true?            {02}

N1 > N2

N1 < N2

In order to purge (empty) both relations, one must purge in the order {R1, R2}

In order to purge (empty) both relations, one must purge in the order {R2, R1}

How many standard normal forms are there?     {01}

Three

Four

Five

Six

A relation that is in BCNF is automatically in:  {02}

3NF, 2NF and 1NF

3NF and 2NF

4NF and 5NF

None of the above

In database systems, a surrogate:      {02}

Is a relational table that is not related to other tables in the database

Is a system-controlled primary key, introduced by the designer

Is sometimes introduced to avoid using a cumbersome composite primary key.

Is a database designer that gets no credit for his/her work

An ERD is useful to:        {01}

Show the interrelationships among application programs of a system

Represent the relationships among application programs and database tables

Illustrate the various external view that users of a database might have

Illustrate the main entities and relationships of a database system

Section B: Answer one question from this section.

16a.    Give a formal definition of a relation. Clarify any ambiguity that may exist between an entityand a relation.  {04}

16b.    State three (3) kinds of relations which may be found in a database; for each type, explain what it is and give an appropriate example.        {06}

16c-16e.    The following is an excerpt from a college database design proposal (Figure 26-10):

Figure 26-10. Except From a College Database

16c.    Based on the Figure 26-10, clearly identify all relationships and foreign keys.        {06}

16d.    Based on Figure 26-10, describe, how you could obtain a list showing course names and the corresponding names of lecturers who teach these courses.        {02}

16e.    Based on Figure 26-10, describe, how you could obtain a list showing program names and the corresponding course names in appropriate order for the college bulletin.  {02}

17a.    Define the following terms:

Functional Dependence        {02}

Transitive Dependence           {02}

Non-loss decomposition            {02}

Boyce-Codd Normal Form              {02}

Multi-valued Dependence              {02}

Fourth Normal Form                {02}

17b.    As Database Manager or Information Systems Manager in a software engineering firm, what standards would you set in respect of logical database design? Justify your position.          {08}]

Section C: Answer one question from this section.

Figure 26-2, which provides an excerpt from the specification of a music database, is repeated here (as Figure 26-11) for ease of reference. Carefully analyze the figure, and answer the questions that follow.

Figure 26-11. Basic Specifications of Music Database

Provide sample data of the above relations.                     {bonus 04}

Write relational calculus statements to realize the following:

18a.    List registered musicians from CUB or CAN (where “CUB” and “CAN” are abbreviated codes for Cuba and Canada respectively).              {03}

18b.    Give the ENO of every ensemble that includes a Violin or Violin player.          {03}

18c.    Give the ENO of every ensemble that includes a Violin player but not a Guitar player.      {04}

18d.    List all compositions (CNO and TITLE) by David Foster.              {04}

18e.    List all performances (PNO CNO MNO & PCOUNTRY) of compositions that have been performed in the country of origin.        {06}

Provide sample data of the above relations.                     {bonus 04}
Write relational algebra statements corresponding  to specifications of 18a – 18e above.        {20}

Section D: Answer one question from this section.

20a.    How important are logical views in a database?  {04}

20a-20f.    The college database of question 16 is repeated here for ease of reference.

Based on the schema provided in Figure 26-12, write SQL statements to do the following:

20b.    Create a view which will allow three of the above tables to be joined as one, so as to provide information for the college bulletin. The information required should include Program Code, Program Name,  Course Sequence, Course Number, and Course Name.        {04}

20c.    Assuming the existence of a program with code “P01” and name “BS. in Mathematics,” extract from the system, the program structure for this program with courses listed in order of sequence number. The data fields required are outlined in 20b above.             {04}

Figure 26-12. Except From a College Database

20d.    Establish a class-course-lecturer relationship, stored as a view, so that for each class, it will be known what course is offered, and who is the lecturer.      {04}

20e.    List all courses offered by the college.    {01}

20f.    Create a view for chemistry majors, which enables them to have read-only access to the chemistry curriculum only (assume that chemistry course codes are prefixed by the string “CHEM”).         {03}

21a.    Write SQL statement(s) to create a role called Developer and give it all the privileges required for application development.              {04}

21b.    Write SQL statement(s) to assign the Developer role to users GrantFord and HarryLim, and deny the user Stalker all privileges.              {03}

21c.    Assume that your database contains a table called Payroll, which has confidential information about the salary of employees. As the DBA, you want to restrict access to this table as follows: User JohnHenry must have query access only; user BruceJones must have query as well as modification access; no one else must have any access to the table. Write SQL statement(s) to realize this.      {06}

21d.    Write an SQL statement to query the Oracle 10G system catalog and provide a list of all tables in a given database (your display must include the name of the table, the owner of the table and the tablespace it belongs to).    {03}

21e.    Describe two limitations of SQL and briefly explain how they can be circumvented.        {04}

26.14  Sample Final Examination C

This examination consists of four (4) sections. You are required to do all questions in section A, and three additional questions — one from each of the other three sections.

Section A: Answer all questions in this section

State six (6) significant objectives of a database system.               {06}

State the components of a database system.        {05}

State four (4) primary functions of a DBMS.    {04}

Give the meaning of the acronyms DSL, DDL, DML and DCL. Briefly explain their importance in a database system.  Provide an example of a DSL.            {05}

DSL:

DDL:

DML:

DCL:

Example:

Give three (3) examples of popular and modern DBMS suites.                  {03}

State and briefly clarify three (3) types of relations that are likely to be found in a database.                  {03}

Clarify the terms relation, foreign key, candidate key, and degree.              {04}

Relation:

Foreign Key:

Candidate Key:

Degree:

State four (4) types of objects that are typically defined in a database that supports SQL.      {04}

State two (2) strong points about SQL and describe two (2) limitations of the language.      {04}

What are the two dominant contemporary models for database design, implementation and management?                      {02}

Section B: Answer one question from this section

Precision Software Inc. is a software engineering firm contracted to develop and implement a Human Resource Management System (HRMS) for Zealot Industries Inc. After a preliminary meeting with Zealot’s senior management, the lead software engineer on the Precision Software team documents his findings as follows:

A detailed employee profile consists of the employee’s personal information, employment history, education history and beneficiary information.

Every employee belongs to a department, and is assigned a specific job description (there may be more than one employees with the same job description).

Employees are classified according to the salary range that they fall in.

Each employee has a compensation package which outlines basic salary and other benefits. Benefits include health insurance, retirement plan, life insurance, professional development allowance, housing allowance, traveling allowance, entertainment allowance, vacation allowance, and education allowance.

A payroll log keeps track of remuneration paid out to employees.

Each employee is assigned to at least one project; the number of employees assigned to a project depends on the size and complexity of the project.

11a.    Identify all information entities (object types) mentioned or implied in the case.        {05}

11b.    Identify all relationships among the entities (or object types) and represent them on an ERD or ORD.    {15}

A scientific research est.  For each experiment, the following is recorded:

Project number

Project manager

Project name

Experiment number

Experiment name

Lab in which experiment is to be conducted

List of scientists to work on the project

Note: A project manager may work on an experiment. A scientist may work on more than one experiment, provided that they are done in the same lab. A manager may manage more than one project.

From the information given, use the XR model to derive a set of BCNF relations for the system.

Use self-explanatory relation and attribute names (you may introduce new attributes).            {20}

Section C: Answer one question from this section

Examine the section of a college/university database, shown in Figure 26-13, and answer one question from this section.

Write relational calculus statements that will produce the results specified below:

13a.    List all students (name, ID, and gender) who share surname with at least another student      {03}

13b.    List all program-names and related course-names  {04}

13c.    List the names of all programs that include the course (code) ‘CS100’        {03}

13d.    List of the names of all programs that do not include the course (code) ‘ENGL101’      {03}

13e.    List all Canadian (country code ‘CAN’) students (name, ID, Hall-name) enrolled in the B.S. in Computer Science (program code ‘P11SC’)         {04}

13f.    List the name of each department along with the name of its chairperson         {03}

Write relational algebra statements that will produce the results specified below:

Write relational calculus statements that will produce the results specified below:

14a.    List all students (name, ID, and gender) who share surname with at least another student    {03}

14b.    List all program-names and related course-names  {04}

14c.    List the names of all programs that include the course (code) ‘CS100’        {03}

14d.    List of the names of all programs that do not include the course (code) ‘ENGL101’        {03}

14e.    List all Canadian (country code ‘CAN’) students (name, ID, Hall-name) enrolled in the B.S. in Computer Science (program code ‘P11SC’)       {04}

14f.    List the name of each department along with the name of its chairperson          {03}

Section D: Answer one question from this section

Examine the section of a college/university database, shown in Figure 26-13, and answer one question from this section.

Write SQL statements that will produce the results specified below:

15a.    List all students (name, ID, and gender) who share surname with at least another student       {03}

15b.    List all program-names and related course-names  {04}

15c.    List the names of all programs that include the course (code) ‘CS100’        {03}

15d.    List of the names of all programs that do not include the course (code) ‘ENGL101’        {03}

15e.    List all Canadian (country code ‘CAN’) students (name, ID, Hall-name) enrolled in the B.S. in Computer Science (program code ‘P11SC’)     {04}

15f.    List the name of each department along with the name of its chairperson        {03}

16a.    State three (3) advantages of logical views.      {03}

16b.    State three (3) scenarios that could render a logical view non-updateable.            {03}

16c.    Referring to the college database of Figure 26-13, define a view that “stores” computer science courses only (assume that all computer science course-codes begin with the acronym ‘CS’).        {03}

16d.    Assuming that the program code for Computer Science is ‘P11SC’, construct a second logical view for computer science majors only. Your view must include all attributes of the student record, the full descriptive name of the computer science program, and the descriptive name of the hall dormitory that the students reside in.     {05}

16e.    Provide the SQL statement that will create the Student table of Figure 26-13, with its primary key and foreign keys appropriately defined (assume that referenced tables have been previously defined).        {06}

Figure 26-13. Section of a College Database

PART G

Appendices

This final division contains three review topics from your data structures and software engineering courses:

Appendix 1 — Review of Trees

Appendix 2 — Review of Hashing

Appendix 3 — Review of Information Gathering Strategies

APPENDIX 1

Review of Trees

This appendix provides a brief review of trees. You should pay specific attention to the section on B-trees, since most DBMS suites implement them by default. Note: This appendix is not meant to replace a full course (and text) in data structures. It should therefore be regarded as an overview, not a final authority on the subject matter. This review covers the following sub-topics:

Introduction to Trees

Binary Trees

Threaded Binary Trees

Binary Search Trees

Height-Balanced Trees

Heaps

M-way Search Trees and B-trees

Summary and Concluding Remarks

A1.1 Introduction to Trees

The main difference between O(N²) sorting algorithms and O(N log N) sorting algorithms is that the latter repeatedly reduce (by approximately one half) the number of keys remaining to be compared with each other, while the former does not. Trees are excellent sources of f (N log N) sorts.

As you are no doubt aware, in computer science, we use trees (and graphs) to represent and implement complex data structures. Here is a working definition of a tree: A tree T is a finite set of nodes (V₁, V₂ …Vn) such that:

There is one designated node called the root

The remaining nodes are partitioned into M ≥ 0 disjoint sets T₁, T₂ …Tn such that each Ti is itself a tree.

Except for the root, each node has a parent node.

Figure A1-1. An Example of a Tree

Here are a few additional conventions about trees:

The leaves are the terminal nodes.

Parent nodes are said to have siblings.

A forest is a combination (abundance) trees.

Level, Height and Weight of a Tree

The root is at level zero (0).

The height is therefore determined by the formula

Weight = Number of leaves

A1.2 Binary Trees

A1.2.1 Overview of Binary Trees

A binary tree is a tree in which each node is either empty or consists of two disjoint binary trees: the left sub-tree and the right sub-tree. Figure A1-2 illustrates this concept.

Figure A1-2. Illustrating a Binary Tree

Note:

No parent has more than two children.

A full (also called perfect) binary tree is a binary tree that contains the maximum number of nodes possible. Figure A1-3 shows a complete binary tree.

A complete binary tree is a binary tree that is full down to the penultimate level, and with nodes at the final level filled in from left to right.

Figure A1-3. Illustration of a Full Binary Tree and Complete Binary Tree

The maximum number of nodes of a tree may be determined as follows:

Number of nodes at the ith level is 2i

Maximum number of nodes = ∑ [0..k] of 2i

Observe that this is a geometric series of first term 1 and common ration 2 (review your discreet mathematics). The sum is therefore 2k - 1. So for a binary tree of height k,

Example 1: In the example above (Figure A1-3), maximum nodes = 2³ – 1 = 7.

A1.2.2 Representation of Binary Trees

Binary trees may be represented by arrays or preferably linked lists. Figure A1-4a illustrates a tabular representation of a binary tree while Figure A1-4b illustrates the graphic representation.

Figure A1-4a. Tabular Representation of a Binary Tree

Figure A1-4b. Graphic Representation of a Binary Tree

A1.2.3 Application of Binary Trees

Binary trees are typically applied in the following ways:

Binary trees (and extensions of binary trees) are used extensively in database management systems (DBMS) and operating system (OS) construction to effectively manage indexes of data files.

Binary trees are used in calculation of expressions during compilation.

Binary trees are used in data compressions, example, Huffman Coding Tree.

A1.2.4 Operations on Binary Trees

The following are the main operations that are normally defined on a binary tree:

Creation: At creation the tree either has no nodes or a dedicated root-node.

Insertion: We may allow insertion at the root, after terminal nodes only, or we may allow insertion anywhere in tree.

Deletion: Again, we may allow deletion of terminal nodes only, or we may allow deletion of nodes from anywhere in tree.

Clear: Remove all nodes from the binary tree, thus leaving it empty.

Check-Size: Return the size of the tree.

Check-Empty: Check if the tree is empty.

In-order: In-order traversal of the tree.

Pre-order: Pre-order traversal of the tree.

Post-order: Post-order traversal of the tree.

A1.2.5 Implementation of Binary Trees

Suppose that you desire a binary tree of LibraryPatron objects. The LibraryPatron class includes data items including patron number, name, major, and status; it also includes methods for manipulating instances of the class. You can implement this tree using an array, array-list, or linked-list. The preference is for the linked-list implementation, since it provides more flexibility. Figure A1-5 shows the UML diagrams for implementing this binary tree. It is assumed that you have mastery of fundamental programming principles; therefore, elaboration on these diagrams is not considered necessary. Note also that Figure A1-5c lists a number of methods that you may choose to implement. Except for traversal methods (the last three), you should be able to figure out the required logic for these methods ob your own. Figure A1-6 provides the pseudo-code for these methods. The logic for the traversal methods will be discussed in the next subsection.

Figure A1-5a. The UML Diagram of the LibraryPatron Class

Figure A1-5b. The UML Diagram of the PatronNode Class

Figure A1-5c. The UML Diagram of the PatronsBinaryTree Class

Figure A1-5d. The UML Diagram of the PatronsBinaryTreeMonitor Class

Figure A1-6. Binary Tree Algorithms

A1.2.6 Binary Tree Traversals

There are three traversal algorithms for binary trees: Pre-order, In-order and Post-order.

Pre-order traversal (also called prefix walk or polish notation) obeys the algorithm shown in Figures A1-7a and A1-7b.

Figure A1-7a. Summary of the Pre-order Traversal Algorithm

Figure A1-7b. Detailed Pre-order Traversal Algorithm

Example 2: Figure A1-7c provides an example of the application of the pre-order algorithm.

Figure A1-7c. Example of Pre-order Traversal Algorithm

In-order traversal (also called symmetric order or infix notation) obeys the algorithm shown in Figures A1-8a and A1-8b.

Figure A1-8a. Summarized In-order Traversal Algorithm

Figure A1-8b. Detailed In-order Traversal Algorithm

Example 3: Figure A1-8c provides an example of the application of the in-order algorithm.

Figure A1-8c. Example of In-order Traversal Algorithm

Post-order traversal (also called suffix walk or reverse polish notation) obeys the algorithm shown in Figures A1-9a and A1-9b.

Figure A1-9a. Summarized Post-order Traversal Algorithm

Figure A1-9b. Detailed Post-order Traversal Algorithm

Example 4: Figure A1-9c provides an example of the application of the post-order algorithm.

Figure A1-9c. Example of Post-order Traversal Algorithm

A1.2.7 Using Binary Tree to Evaluate Expressions

We can use binary trees to evaluate expressions by following a simple convention: All operands are leaves of the tree and the operators are parent nodes.

Example 5: The expression (a - b) * c is represented on a binary tree as shown in Figure A1-10a. To load an expression to tree, repeatedly select the operator that divides the expression into two, as the root. In the end leaves are operands and roots are operators.

Figure A1-10a. Using Binary Tree to Represent Arithmetic Expressions

Example 6: The expression [(((a+b)*c)/d) + ef]/g is represented on a binary tree as shown in Figure A1-10b. Note that in the figure the symbol ↑ is used for exponent.

Figure A1-10b. Using Binary Tree to Represent Arithmetic Expressions

Observe: You should be able to convince yourself that a post-order traversal of a binary tree for arithmetic expressions of this sort, produces the expression in postfix notation. This is left as an exercise for you.

A1.3 Threaded Binary Trees

In order to facilitate easy traversal of a binary tree, the tree may be threaded with pointers that explicitly show a traversal ordering. The threads link the nodes in the sequence of the traversal method.

Types of Threads:

Right thread links a node to its successor

Left thread links a node to its predecessor

A tree can only be threaded according to one traversal method at a time.

Threaded for In-order Traversal

To illustrate, if a binary tree is threaded for In-order processing, the following conventions are observed:

Left threads (except for leftmost leaf) point to predecessor nodes in in-order.

Left thread from leftmost leaf points to the root.

Right threads (except for rightmost leaf) point to successor nodes in in-order.

Right thread of rightmost leaf is NULL.

A1.4 Binary Search Trees

A binary search tree (BST) is a binary tree in which the following properties hold:

All nodes in the left sub-tree of Ri precede (by way of ordering) Ri so that, If Rj = Ri.Left, then Rj.Info < Ri.Info

All nodes in the right sub-tree of Ri succeed (by way of ordering) Ri so that, If Rj = Ri.Right, then Rj.Info >= Ri.Info

Note: In-order traversal of a BST produces a sorted list (in ascending values). Moreover, it can be shown that this sort algorithm is an O(N Log N) algorithm on the average.

A BST provides the following advantages:

It facilitates an O(N Log N) sort, which is more efficient sort than N² sort algorithms.

It facilitates faster (binary) search than searching a linear linked list.

The disadvantages associated with a BST are as follows:

It takes up more space than linear link list.

It can degenerate into a linked list (see next section).

As nodes are added, there is no control on the height (hence structure) of the tree. This is often undesirable.

Example 7: Suppose we wish to load the string HBXAM to a BST. We obtain a BST as follows:

Figure A1-11. Example of BST

Assuming the conventions of Figure A1-5, the algorithms of Figure A1-6 would still be applicable but with two extensions: We need to introduce an algorithm for inserting a node in the tree while preserving its properties. Secondly, we need to be able to search for a particular value in the tree. Figures A1-12 and A1-13 provide the insertion algorithm, and the search algorithm respectively. Note that the order in which nodes are added to the BST affects the structure of the tree. In fact, if a sorted list is added to the BST, it degrades to a linear linked list.

Figure A1-12a. Algorithm to Insert a Node in the BST

Figure A1-12b. Algorithm to Find the Insertion Point in the BST

Figure A1-13. Algorithm to Search the BST

Example 8: Adding the sorted list A B C D E F G to a BST results in the following:

Figure A1-14. Loading a Sorted List to a BST

Searching for a node in this BST would be reduced to searching a linear linked list! Solving this problem is not trivial. It is generally referred to as balancing the tree. Several algorithms have been proposed for balancing a BST.

A1.5 Height-Balanced Trees

A height-balanced k-tree (denoted (HB (k)) is a BST in which the maximum allowable difference in height between any two sub-trees sharing a common root (not necessary a parent) is k. Put another way, the maximum possible difference in height between leaves of the tree is k.

An AVL tree (named after Adelson-Velskii and Landis, the Russian founders) is a HB (1) tree, that is, the maximum possible difference in height between any two sub-trees sharing a common root is 1. Put another way, leaves are either at level m or level m + 1. Figure A1-14 provides some illustrations. Heaps (discussed next) are also examples of HB trees.

Figure A1-15. Illustrating AVL and non-AVL Trees

A1.6 Heaps

A heap is an almost complete binary tree, such that:

Every leaf of the tree is at level m or m + 1, where m is an integer (AVL requirement).

If a node has a right descendant at level l, it also has a left descendant at level l (converse, not necessarily true).

There is some established relationship between the parent-value and each child-value.

Figure A1-16 provides some illustrations. From the definition, observe that there are two types of heaps: max-heaps and min-heaps:

Max-heap: Every node stores a value that is greater than or equal to the value of either of its children. The root therefore stores maximum value of all nodes in the tree.

Min-heap: Every node stores a value that us less than or equal to the value of either of its children. The root therefore stores the minimum value of all nodes in the tree.

Figure A1-16. Illustrations of Heap

The heap-sort algorithm utilizes a max-heap. Heap-sort involves two phases:

Creating (building) the heap

Processing the heap

A1.6.1 Building the Heap

Figure A1-17 provides a summarized form of the algorithm to construct a heap.

Figure A1-17. Heap Construction Algorithm

Example 9: Figure A1-18 shows how a maximum heap is constructed with the following nodes:6 25 7 2 14 10 9 22 3 16.

Figure A1-18. Illustration of Heap Construction

A1.6.2 Processing the Heap (Heap Sort)

The algorithm for processing the heap (also called heap-sort because it produces a sorted list) is shown in Figure A1-19. The algorithm shown assumes a max-heap; it progressively removes the root of the heap until it is empty. Heap-sort performs as well as quick-sort on the average and better than quick-sort at worst; it is an O(N Log N) algorithm.

Figure A1-19. Heap-sort Algorithm

Example 10: Figure A1-20 illustrates how the heap of Example 9 would be processed to yield a sorted list.

Figure A1-20. Illustrating Heap-sort

A1.7  M-Way Search Trees and B-Trees

An m-way search tree is a tree in which each node has out-degree ≤ m. The out-degree of a node is the number of sub-trees that it has. Figure A1-21 illustrates an m-way search tree in which the root has out-degree 2, the left node has out-degree 3 and the right node has out-degree 3. The leaves (always) have out-degree 0. Figure A1-22 illustrates a 3–way search tree.

Figure A1-21. Illustrating an m-way Search Tree

The M-Way tree has the following properties (which are illustrated in Figure A1-22):

Each node is of the form where Po, P₁ … Pn are pointers to successor nodes and Ko, K₁ … Kn-1 are key values.

The key values in a node are in ascending order so that Ki ≤ Ki + 1.

All key values in nodes of the sub-tree pointed to by Pi are less than the key value Ki.

All key values in nodes of the sub-tree pointed to by Pi are greater than the key value ki-1.

The sub-tree pointed to by Pi are also m-way search trees.

Figure A1-22. A Three-way Search Tree

A1.7.1 Definition of B-tree

A B-tree of order m is an m-way search tree with the following properties:

Except for the root and leaves, each node of the tree has at least [m/2] sub-trees and no more than m sub-trees so that [m/2] ≤ number of sub-trees ≤ m. Note: [x] the smallest integer greater than x (e.g. [1.5] = 2).

The root of the tree has at least two sub-trees, unless it is itself a leaf.

All leaves of the tree are at the same level.

Figure A1-23 illustrates a B-tree of order 3, constructed from the 3-way search tree of Figure A1-22.

Figure A1-23. B-tree of order 3 (Corresponding to 3-way Search Tree)

A1.7.2 Implementation of the B-tree

The main operations we desire for the B-tree include:

Creation

Direct Search

Sequential Search

Insertion

Deletion

A possible implementation of a B-tree of LibraryPatron objects is shown in Figure A1-24. As you may well imagine, maintaining such a tree is much more challenging than a BST or heap, but by no means insurmountable. We will look more closely at some of these algorithms shortly.

Figure A1-24a. The UML Diagram of the PatronBTNode Class

Figure A1-24b. The UML Diagram of the PatronsBTree Class

Sequential search of the B-tree is achieved by an In-order traversal. Several values may be sought simultaneously. Note however, that internal nodes will be visited more than once since they contain several keys. Performance is therefore poor.

The direct search algorithm is shown in Figure A1-25. It is used to find a specific node in the tree. It is a very efficient algorithm. On the average, searching for an item among 1,000,000 items takes just about 20 comparisons! B-trees are also excellent for storing large volumes of data without deteriorating. For these reasons, B-trees are widely used as the file systems for compilers, database management systems, and operating systems.

Figure A1-25. Direct Search Algorithm

Figure A1-26 provides the summarized form of the B-tree insertion algorithm. This algorithm is easier described than implemented. Fortunately, you do not need to implement it in a typical data structures course. However, you do need to be able to demonstrate an understanding of the algorithm, so let us take an example:

Figure A1-26. Summary of B-tree Insertion Algorithm

Example 11: Referring to the B-tree of Figure A1-23, consider inserting the following items: 22, 41, 59, 57, 54. Figure A1-27 illustrates how these items would be added.

Figure A1-27. Illustrating Addition of Items to the B-tree

The summarized item deletion algorithm is shown in Figure A1-28. Note that as in the case of insertion, deletion may lead to significant adjustment of the tree.

Figure A1-28. B-tree Item Deletion Algorithm

Example 12: Referring to the B-tree of Figure A1-23, consider deleting the following items:

65 is in node p, remove it: p

7 is in node j, remove it:     j

16 is in node d, remove it. Move up 15 from node j and place a dummy (null) value at node j.

A1.8  Summary and Concluding Remarks

Here is a summary of what has been discussed in this chapter:

A tree has one root and zero or more other sub-trees connected to it. Except for the root, each node has a parent node.

A binary tree (BST) is a tree in which each node is either empty or consists of two disjoint binary trees: the left sub-tree and the right sub-tree. The BST may be traversed in-order, pre-order, or post-order.

A binary search tree is a binary tree in which all nodes to the left of a node are less then or equal to that node; all nodes to the right of a given node are greater than or equal to that node.

A height-balanced k-tree (denoted (HB (k)) is a BST in which the maximum allowable difference in height between any two sub-trees sharing a common root (not necessary a parent) is k. For AVL trees, k = 1.

A heap is an almost complete binary tree, such that: Every leaf of the tree is at level m or m + 1, where m is an integer; if a node has a right descendant at level l, it also has a left descendant at level l; there is some established relationship between the parent-value and its two children-values.

An m-way search tree is a tree of out-degree less than or equal to m, i.e. a maximum of m pointers point from a node. The tree has the following properties: Each node has a maximum of m pointers and m-1 key values; in any node, the key values are in ascending order; the pointer to the left of key value k[i] points to a node with values less than or equal to k[i]; the pointer to the right of key value k[i] points to a node with values greater than or equal to k[i].

A B-tree is a special type of m-way search tree with the following properties: Except for the root and leaves, each node of the tree has at least m/2 sub-trees and no more than m sub-trees; the root of the tree has at least two sub-trees, unless it is itself a leaf; all leaves of the tree are at the same level.

Trees (and particularly BSTs, heaps, and B-trees) are widely used in software engineering to solve various programming problems related to the organization and retrieval of data.

A1.9 References and/or Recommended Readings

[Carrano, 2006] Carrano, Frank and Janet Prichard. Data Abstraction & Problem Solving with Java. Boston, MA: Addison-Wesley, 2006. See Chapter 11.

[Carrano, 2007] Carrano, Frank. Data Structures & Abstraction with Java 2nd ed. Upper Saddle River, NJ: Prentice Hall, 2007. See Chapter 25 – 29.

[Drake, 2006] Drake, Peter. Data Structures and Algorithms in Java. Upper Saddle River, NJ: Prentice Hall, 2006. See Chapter 10.

[Folk, 1998] Folk, Michael, Bill Zoellick & Greg Riccardi. File Structures: An Object-Oriented Approach with C++, 3rd ed. Reading, Massachusetts: Addison-Wesley, 1998. See Chapters 8 and 9.

[Ford, 2005] Ford, William H. and William R, Topp. Data Structures with Java. Upper Saddle River, NJ: Prentice Hall, 2005. See Chapters 16 – 18, 22.

[Knuth, 1973] Knuth, D. The Art of Computer Programming volume 3, “Searching and Sorting”. Reading, Massachusetts: Addison-Wesley, 1973.

[Kruse, 1999] Kruse, Robert and Alex Ryba. Data Structures and Program Design in C++ 1st ed. Upper Saddle River, New Jersey: Prentice Hall, 1999.

[Langsam, 2003] Langsam, Yedidya, Moshe Augenstein, and Aaron M. Tanenbaum. Data Structures Using Java. Upper Saddle River, NJ: Prentice Hall, 2003. See Chapter 5.

[Main, 2006] Main, Michael. Data Abstraction & Other Objects Using Java. Boston, MA: Addison-Wesley, 2006. See Chapters 9 and 10.

[Standish, 1980] Standish, T. A. Data Structure Techniques. Reading, Massachusetts: Addison-Wesley, 1980.

[Venugopal, 2007] Venugopal, Sesh. Data Structures Outside In with Java. Upper Saddle River, NJ: Prentice Hall, 2007. See Chapters 9 – 11.

[Weis, 2007] Weiss, Marl Allen. Data Structures and Algorithm Analysis in Java 2nd ed. Boston, MA: Addison-Wesley, 2007. See Chapter 4 and 7.

APPENDIX 2

Review of Hashing

Hashing is a technique for mapping data from a large set to limited space in a much smaller set. This appendix provides an overview of the subject. The appendix covers the fundamentals of hashing under the following captions:

Introduction

Hash Functions

Collision Resolution

Hashing in Java

Summary and Concluding Remarks

A2.1 Introduction

In computer science, it is always important to establish mapping functions between (what is called) the conceptual user view of data and actual physical storage reality in the computer system. The former is what the end-user sees; the latter is what is actually stored on the storage media. For reasons that will become clear in your courses in software engineering, database systems, and operating systems, the two perspectives are seldom identical.

Another equally important issue is how to find data that is stored. For example, consider a transaction file for a financial institution, with millions of records. As you can well imagine (based on earlier discussions), sequential access of records in this file, would be suitable for a non-interactive program running in batch mode. However, it would not be suitable for a program that needs to provide interactive responses to the end used, based on different customers. You certainly would not want to process this file relying solely on an array, linked list, queue, or stack. You could use a BST, a heap, or better yet, a B-tree! But how would you determine where on the storage media to actually store records of your file so they can be easily retrieved when needed? Hashing is one technique for addressing this problem.

Hashing is a technique that addresses the problems of where (on a physical storage medium) to store data and how to retrieve the data stored. In hashing, there is a predictable relationship between a key value used to identify a record, and the record’s location on a storage medium.

A hash function takes a key value, applies an algorithm to determine its location. Mathematically, we represent this as follows (Figure A2-1):

Figure A2-1. Basic Concept of Hashing

The hash function maps key values from a relatively large domain to a relatively smaller range of address locations. Because of this, collision may occur and must be resolved. Collision occurs when two or more keys map to the same address, as illustrated in Figure A2-2.

Figure A2-2. Illustrating Collision

Since two records cannot be stored in the same location, the collision must be resolved. Various collision resolution techniques have been proposed. These will be discussed shortly.

A2.2 Hash Functions

A good hash function should exhibit two important features: Firstly, it should be easy and fast to compute. Secondly, it must attempt to scatter the key values (random or non-random) evenly over the address space. Various hash functions have been proposed and tested with varying results. Among the commonly discussed hash functions are the following: absolute addressing, direct table lookup, division-remainder, mid-square, folding, and truncation. We will briefly discuss each technique.

A2.2.1 Absolute Addressing

In absolute addressing, the address is the key value. Thus,

This is a simplistic approach to hashing; it will only work for trivial cases. In most practical cases, this approach will be inadequate, since storage will be limited.

A2.2.2 Direct Table Lookup

In direct table lookup, the following steps are taken:

The addresses are initially generated via some method. The method used to generate these addresses could be any of the techniques discussed in this section, or some other technique.

The keys and addresses are stored as a separate index (directory).

To access a record, the index is first consulted to determine its address.

Figure A2-3 provides an example of an index lookup table for an employee file. The assumptions here are as follows:

The file is keyed on the employee’s identification number.

Each employee record occupies 128 bytes of space.

A means of determining the address for each record is in place.

The notation TnSn is used to denote the track number and sector number on a hard drive (for instance T1S1 means track 1, sector 1).

Each sector can store up to 4096 bytes of data.

Figure A2-3. Example of Table Lookup for Employee Records

The technique is applied as follows:

When a record is first written, its address is determined and logged into the index (lookup) table.

When a record is to be retrieved, the index is consulted for its exact location.

The technique has three significant advantages: Table lookup is a very efficient way of storing small to medium sized files. Additionally, the technique is simple and easy to implement. Finally, the technique avoids the problem of collision (by presumably addressing it up front). For these reasons, the technique is widely used in database management system (DBMS) suites and operating systems.

The approach has two problems: Firstly, storing a data file requires management of two separate physical files on the storage medium. Secondly, as the size of the data file grows, so does the size of the index (table lookup) file.

A2.2.3 Division-Remainder

The division-remainder technique prescribes that you divide the key value by an appropriate number, then use the remainder of the division as the address for the record. In computer science, we refer to the operation that yields the remainder of an integer division as the modulus (commonly abbreviated as mod). Thus,

Note:

If the divisor is N, then address space must be of minimum size N (that is 0 to N-1).

To minimize the number of collisions, the divisor is typically a large number that does not contain any prime factors less than 20. For example, 997 or 1011 is preferred to 1024.

The technique is applied as follows: When a record is to be stored or retrieved, its address is first determined via the hash function.

The technique has three significant advantages:

It is simple and easy to implement.

It is very efficient for small files.

It avoids using two physical files for a single data file.

The technique has three significant disadvantages that discourage widespread use as a primary hash function:

It offers no guarantee against collision (though it can be minimized by carefully choosing the divisor).

It does not perform very well in attempting to distribute keys evenly across the address space.

As the size of the data set increases, so does the likelihood of collision.

A2.2.4 Mid-Square

In the mid-square technique, the key value is squared, and then a specific number of digits is extracted from the “middle” of the result to yield the hash address. If an address space of N digits is desired, then digits are truncated at both ends of the squared key value, leaving N digits from the middle.

As an example, suppose that we need a 4-digit address, and we have a key value of 7895. The address could be determined as follows:

Like the division-remainder technique, when a record is to be stored or retrieved, its address is first determined via the hash function.

The technique is a slight improvement over the division-remainder technique, providing the following advantages:

It is simple and easy to implement.

It is very efficient for small files as well as large files.

It performs reasonably well in attempting to distribute the keys evenly over the address space.

It avoids using two physical files for a single data file.

The technique has one significant disadvantage: It offers no guarantee against collision (but the performance is better than the division-remainder technique).

A2.2.5 Folding

The folding technique involves partitioning the key into several parts and combining the parts in a convenient way (via multiplication, addition or subtraction) to obtain a hash address.

To illustrate this technique, suppose that the folding method is addition and the address space is for four digits. Then, we may derive a hash address from the key value of 625149 via one of the following means:

The advantages and disadvantages of folding are similar to those of the mid-square technique. Figure A2-4 shows a comparison of how both techniques forms on data set of key-values. For this particular data set, folding appears to distribute the key-values more gracefully over the address space. On the other hand, the mid-square technique appears to distribute the key-values more randomly over the address space. Of course, this simple experiment is not enough for us to make conclusion as to how the techniques will perform on other data sets.

Figure A2-4. Comparing Folding with Mid-square on a Data Set

A2.2.6 Truncation

In the truncation technique, part of the key is ignored, and the other part is used as the hash address. To illustrate this technique, suppose that the address space is for four digits. Then, we may derive a hash address from the key value of 625149 via one of the following means:

Truncation, like absolute addressing is too simplistic for practical purposes. It could result in repeated collisions, and it does not come close to distributing the keys evenly over the address space.

A2.2.7 Treating Alphanumeric Key Values

In many cases, key values will be alphanumeric instead of numeric. For these situations, you will need to convert the alphanumeric data to numeric form. There are several ways to do this, so there is no need to panic. Figure A2-5 provides a summary of two commonly used techniques that have been proposed. Of course, you can come up with your own.

Figure A2-5. Converting Alphanumeric Keys to Numeric

A2.3 Collision Resolution

As mentioned earlier, when hash functions are applied, collisions are likely to occur. Three collision resolution techniques are prevalent: linear probing, synonym chaining, and rehashing.

A2.3.1 Linear Probing

The linear probing strategy prescribes that whenever a collision occurs, the empty location that is closest to the hash address is found and used. If the end of the table is reached, wrap-around is allowed. Figure A2-6 illustrates this. In the figure, collision has been resolved for nodes D, E, and G.

Figure A2-6. Illustrating Linear Probing

The load factor f is defined as the number of records (to be) stored in the table to the actual size of the table. Thus,

As items are added to the table, blocks of occupied cells start forming. This is referred to as primary clustering. The result is increased likelihood of collision and lengthened probes for subsequent items to be inserted. As the table fills up these searches become longer.

For insertions and unsuccessful searches, the expected number of probes is ½[1 + 1/(1 – f)²]. For successful searches, the expected number of probes is ½[1 + 1/(1 – f)]. The derivation of these formulae is beyond the scope of this course.

Linear probing has the following advantages:

The technique is conceptually simple and easy to implement.

The amount of calculation done at each stage is very small.

Disadvantages of linear probing include the following:

Primary clustering — the hash-table fills up in clusters, rather than in even distribution over the address space.

Primary clustering leads to longer search and increased likelihood of collision, particularly as the file becomes full.

Linear probing sometimes described as an open addressing strategy. Another commonly discussed open addressing strategy is quadratic probing. Since its performance is not much different from linear probing, it will not be discussed any further. Rehashing is another open addressing strategy; it will be discussed shortly.

A2.3.2 Synonym Chaining

The synonym chaining technique uses the hash address, not as the actual storage location for the record, but as the index into a list (implemented as a linked list or array-list). This way, we can facilitate multiple records with the same hash address! Figure A2-7 provides a graphic representation of this approach.

Figure A2-7. Graphic Representation of Synonym Chaining

Programmatically, the hash table may be implemented via any of the following strategies (you are encouraged to try implementing this on your own):

An array of linked lists or queues

An array-list of linked lists or queues

An array-list of array-lists

Observe:

The table size s is the number of linked lists (chains), not the number of nodes.

The average length of each chain is n/s where n is the number of records (nodes).

The load factor s is therefore the average length of each chain.

An unsuccessful search will involve the examination of f items; a successful search will require an average of 1 + f/2 examinations. So as expected, the performance is significantly better than linear probing.

Synonym chaining has the following advantages:

The technique reduces access time for records with collision addresses.

It is relatively easy to insert nodes in a given chain.

There is even distribution of keys (nodes) over address space.

The technique has two main disadvantages:

Additional space is required for the linked lists. However, since linked lists are dynamic structures, this is not significant.

Linked lists cannot be accessed randomly; they are sequential access structures.

A2.3.3 Rehashing

Rehashing is another open addressing strategy that significantly reduces clustering. The technique involves the application of a second hash function to resolve any collision that might have occurred from the first hash function. In principle, rehashing can continue to other levels until collision is resolved. However, in practice, if you have to hash more than twice to resolve collision, you probably need to change your hash function(s).

The main advantage of rehashing is that it tends to produce an even spread of key values over the address space. One significant disadvantage is that records may be moved some distance from their home address, thus violating the locality principle.

A2.4 Hashing in Java

You can develop and implement hash functions in just about any programming language. However, as usual, Java provides you with some nice features that facilitate this. In particular, there are three Java classes that you should be familiar with — the Hashtable, HashMap, and the TreeMap classes. These classes reside in the java.util package.

The Hashtable class implements a hash-table that maps keys to values. A key can be any valid object. Each instance of Hashtable class has two properties that affect its performance: the initial capacity and load factor. The capacity is the number of buckets in the hash table, and the initial capacity is simply the capacity at the time the hash table is created. A bucket is simply a holding area for key values that have the same address (similar to a chain); the bucket is searched sequentially. The load factor is a measure of how full the hash table is allowed to get before its capacity is automatically increased. The conventional wisdom is 0.75. Figure A2-8 provides the UML diagrams for the class.

Figure A2-8. UML Diagram for the Hastable Class

The HashMap class is roughly equivalent to Hashtable, except that it is unsynchronized and permits nulls. Additionally, there is no guarantee that the order of the hash-table will remain constant over time.

The TreeMap class implements a non-synchronized map (similar to a hash-table), in which keys are sorted. Figure A2-9 provides the UML diagrams for the class.

Figure A2-9. UML Diagram for the TreeMap Class

A2.5 Summary and Concluding Remarks

Let us summarize what we have covered in this chapter:

A hash function takes a key value, applies an algorithm to determine its location.

The hash function maps key values from a relatively large domain to a relatively smaller range of address locations. Because of this, collision may occur and must be resolved.

Among the commonly used hashing techniques are absolute addressing, direct table lookup, division-remainder, mid-square, folding, and truncation.

Three commonly used collision resolution strategies are linear probing, synonym chaining, and rehashing.

In linear probing, consecutive address locations are checked until a free location is found for insertion, or the sought item is found.

In synonym chaining, each hash address is a reference to a linked list of items that have that hash address.

In rehashing, successive hash functions are applied until the collision is resolved.

Java facilitates management of hash tables via the Hashtable, HashMap, and TreeMap classes.

Hashing is widely used in the implementation of filing systems for compilers, operating systems and database management systems.

A2.6 References and/or Recommended Readings

[Carrano, 2006] Carrano, Frank and Janet Prichard. Data Abstraction & Problem Solving with Java. Boston, MA: Addison-Wesley, 2006. See Chapter 13.

[Carrano, 2007] Carrano, Frank. Data Structures & Abstraction with Java 2nd ed. Upper Saddle River, NJ: Prentice Hall, 2007. See Chapter 19 and 20.

[Drake, 2006] Drake, Peter. Data Structures and Algorithms in Java. Upper Saddle River, NJ: Prentice Hall, 2006. See Chapter 11.

[Ford, 2005] Ford, William H. and William R, Topp. Data Structures with Java. Upper Saddle River, NJ: Prentice Hall, 2005. See Chapter 19 and 21.

[Kruse, 1994] Kruse, Robert. Data Structures and Program Design 3rd ed. Eaglewood Cliffs, New Jersey: Prentice Hall, 1994. See Chapter 9.

[Langsam, 2003] Langsam, Yedidya, Moshe Augenstein, and Aaron M. Tanenbaum. Data Structures Using Java. Upper Saddle River, NJ: Prentice Hall, 2003. See Chapter 4.

[Lewis, 1982] Lewis, T.G. and M.Z. Smith. Applying Data Structures 2nd ed. Hopell, New Jersey: Houghton Miflin, 1982. See Chapter 8.

[Shaffer, 1997] Shaffer, Clifford A. A Practical Introduction to Data Structures and Algorithm Analysis. Upper Saddle River, New Jersey: Prentice Hall, 1997. See Chapter 10.

[Venugopal, 2007] Venugopal, Sesh. Data Structures Outside In with Java. Upper Saddle River, NJ: Prentice Hall, 2007. See Chapter 12.

[Weis, 2007] Weiss, Marl Allen. Data Structures and Algorithm Analysis in Java 2nd ed. Boston, MA: Addison-Wesley, 2007. See Chapter 5.

APPENDIX 3

Review of Information Gathering Techniques

This appendix provides a brief review of information gathering techniques. You would normally learn these techniques in a course in software engineering or systems analysis. Mastery of the techniques discussed will enable you to readily identify information entities, as you model and design a database system. Note, this appendix is not meant to replace a full course (and text) in software engineering or systems analysis. It should therefore be regarded as an overview, not a final authority on the subject matter.

In order to accurately and comprehensively specify the system, the software engineer gathers and analyzes information via various methodologies. This Chapter discusses these methodologies as outlined below:

Rationale for Information Gathering

Interviews

Questionnaires and Surveys

Sampling and Experimenting

Observation and Document Review

Prototyping

Brainstorming and Mathematical Proof

Object Identification

Summary and Concluding Remarks

A3.1 Rationale for Information Gathering

What kind of information is the software engineer looking for? The answer is simple, but profound: You are looking for information that will help to accurately and comprehensively define the requirements of the software to be constructed. The process is referred to as requirements analysis, and involves a range of activities that eventually lead to the deliverable we call the requirements specification (RS). In particular, the software engineer must determine the following:

Synergistic interrelationships of the system components: This relates to the components and how they (should) fit together.

System information entities (object types) and their interrelatedness: An entity refers to an object or concept about data is to be stored and managed.

System operations and their interrelatedness: Operations are programmed instructions that enable the requirements of the system to be met. Some operations are system-based and may be oblivious to the end user; others facilitate user interaction with the software; others are internal and often operate in a manner that is transparent to the end user.

System business rules: Business rules are guidelines that specify how the system should operate. These relate to data access, data flow, relationships among entities, and the behavior of system operations.

System security mechanism(s) that must be in place: It will be necessary to allow authorized users to access the system while denying access to unauthorized users. Additionally, the privileges of authorized users may be further constrained to ensure that they have access only to resources that they need. These measures protect the integrity and reliability of the system.

As the software engineer embarks on the path towards preparation of the requirements specification, these objectives must be constantly borne in mind. In the early stages of the research, the following questions should yield useful pointers:

WHAT are the (major) categories of information handled? Further probing will be necessary, but you should continue your pursuit until this question is satisfactorily answered.

WHERE does this information come from? Does it come from an internal department or from an external organization?

WHERE does this information go after leaving this office? Does it go to an internal department or to an external organization?

HOW and in WHAT way is this information used? Obtaining answers to these questions will help you to identify business rules and operations.

WHAT are the main activities of this unit? A unit may be a division or department or section of the organization. Obtaining the answer to this question will also help you to gain further insights into the operations.

WHAT information is needed to carry out this activity? Again here, you are trying to refine the requirements of each operation by identifying its input(s).

WHAT does this activity involve? Obtaining the answer to this question will help you to further refine the operation in question.

WHEN is it normally done? Obtaining the answer to this question will help you to further refine the operation in question by determining whether there is a time constraint on an operation.

WHY is this important? WHY is this done? WHY…? Obtaining answers to these probes will help you to gain a better understanding of the requirements of the software system.

Of course, your approach to obtaining answers to these probing questions will be influenced by whether the software system being researched is to be used for in-house purposes, or marketed to the public. The next few sections will examine the commonly used information gathering strategies.

A3.2 Interviewing

Interviewing is the most frequent method of information gathering. It can be very effective if carefully planned and well conducted. It is useful when the information needed must be elaborate, or clarification on various issues is required. The interview also provides an opportunity for the software engineer to win the confidence and trust of clients. It should therefore not be squandered.

Steps in Planning the Interview

In panning to conduct an interview, please observe the following steps:

Read background information.

Establish objectives.

Decide whom to interview.

Prepare the interviewee(s).

Decide on structure and questions.

Basic Guidelines for Interviews

Figure A3-1 provides some guidelines for successfully planning and conducting an interview. These guidelines are listed in the form of a do-list, and a don’t-list.

Figure A3-1. Basic Guidelines for Interview

A3.3 Questionnaires and Surveys

A questionnaire is applicable when any of the following situations hold:

A small amount of information is required of a large population.

The time frame is short but a vast area (and/or dispersed population) must be covered.

Simple answers are required to a number of standard questions.

Guidelines for Questionnaires

Figure A3-2 provides a set of basic guidelines for preparing a questionnaire.

Figure A3-2. Guidelines for Questionnaires

Using Scales in Questionnaires

Scales may be used to measure the attitudes and characteristics of respondents, or have respondents judge the subject matter in question. There are four forms of measurement as outlined below:

Nominal Scale: Used to classify things. A number represents a choice. One can obtain a total for each classification.

Ordinal Scale: Similar to nominal, but here the number implies ordering or ranking.

Example 1:

Interval Scale: Ordinal with equal intervals.

Example 2: Usage of a particular software product by number of modules used (10 means high):

Ratio Scale: Interval scale with absolute zero.

Example 3: Distance traveled to obtain a system report:

Example 4: Average response time of the system:

Administering the Questionnaire

Options for administering the questionnaire include the following:

Convening respondents together at one time.

Personally handing out blank questionnaires and collecting completed ones.

Allowing respondents to self-administer the questionnaire at work and leave it at a centrally located place.

Mailing questionnaires with instructions, deadlines, and return postage.

Using the facilities of the World Wide Web (WWW), for example e-mail, user forums and chat rooms.

A3.4 Sampling and Experimenting

Sampling is useful when the information required is of a quantitative nature or can be quantified, no precise detail is available, and it is not likely that such details will be obtained via other methods. Figure A3-3 provides an example of a situation in which sampling is relevant.

Figure A3-3. Examining the Delivery of Orders After Customer Complaints

Sampling theory describes two broad categories of samples:

Probability sampling involving random selection of elements.

Non-probability sampling where judgment is applied in selection of elements.

A3.4.1 Probability Sampling Techniques

There are four types of probability sampling techniques:

Simple Random Sampling uses a random method of selection of elements.

Systematic Random Sampling involves selection of elements at constant intervals. Interval = N/n where N is the population size and n is the sample size.

Stratified Sampling involves grouping of the data in strata. Random sampling is employed within each stratum.

Cluster Sampling: The population is divided into (geographic) clusters. A random sample is taken from each cluster.

Note: Techniques (b), (c), (d) constitute Quasi Random Sampling. The reason for this is that they are not regarded as perfectly random sampling.

A3.4.2 Non-Probability sampling Techniques

There are four types of non-probability sampling techniques:

Convenience Sampling: Items are selected in the most convenient manner available.

Judgment Sampling: An experienced individual selects a sample (e.g. a market research).

Quota Sampling: A subgroup is selected until a limit is reached (e.g. every other employee up to 500).

Snowball Sampling: An initial set of respondents is selected. They in turn select other respondents; this continues until an acceptable sample size is reached.

A3.4.3 Sample Calculations

Figure A3-4 provides a summary of the formulas that are often used in performing calculations about samples:

Figure A3-4. Formulas for Sample Calculations

These formulas are best explained by examining the normal distribution curve (Figure A3-5). From the curve, observe that:

Prob (-1 <= Z <= 1) = 68.27%

Prob (-2 <= Z <= 2) = 95.25%

Prob (-3 <= Z <= 3) = 99.73%

Figure A3-5. The Normal Distribution Table

The confidence limit of a population mean is normally given by X' +/- ZSE where Z is determined by the normal distribution of the curve, considering the percentage confidence limit required. The following Z values should be memorized:

68% confidence => Z = 1.64

95% confidence => Z = 1.96

99% confidence => Z = 2.58

The confidence limit defines where an occurrence X may lie in the range (X' - ZSE) < X < (X' + ZSE), given a certain confidence. As you practice solving sampling problems, your confidence in using the formulas will improve.

A3.5 Observation and Document Review

Review of source documents will provide useful information about the input, data storage, and output requirements of the software system. This activity could also provide useful information on the processing requirements of the software system. To illustrate, get a hold of an application form at your organization, and attempt to identify information entities (object types) represented on the form. With a little thought, you should be able to identify some or all of the following entities:

Personal Information

Family/Kin Contact Information

Education History

Employment History

Professional References

Extra Curricular Activities

Internal documents include forms, reports and other internal publications; external documents are mainly in the form of journals and other professional publications. These source documents are the raw materials for gaining insights into the requirements of the system, so you want to pay keen attention to them.

With respect to observation, it is always useful to check what you have been told against what you observe and obtain clarification where deviations exist. Through keen observation, the software engineer could gather useful information not obtained by other means.

A3.6 Prototyping

In prototyping, the user gets a “trial model” of the software and is allowed to critique it. User responses are used as feedback information to revise the system. This process continues until a software system meeting user satisfaction is obtained.

The following are some basic guidelines for developing a prototype:

Work in manageable modules.

Build prototype rapidly.

Modify prototype in successive iterations.

Emphasize the user interface—it should be friendly and meeting user requirements.

Kinds of Prototypes

There are various types of prototypes that you will find in software engineering. Among the various categories are the following:

Patched-up Prototype or Production Model: This prototype is functional the first time, albeit inefficiently constructed. Further enhancements can be made with time.

Non-operational or Interactive Prototype: This is a prototype that is intended to be tested, in order to obtain user feedback about the requirements of the system represented. A good example is where screens of the proposed system are designed; the user is allowed to pass through these screens, but no actual processing is done. This approach is particularly useful in user interface design (see Chapter 6).

First of Series Prototype: This is an operational prototype. Subsequent releases are intended to have identical features, but without glitches that the users may identify. This prototype is typically used as a marketing experiment: it is distributed free of charge, or for a nominal fee; users are encouraged to use it and submit their comments about the product. These comments are then used to refine the product before subsequent release.

Selected Features Prototype or Working Model: In this prototype, not all intended features of the (represented) software are included. Subsequent releases are intended to be enhancements with additional features. For this reason, it is sometimes referred to as evolutionary prototype. The initial prototype is progressively refined until an acceptable system is obtained.

Throw-away Prototype: An initial model is proposed for the sole purpose of eliciting criticism. The criticisms are then used to develop a more acceptable model, and the initial prototype is abandoned.

A3.7 Brainstorming and Mathematical Proof

The methodologies discussed so far all assume that there is readily available information which when analyzed, will lead to accurate capture of the requirements of the desired software. However, this is not always the case. There are many situations in which software systems are required, but there is no readily available information that would lead to the specification of the requirements of such systems. Examples include (but are not confined to) the following:

Writing a new compiler

Writing a new operating system

Writing a new CASE tool, RAD tool, or DBMS

Developing certain expert systems

Developing a business in a problem domain for which there is no perfect frame of reference

For these kinds of scenarios, a non-standard approach to information gathering is required. Brainstorming is particularly useful here. A close to accurate coverage of the requirements of an original software product may be obtained through brainstorming: a group of software engineering experts and prospective users come together, and through several stages of discussion, hammer out the essential requirements of the proposed software. The requirements are then documented, and through various review processes, are further refined. A prototype of the system can then be developed and subjected to further scrutiny.

Even where more conventional approaches have been employed, brainstorming is still relevant, as it forces the software engineering team to really think about the requirements identified so far, and ask tough questions to ascertain whether the requirements have been comprehensively and accurately defined.

Mathematical proofs can also be used to provide useful revelations about the required computer software. This method is particularly useful in an environment where formal methods are used for software requirements specification. This approach is often used in the synthesis of integrated circuits and chips, where there is a high demand for precision and negligible room for error.

A3.8 Object Identification

We have discussed six different information-gathering strategies. As mentioned in section A3.1, these strategies are to be used to identify the core requirements of the software system. As mentioned then, one aspect that we are seeking to define is the set of information entities (object types). Notice that the term information entity is used as an alternative to object type. The two terms are not identical, but for most practical purposes, they are similar. An information entity is a concept, object or thing about which data is to be stored. An object type is a concept, object or thing about which data is to be stored, and upon which a set of operations is to be defined.

In object-oriented environments, the term object type is preferred to information entity. However, as you have seen, in most situations, the software system is likely to be implemented in a hybrid environment as an object-oriented (OO) user interface superimposed on a relational database.

Early identification of information entities is critical to successful software engineering in the OO paradigm. This is so because your software will be defined in terms of objects and their interactions. For each object type, you want to be able to describe the data that it will host, and related operations that will act on that data. Approaching the software planning in this way yields a number of significant advantages; moreover, even if it turns out that the software development environment is not object-oriented, the effort is not lost (in light of the previous paragraph).

Several object identification techniques have been proposed by the software engineering industry. Among the approaches that have been proposed are the following:

Using Things to be Modeled

Using Definitions of Objects, Categories and Types

Using Decomposition

Using Generalizations and Subclasses

Using OO Domain Analysis or Application Framework

Reusing Individual Hierarchies, Objects and Classes

Using Personal Experience

Using the Descriptive Narrative Approach

Using Class-Responsibility-Collaboration Card

Using the Rule-of-Thumb Method

These methodologies are discussed in the companion text on software engineering (see [Foster, 2010]). To get you adjusted to the idea of object identification, two of the approaches are summarized here.

A3.8.1 The Descriptive Narrative Approach

To use the descriptive narrative approach, start with a descriptive overview of the system (if it is small), or each component subsystem (in the case of a large or complex system). From each descriptive overview, identify nouns (objects) and verbs (operations). Repeatedly refine the process until all nouns and verbs are identified. Represent nouns as object types and verbs as operations, avoiding duplication of effort.

To illustrate, the Purchase Order and Receipt Subsystem of an Inventory Management System might have the following overview (Figure A3-6):

Figure A3-6. Descriptive Narrative of Purchase Order and Invoice Receipt Subsystem

From this narrative, an initial list of object types and associated operations can be constructed, as shown below (Figure A3-7). Further refinement would be required; for instance, additional operations may be defined for each object type (left as an exercise); also, the data description can be further refined.

Figure A3-7. Object Types and Operations for Purchase Order and Invoice Receipt Subsystem

A3.8.2 The Rule-of-Thumb Approach

As an alternative to the descriptive narrative strategy, you may adopt an intuitive approach as follows: Using principles discussed earlier, identify the main information entities (object types) that will make up the system. Most information entities that make up a system will be subject to some combination of the following basic operations:

ADD: Addition of data items

MODIFY: Update of existing data items

DELETE: Deletion of existing data items

INQUIRE/ANALYZE: Inquiry and/or analysis on existing information

REPORT/ANALYZE: Reporting and/or analysis of existing information

RETRIEVE: Retrieval of existing data

FORECAST: Predict future data based on analysis of existing data

Obviously, not all operations will apply for all object types (data entities); also, some object types (entities) may require additional operations. The software engineer makes intelligent decisions about these exceptions, depending on the situation. Additionally, the level of complexity of each operation will depend on the object type (data entity).

In a truly OO environment, the operations may be included as part of the object’s services. In a hybrid environment, the information entities may be implemented as part of a relational database. The operations would be implemented as user interface objects (windows, forms, etc.).

A3.9 Summary and Concluding Remarks

Here is a summary of what we have covered in this Chapter:

It is important to conduct a research on the requirements of a software system to be developed. By so doing, we determine the synergistic interrelationships, information entities, operations, business rules, and security mechanisms.

In conducting the software requirements research, obtaining answers to questions commencing with the words WHAT, WHERE, HOW, WHEN and WHY is very important.

Information gathering strategies include interviews, questionnaires and surveys, sampling and experimenting, observation and document review, prototyping, brainstorming and mathematical proofs.

The interview is useful when the information needed must be elaborate, or clarification on various issues is required. The interview also provides an opportunity for the software engineer to win the confidence and trust of clients. In preparing to conduct an interview, the software engineer must be thoroughly prepared, and must follow well-known interviewing norms.

A questionnaire is viable when any of the following situations hold: A small amount of information is required of a large population; the time frame is short but a vast area (and/or dispersed population) must be covered; simple answers are required to a number of standard questions. The software engineer must follow established norms in preparing and administering a questionnaire.

Sampling is useful when the information required is of a quantitative nature or can be quantified, no precise detail is available, and it is not likely that such details will be obtained via other methods. The software engineer must be familiar with various sampling techniques, and know when to use a particular technique.

Review of source documents will provide useful information about the input, data storage, and output requirements of the software system. This activity could also provide useful information on the processing requirements of the software system.

Prototyping involves providing a trial model of the software for user critique. User responses are used as feedback information to revise the system. This process continues until a software system meeting user satisfaction is obtained. The software engineer should be familiar with the different types of prototypes.

Brainstorming is useful in situations in which software systems are required, but there is no readily available information that would lead to the specification of the requirements of such systems. Brainstorming involves a number of software engineers coming together to discuss and hammer out the requirements of a software system.

Mathematical proof is particularly useful in an environment where formal methods are used for software requirements specification.

One primary objective of these techniques is the identification and accurate specification of the information entities (or object types) comprising the software system.

Accurate and comprehensive information gathering is critical to the success of a software engineering venture. In fact, the success of the venture depends to a large extent on this. Your information fathering skills will improve with practice and experience.

A3.10 References and/or Recommended Readings

[Daniel, 1989] Daniel, Wayne, and Terrel, James. Business Statistics for Management and Economics 5th ed. Boston, MA: Houghton Mifflin Co., 1989.

[DeGroot, 1986] DeGroot, Morris H. Probability and Statistics 2nd ed. Reading, MA: Addison-Wesley, 1986.

[Foster, 2010] Foster, Elvis. Software Engineering — A Methodical Approach. Bloomington, IN: Xlibris Publishing, 2010. See Chapter 5 and Appendix 4.

[Harris, 1995] Harris, David. Systems Analysis and Design: A Project Approach. Fort Worth, TX: Dryden Press, 1995. See Chapters 3, 4.

[Kendall, 1999] Kendall, Kenneth E. and Julia E. Kendall. Systems Analysis and Design 4th ed. Upper Saddle River, NJ: Prentice Hall, 1999. See Chapters 4 – 8.

[Long, 1989] Long, Larry. Management Information Systems. Eaglewood Cliffs, NJ: Prentice Hall, 1989. See Chapter 13.

[Pfleeger, 2006] Pfleeger, Shari Lawrence. Software Engineering Theory and Practice 3rd ed. Upper Saddle River, NJ: Prentice Hall, 2006. See Chapter 4.

[Sommerville, 2001] Sommerville, Ian. Software Engineering 6th ed. Reading, MA: Addison-Wesley, 2001. See Chapter 8.

[Sommerville, 2006] Sommerville, Ian. Software Engineering 8th ed. Reading, MA: Addison-Wesley, 2006. See Chapter 7.

[Van Vliet, 2000] Van Vliet, Hans. Software Engineering 2nd ed. New York, NY: John Wiley & Sons, 2000. See Chapter 9.

Index

images  A

The American National Standard Institute (ANSI) method

advantage

formats

modified From-Clause

ANSI method. See The American National Standard Institute (ANSI) method

images  B

Binary search tree (BST)

advantages and disadvantages

insertion algorithm

properties

search algorithm

Binary trees

algorithms

application

expressions

full and complete

in-order traversal

left and right sub-tree

LibraryPatron class

operations

post-order traversal

pre-order traversal

tabular and graphic representations

threads

Boyce-Codd normal form (BCNF)

BST. See Binary search tree (BST)

B-tree

definition

deletion algorithm

direct search algorithm

insertion algorithm

LibraryPatron objects

images  C

Catalog tables

commonly used catalog views

manipulation

in sample college database

Tab_Columns

User_Indexes

User_Tables

Codd’s twelve rules

images  D

Database administrator (DBA)

database backups

database removal

description

installation, creation and configuration

management

Oracle database (see Oracle database)

security mechanism

Database design

description

E-R model

O/ESG (see Object/entity specification grid (O/ESG))

XR model (see Extended relational (XR) model)

Database development life cycle (DDLC)

Database management system (DBMS)

application development subsystem (ADS)

components

data administration subsystem (DAS)

data communications manager (DCM)

data definition subsystem (DDS)

data dictionary (DD)

engine

functions

steps, application program

user interface subsystem (UIS)

utilities subsystem

Database model

description

entity-relationship (E-R) model

tools

XR model (see Extended relational (XR) model)

Database system (DBS)

advantages

approaches

architecture

components

DBA

DBMS

definition

features

front-end and back-end systems

inter-level mappings

levels of architecture

need, software category

objectives

SDLC and DDLC

Database tuning

components

database application

database contention

database design

goals

memory and I/O management

Database user interface design

category

command-based user interface

description

development and implementation

menu/graphical user interface

relational table

software development tool

Data dictionary. See System catalog

Data independence

Data mining

Data warehousing

architectures

basic

staging area and data marts

business requirement

construction

data modifications

definition

ETL

information systems (IS)

integration

nonvolatile

rules

schema design

subject oriented

time span, granularity and dimension

time-variant

typical operations

user constraints

workload

WWW and Web-accessible databases

DB2 suite

components

DB2 Universal Database Core

dominant database models

IBM Data Studio

IBM InfoSphere Information Server

IBM InfoSphere Warehouse

inventions

modern operating systems

shortcomings

UDB

“universal”

DB2 Universal Database Core

DB2 Universal Database (DB2 UDB)

advantages

components

data warehousing software

product editions

version 9.7

DBA. See Database administrator (DBA)

DBMS. See Database management system (DBMS)

DBS. See Database system (DBS)

DDLC. See Database development life cycle (DDLC)

DDL statements

Delphi

application building blocks

Borland Database Engine (BDE)

Component Library for Cross-Referencing (CLX)

components

database development environment

description

documentation

Enterprise Core Object (ECO) subsystem

Interactive Development Environment (IDE)

inventions

observed flaws

projects, types of

strategies

strength

Descriptive narrative approach

Distributed database systems

autonomous

benefits

catalog management

communication network

concurrency

connectivity

continuous operation

database gateways

DBMS suites

electronic communication systems (ECS)

fragmentation

hardware platforms

object technology (OT)

operating system and networking

query optimization

query processing

replication

transaction

transparency

update propagation

DKNF. See Domain-key normal form (DKNF)

DML and DCL statements

Domain-key normal form (DKNF)

Dynamic queries, SQL statements

application program

database administrator

description

front-end system

inquiry screen

tablespaces backup in a database

user

images  E

Editions, MS SQL Server

Compact Edition

Developer Edition

Enterprise Edition

Express Edition

Microsoft English Query

Microsoft SQL Server Analysis Services

Standard Edition

Workgroup Edition

Electronic communication systems (ECS)

Embedded SQL statements

database access

general convention

general syntax

PL/SQL program block

Entity integrity rule

Entity-relationship diagram (ERD)

Crows-Foot notation

symbols

ternary supplier-schedule relationship

ETL. See Extraction, transformation and loading (ETL)

Extended relational (XR) model

associations

characteristic entities

component entities

database design procedure

designative entities

entity classifications

E-relations and P-relations

integrity rules

kernel entities

operators

properties

subtype-super-type relationships

surrogates

Extensible markup language (XML)

Extraction, transformation and loading (ETL)

database systems and applications

DB2 tools

methodologies and tasks

Oracle tools

SQL statement

transactions

images  F, G

Fagin’s theorem

Fifth normal form (5NF)

Fagin’s theorem

join dependency (JD) constraint

SupplierSchedule relationship

First normal form (1NF)

Foreign key (FK)

Fourth normal form (4NF)

images  H

Hashing

absolute addressing

alphanumeric key values

collision resolution

linear probing

rehashing

synonym chaining

description

direct table lookup

division-remainder

folding

Java

Hashtable class and HashMap class

TreeMap class

key value

mid-square

transaction file

truncation

Heaps

construction algorithm

description

heap-sort

types

Height-balanced trees

HTTP. See Hypertext transport protocol (HTTP)

Hypertext transport protocol (HTTP)

images  I

IBM Data Studio

IBM InfoSphere Information Server

Information gathering techniques

brainstorming

description

interview

mathematical proofs

non-probability sampling techniques

object identification

observation and document review

probability sampling techniques

prototyping

questionnaires and surveys

sample calculations

software engineer

Insertion statement, SQL statements

abridged form

create-table statement

execution-time variable

facilitation

images  J, K

Java database connectivity (JDBC)

JDBC. See Java database connectivity (JDBC)

images  L

Limitations, SQL

calculated column

data manipulation

foreign key constraint

if-then, format

Order-By-Clause

programming limitations

referential integrity, superfluous enforcement

images  M

Materialized views, SQL

Alter-Materialized-View statement

Create-Materialized-View statement

description

in replication environments

master tables and databases

privileges

MS SQL Server

analysis services

client-server software system

components, suite

database support

default and named instance

default databases

default login

description

development tools

editions (see Editions, MS SQL Server)

installation and usage

integration services

interoperability

management tools

notification services

operating system

remove SQL Server

replication services

reporting services

shortcomings

SQL Server 7.0

T-SQL

MS SQL Server suite

client connectivity

code samples

development tools

management tools

server components

Multi-valued dependency (MVD)

MVD. See Multi-valued dependency (MVD)

M-way search trees

MySQL

advantages

characteristic services and features

components

description

editions

limitations

on joins and views

on server-side cursors

sub-queries

XA transactions

underlying features

images  N

Nested query, SQL statements

ANY(or SOME) and ALL operator

equal and IN operator

EXISTS and NOT EXISTS

multiple-row sub-query

Oracle employee-department database

rules

single-row sub-query

sub-query

Normalization

candidate keys

database design

description

functional dependence (FD)

mountaineering problem

non-loss decomposition (NLD)

procedure, normal forms

images  O, P

Object databases

hybrid approaches

object-oriented DBMS (OO DBMS)

object technology (OT)

Object/entity specification grid (O/ESG)

conventions

description

manufacturing environment

Object-oriented DBMS (OO DBMS)

Object-oriented software engineering (OOSE)

Object technology (OT)

ODBC. See Open database connectivity (ODBC)

Open database connectivity (ODBC)

Oracle 10G

Oracle 11G

Oracle database

backups

recovery

basic concept

rolling forward and rolling back

RMAN and user-managed methods

types of failures

Oracle Database Configuration Assistant (DBCA)

Oracle DBMS suite

advantages

components

description

editions

inventions

modern operating systems

OEM Database Control

Oracle Database Configuration Assistant (DBCA)

Oracle Developer

Oracle Enterprise Manager Grid Control

Oracle Server

OSQLD

OWB

PL/SQL and SQL *Plus

shortcomings

Oracle Enterprise Manager Grid Control

Oracle Enterprise Manager (OEM) Database Control

Oracle SQL Developer (OSQLD)

Oracle system catalog

referenced objects

views

Oracle Warehouse Builder (OWB)

images  Q

Queries involving multiple tables

ANSI method

traditional method

Queries involving SQL functions

aggregation functions

data conversion functions

date functions

programmer-defined functions

row functions

images  R

RAL. See Relation-attribute list (RAL)

Ramifications, relational model

Codd’s early benchmark

far reaching consequences

revised definition

Recovery Manager (RMAN)

Referential integrity rule

Relational algebra

Alias operation

assignment operation

objectives

operations

operators

primary and secondary operations

Codd’s original classifications

nested operations

rename operation

syntax

Cartesian product

division statement

natural join statement

projection statement

sample data

select statement

theta join

union and intersection and statements

Relational calculus

domain oriented relational calculus

explicit operations

free and bound variables

notation

quantifiers

QUEL

query optimization

salient syntactical components

SQL

substitution and standardization rules

well-formed formula

Relational model

attributes

basic concepts

categorization

database network

database tree (hierarchy)

definition

domains

ERD. See Entity-relationship diagram (ERD)

features

implementation

multiplicity

non-relational approaches

object-relationship diagram (ORD)

properties

RDBS

types of relationships

Relational Model, ramifications (see Ramifications, relational model)

Relation-attribute list (RAL)

Relationship list (RL)

Rule-of-thumb approach

images  S

Sample exercises and examination questions

SDLC. See Software development life cycle (SDLC)

Second normal form (2NF)

Shortcomings, DB2

affordability

backup and recovery capabilities

combined effect

documentation

support domain-based calculus

Shortcomings, MS SQL Server

load balancing capabilities

non-support stance on Java

no versioning support

performance and configuration

robustness and stability

support for MS Windows

transaction logs

Shortcomings, Oracle

affordability

code generation

combined effect

database management

Oracle Forms Developer (OFD)

PL/SQL support

SQL environment

Simple object access protocol (SOAP)

Sixth normal form (6NF)

Software development life cycle (SDLC)

SQL. See Structure query language (SQL)

SQL data manipulation statements

commit and rollback

core statements

delete and truncate

dynamic queries

embedded SQL

insertion statement

LIKE, BETWEEN and IN operator

nested query

queries involving multiple tables

queries involving set operators

queries, SQL functions (see Queries involving SQL functions)

runtime variables

select statement

set queries involving operators

set-transaction and save-point

simple query

SQL Plus format commands

update statements

SQL definition statements

alter-index statement

altering and dropping sequences

alter-table statement

create-index statement

creating and managing sequences

database creation

database management

drop-index statement

drop-table statement

managing synonyms and creation

Oracle

student table

table creation statement

tablespace creation

tablespace management

types, database objects

SQL view, limitations (see Limitations, SQL)

SQL views

and system security (see System security, SQL)

materialized views

traditional logical view (see Traditional logical view, SQL)

Structure query language (SQL)

advantages

DDL statements

description

DML and DCL statements

syntax convention

System catalog

benefits

catalog tables (see Catalog tables)

metadata

querying

updating

System security, SQL

access to data

database design

logical views

object privileges

access to resources

developer

development privileges

Grant and Revoke statement

objects

roles

access to the system

Alter-User statement

alter-View and drop-View statement

Bremar

Create-Profile statement

Create-User statement

InventoryProfile

profiles and user accounts

levels

images  T

Third normal form (3NF)

Traditional logical view, SQL

alter-View and drop-View statement

Cascade-Constraints-Clause

create and drop statements

create-view statement

CSCourses

data modification

derivable table

description

storage

usefulness and manipulation

virtual, named and derived

Transact-SQL (T-SQL)

Trees

binary (see Binary trees)

BST

level, height and weight

root

images  U, V

UML. See Unified modeling language (UML)

Unified modeling language (UML)

college database

database design procedure

development

marketing company

symbols

Update statements, SQL statements

abridged general formats

description

facilitation

multiple update

simple update

images  W, X, Y, Z

Web-accessible databases

ActiveX

benefits

business-to-business (B2B)

CGI and API

consumer-to-business (C2B)

cookies

DB2 implementation

description

front-end and back-end tools

HTTP

Java

JavaScript

Oracle implementation

plug-in

server-side extension

SOAP

three-tiered approach

two-tiered approach

VBScript

Web server

WSDL

XML

Web services description language (WSDL)

WSDL. See Web services description language (WSDL)