Rethinking Agility in Databases

view as a single page

By Max Guernsey, III - Managing Member, Hexagon Software LLC

Introduction

Yesterday (May 8th, 2008), I did another Data Structure Metamorphism seminar. At the end, one of the people attending commented on the distinction I draw between structure and information. His contention was that there was value in the design, itself. He actually said that there is more value in the model than the content. Even though he changed my mind a little, I still would not take it that far.

The essence of his argument was correct, though: There is value in the structure. That’s why we have a database structure: it does something for us that we want done. The structure is still not as valuable as the content – so long as the content’s existence is not dependent on its structure – but I don’t think his real contention was with my valuing information over structure. I think it was with the implied effort to demote structure to a “second-class citizen.”

It all got me thinking: “maybe it’s time to reframe that distinction.”

Related Links: Introduction to this Series, Discussion Board (on Yahoo!), DataConstructor

view as a single page

By Max Guernsey, III - Managing Member, Hexagon Software LLC

Introduction

This episode is definitely a divergence from this series. At least, in a sense… in previous installments we talk about how Agile practices in the database world must differ from their software counterparts. Here, we will talk about how an Agile practice (XP really deserves the credit, if I’m not mistaken) can be ported straight across to databases as a way of reinforcing the other methodologies we’ve discussed.

While databases differ from programs significantly, the people who work on them do not. Programmers are people. Testers are people. Business analysts are people. Guess what? DBAs are people, too.

As you are probably aware, by virtue of monitoring this series of articles, we recently put on a seminar on changing Agility in the database world. This series went very well – better than we could have hoped, really – and one of the great unforeseen products was the set of questions asked.

One of the attendees seemed to be sold on the methodologies and raised a concern that I had not really considered. It’s kind of funny because I’ve dealt with it, I just didn’t consider it in all of my preparations, in the development of DataConstructor, or in collaboration with any of my friends. The problem is that he has to get buy-in from the people who currently have the keys to the databases.

Related Links: Introduction to this Series, Discussion Board (on Yahoo!), DataConstructor

view as a single page

by Max Guernsey, III - Managing Member, Hexagon Software LLC

Introduction

This installment breaks from the trend set by its predecessors a little bit. Instead of digging into a specific practice, we are going to look at a paradigm. The reason for this is that the thinking we are going to examine is one of the underpinnings for most of the Agile practices that we have developed.

We are going to talk about change; the kinds of change we have to address and how we embrace change with Agility and Lean. The concepts herein are linked to Part I (Evolution), but you can probably read either article in any order.

Related Links: Introduction to this Series, Part I: Evolution, Discussion Board (on Yahoo!), DataConstructor

view as a single page

by Max Guernsey, III - Managing Member, Hexagon Software LLC

Introduction

This article builds upon the concepts laid out in its predecessors. If you have not read the introduction, Part I, and Part II, it is recommended that you do so now.

For those you who have, here is a quick refresher on Part I and Part II of this series:

In Part I, we discussed evolutionary thinking in software development – a major player in emergent design – and found that it is a great fit for software but doesn’t really work for databases. A better analogy is metamorphosis, which also permits emergent design.

In Part II, we discussed deployment. We saw that most of the mechanisms used were fundamentally unreliable or manual. We hypothesized that this might be because we are trying to apply deployment models developed for software – where builds are reliable and generally happen as a separate step from deployment. Finally, we came to the conclusion that database deploys are database builds. Because of this fact, a “build in place” mentality might be more useful than a “deploy to a place” mindset.

In many respects, this installment is a continuation of Part II that “closes the loop” with Part I. We will cover how testing works in software and how it works in databases. Of course, we will find the differences, and see if there is a way we can reinvent the practice of testing software in the database domain.

Related Links: Introduction to this Series, Part I: Evolution, Part II: Deployment, DataConstructor

view as a single page

by Max Guernsey, III - Managing Member, Hexagon Software LLC

Introduction

If you haven’t read the introduction for this channel of articles, I highly recommend you do so now. For those who just need a brief reminder, the point is this:

In each article, we pick a practice used in traditional, program-centric Agility. We deconstruct that practice until we have reduced it to its essence: principles and values. Then, we put Humpty-Dumpty together again, with data in mind. The result should be a new practice which fills the same need as the one with which we started but is applicable to persistent data stores.

You will also need to understand the concepts from Part I: Evolution. If you have not already done so, you probably should read that now. If you just need a refresher, here is the gist:

Source code does and should evolve. Evolution in source code permits emergent design without which Agility could not function. However, the process of evolution operates on groups of things and functions by the destruction of one generation and creation of another. This is not possible with databases, because the information in an individual database has too much value in it to permit its destruction and replacement.

In this installment, we’re going to talk about deployment in the database world. In the software world, we have highly reliable deployment mechanisms and techniques. These are important because they let us distribute the software we write. If you cannot distribute a program to its users, it is very difficult to realize any of its potential value. While the techniques in the software world are highly refined and usually pretty reliable, the processes we use for databases are largely manual and/or unreliable. We are going to try and figure out why deployment of programs and components is so reliable and deployment of database schemas is, by comparison, unreliable.

Related Links: Introduction to this Series, Part I: Evolution, DataConstructor

view as a single page

by Max Guernsey, III - Managing Member, Hexagon Software LLC

Introduction

If you haven’t read the introduction for this channel of articles, I highly recommend you do so now. For those who just need a brief reminder, the point is this:

In each article, we pick a practice used in traditional, program-centric Agility. We deconstruct that practice until we have reduced it to its essence: principles and values. Then, we put Humpty-Dumpty together again, with data in mind. The result should be a new practice which fills the same need as the one with which we started but is applicable to persistent data stores.

In this installment we discuss what I think is one of the most central but least portable practices: evolutionary software development. Why is this so important? It's the practice that let's designs emerge. Without the ability to change our design for the better, we have to plan for every kind of variation; if not every variant. Why does it not work with data? Simple: databases can't evolve.

Related Links: Introduction to this Series, DataConstructor, Emergent Design (Bain)

Author: Max Guernsey, III - Managing Member, Hexagon Software LLC

Over the last decade or so, we've developed a number of practices to facilitate Agility. For example: test-driven development, design patterns, and Scrum. Many of us, including myself, have come to think of these things as components of Agility.

...but they are not. These practices are all just tools that we use to make Lean/Agile software development possible and effective. Yet, so long as the practices continue to work, there is no reason for us to challenge the analogy. At Hexagon Software, we believe that the practices used for software development do not facilitate Lean/Agile development of data stores.

In this series of articles, I'm going to try and deconstruct various practices of Agility into the underlying principles. To do this, we will rely heavily upon Lean which, to a large extent, has already done this through a number of other avenues. We will then take that essence of each successful behavior and rebuild from it new practices which are more applicable to data.

Related Links: DataConstructor