Metaprogramming is writing code that writes code.
There are a lot of ways of going about it, and a lot of purposes to which it can be put. This series is intended to capture my thoughts on metaprogramming, based on over ten years of experience with various ways of doing it.
My first metaprogramming experience came as a user of UML-based code generation tools, which were big in the '90's when UML was the silver bullet du jour. These tools all suffered from some fairly problematic features. In particular:
1) They didn't think like developers
2) They didn't act like developers
3) They didn't write code like developers
4) They barely read code at all
The last point is the most important, and still the most challenging for metaprogramming tools.
By the late '90's I was writing my own code generators, and since then have accumulated a great deal of experience on what works and what doesn't. This experience has brought me to the point where I sincerely believe we should be aiming at tools that can replace junior developers, although that is a goal we haven't quite reached yet.
The Industrial Revolution was successful not because it made certain occupations more efficient, but because it elimiated many occupations entirely. To take a recent example, there used to be a job position called a "computer". These were people--mostly women--who worked in teams to do complex calculations. Looking backward, we can see each one as little more than a single-operation chip, like a multiplier, say. They would take two numbers in, multiply them, and pass the result on to the next person in line.
Mainframes and then desktop computation didn't make these jobs easier: they wiped them out. This is the way mechanization should work, but we aren't there yet.
This is in part because much of what humans do is hard. CAD programs face this challenge. A typical CAD program replaces the draughting table. What we want it to do is replace the draughtsman, and eventually the architect. After all, there are only so many ways to put together a building. There are algorithms for layout, and simulations to optimize against expected use patterns. There will still be a role for humans in setting the goals and deciding the priorities and maybe working up the decor, but a huge number of the choices architects make can be automated, and any task that can be automated, should be.
The same is true of software development. The goal of metaprogramming is to bring us as close as possible to the ideal world of Grady Booch's Rule 122: Never write code unless you absolutely, positively must. If we can write code that writes code, we can get a lot closer to this ideal.
The problem with automating software development is the problem with automating any human task: we need to understand in detail what the humans who perform that task are actually doing. Most of the first generation of metaprogramming tools failed miserably at this, and so the tools didn't really satisfy anyone. Except in a few narrowly defined domains, metaprogramming has failed to realize its promise.
Currently, most practical metaprogamming systems are specifically for generative programming, which is mostly what I'll be talking about. But those systems all focus on making certain tasks that every individual developer does easier. This is the same model that CAD systems use: make the draughtsman's job easier by automating parts of it. While a good thing, this isn't the goal. The goal is to eliminate the entire job category.
One of the things that makes it difficult to eliminate entire categories of development jobs is that software development as it is currently practised is still incredibly heterogenous. That makes it hard to capture invariants that can be automated. However, within a given development context, it is possible to use metaprogramming to ensure that no developer will ever do certain jobs, effectively eliminating the category.
This is a akin to a CAD program that eliminates line-drawing from an architect or draughtsman's job. Imagine an program that could take a specification in terms of patterns of use, say, and generate plans on that basis. Likewise, imagine a program that could take a software specification in terms of desired behaviours and generate the code on that basis. We aren't there by a long shot, but there is no reason to suppose that we won't be there someday, and sooner rather than later.
There are a lot of ways of going about it, and a lot of purposes to which it can be put. This series is intended to capture my thoughts on metaprogramming, based on over ten years of experience with various ways of doing it.
My first metaprogramming experience came as a user of UML-based code generation tools, which were big in the '90's when UML was the silver bullet du jour. These tools all suffered from some fairly problematic features. In particular:
1) They didn't think like developers
2) They didn't act like developers
3) They didn't write code like developers
4) They barely read code at all
The last point is the most important, and still the most challenging for metaprogramming tools.
By the late '90's I was writing my own code generators, and since then have accumulated a great deal of experience on what works and what doesn't. This experience has brought me to the point where I sincerely believe we should be aiming at tools that can replace junior developers, although that is a goal we haven't quite reached yet.
The Industrial Revolution was successful not because it made certain occupations more efficient, but because it elimiated many occupations entirely. To take a recent example, there used to be a job position called a "computer". These were people--mostly women--who worked in teams to do complex calculations. Looking backward, we can see each one as little more than a single-operation chip, like a multiplier, say. They would take two numbers in, multiply them, and pass the result on to the next person in line.
Mainframes and then desktop computation didn't make these jobs easier: they wiped them out. This is the way mechanization should work, but we aren't there yet.
This is in part because much of what humans do is hard. CAD programs face this challenge. A typical CAD program replaces the draughting table. What we want it to do is replace the draughtsman, and eventually the architect. After all, there are only so many ways to put together a building. There are algorithms for layout, and simulations to optimize against expected use patterns. There will still be a role for humans in setting the goals and deciding the priorities and maybe working up the decor, but a huge number of the choices architects make can be automated, and any task that can be automated, should be.
The same is true of software development. The goal of metaprogramming is to bring us as close as possible to the ideal world of Grady Booch's Rule 122: Never write code unless you absolutely, positively must. If we can write code that writes code, we can get a lot closer to this ideal.
The problem with automating software development is the problem with automating any human task: we need to understand in detail what the humans who perform that task are actually doing. Most of the first generation of metaprogramming tools failed miserably at this, and so the tools didn't really satisfy anyone. Except in a few narrowly defined domains, metaprogramming has failed to realize its promise.
Currently, most practical metaprogamming systems are specifically for generative programming, which is mostly what I'll be talking about. But those systems all focus on making certain tasks that every individual developer does easier. This is the same model that CAD systems use: make the draughtsman's job easier by automating parts of it. While a good thing, this isn't the goal. The goal is to eliminate the entire job category.
One of the things that makes it difficult to eliminate entire categories of development jobs is that software development as it is currently practised is still incredibly heterogenous. That makes it hard to capture invariants that can be automated. However, within a given development context, it is possible to use metaprogramming to ensure that no developer will ever do certain jobs, effectively eliminating the category.
This is a akin to a CAD program that eliminates line-drawing from an architect or draughtsman's job. Imagine an program that could take a specification in terms of patterns of use, say, and generate plans on that basis. Likewise, imagine a program that could take a software specification in terms of desired behaviours and generate the code on that basis. We aren't there by a long shot, but there is no reason to suppose that we won't be there someday, and sooner rather than later.
