It was an experiment -- and it went much better than I had imagined: I used Factor (a concatenative programming language) as the subject of study in a project week at Heilbronn University in a course called "Software Engineering of Complex Systems" (SECS). Maybe we are the first university in the world, where concatenative languages in general and Factor in specific are used and studied. Factor is the most mature concatenative programming language around. Its creator, Slava Pestov, and some few developers have done an excellent job.
Why concatenative programming? Why Factor?
Over the years I experimented with a lot of different languages and approaches. I ran experiments using Python, Scheme and also Prolog in my course. It turned out that I found myself mainly teaching how to program in Python, Scheme or Prolog (which still is something valuable for the students) instead of covering my main issue of concern: mastering complexity. In another approach I used XML as a tool for lightweight modeling to explore and study some techniques. The approach is innovative and still worth to be developed further but I wasn't satisfied.
My goal in the course "Software Engineering of Complex Systems" is to present and discuss practical techniques to conquer complexity in software systems. I did and still do a lot of research in this area. To make a long story short, I have come to the conclusion that Language-Driven Software Engineering (LDSE) is a very powerful and promising approach to conquer complexity. It's more than creating and using Domain Specific Languages (DSLs). It's consistently designing and creating languages throughout all levels and layers of a software implementation.
During my research I stumbled across Joy and Factor and learned about the concatenative paradigm. A series of excellent articles written by Manfred von Thun, creator of Joy, taught me the theory and fundamentals of concatenative languages. Factor, Slava Pestov's implementation of a practical concatenative language, turned out to be the best showcase I could think of: Factor is almost self-contained and extends itself by creating vocabularies of words using other vocabularies of words. Programs in Factor are written the very same style. Factor is seeing LDSE in action. I realized that it's the concatenative paradigm which enforces you to design software from a language-driven point of view.
How did the course go?
First I discussed the issue of complexity in software systems. Before I used Factor in the project week, I introduced the concatenative paradigm in the course. I presented it's mathematical foundation and used a pattern-driven approach to define the semantics of concatenative words as syntactic transformations. Finally, we defined a simplified grammar of Factor which we extended to cover pattern matching and template instantiation. All this served the purpose to smoothly prepare the ground for Factor. We also reflected and discussed a lot about what complexity is about and how it can be managed.
During the project week, my students (about 30 persons) worked with Factor four out of five days almost 8 hours per day. For my students it was full contact combat with an almost unknown programming paradigm and an exotic language. But they did really well. On day 1, the students worked through the introductory material supplied with Factor. On day 2, we studied types and object-orientation in Factor. On day 3, parsing and macros were studied in Factor. For these two days, the students worked with tutorials and worked their way through a number of exercise, which required them to write tiny programs in Factor to pass unit tests. On day 4, we worked on a topic unrelated to Factor. On day 5, two students thoroughly presented their project work, a real-world application in Factor, they had done in another course in the previous semester. We concluded the week with discussing and reflecting Factor's capabilities and the power of the concatenative paradigm in general.
Before I forget to mentioned it: Tim, research assistant in the software engineering department and PhD student, created the tutorials and the exercises and helped out a lot in class. Without him, the course wouldn't have been possible!
The students enjoyed the week very much. The evaluation of the course shows that they liked getting a new and different viewpoint on software development, object-orientation, parsing etc. They definitely realized and experienced that Factor helps them becoming better software engineers although Java is their main language.
Isn't Facor, aren't concatenative languages too esoteric to be useful?
Yes and no. There is no question that Factor is a niche language no one in industry shows interest for (besides Google, so far ;-). There might be some companies out there which use Forth and might be open for "concatenative thinking". However, even though the concatenative paradigm is almost unknown, concatenative languages are functional languages and functional languages are gaining in popularity. There's little doubt that learning functional programming broadens your scope and complements a student's skill set.
The fun part is that the concatenative approach to functional programming is much more simpler than the lambda calculus, which is traditionally taught. The math is simple and no intellectual barrier and formal transformations are easy to understand since there are no variable bindings and nesting scopes. Key concepts are stripped to their bare minimum. Did you ever try to explain the idea of continuations in Scheme? You might spend a good amount of time explaining continuations and running exercises. It's not unlikely that some students still don't get it. Continuations seem to be an extremely complex thing and appear to be somewhat mystical. In Factor, and in concatenative languages in general, continuations are a triviality! In principle, it's a snapshot of the data and the call stack. No big deal, since you juggle around with both stacks all the time. Are generic functions a specialty of CLOS. They come out naturally in a pattern-based approach to define concatenative words.
But my point goes beyond that. The way you create abstractions and refactor your programs in a concatenative language enforces you to continuously reflect about your design decisions. You have an enormous freedom of how you shape and constrain the design space of options at hand. It lets you think about words and vocabularies of words. It is thinking about creating and using languages. It combines software engineering and software programming in a way I haven't experienced in any other paradigm. That's why I introduced Factor in my course: You will start to engineer software, you'll explore new ways of creating abstractions and design frameworks.
Factor itself is an excellent case study for this approach. Factor starts from a relatively small kernel (which I -- admittedly -- haven't cleanly dissected, yet) and then consequently adds feature by feature with using Factor to extend Factor. A neat concatenative kernel turns itself into a powerful piece of software using a language-driven approach right from the start. Slava Pestov proves that this approach does result in a fast, interactive and highly reflective language. For me, Factor is a masterpiece of software engineering! It's definitely worth studying it!
Conclusion
What I experienced over the last two semesters is that some students become deeply attracted by Factor. Even if not, almost all students sense that there is a new world worth entering that takes them to a new level of understanding. It broadens their scope and skill set. Eventually, they'll leave the concatenative path for doing their Java/C# assignments in other courses or when they do some programming for a living. Still, I'm convinced that concatenative programming has an impact that lasts.
Do I sound too enthusiastic? Possibly, but I prefer to teach things I'm enthusiastic about! I'm still a student regarding the concatenative paradigm myself, I'm learning a lot each and every day about this paradigm. And one is for sure: I will continue to use Factor in the next semesters.
---
Update (2010-01-29) I received quite some requests to publish the Factor material we produced for the project week. The material is in German. Comments, ideas, corrections and improvements are welcome!
Day 1 - Intro: Getting started (Factor docu), Q&As
Day 2 - Object-Orientation: Intro, Tutorial, Q&As
Day 3 - Parsing and Macros: Intro, Tutorial, Q&As
Day 4 - Unrelated Topic:
Day 5 - Real-World Application in Factor: Presentation, Report, Sources (thanks to Andreas Maier and Marcel Steinle)
By the way, Daniel Ehrenberg indicated that Heilbronn University is not the first using Factor in a course. That's great to hear. Factor starts spreading!
In case you are interested in our research on concatenative languages, there is a paper available: "Concatenative Programming: An Overlooked Paradigm in Functional Programming".
Why concatenative programming? Why Factor?
Over the years I experimented with a lot of different languages and approaches. I ran experiments using Python, Scheme and also Prolog in my course. It turned out that I found myself mainly teaching how to program in Python, Scheme or Prolog (which still is something valuable for the students) instead of covering my main issue of concern: mastering complexity. In another approach I used XML as a tool for lightweight modeling to explore and study some techniques. The approach is innovative and still worth to be developed further but I wasn't satisfied.
My goal in the course "Software Engineering of Complex Systems" is to present and discuss practical techniques to conquer complexity in software systems. I did and still do a lot of research in this area. To make a long story short, I have come to the conclusion that Language-Driven Software Engineering (LDSE) is a very powerful and promising approach to conquer complexity. It's more than creating and using Domain Specific Languages (DSLs). It's consistently designing and creating languages throughout all levels and layers of a software implementation.
During my research I stumbled across Joy and Factor and learned about the concatenative paradigm. A series of excellent articles written by Manfred von Thun, creator of Joy, taught me the theory and fundamentals of concatenative languages. Factor, Slava Pestov's implementation of a practical concatenative language, turned out to be the best showcase I could think of: Factor is almost self-contained and extends itself by creating vocabularies of words using other vocabularies of words. Programs in Factor are written the very same style. Factor is seeing LDSE in action. I realized that it's the concatenative paradigm which enforces you to design software from a language-driven point of view.
How did the course go?
First I discussed the issue of complexity in software systems. Before I used Factor in the project week, I introduced the concatenative paradigm in the course. I presented it's mathematical foundation and used a pattern-driven approach to define the semantics of concatenative words as syntactic transformations. Finally, we defined a simplified grammar of Factor which we extended to cover pattern matching and template instantiation. All this served the purpose to smoothly prepare the ground for Factor. We also reflected and discussed a lot about what complexity is about and how it can be managed.
During the project week, my students (about 30 persons) worked with Factor four out of five days almost 8 hours per day. For my students it was full contact combat with an almost unknown programming paradigm and an exotic language. But they did really well. On day 1, the students worked through the introductory material supplied with Factor. On day 2, we studied types and object-orientation in Factor. On day 3, parsing and macros were studied in Factor. For these two days, the students worked with tutorials and worked their way through a number of exercise, which required them to write tiny programs in Factor to pass unit tests. On day 4, we worked on a topic unrelated to Factor. On day 5, two students thoroughly presented their project work, a real-world application in Factor, they had done in another course in the previous semester. We concluded the week with discussing and reflecting Factor's capabilities and the power of the concatenative paradigm in general.
Before I forget to mentioned it: Tim, research assistant in the software engineering department and PhD student, created the tutorials and the exercises and helped out a lot in class. Without him, the course wouldn't have been possible!
The students enjoyed the week very much. The evaluation of the course shows that they liked getting a new and different viewpoint on software development, object-orientation, parsing etc. They definitely realized and experienced that Factor helps them becoming better software engineers although Java is their main language.
Isn't Facor, aren't concatenative languages too esoteric to be useful?
Yes and no. There is no question that Factor is a niche language no one in industry shows interest for (besides Google, so far ;-). There might be some companies out there which use Forth and might be open for "concatenative thinking". However, even though the concatenative paradigm is almost unknown, concatenative languages are functional languages and functional languages are gaining in popularity. There's little doubt that learning functional programming broadens your scope and complements a student's skill set.
The fun part is that the concatenative approach to functional programming is much more simpler than the lambda calculus, which is traditionally taught. The math is simple and no intellectual barrier and formal transformations are easy to understand since there are no variable bindings and nesting scopes. Key concepts are stripped to their bare minimum. Did you ever try to explain the idea of continuations in Scheme? You might spend a good amount of time explaining continuations and running exercises. It's not unlikely that some students still don't get it. Continuations seem to be an extremely complex thing and appear to be somewhat mystical. In Factor, and in concatenative languages in general, continuations are a triviality! In principle, it's a snapshot of the data and the call stack. No big deal, since you juggle around with both stacks all the time. Are generic functions a specialty of CLOS. They come out naturally in a pattern-based approach to define concatenative words.
But my point goes beyond that. The way you create abstractions and refactor your programs in a concatenative language enforces you to continuously reflect about your design decisions. You have an enormous freedom of how you shape and constrain the design space of options at hand. It lets you think about words and vocabularies of words. It is thinking about creating and using languages. It combines software engineering and software programming in a way I haven't experienced in any other paradigm. That's why I introduced Factor in my course: You will start to engineer software, you'll explore new ways of creating abstractions and design frameworks.
Factor itself is an excellent case study for this approach. Factor starts from a relatively small kernel (which I -- admittedly -- haven't cleanly dissected, yet) and then consequently adds feature by feature with using Factor to extend Factor. A neat concatenative kernel turns itself into a powerful piece of software using a language-driven approach right from the start. Slava Pestov proves that this approach does result in a fast, interactive and highly reflective language. For me, Factor is a masterpiece of software engineering! It's definitely worth studying it!
Conclusion
What I experienced over the last two semesters is that some students become deeply attracted by Factor. Even if not, almost all students sense that there is a new world worth entering that takes them to a new level of understanding. It broadens their scope and skill set. Eventually, they'll leave the concatenative path for doing their Java/C# assignments in other courses or when they do some programming for a living. Still, I'm convinced that concatenative programming has an impact that lasts.
Do I sound too enthusiastic? Possibly, but I prefer to teach things I'm enthusiastic about! I'm still a student regarding the concatenative paradigm myself, I'm learning a lot each and every day about this paradigm. And one is for sure: I will continue to use Factor in the next semesters.
---
Update (2010-01-29) I received quite some requests to publish the Factor material we produced for the project week. The material is in German. Comments, ideas, corrections and improvements are welcome!
Day 1 - Intro: Getting started (Factor docu), Q&As
Day 2 - Object-Orientation: Intro, Tutorial, Q&As
Day 3 - Parsing and Macros: Intro, Tutorial, Q&As
Day 4 - Unrelated Topic:
Day 5 - Real-World Application in Factor: Presentation, Report, Sources (thanks to Andreas Maier and Marcel Steinle)
By the way, Daniel Ehrenberg indicated that Heilbronn University is not the first using Factor in a course. That's great to hear. Factor starts spreading!
In case you are interested in our research on concatenative languages, there is a paper available: "Concatenative Programming: An Overlooked Paradigm in Functional Programming".