Capstone project: stepping back for a moment

We have a capstone project, a reworking in Clojure of some of the mashup features of Yahoo Pipes. At our last meeting, Luke VanderHart presented a framework for building and connecting components. I’m going to recap the framework as well as a few philosophical decisions we hammered out. But I’m also going to invite you to take a look at the code with me, with an eye towards 1) understanding each bit and 2) replaying the development of the framework by starting with simple building blocks. Luke and I both agree that the current framework, although awesome, has leapfrogged the kind of discussion and collaboration we want to foster in the study group. So I’m going to try an experiment with coding in public. On this blog, and soon on github, we’re going to build the framework again, together, feature by feature, always embracing the principle of the simplest thing that could possibly work. We’ll build up the might of Luke’s framework, but in a way that we get there together. And we’ll probably make it better from all being involved.

Unofficial Clojure 1.0 release candidate

Clojure 1.0 is on the way, and as a result a lot of activity is occurring in the Subversion repository.  Revision 1357 of the Clojure source was proclaimed the likely release candidate, although a few minor changes have been checked in since.  Likewise, clojure-contrib revision 723 is the likely release candidate corresponding to 1.0. It should be interesting to watch what unfolds in the source until the actual release.

It’s not too late to contribute, and no contribution is too small.

-m

Meeting recap, Pipes/Tubes/Clinks, and next meeting info

At our latest meeting, we discussed macros and adopted a capstone project, an open-source Clojure implementation of some of the functions of Yahoo Pipes.

Macros

Regarding macros, we touched on the chain of events when a Clojure program is evaluated: evaluation runs from top to bottom, and each S-expression is macro-expanded (if possible) and then evaluated. If an expression expands into something that itself contains a macro, the the new macro expression will be expanded before it is evaluated. This allows a macro to use a previously defined function in the determination of its expansion.

“Macros” in Lisp are very different from “macros” in other languages, as they allow the programmer to use all the power of Lisp to determine the macro’s expansion. Macros are not first-class members of the Clojure language, however. You cannot use them as you use functions–you can not use them with map or apply, for example–because macroexpansion happens before an expression can be used in this way.

Seeing stars

We also talked about “star” functions and macros. The macroexpansion of an expression that uses ‘and‘ includes something called let*:

(macroexpand '(and 1 2 3))
->(let* [and__2863 1]
(if and__2863 (clojure.core/and 2 3) and__2863))

We concluded that you really aren’t supposed to see let*. You only do in this case because macroexpand completely expands the expression it is given, and it turns out that let is a macro whose definition includes the mysterious let*. In fact, a lot of Clojure’s core macros use “star” constructs that appear to be special forms. Some digging at the REPL confirmed that they are not symbols. Likely the “star” forms are transformed by the Clojure parser directly into Java. If you have some special knowledge about this, please post a comment or join our discussion group. We’d like to know more.

Capstone project

Luke presented the case for a project that implemented some of the Yahoo Pipes application. By general acclamation we adopted this project as our capstone. We’ve decided to spend the next two weeks hacking on the basic problems of the project ourselves, keeping in touch through the group mailing list. David set up a github repo for us, with a wiki for collaboration. There you can find the summary of our initial work on defining the project, thanks to Keith. Serge dug up Yahoo’s Pipes documentation.

We’re calling this Tubes right now (as in “teh tubes”), but a final decision on the name (I’m not saying I don’t like Tubes) is one of many tasks we’re deferring. Right now we just want to get things moving. Personally, I expect Paul to have it all done by this afternoon.

Multimethods on Sunday, April 19th, 1pm

Our next meeting will be at least partly devoted to the final topic in Stu’s book: multimethods. Read chapter 8 of Programming Clojure and bring any questions or comments you’d like to discuss. Please notice two things about the next meeting date: 1) it is in two weeks, not three; and 2) it is on a Sunday. The location is still HacDC’s meeting space at St. Stephens, and the time is still 1pm. We hope to see you there.