Thanks to Paul for pointing this out: the Pragmatic Programmers training series is offering a 3-day training course in Reston, VA next year (date TBD). Clojure’s creator Rich Hickey and Stuart Halloway will be instructing. More information is at the Pragmatic Programmers site.

It looks like the course will assume you know something about Lisp or Java and functional programming, and possibly concurrency, but nothing about Clojure. It’s going to go well beyond basics, though, and it’s a chance to ask Rick Hickey questions directly.

It’s going to be expensive, with the alumni/group discount rate still a hefty $1195. The early bird price is $1495. I’ve been to the Advanced Ruby course of the same dimensions and cost, and it was worth it. If you all want to join up, 3 or more of us can get the group rate and save a few hundred dollars each.

I’m chipping away at my Mashup project tasks, namely an OAuth library for Twitter and other providers of protected resources. But I wanted to pass on a concrete contribution, so here are some tips about taking advantage of Clojure’s availability through git, and how to put your clojure-contrib library in sync with the big Clojure 1.0 release.

Clojure first

Clojure 1.0 is released, and you can get it as a ZIP file:
http://clojure.googlecode.com/files/clojure_1.0.0.zip

Unzip it, and everything you need is there, including a JAR file, clojure_1.0.0.jar.

I had to rename a few things, since my CLASSPATH expects a JAR called simply ‘clojure.jar’ in a folder called ‘clojure’. After I renamed/backed up my existing clojure directory, I copied the unzipped clojure_1.0.0 folder into its place and renamed it. I also created a clojure.jar link, via ln to clojure_1.0.0.jar.

clojure-contrib

Now for the clojure-contrib. Stuart Sierra did us the favor of setting up a branch of clojure-contrib development that is intended to be compatible with the Clojure 1.0 release. Here’s Stuart’s announcement.

First, I needed to clone the git repo. I renamed/backed up my old clojure-contrib folder and then issued this command:

> git clone git://github.com/richhickey/clojure-contrib.git

This initialized a repository tracking the master branch. To get the latest 1.0-compatible code, I needed to track the branch ‘clojure-1.0-compatible’. Following advice from ENTP, I determined the form of the command to issue was:

git branch --track local origin/remote && git checkout local

For example, I decided to call my tracking branch ‘1.0′, so I issued:

> git branch --track 1.0 origin/clojure-1.0-compatible && git checkout 1.0

Then, to pull down a copy of this branch:

> git pull

That’s all there is too it. Then I was ready to build my JAR’s:

> ant -Dclojure.jar=../clojure/clojure.jar

First entirely painless Clojure update ever!

If you’re comfy with git, you’ll want to decide on whether to ignore the generated files or commit them. Of course, you can’t push to the remote source, so it really doesn’t matter.

Happy hacking!

I’m posting the code I’ve written for date management in the mashup project. It’s just a sketch, really. There’s plenty of work to do to fill it out. But I wanted to post what I’ve got now.

Thanks to Keith, for his suggestion I use the joda.time library.

Organization: namespaces

I’ve broken the code out into a couple of namespaces, and thus into different files. The files live in folders under a src directory that is on my development classpath. For instance, the date time utility functions in namespace org.clojurestudydc.mashups.util.datetime are in a file src/org/clojurestudydc/mashups/util/datetime.clj.

Datetime wrangling

So date parsing is a bitch because dates are expressed in so many formats. The joda.time library offers a way to parse datetime strings by describing them using different characters for different parts of the date and time. The details are here:
http://joda-time.sourceforge.net/api-release/org/joda/time/format/DateTimeFormat.html

The org.joda.time.format.DateTimeFormat class has a factory method forPattern that takes a descriptive string in this format and returns a DateTimeFormat object ready to use on datetime strings that match the description. In Clojure, to make a DateTimeFormat object that handles RSS 1.0 datetime strings:

user> (def rss-10-fmt (org.joda.time.format.DateTimeFormat/forPattern
                     "yyyy-MM-dd'T'HH:mm:ssZZ"))
#'user/rss-10-fmt

Then use the object’s parseDateTime method on a timestamp:

user> (.parseDateTime rss-10-fmt "2005-10-02T09:01:15-07:00")
#>DateTime 2005-10-02T12:01:15.000-04:00>

One serious shortcoming of the DateTimeFormat class is that you can’t ask it to parse a timezone expressed as an acronym (EDT, e.g.). So, I have to search timestamps for these and convert them into numerical timezone descriptors (e.g. -0400). So I wrote a function replace-timezone-abbreviation and a datamap timezones for this. (Code follows below.)

Then I built a vector of descriptions of datetime formats. I only put in two, for illustration purposes: RFC822, which RSS 2.0 holds to (mostly), and ISO8601, the closest format to RSS 1.0. Each format is expressed as a datamap with a name, a regex (:rpattern), and a DateTimeFormat pattern (:dpattern). The rest of the code I wrote builds up functions that can process a seq of datamaps and return a copy with each datamap’s value for a given key converted from a string to a joda.time.DateTime object. The code uses the formats’ regexes to select a format to use. I have placeholders in the code for “hints” of various kinds. The idea is that you could pass a format’s name, and the code would try that format out first. Pass a format, or seq of them, and the code would try to match with them first. Pass a function that can be applied to a string value to derive a joda.time.DateTime object, and the code would try applying it and only go on to the formats if some exception were returned.

OK, so here’s the code

(ns org.clojurestudydc.mashups.util.datetime
    (:import (org.joda.time DateTime DateTimeComparator)
	     (org.joda.time.format DateTimeFormat)))

(defn date-comparator [] (DateTimeComparator/getInstance))

(def timezones {"AST" "-0400" "ADT" "-0300" "EST" "-0500" "EDT" "-0400" "CST" "-0600" "CDT" "-0500"
"MST" "-0700" "MDT" "-0600" "PST" "-0800" "PDT" "-0700"
"AST" "-0900" "ADT" "-0800" "HST" "-1000" "HDT" "-0900"})

(defn replace-timezone-abbreviation [datestring]
	(let [ptn   (re-pattern (apply str (interpose "|" (keys timezones))))
	      match (re-find ptn datestring)]
	  (if match
	    (.replace datestring match (timezones match))
	    datestring)))

(defstruct date-format-entry :name :rpattern :dpattern)

(def date-format-entries
     [(struct date-format-entry :RFC822
             #"\w\w\w, \d\d \w\w\w \d\d\d\d \d\d:\d\d:\d\d"
	      "EEE, dd MMM yyyy HH:mm:ss ZZ")
      (struct date-format-entry :ISO8601
             #"\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\d-\d\d:\d\d"
	      "yyyy-MM-dd'T'HH:mm:ssZZ")])

(defn find-fmt-by-name [name & entries]
  (filter #(= name (:name %)) (concat date-format-entries (first entries))))

(defn find-fmt-by-regex-match [datestring & xtra-entries]
  (let [entries (concat date-format-entries (first xtra-entries))]
	(first (filter (complement #(empty?
                (re-seq (:rpattern %) datestring))) entries))))

(defn parse-string [datestring dpattern]
  (.parseDateTime (DateTimeFormat/forPattern dpattern)
                            (replace-timezone-abbreviation datestring)))

(defn parse-item [item datekey]
  (let [date-format-schema (find-fmt-by-regex-match (datekey item))]
    (if (nil? date-format-schema)
      item ;better to throw an exception
      (let [datestring (datekey item)
	    dpattern (:dpattern date-format-schema)
	    dvalue (parse-string datestring dpattern)]
	(assoc item datekey dvalue)))))

(defn parse-mapseq-date-values [mapseq datekey & hints]
  (map #(parse-item % datekey) mapseq))

Here’s how you’d use it on a seq of rss items like the cleaned-rss seq we built earlier:

user> (use 'org.clojurestudydc.mashups.util.datetime)
nil
user> (def date-corrected-rss
                     (parse-mapseq-date-values cleaned-rss :pubDate))
#'user/date-corrected-rss
user> (:pubDate (first date-corrected-rss))
#<DateTime 2009-06-23T14:37:54.000-04:00>

Sorting

And here’s how I use the datetime functions to sort. I have more code in a namespace under ‘….alterations.sorting’, as this code conceptually makes a copy of the structure with its items in a different order:

(ns org.clojurestudydc.mashups.alterations.sorting
    (:require [org.clojurestudydc.mashups.util.datetime :as dt  ]))
...

(defn configure-sort-by-date-fn [date-key]
	(let [dt-comparator (dt/date-comparator)]
	  (fn [coll] (sort #(.compare
               dt-comparator (date-key %2) (date-key %1)) coll))))

The function configure-sort-by-date-fn takes a key to use to locate datetime values in a datamap and returns a function that’s ready to sort a collection (seq) of datamaps. Here’s how it’s used:

user> (use 'org.clojurestudydc.mashups.alterations.sorting)
nil
user> (def sort-rss-items (configure-sort-by-date-fn :pubDate))
#'user/sort-rss-items
user> (map :pubDate (take 5 (sort-rss-items date-corrected-rss)))
(#<DateTime 2009-06-23T15:28:45.000-04:00>
 #<DateTime 2009-06-23T15:06:41.000-04:00>
 #<DateTime 2009-06-23T15:02:38.000-04:00>
 #<DateTime 2009-06-23T15:02:27.000-04:00>
 #<DateTime 2009-06-23T14:52:36.000-04:00>)

Marking structures generated by builders

As a closing note, another “hint” for a transformer like the datetime parser could be a metadata tag denoting the origin of the structure (and thus the likely format of its values). So here’s an edit to the RSS builder that marks our collection of RSS items with origin ‘rss:

(ns org.clojurestudydc.mashups.builders.rss)
...

(defn rss-reader [url]
  (let [xml (xml/parse url)
          zipper (zip/xml-zip xml)
          elements (-> zipper zip/down zip/children)
          items (filter #(= :item (:tag %)) elements)]
    (with-meta (map select-contents items)
	       {:origin 'rss})))

We’ll need something better than concat now to combine these maps: we have to combine their metadata too.

Just a quick post to mention how Clojure was mentioned at the RubyNation conference in Reston last weekend.

I gave a lightning talk on day 2. Slides are posted. I only had 6 minutes, so I kept it brief. I don’t think I convinced a lot of the audience members to try it out. Afterwards, though, I got to talk to Aaron Bedra of Relevance, who was a technical editor for Stu’s book. He reports Clojure is actually in use on commercial projects. He’s a swell fellow.

Russ Olsen gave an impromptu talk at the end of the day about Clojure, Scala, Erlang, and Reia. Two things struck me about his approach to new languages. First, he gives a lot of important to “curb appeal”, or how nice the code looks. The parentheses bother him about Clojure and other lisps, same old story. The second thing, though, is the importance given to metaprogramming, which Ruby has brought into the forefront of a lot of developers’ minds. As Russ pointed out, any Lisp makes metaprogramming easy. So that may be a selling point for Clojure among Rubyists.

We’re meeting face-to-face again on Saturday, May 16th at 1pm, at HacDC’s meeting space in DC. We’ll be meeting to hack some code on our mashup project. Check out the group’s google group for discussion and code produced so far.

We have a capstone project, a reworking in Clojure of some of the mashup features of Yahoo Pipes. At our last meeting, Luke VanderHart presented a framework for building and connecting components. I’m going to recap the framework as well as a few philosophical decisions we hammered out. But I’m also going to invite you to take a look at the code with me, with an eye towards 1) understanding each bit and 2) replaying the development of the framework by starting with simple building blocks. Luke and I both agree that the current framework, although awesome, has leapfrogged the kind of discussion and collaboration we want to foster in the study group. So I’m going to try an experiment with coding in public. On this blog, and soon on github, we’re going to build the framework again, together, feature by feature, always embracing the principle of the simplest thing that could possibly work. We’ll build up the might of Luke’s framework, but in a way that we get there together. And we’ll probably make it better from all being involved.

First off, we haven’t agreed on a name for the capstone project, so I’m just going to call it the clojure mashup project, or the mashup project, or just the project. Luke has named his repo “flowjure”, and there’s a lot of “tubes” and “pipes” in the namespaces, but that’s immaterial right now. Don’t worry about the names.

Next, you can find Luke’s code on github: http://github.com/levand/flowjure/tree/master

1. If you don’t have git installed, install it
2. Decide where you want to put the cloned repository. I have a ‘clojure-dev’ directory in my home directory.
3. clone the repository in that location: clojure-dev$> git clone git://github.com/levand/flowjure.git
4. Make sure your classpath — either your environment variable if you’re doing Clojure form the command line or in a configuration file for emacs or other editor — contains the path to flowjure/src/. In my case, that’s /Users/michael/clojure-dev/flowjure/src

Now let’s get started by going over a few things from the last meeting.

• The mashup project is component-based. For now, we’re trying out a strict policy of every component taking one or more inputs, either external resources like URIs or other project components, put only yielding one output, either a Clojure data structure or something intended for output from (and thus marking the completion of) the component chain.
• The component chain operates in a “pull” style, with each component in the chain asking the component(s) immediately “upstream” to give them data.
• Because of the amny-in/one-out structure of the components and the “pull” metaphor, the component chain can’t branch. Data funnels down to a single output

Furthermore, Paul Barry described the different kinds of objects in the project in five categories:

1. source (something external like an RSS feed or file),
2. builder (any format into internal representation),
3. collector/combiner (multiple inputs to one output),
4. transform (alter one input into one output),
5. formater/outputer (internal representation into any format)

With this in mind, let’s look as the RSS reader in the file src/flowjure/components/rss.clj:

(ns flowjure.components.rss
  (:require [clojure.xml :as xml]
             [clojure.zip :as zip])
  (:use flowjure.engine))

(defn create-map
  "Creates a map object from an RSS 'item' entry"
  [item]
  (reduce (fn [acc it]
            (assoc acc (keyword-to-str (:tag it))
              (first (:content it)))) {} (:content item)))

(def-component {:name "rss"
                :category "Input"
                :description "Loads an RSS Feed"
                :output-type "Record Sequence"
                :args {"url" {:doc "The URL of the RSS feed"
                              :type "String"
                              :min-required 1
                              :max-required 1}}}
  (fn [pipe pipe-args args]
    (let [xml (xml/parse (args "url"))
          zipper (zip/xml-zip xml)
          elements (-> zipper zip/down zip/children)
          items (filter #(= :item (:tag %)) elements)]
      (map create-map items))))

Part 1: Examining the code

Now, fire up a REPL, because I’m going to walk you through the execution of the meat of this code, the five lines at the end. We’re going to have to wrangle with namespaces a bit, and I’m going to advocate using the pretty printer in clojure.contrib during our REPL session.

We start out with the xml and zip namespaces already available to us, as using resolve demonstrates:

user> (resolve 'clojure.xml/parse)
#'clojure.xml/parse
user> (resolve 'clojure.zip/zipper)
#'clojure.zip/zipper
user> (resolve 'clojure.zip/down)
#'clojure.zip/down

The pretty-printer is in clojure.contrib, so we’ll need to get it into our current namespace with use:

user> (resolve 'clojure.contrib.pprint/pprint)
nil
user> (use 'clojure.contrib.pprint)
nil
user> (resolve 'clojure.contrib.pprint/pprint)
#'clojure.contrib.pprint/pprint

See http://bc.tech.coop/blog/081029.html for more basics on namespaces.

See http://code.google.com/p/clojure-contrib/wiki/PprintApiDoc for API docs on the pretty printer.

Back to the code. Here are the lines I’m interested in right now:

(let [xml (xml/parse url)
        zipper (zip/xml-zip xml)
        elements (-> zipper zip/down zip/children)
        items (filter #(= :item (:tag %)) elements)] ...)

Let’s call the lines in the function one at a time in the REPL.

First, use xml/parse to read the RSS available at a URL, let’s use “http://rss.cnn.com/rss/cnn_topstories.rss”, and convert it into a Clojure data map. That’s a powerful function!

user> (clojure.xml/parse "http://rss.cnn.com/rss/cnn_topstories.rss")
{:tag :rss, :attrs {:xmlns:media "http://search.yahoo.com/mrss/", :xmlns:feedburner "http://rssnamespace.org/feedburner/ext/1.0", :version "2.0"}, :content [{:tag :channel, :attrs nil, :content [{:tag :title, :attrs nil, :content ["CNN.com"]} {:tag :link, :attrs nil, :content ["http://www.cnn.com/?eref=rss
[snip]

Well, it’s a map, but it’s sure hard to read. Let’s take advantage of one of the pretty printers awesomer tools, the pp macro, which when called pretty-prints the last thing output (our ugly map)

user> (pp)
{:tag :rss,
 :attrs
 {:xmlns:media "http://search.yahoo.com/mrss/",
  :xmlns:feedburner "http://rssnamespace.org/feedburner/ext/1.0",
  :version "2.0"},
 :content
 [{:tag :channel,
   :attrs nil,
   :content
[snip]

That’s better, although this thing is enormous. The thing to look for is “:tag :item” — that denotes an item, the elements that we’re after.

[snip]
{:tag :item,
     :attrs nil,
     :content
     [{:tag :title,
       :attrs nil,
       :content ["Mexico City shuts down venues due to swine flu"]}
      {:tag :guid,
       :attrs {:isPermaLink "false"},
       :content
       ["http://www.cnn.com/2009/HEALTH/04/28/swine.flu/index.html?eref=rss_topstories"]}
      {:tag :link,
       :attrs nil,
       :content
       ["http://rss.cnn.com/~r/rss/cnn_topstories/~3/kjZP0YJPYI0/index.html"]}
[snip]

Now, Luke uses xml.zip/xml-zip to assemble the value for ‘zipper.’ What’s xml-zip do? Let’s check the API docs:

(xml-zip root)
Returns a zipper for xml elements (as from xml/parse), given a root element

Huh. OK. Maybe we should evaluate our parsed XML and see what we get.

user> (pprint (clojure.zip/xml-zip (clojure.xml/parse "http://rss.cnn.com/rss/cnn_topstories.rss")))
[{:tag :rss,
  :attrs
  {:xmlns:media "http://search.yahoo.com/mrss/",
   :xmlns:feedburner "http://rssnamespace.org/feedburner/ext/1.0",
   :version "2.0"},
  :content
[snip]

That looks pretty familiar. Zippers must print out as maps. We’ll have to take it on faith that internally it’s fancier than a map.

For convenience, let’s def a zipper variable to hold our Zipper.

user> (def zipper (clojure.zip/xml-zip (clojure.xml/parse "http://rss.cnn.com/rss/cnn_topstories.rss")))
#'user/zipper

Now we use the -> macro (called thread). Clojure’s English descriptions suffer from namespace collisions: thread, map — these things have multiple meanings. The thread macro is a way to line up functions:
(-> zipper clojure.zip/down clojure.zip/children) is the same as (clojure.zip/children (clojure.zip/down zipper)).

What do these do? According to the docs:

(children loc)
Returns a seq of the children of node at loc, which must be a branch

(down loc)
Returns the loc of the leftmost child of the node at this loc, or nil if no children

In other words, we get the children of the first child of the RSS XML root element as a seq. This is important: we have all the tag maps we want at the top level of the seq. We don’t have to do any digging now. This is what the seq looks like:

({:tag :title, :attrs nil, :content ["CNN.com"]} [snip] {:tag :item, :attrs nil, :content [{:tag :title, :attrs nil, :content ["HHS secretary confirmed in midst of outbreak"]}]} [snip])

Now we filter the tags for those with :tag value :item:

user> (def items (filter #(= :item (:tag %)) elements))
#'user/items
user> (pprint (first items))
{:tag :item,
 :attrs nil,
 :content
 [{:tag :title,
   :attrs nil,
   :content ["HHS secretary confirmed in midst of outbreak"]}
  {:tag :guid,
   :attrs {:isPermaLink "false"},
   :content
   ["http://www.cnn.com/2009/POLITICS/04/28/sebelius.confirmation/index.html?eref=rss_topstories"]}
  {:tag :link,
   :attrs nil,
   :content
   ["http://rss.cnn.com/~r/rss/cnn_topstories/~3/dcHTwVYr0QU/index.html"]}
  {:tag :description,
   :attrs nil,
   :content
   ["The Senate today confirmed Kansas Gov. Kathleen Sebelius as secretary of
health and human services on a 65-31 vote. Sebelius takes office as swine flu
numbers climb worldwide. But confirming Sebelius, who met several obstacles
during confirmation hearings, doesn't bring the health team to 100 percent.
There are still no appointees in place for any of the department's 18 key jobs.
[snip]"]}
  {:tag :pubDate,
   :attrs nil,
   :content ["Tue, 28 Apr 2009 10:10:23 EDT"]}
  {:tag :feedburner:origLink,
   :attrs nil,
   :content
   ["http://www.cnn.com/2009/POLITICS/04/28/sebelius.confirmation/index.html?eref=rss_topstories"]}]}

Next, the create-map helper function is applied to each element in the items collection (via map–again, many meanings) to get to the content in each of these {:tag ...} data maps.

Let’s just call the function’s body without defining it. And I’m removing the keyword-to-str call within Luke’s method to make things simpler:

user> (pprint (reduce (fn [acc it]
            (assoc acc (:tag it)
              (first (:content it)))) {} (:content (first items))))
{:feedburner:origLink
 "http://www.cnn.com/2009/POLITICS/04/28/sebelius.confirmation/index.html?eref=rss_topstories",
 :pubDate "Tue, 28 Apr 2009 10:10:23 EDT",
 :description
 "The Senate today confirmed Kansas Gov. Kathleen Sebelius as secretary of health and human services on a 65-31 vote. Sebelius takes office as swine flu numbers climb worldwide. But confirming Sebelius, who met several obstacles during confirmation hearings, doesn't bring the health team to 100 percent. There are still no appointees in place for any of the department's 18 key jobs.
\n ...",
 :link
 "http://rss.cnn.com/~r/rss/cnn_topstories/~3/dcHTwVYr0QU/index.html",
 :guid
 "http://www.cnn.com/2009/POLITICS/04/28/sebelius.confirmation/index.html?eref=rss_topstories",
 :title "HHS secretary confirmed in midst of outbreak"}

A better name for the create-map helper function might be “select-contents”

Part 2: The simplest thing

The current code in the tubes.components.rss namespace adheres to a pretty highly generalized interface: the defcomponent macro takes a bunch of metadata, some of which is necessary for the rss component but some of which is not (we don’t really have to specify that a function that reads a url requires exactly one url string). The interface is complicated because of all the other components defcompenent could and will create. Similarly, the function that does the work, that for our use *is* the rss component, takes three arguments: a pipe, pipe-args, and plain old args. Again, these components have a consistent and sophisticated interface, because of what we may ask them to do tomorrow. But let’s not worry about tomorrow just now.

What’s the simplest thing that could work? Well, what do we need? We need to pass a URL to some component and get back a list of data maps, each containing the different pieces of content from each item tag in the feed.

OK, how about

(ns tubes.components.rss
  (:require [clojure.xml :as xml]
             [clojure.zip :as zip]))

(defn select-contents
  "Creates a data map of the contents of an RSS 'item' entry"
  [item]
  (reduce (fn [acc it]
            (assoc acc (:tag it)
              (first (:content it)))) {} (:content item)))

(defn rss-reader [url]
  (let [xml (xml/parse url)
          zipper (zip/xml-zip xml)
          elements (-> zipper zip/down zip/children)
          items (filter #(= :item (:tag %)) elements)]
      (map select-contents items)))

Then, from the REPL:

user> (use 'tubes.components.rss)
nil
user> (def cnn-rss (rss-reader "http://rss.cnn.com/rss/cnn_topstories.rss"))
#'user/cnn-rss

Sure, it isn’t lazy. And it doesn’t support any kind of reuse — it’s use specific. But let’s add those things in as we need them. For now, we’re just exploring how the components work when we need them.

What’s next?

I’m going to propose a discussion on the mailing list for the next week or so. I propose we take on one thing at a time, and pass code around. Please join us there. You don’t have to be a DC native at this point: the action is online.

Clojure 1.0 is on the way, and as a result a lot of activity is occurring in the Subversion repository.  Revision 1357 of the Clojure source was proclaimed the likely release candidate, although a few minor changes have been checked in since.  Likewise, clojure-contrib revision 723 is the likely release candidate corresponding to 1.0. It should be interesting to watch what unfolds in the source until the actual release.

It’s not too late to contribute, and no contribution is too small.

-m

At our latest meeting, we discussed macros and adopted a capstone project, an open-source Clojure implementation of some of the functions of Yahoo Pipes.

Macros

Regarding macros, we touched on the chain of events when a Clojure program is evaluated: evaluation runs from top to bottom, and each S-expression is macro-expanded (if possible) and then evaluated. If an expression expands into something that itself contains a macro, the the new macro expression will be expanded before it is evaluated. This allows a macro to use a previously defined function in the determination of its expansion.

“Macros” in Lisp are very different from “macros” in other languages, as they allow the programmer to use all the power of Lisp to determine the macro’s expansion. Macros are not first-class members of the Clojure language, however. You cannot use them as you use functions–you can not use them with map or apply, for example–because macroexpansion happens before an expression can be used in this way.

Seeing stars

We also talked about “star” functions and macros. The macroexpansion of an expression that uses ‘and‘ includes something called let*:

(macroexpand '(and 1 2 3))
->(let* [and__2863 1]
(if and__2863 (clojure.core/and 2 3) and__2863))

We concluded that you really aren’t supposed to see let*. You only do in this case because macroexpand completely expands the expression it is given, and it turns out that let is a macro whose definition includes the mysterious let*. In fact, a lot of Clojure’s core macros use “star” constructs that appear to be special forms. Some digging at the REPL confirmed that they are not symbols. Likely the “star” forms are transformed by the Clojure parser directly into Java. If you have some special knowledge about this, please post a comment or join our discussion group. We’d like to know more.

Capstone project

Luke presented the case for a project that implemented some of the Yahoo Pipes application. By general acclamation we adopted this project as our capstone. We’ve decided to spend the next two weeks hacking on the basic problems of the project ourselves, keeping in touch through the group mailing list. David set up a github repo for us, with a wiki for collaboration. There you can find the summary of our initial work on defining the project, thanks to Keith. Serge dug up Yahoo’s Pipes documentation.

We’re calling this Tubes right now (as in “teh tubes”), but a final decision on the name (I’m not saying I don’t like Tubes) is one of many tasks we’re deferring. Right now we just want to get things moving. Personally, I expect Paul to have it all done by this afternoon.

Multimethods on Sunday, April 19th, 1pm

Our next meeting will be at least partly devoted to the final topic in Stu’s book: multimethods. Read chapter 8 of Programming Clojure and bring any questions or comments you’d like to discuss. Please notice two things about the next meeting date: 1) it is in two weeks, not three; and 2) it is on a Sunday. The location is still HacDC’s meeting space at St. Stephens, and the time is still 1pm. We hope to see you there.

This Saturday, April 4, at our face-to-face meeting at HacDC, member Luke VanderHart will informally present a case for a Yahoo Pipes-like content aggregator as a possible capstone project. Interested?

Read up on Pipes.

Read Luke’s initial comments.

Of course, we’ll also discuss macros, including the material in Chapter 7 of Programming Clojure.

We hope to see you there: 1PM at HacDC.

We had another successful meeting at HacDC’s workshop on Saturday. We covered functional programming, walking through the concepts in the chapter from Programming Clojure and then putting them to work in translating a decision tree modeling program from Python to Clojure: watch for a separate announcement on how we will continue to work on that. We had an especially good time unraveling this bit of code…

(def head-fibo (lazy-cat [0 1] (map + head-fibo
                                      (rest head-fibo))))

…and understanding what’s less than optimal about it: that it keeps a reference to the head of a lazy sequence, and thus (nth head-fibo x) fails for large values of x.

Next, we’ll tackle macros on April 4th, at 1PM, at HacDC’s workshop. The reading is Chapter 7 of Programming Clojure. We’ll be looking at simple implementations of defmacro and reviewing different uses for macros. We may also tackle a bit of Paul’s proposed templating language, putting what we’ve learned to use. Hope to see you there!