A new appreciation for convention over configuration

At work, I'm working on revamping our logging infrastructure to use Hypertable (an opensource Bigtable clone) for storage, and Hadoop (an opensource Map/Reduce framework) for log processing.

I'm extremely excited about the potential these tools have for allowing us to do all sorts of ridiculously cool analyses on our logs, and at some point I'll write about some of the things we are doing with them. However, in the meantime I just wanted to share my renewed appreciation for the simplicity of convention over configuration that I've gotten used to with Rails.

You see, Hadoop is written in Java, and uses Apache Ant for builds. We're talking 60-100 line XML files for to configure simple example jobs. Why would anyone voluntarily set their system up this way? To paraphrase one of my coworkers... enterprise-quality appears to be roughly translatable as 'over-engineer everything'!

Flash of Insight about Prototypal Inheritance

I've always been confused about how JavaScript inheritance worked, and what
all of these prototype things actually are. Reading article after article after article about prototypal inheritance gave me some more details about how to use it, as well as opinions on its value, but the fundamental model was still missing. But on the train this evening I was reading through the Wikipedia article on Prototypal inheritance when suddenly I had a flash of insight! The reality is so much simpler than anything I'd imagined. Without further ado, here it is:

A prototype is an object to be copied. Thats it. When you create a new object in a prototypal language, it starts out as a copy of its prototype.

Its common to see javascript that looks something like:

function A(){};
A.prototype.method1 = function() {foo};
A.prototype.method2 = function() {bar};
A.prototype.method3 = function() {blah};

Now any new instance of A will have method1, method2, and method3 defined. I guess I'd always thought of this as some sort of magic. But really, its equivalent to

function B(){};
B.method1 = function() {foo};
B.method2 = function() {bar};
B.method3 = function() {blah};
A.prototype = B;
function A(){};

In other words... here's this object B, it has these methods. When you create an object of type A? Just copy B. So simple! So flexible! Why did I not understand this before?

There is, of course a caveat. In JavaScript, we aren't really copying the methods, as that would be incredibly inefficient. There is a hidden pointer (accessible in some browsers as A.__proto__) that points back to the prototype, and the runtime then takes care of delegation. But conceptually, its just a copy.

Name change & feedback request

I realized that the original name I had for this blog (Ruby, Rails, and other coding thoughts...) was not turning out to be representative of what I was posting. I've been writing almost entirely Shoes related content, with a few divergences into process related things and other microframeworks.

So I've decided to change the name to something more representative of what I'm using this blog for: writing about the coding related stuff I'm doing for fun. Since I develop a Rails app for work (this app, company blog here), I've been doing very little playing with Rails outside of work.

The new name, 'Ruby Merriment', expresses more clearly what this is about. Having fun playing with my favorite programming language.

That said, one thing I'm excited about is trying to engage in an ongoing conversation with other people who are excited about Ruby. I'm not quite clear what sets of things I write about are interesting to other people; It might be interesting to explore some more Rails related ideas, if people want to talk about those. So give me some feedback: Do you like what you're reading? Are there other things you'd like to talk about? If you have a blog where you talk about coding, point me at it so I can take a look and respond.

Thanks

More Shoes Structure

When I first started using Shoes, it took me a while to get the various pieces set in my head. There's a couple of different levels to understand. There are a number of tutorials about layout, the most fundamental being Stacks and Flows. This is critical to understanding to design your GUI, but it wasn't enough for me to really understand what was going on.

After hacking around, reading through the source code, and writing some addons I think I'm starting to get my head around it, so here's an attempt to explain some more of the structure of Shoes.

The most basic piece is Shoes.app. This is the method you use to open a window for a new Shoes application; hello world in shoes is something like.

Shoes.app do
  para "Hello World"
end

What this does is twofold... it instantiates a new Shoes::App object, and sets self inside the block to point to that object. The Shoes::App object provides all of the helper methods for creating different components, such as para, button, stack, flow, etc, and is responsible for actually rendering them. The objects themselves don't render, they just define a structure that the Shoes::App object will then render. This is why, when trying to do something in a module or class outside of the block, you need to pass in the app object, or access it through the app method on some already rendered object (all shoes objects have an app method to get back to the app they are rendered in).

How does the structure that Shoes::App builds up look? It is a tree format, that actually looks remarkably similar to the DOM in the web world.

At the root of the tree is the slot associated with the original Shoes::App object. This slot behaves like a flow, and can be accessed via the slot method on the Shoes::App object; within the Shoes.app block that would be self.slot. (This is important; after reading that the app object was a flow, I originally thought I'd be able to use all of the flow methods immediately on it, but you actually need to call them on self.slot.)

Every node in the tree has an accessor called parent and one called children. Lets examine what these look like in the hello world example above, by adding some debug output:

app = Shoes.app do
  para "Hello World"
end
Shoes.debug "Parent is #{app.slot.parent.inspect}"
Shoes.debug "Children are #{app.slot.children.inspect}"

The console now shows:



which is pretty much what we'd expect. The root should have no parent, and the paragraph is pretty much the only element, so it should be the first child. What does that paragraph look like? Lets try

Shoes.app do
  p = para "Hello World"
  Shoes.debug "Para Parent is #{p.parent.inspect}"
  Shoes.debug "Para Children are #{p.children.inspect}"
end

Now we see:



The parent is pointing back to the app.slot (which we now see is not actually a flow, but an instance of the Shoes class, which flows inherit from.) And the children of a Para object actually contain the text of the paragraph.

Understanding this tree structure was one of the biggest breakthroughs for me, because it means that many of the things I know from manipulating the DOM with javascript in the web world are suddenly applicable to Shoes. I'm no javascript genius, but I have already climbed some of the learning curve there, and being able to apply that to Shoes was eye opening.

One of the things the DOM does really well (and the Shoes model should as well) is allow you to navigate, find, and manipulate things that have been generated someplace else. This is where I got the idea for ShoeQuery, and will prove extremely useful in my efforts to build a Shoes debugger.

I think this similarity could make Shoes the toolkit of choice for web developers coming to the desktop.

Event Handler chains in Shoes

One of the beautiful things about Shoes is how simple and easy it is to set up event handlers on objects. As I talked about in a previous post creating a handler on a button is as simple as

b = button "Click Me"
b.click { alert "You clicked the button" }

This is great for simple things, but I hadn't been hacking on Shoes for very long before I wanted to do more than this. In particular, since one of my projects is creating an application to preview Shoes applications, I wanted to be able to add handlers that call back into my main application without worrying about their effect on the code being previewed.

To do this, you need to be able to have multiple handlers on an element at once, so that you can add new event handlers without disturbing the existing ones. I spent a while trying to understand the c-based implementation of event handlers, and figuring out all of the things I'd have to do to change it to a chain based implementation, but then realized that given the ruby nature of Shoes, this was a prime opportunity for monkeypatching.

Monkeypatching may be a little less performant, but this is for a debugger-style thing anyway (I'm thinking I want to get to something a lot like Firebug is for web development...), so who cares if its slow?

So in I dove. I'm calling the module ShoeShine, because its cleaning up my Shoes without changing the inside at all, and it turned out to be pretty simple to implement. First, I define a few methods to add, remove, and call handlers:

  def add_handler(type, &block)
    @_handlers ||= {}
    @_handlers[type] ||= []
    @_handlers[type].push block
    # set up callbacks from old handlers invocation into
    # new handler chain
    self.send "#{type}_without_chains".to_sym do |*args|
      self.call_handlers(type, *args)
    end
  end

  def clear_handlers(type)
    if @_handlers
      if @_handlers.delete(type)
        _handler_deleted(type)
      end
    end
  end

  def call_handlers(type, *args)
    return unless @_handlers[type
    @_handlers[type].each do |block| 
      block.call(*args)
    end
  end

Then I do some metaprogramming and monkeypatching to plug this in between handler registration and handler callback:

  HANDLERS.each do |handler|
    eval %(
      def #{handler}_with_chains(&block)
        # need to clear handlers to maintain old semantics
        clear_handlers(:#{handler})
        add_handler(:#{handler}, &block)
      end
    )
  end

  def self.included(klass)
    klass.instance_eval do
      ShoeShine::HANDLERS.each do |handler|
        if klass.instance_methods.include? handler.to_s
          # Replace old handlers with new handlers
          alias_method "#{handler}_without_chains".to_sym, handler       
          alias_method handler, "#{handler}_with_chains".to_sym
        end
      end
    end
  end

Finally, I include this module into all interesting Shoes classes:

Shoes.constants.each do |c|
  k = Shoes.const_get(c)
  if k.is_a? Class
    k.send :include, ShoeShine
  end
end

Voila! Suddenly, as well as having the traditional Shoes interface to handlers, you can also add additional handlers onto existing objects without disturbing the originals. This will open the door to all sorts of interesting things. I've already mentioned my intent to use this for debugging, but I'm also exploring using this as a way to get around the current Shoes limitations on click, hover, leave, etc handlers of only working on slots, allowing them to trickle down to paragraphs, widgets, etc.

I've got some early versions of this working, but its definitely not ready for prime time yet. For any who are interested, you can find both the ShoeShine code and my early trickle down work at github in the shoes-preview tree.

Can I just say how much I love method_missing?

I wanted to follow up on my previous explorations with ShoeQuery to add some helper methods to access pieces of shoes functionality... thinking I'd have to add pieces one by one. However, it turns out that due to the magic of method_missing, I was able to add access to basically all of them with a single 22-line method. It looks like:

  def method_missing(method, *args, &block)
    if args.empty? && !block_given?
      # if you have no arguments and no block, think of it as a query
      # and return the answer for the first element.
      if first.respond_to? method
        return first.send(method)
      else
        return elem.style(method)
      end
    elsif block_given?
      each {|elem| elem.send(method, *args, &block)}
    else
      # Otherwise, think of it as a setter
      setter = "#{method}=".to_sym
      if first.respond_to? setter
        each {|elem| elem.send(setter, *args, &block)}
      else
        each {|elem| elem.style(method => args[0], &block)}
      end
    end
    self
  end

And now, thanks to this I can write lines in shoes as follows:

# find all paragraphs, change their text to 'all ps', and color 
# them red
shoe_query('para').text("all ps").stroke(red)

# replace the click handlers on all buttons
shoe_query('button').click do
  para "All your button are belong to us now
end

Introducing ShoeQuery

ShoeQuery (better name needed) was born out of a desire to be able to find and manipulate elements in Shoes without having stashed away their locations ahead of time. It was inspired by jQuery's mechanisms for finding and manipulating DOM objects in Javascript.

The fundamental object (a ShoeQuery object) is pretty much an array of Shoes Elements, augmented with some extra functions. ShoeQuery objects are created via a helper (unsurprisingly named shoe_query) that lives in the Shoes::App object, and so should be accessible anywhere you want it. You can pass into that helper a selector, and it will return a ShoeQuery object containing the elements that match. You can also pass in an array of elements you already have handy, or an existing element, and it will also return a ShoeQuery object containing those elements. This form should be familiar for those who've used jQuery.

Right now, the things it lets you do are pretty limited, though a lot of the functionality jQuery specially provides already exists in Ruby. The one big feature is the ability to find elements by type, including css-like hierarchy. You can do this either by invoking shoe_query directly, or calling find on an existing shoe_query object (to search within its children). For example, given the demo app:

load 'shoe_query.rb'
Shoes.app do
  app = Shoes.app do
  para "foo"
  f = flow do
    stack do
      para "bar"
      para 'blah'
    end
  end
  b = button "click me to change all paragraphs"
  b.click do
    shoe_query('para').each do |p|
      p.text = "all ps"
    end
  end
  b2 = button "click me to just change the paragraphs inside the flow"
  b2.click do
    shoe_query('flow').find('para').each do |p|
      p.text = "flow ps"
    end
  end
  b3 = button "click me to change the button inside the stack"
  b3.click do
    shoe_query('stack para').each do |p|
      p.text = "stack p"
    end
  end

end

In this case, the first button modifies all three paragraph elements, while the latter two use two different mechanisms to scope their changes to within the flow and within the stack respectively

As I play with this more, I'll be extending it to include more functionality. Let me know if there's anything you want!

You can get ShoeQuery at github at http://github.com/kball/shoequery/tree/master