Flourish

Ultraviolet syntax highlighting in Mephisto

Being a mac rails user, I love textmate. Naturally, I went crazy when I found out that Ultraviolet is a syntax highlighting gem for ruby that reads textmate theme and textmate syntax files to create xhtml highlighted code. After using this in my tictactoe project, I integrated it into mephisto. Because I wanted to use the typo:code tag that's normally used by coderay, I did this by deleting all the files in the filtered_column_code_macro/lib plugin except code_macro.rb, which I replaced as follows

require 'uv'
class CodeMacro < FilteredColumn::Macros::Base
  def self.filter(attributes, inner_text = '', text = '')
    lang = attributes.delete(:lang) || "ruby_on_rails"
    theme = attributes.delete(:theme) || "blackboard"
    begin
      Uv.parse( inner_text, "xhtml", lang, false, theme) 
    rescue
        RAILS_DEFAULT_LOGGER.warn "UltraViolet Error: #{$!.message}"
        RAILS_DEFAULT_LOGGER.debug $!.backtrace.join("\n")
    end
  end
end

Then I copied in the xhtml CSS files for Ultraviolet xhtml rendering (as shown in the UV instructions) into my stylesheets directory. I included the themes I liked into my layout.liquid as follows:

<link rel="stylesheet" href="/stylesheets/uv/blackboard.css" type="text/css" media="screen" />
<link rel="stylesheet" href="/stylesheets/uv/dawn.css" type="text/css" media="screen" />
<link rel="stylesheet" href="/stylesheets/uv/blackboard.css" type="text/css" media="screen" />

Finally, you have to remove a reference to 'include "coderay"' in environment.rb. For each code block in your blog, you can change the theme (there's about 12 included themes):

import java.net;

public class Bob extends BobParent implements Bobliness {
  private final int SOMETHING=0;
  public static void main(String[] args) {
    System.out.println("Hello world!" + 5.5)
  }
}

UV comes with syntax highlightinb for 158 languages:

actionscript, active4d_html, active4d_ini, active4d_library, active4d, ada, antlr, apache, applescript, asp, asp_vb.net, bibtex, blog_html, blog_markdown, blog_textile, blog_text, build, bulletin_board, cake, camlp4, cm, coldfusion, context_free, css_experimental, css, cs, csv, c, c++, diff, dokuwiki, dot, doxygen, d, dylan, eiffel, erlang, fortran, f-script, fxscript, greasemonkey, gri, groovy, gtdalt, gtd, haml, haskell, html-asp, html_django, html_for_asp.net, html_mason, html_rails, html, html_tcl, icalendar, inform, ini, installer_distribution_script, io, javaproperties, javascript_+_prototype_bracketed, javascript_+_prototype, javascript, java, jquery_javascript, json, languagedefinition, latex_beamer, latex_log, latex_memoir, latex, lexflex, lighttpd, lilypond, lisp, literate_haskell, logo, logtalk, lua, macports_portfile, mail, makefile, man, markdown, mediawiki, mel, mips, mod_perl, modula-3, moinmoin, mootools, movable_type, m, multimarkdown, objective-c, objective-c++, ocamllex, ocaml, ocamlyacc, opengl, pascal, perl, php, plain_text, pmwiki, postscript, processing, prolog, property_list, python_django, python, qmake_project, qt_c++, quake3_config, ragel, r_console, rd_r_documentation, regexp, regular_expressions_oniguruma, regular_expressions_python, release_notes, remind, restructuredtext, rez, r, ruby_experimental, ruby_on_rails, ruby, s5, scheme, scilab, setext, shell-unix-generic, slate, smarty, sql_rails, sql, ssh-config, standard_ml, strings_file, subversion_commit_message, sweave, swig, tcl, template_toolkit, tex_math, tex, textile, tsv, twiki, txt2tags, vectorscript, xhtml_1.0, xml_strict, xml, xsl, yaml, yui_javascript

Also posted on ELC

I work at ELC

I've been working at ELC Technologies/Rightcart.com since February of 2006... and I love it there! In this section of my blog, you'll find all the posts that I did on the request of ELC.

Welcome to flouri.sh

I finally have a blog!

Anyone who knows me knows that I've despised blogs my entire life. However, I feel it's time to move on from my backwards ideals and embrace a new technology. I divided it into these sections:

• Blog: a running timeline of the next 3 sections
• Personal: information about my life.
• Projects: full projects that I've done outside my current job.
• Technology: smaller technological tidbits not related to work.
• Work: things that I've done at work.

I chose Mephisto as my blogging system because it's rails based. I've already made several modifications which I'm going to blog about in later posts.

Safely exposing your app to a ruby Sandbox

Creating wrapper classes for the sandbox

When creating my sandboxed game of Tictactoe (where a user can upload a new algorithm and play tictactoe against it), I wanted to expose only a small part of my application to user uploaded code. In the follow code, for example, I would want to provide user access to only a few methods of the Board class:

class Board < ActiveRecord::Base
  has_many :moves
  belongs_to :algorithm_x, :class_name => "Algorithm", :foreign_key => "algorithm_x_id"
  belongs_to :algorithm_o, :class_name => "Algorithm", :foreign_key => "algorithm_o_id"

  def make_move!(x, y)...
  def move_matrix...
  def log_info(msg)...
  def winner...
  def game_over...
  def make_computer_move!...
  def human_turn?...
end

If I want to allow the user's code to access make_move, moves, move_matrix, log_info only, I'd create a wrapper class as follows:

class BoardWrapper
  def initialize(board); @board = board; end
  def make_move(x,y); @board.make_move(x,y); end
  def moves; @board.moves.collect {|m| MoveWrapper.new(m) }; end
  def move_matrix; @board.move_matrix; end
  def log_info(msg); @board.log_info(msg); end
end

acts_as_wrapped_class

This is pretty cumbersome to build, so I built acts_as_wrapped_class to make creating these wrappers easy. It does the following:

• Automatically generate a wrapper class for each class marked as acts_as_wrapped_class
• Dispatch methods that match (or don't match) a safelist or blacklist
• Finds appropriate wrappers for return results (meaning if Board returns a Move then BoardWrapper returns a MoveWrapper)
• Wrap the contents of arrays and hashes (same as above, but will work with arrays of Move, and Hashes containing Move)
• Dispatch ===, hash, <=> methods directly to the wrapped objects. Compare two wrappers objects and get the same results as the two wrapped objects.

The above example is much shorter when written with acts_as_wrapped_class:

class Board < ActiveRecord::Base
  acts_as_wrapped_class :methods => [:moves, :make_move!, :move_matrix, :log_info]

  def make_move!(x, y)...
  def move_matrix...
  ...
end

class Move < ActiveRecord::Base
  belongs_to :board
    
  acts_as_wrapped_class :methods => [:x_pos, :y_pos, :is_x, :created_at]
end

Simple executing acts_as_wrapped_class inside the definition of Board automatically defines the BoardWrapper class with checks on which methods are called. This is accomplished through undefining all the methods of BoardWrapper and defining a method_missing which checks the safelist/blacklist before dispatching the method call.

Try to access winner on a BoardWrapper and it will throw an exception, because :winner isn't on the list of approved classes. Of course, you can call wrapper._wrapped_class and get access to the original Board object, but if you've set up your sandbox correctly, the class Board will not even be defined in the sandbox and will raise an exception.

View the RDOC for acts_as_wrapped_class for more detail.

acts_as_runnable_code

In order to make sandboxing user code even easier, I created another gem: acts_as_runnable_code. This gem helps you with the creation of the sandbox, the referencing of the wrapper classes, and automatic wrapping/unwrapping of data as it flows in and out of the sandbox. It assumes the following about your application

• you have objects that store user uploaded code in them
• you want to use your classes in the sandbox with reduced functionality provided by acts_as_wrapped_class
• you want to evaluate an instance of user uploaded code within the context of some instance of a wrapped class

When writing tictactoe, I created an Algorithm model which stored user uploaded code in a database TEXT field. I also wanted to evaluate that code using the binding of the Board object on which the game was being played (meaning the user code looks like "make_move!(1,1)" rather than "@board.make_move(1,1)").

class Algorithm < ActiveRecord::Base
  acts_as_runnable_code
end

@board = Board.find(id)
@board.algorithm_x.run_code(@board, :timeout => 1.0)

View the RDOC for acts_as_runnable_code gem.

To see tictactoe in action, create your own algorithm, and test the safety of the sandbox (scary!) visit tictactoe.mapleton.net

I originally posted this on ELC's blog:

Sandboxing in ruby

A few weeks ago, I decided to make a rails-based game. I wanted to bring the strength of ruby's metaprogramming into the game world, so I investigated sandboxing user uploaded code blocks. The only ruby sandbox was written by Why the Lucky Stiff, and you can find complete details on it here:

The Freaky Freaky Sandbox

The sandbox is an amazing hack on ruby's lookup tables to essentially allow a completely separate execution context with its completely own set of classes. The interesting part is how it interfaces with the outside world (the "Jungle"):

• Classes can be copied in from the Jungle using Sandbox.import, and exist in both places with separate definitions. The sandbox automatically does this with simple essentially classes like String, Object, Hash, Array, etc.
• Classes can be proxied in from the Jungle using Sandbox.ref. In this case, a proxy class is defined in the sandbox with exactly the same name as the outside class, but with only a two methods: const_missing & method_missing. When a method is called on the proxy, the sandbox is disabled and the actual method executes outside the sandbox. The result of the method is Marshalled into the sandbox, and it is enabled again.
• Objects can be copied into the sandbox using Sandbox.set as long as they're defined there. This is accomplished by marshalling
• Objects can be returned from the sandbox at the end of a Sandbox.eval call. This is accomplished by marshalling

Now you're ready to start writing your own applications using the sandbox. It's a pain to install in ruby 1.8.6, because it requires a small patch, but Why says that it works without patch in ruby 1.9.

I posted this originally on ELC's Blog

Tictactoe

http://tictactoe.mapleton.net

I designed tictactoe as the simplest place to showcase some of my gem work with the ruby sandbox. The rules of the game are simple enough that anyone could write an alogrithm for it.

It uses the following gems, which I wrote, to make the sandboxing and code editing experience easier:

• acts_as_wrapped_class (gem)
• acts_as_runnable_code (gem)
• acts_as_runnable_code_on_rails (plugin)

Nesting document.write

Not a good idea - IE and Firefox treat differently

I'm not an advocate of document.write by any means, but it seems unavoidable when dealing with 3rd parties. Ad networks especially seem to rely on delivering ads using a remote script tag and document.write. They typically do not provide an alternative using the more modern and well defined XML DOM manipulation functions such as createElement and appendChild.

Let's examine two simple examples of using nested document.writes, and see how they are treated in both IE and firefox:

<div id="before_main_script">Before Main Script</div>
<div id="main_script">
 <script>
  document.write("<div id='before_sub_script'>Before Sub Script</div>");
  document.write("<scr"+"ipt>document.write('<div id="hello_div">hello!</div>');</scr"+"ipt>")
  document.write("<div id='before_sub_script'>After Sub Script</div>");
 </script>
</div>
<div id="after_main_script">After Main Script</div>

You can see how I escape the inside <script> and </script> tag so the browser doesn't think I'm closing the outside script tag. In both IE and firefox, this renders how you'd expect:

Before Main Script
Before Sub Script
hello!
After Sub Script
After Main Script

Now let's make the example more complicated and watch things break down. All we've done is refactored the embedded script tag into an external script with a src attribute:

<div id="before_main_script">Before Main Script</div>
<div id="main_script">
 <script>
  document.write("<div id='before_sub_script'>Before Sub Script</div>");
  document.write("<scr"+"ipt src='/say_hello.js'></scr"+"ipt>")
  document.write("<div id='after_sub_script'>After Sub Script</div>");
 </script>
</div>
<div id="after_main_script">After Main Script</div>

Before Main Script
Before Sub Script
hello!
After Sub Script
After Main Script

Before Main Script
Before Sub Script
After Sub Script
hello!
After Main Script

Unexpectedly, IE7 did not process the nested script tag as it was being written into the document stream, instead waiting until it had finished processing the parent script. As browser implementation goes, this approach should be easier, because you wouldn't have to save the running script context temporarily to being processing a new one, and finally resume the saved context. We've also tried the same code without the nested script tags, and this weird behavior goes away. Naturally, this can get you into serious trouble if you're including a remote script that defines certain variables which you make use of immediately!

You might be skeptical that this could ever become an issue in the real world. One of our clients had to generate some javascript to skin a 3rd party site to look like theirs, and they wanted to put double-click ads in. This requirement resulted in a javascript which executed a series of document.writes to skin the 3rd party site, including writing out a 2 inline script tags and 1 with SRC to doubleclick. The doubleclick script then used document.write to print out the ad. We first noticed the difference in placements of the ads in IE7 and Firefox, and ran the above tests to determine what the problem is. Here's how we solved it:

document.write("<scr"+"ipt src='/before_3rd_party.js'></scr"+"ipt>");
document.write("<scr"+"ipt src='3RD_PARTY_URL.js'></scr"+"ipt>");
document.write("<scr"+"ipt src='/after_3rd_party.js'></scr"+"ipt>");

I originally posted this article on ELC

Preloading fixtures

Ever wonder why it takes so gosh darn long to run your tests?

In my experience, long test times are due to two reasons: fixture loading and fixture instantiation

If you don't use instantiated fixtures such as @users_one, and instead use users(:one), then you can safely turn off instantiated fixtures, saving you all the time of finding the data. But what about fixture load time? If you use transactional fixtures, the loading only takes place once per test file, which could be 100-150 times in a given project. Without transactional fixtures, they are loaded before every test, clearly a waste of time!

My solution was to develop a plugin which proloads and preinstantiates the fixtures only once (well, 3 times was the best I could get). It takes advantage of transactions to rollback the state of the database between each individual test (don't try this on MySQL with bad storage engines like MyIASM, kids). It usually takes a few seconds, of course, but cut a down "rake test" for a large project of ours from 20 minutes to 90 seconds. Plus, you don't have to remember which fixtures you need for every single test file anymore! Try adding the globalization plugin, for example, and you'll realize you need 2 fixtures on every single page.

In my opinion, there's no reason not to preload all your fixtures once before running tests, but that's up to you to decide.

Plugin Usage

1. Install the plugin
2. Add the following to test/test_helper.rb

   PreloadFixtures.preload!
   PreloadFixtures.instantiate!(Test::Unit::TestCase)

3. The "fixtures" function in a test is now overloaded to do nothing. All fixtures in the directory will be loaded once and then the tests will be run.

Download Plugin

From our svn respository

./script/plugin install https://wush.net/svn/public/preload_fixtures

Originally posted on ELC

See ELC's other Rails Plugins

Preloading fixtures

Ever wonder why it takes so gosh darn long to run your tests?

In my experience, long test times are due to two reasons: fixture loading and fixture instantiation

If you don't use instantiated fixtures such as @users_one, and instead use users(:one), then you can safely turn off instantiated fixtures, saving you all the time of finding the data. But what about fixture load time? If you use transactional fixtures, the loading only takes place once per test file, which could be 100-150 times in a given project. Without transactional fixtures, they are loaded before every test, clearly a waste of time!

My solution was to develop a plugin which proloads and preinstantiates the fixtures only once (well, 3 times was the best I could get). It takes advantage of transactions to rollback the state of the database between each individual test (don't try this on MySQL with bad storage engines like MyIASM, kids). It usually takes a few seconds, of course, but cut a down "rake test" for a large project of ours from 20 minutes to 90 seconds. Plus, you don't have to remember which fixtures you need for every single test file anymore! Try adding the globalization plugin, for example, and you'll realize you need 2 fixtures on every single page.

In my opinion, there's no reason not to preload all your fixtures once before running tests, but that's up to you to decide.

Plugin Usage

1. Install the plugin
2. Add the following to test/test_helper.rb

   PreloadFixtures.preload!
   PreloadFixtures.instantiate!(Test::Unit::TestCase)

3. The "fixtures" function in a test is now overloaded to do nothing. All fixtures in the directory will be loaded once and then the tests will be run.

Download Plugin

From our svn respository

./script/plugin install https://wush.net/svn/public/preload_fixtures

Originally posted on ELC

See ELC's other Rails Plugins