neovintage

Start Using Shogun Toolbox in Ruby

2011-12-19T07:00:00.000-08:00

Shogun Toolbox is the ultimate in machine learning. It provides a number of different machine learning models like support vector machines (SVM) and hidden Markov models just to name a few. So why use Shogun toolbox? Well, when you look at the SVM landscape, for example, you'll find many different publicly available implementations. Let's say you needed to switch your code to use OCAS instead of LibSVM, the semantics of interacting with each implementation could vary wildly. As a result the authors of Shogun Toolbox state, "the motivation for creating a machine learning toolbox was to provide an easy, unified way for solving certain types of machine learning problems." This is great for us because that means we'll get access to many more machine learning models without having to find or build C extensions for Ruby that interface with the SVM implementation we want to use.

What are we trying to solve with Shogun Toolbox?

According to the wikipedia page for Shogun Toolbox, it was developed with bioinformatics in mind. With an industry like that, you should expect lots of data, and do I mean lots. We're talking about millions and millions of data points. Ok... so it can handle a lot of data, but what else is it good for? The real sweet spot are two- and multiclass classification and regression problems, according to the overview paper. There are some other things packaged with the toolbox but know that if you're looking to do classification this is the package for you.

Compiling Shogun Toolbox

The easiest way to get started would be to download the source from github. Once downloaded, make your way to the src directory so that you can compile the toolbox for your machine.

It's as simple as that. Theres a whole host of options that you can configure when you're compiling the program. I would encourage you to check them out as you use the toolbox. Since we're really interested in doing this for Ruby, we need to do a couple more things before we get the toolbox working. First, install the ruby gem narray to your system. Its pretty simple:

Then when you configure the toolbox you'll need to specify that you also want the ruby_modular interface:

There are other interfaces for languages like Octave, Java and Python. Feel free to include them at this time. You should be good to go from this point, unless you're an RVM user.

Gotchas for RVM users

Unfortunately at this time, when you configure the library, it's not smart enough to know that you need to point to your current ruby instead of the system ruby. You'll see a weird error in the build process like Undefined Symbol: _rb_str_new_cstr. You'll need to collect two different pieces of information.

The first is where your ruby lib lives. You can issue rvm info and find the environment variable MY_RUBY_HOME.
The second is where your platform specific include directory. You can issue ruby -e "puts $:" at the command line and that will show you all of the directories ruby will look to include code. Make sure you've switched to the ruby you want to use! I recommend you put the compiled modshogun lib in the path with site_ruby and the platform specific folder.

Once you get that, all you need to do is slap those on the end of your configuration options:

If you didn't notice when you add the ruby dir to install to, modshogun will be available to all gemsets of the ruby you compiled against. If you need to namespace modshogun to a particular gemset, let me know if you have any ideas.

Running through some examples

Let's go through one of the classifier examples packaged with the source code, namely classifier_libsvm_minimal_modular.rb. I've added my extra notes in the comments:

Resources

Shogun Toolbox: http://www.shogun-toolbox.org
Source Code: https://github.com/shogun-toolbox/shogun/

If you're new to machine learning and/or classification, I would recommend checking out some other articles to get up to speed on the subject before jumping into just a big piece of software like Shogun Toolbox:

Mounting Resque Web in Rails 3 using Warden, Devise and the Asset Pipeline

2011-11-30T12:46:00.001-08:00

Really the title of this post should be, "Don't fear middleware". So here's the set up, I had a Rails 3.1 app that was utilizing the Asset Pipeline, Resque and Devise. Like most people, I wanted to just mount Resque Server into the Rails app behind the authorization that I was already using and be done with it. The easiest and most elegant way of doing so is to do it in your routes. What's so awesome about this solution is that you get authorization and paths pointing to Resque Web just like that. Unfortunately, if you're using the asset pipeline like I was, you'll be unable to get any of the Resque Web assets served by the application in staging and production. By "using the asset pipeline", I mean setting the action_dispatch.x_sendfile_header = 'X-Sendfile' and serve_static_assets = false in the config file. Plus the solution given above for serving the assets from Resque is to copy them to your assets folder. BARF!

To The Middleware

Let's delve into the rackup file. If you're like most Rails projects you probably have a rackup file that looks like this:

Well thats going to have to change. First off you'll need to map the routes and then tell Resque Web how to handle authentication within the scope of rack. What you end up with is a file like this:

The only thing that's really left is creating a new instance of Resque::Server. I decided to do it via Sinatra rather than creating another rack middleware. I have an initializer in config/initializers/resque.rb like this:

There you have it, you can now serve Resque Web without having to copy any assets or do anything funky to the gem. This feels a little bit better to me since we're not really messing with the internal workings of the resque gem.

Text Classification using Support Vector Machines in Ruby

2011-11-14T13:59:00.001-08:00

Support Vector Machines represent a group of supervised learning classifiers that will try to predict the outcome of a feature set. I don't want to bore you with the details but I would recommend that you read Ilya's post about SVMs. It's very informative and it will give you the basics of what you need to understand to get started.

In my case, I've been working on a fairly large scale text classification problem at work when I realized that SVMs would be perfect for the job. Unfortunately, for our needs the current state of Ruby support for SVMs wasn't where we needed it to be.

I am happy to introduce rb-libsvm to the world!

While I can't take credit for the entire gem myself, I did help clean up some pieces of the codebase and gemify the code to make it possible to install from Rubygems.

Basic Usage Example:

As for the future plans for this gem, one of the biggest features I can think of building would be the Marshaling of the Libsvm::Model object. This would be a big performance win since you wouldn't have to re-train the classifier. If you have any other ideas for features, I'd be happy to hear about them.

Configuring Your Rails 3 Plugin

2010-11-14T10:00:00.000-08:00

With Railties, I've found writing plugins for Rails 3 to be much easier than before. Instead of writing a post about how easy it is to create plugins, I figured I would share what I learned about configuring your recently created plugin. After poking through the Rails code, I came across some interesting things that can really clean up and tighten the code in your Rails projects.

If you're looking for more of an introduction on Railties and Rails 3 plugins, I recommend that you read a couple of posts before you continue with mine. These posts should be enough to get you started.

Environment-specific Configuration

In the past, if you needed any type of configuration to be done in your app you'd more than likely create a yaml file and stick what you needed in there. With Rails 3 and Railties, you can move all of your plugin configuration into the environment specific files (application.rb / development.rb / test.rb / production.rb) within your Rails project. I like this so much more because all of the configuration stays in the same place and you don't have to litter your project with a bunch of yaml files.

Throughout this post, I'll refer to a library I created to use MongoDB as a FIFO cache store. Let's start with a snippet of development.rb in one of my current projects.

The important line in the above code is config.mongo_fifo. Did you know that you can create any variable that you want within the config object? The cool thing is that the config object is actually an instance of Rails::Railtie::Configuration and when you specify a method that doesn't exist, that class will actually go through method_missing and add the new method to a hash and assign whatever you wanted to it. So in the example above, mongo_fifo was added to the application's config hash with a new instance of MongoFifo. If you need a little more configurability with your methods, like action_mailer above, you can do that too. So let's say I wanted mongo_fifo to have a slew of config options, I could do this in my Railties class to add that ability:

Coincidentally, that's how ActiveRecord, ActionMailer and ActiveSupport allow for "namespaced" configuration in your Rails app.

Initializers

Now that we moved the configuration into the specific environment files, we'll be able to access it via our initializers. I really wish I had more to write here but it's just as easy as accessing a hash. So in my MongoFifo code, I've got one initializer that will create a global variable within the Rails module so that anywhere within my app, I can read and write to my MongoDB cache. Here's the code from MongoFifo:

Get going!

The beauty of Railties is that you don't necessarily need to develop a whole gem to get the benefits of it's modularity. In my case, I wrote some code that's a slimmed down version of an ActiveSupport cache store that works with MongoDb's capped collections. I plopped this file in my /lib directory. I'm able to put the MongoDB connection configuration in the specific environment (development / test / production) file and keep my overall Rails project that much cleaner. Give it a go!

Using Allocate to Create ActiveRecord Objects When Stubbing find_by_sql

2010-10-17T15:08:00.000-07:00

find_by_sql. What a method. I love the flexibility it gives me when I need to get information from my database with as little fuss as possible. Yet when I want to stub find_by_sql and return real objects in my rspec tests, a little part of me gets frustrated. The real pain comes in when you want to have an object with attributes that aren't part of the original class and typically you're going to run into that when you use find_by_sql.

In my particular use case, I had two different tables, the first was a Shop table and the second was a Job table. In terms of the association, a Shop could have many jobs. The find_by_sql method was aggregating data from the two tables and creating each instance of the result as Job object. So it looked something like this:

I'm sure that you're thinking to yourself that I could have just created some factories or fixtures and then had my tests just query the database for the records. That probably would have been the easy route, but I was wondering if I could pull this off without the database call. So how do we do this?

First, we need to understand how ActiveRecord actually creates new objects. The post from Peter Bui is a great first step in explaining what goes on when ActiveRecord queries a database and creates a new object. The important thing to note in his article is Ruby's ability to create an object without calling the initialize method via allocate.

From there it was time to look at the core ActiveRecord code to see what is going on. Just to note, since Peter's article, ActiveRecord::Base has changed a little bit. First I started with the code for find_by_sql:

You can see in this piece of code that ActiveRecord is hitting the database with your custom SQL and then for each result creating a new instance. Now let's see how that instance is created:

The first line of code is looking for the appropriate class to use to create the object via find_sti_class. Then it allocates an object based upon the class name. The next line is really where the meat is. init_with will actually then create the object with the attributes that you specify. Sweet! I think we struck gold.

Now that we know about the init_with and allocate methods we can now create real objects to use in our tests if we need attributes outside of what the object provides. So in the example that I provided above, I could do something like this:

Excellent! Now I can stub the find_by_sql call and I can create objects that will be in the correct class that I need.

Do note that what I did above is for edge rails. If you're on 3.0.1 or below you're going to have to do this to create your objects:

Also, pay attention to the way the hash was created. You'll need to use strings for the keys otherwise you'll get a bunch of errors.

The Rails Way Means Disruption

2010-08-01T11:10:00.000-07:00

A little less than a month ago, Obie Fernandez posed the question "What is the Rails Way (To You)?" For me, it took a little while to ponder that question and generate a response. This is what I came up with:

I see the Rails way encompassing two different stakeholders, the first being the individual, myself in this case, and the second being the community. I can't explain what 'the way' is without first identifying these two groups.

When talking about the community, I see the Rails way as embracing change. The Rails community seems to embrace a constant state of mild disruption. Some of the best corporations can't even deal with change as well as the Rails community. While some might argue that the precipitation of Rails 3 was a huge disruption, I would say that a Rails 3 type of disruption is quite rare. Mild disruption allows the community to vet ideas and practices and really reinforce what really works within the realm of web development. Something like BDD is an example of the reinforcing practices that have grown with the Rails community.

Personally, what the Rails way means for me is that it can mean anything. If I need a monolithic ERP system using Rails as my backend, I can do it. If I need to create a bunch of RESTful web services that are all interconnected, I can. 'The way' means that when I work with Rails, especially Rails 3, I can mold the framework around my needs knowing that Rails has quality baked in from the start.

Mongodb Stored Javascript Functions - Ruby Edition

2010-07-09T11:47:00.000-07:00

I was writing a couple of different analytics applications recently. My applications did two different things but accessed the same database. I was wondering if there were a way to store javascript functions so that I could share code between two different applications. That's when I stumbled across the server side code execution and Mike Dirolf's post on stored javascript and pymongo. I was hoping for the same type of functionality in the ruby driver as in pymongo, but alas it wasn't.

In the patch that I submitted to the mongo ruby driver, I added a couple of functions that will allow you to add stored Javascript functions to system.js in your database. Let's walk through a use case:

# Assume we're using the mongo driver and db = test
db = Mongo::Connection.new("localhost").db("test")

# Setup the stored function in MongoDB
db.add_stored_function("sum", "function (x, y) { return x + y; }")

sites = db.collection("sites")
sites.insert({:name => "Site 1", :admin_count => 2, :moderator_count => 3})
sites.find("$where" => "sum(this.admin_count, this.moderator_count) == 5")

# When we don't need the stored function any more
db.remove_stored_function("sum")

This is just one example of how to use the stored functions. Don't forget that you can use them in map/reduce, eval, and $where clauses.

Unlike the pymongo version, I did not create an instance of the system.js collection in the db instance. This would have allowed us to use introspection to actually make method calls instead of having to write out an eval method. (See how it works for pymongo at the end of Mike's post.) In my use cases, I used the stored functions within where and map_reduce calls and didn't really need to use eval. If other people need it, I could definitely change it up.

Mongoid Versioning - A Run Down

2010-06-14T06:13:00.000-07:00

In past Rails apps that I've built, I've used plugins such as vestal version or acts_as_versioned to version my ActiveRecord models in MySQL. Using Mongoid, you get that functionality built-in without the need for extra gems / plugins.

For the purposes of this discussion, let's use the following code as a reference point:

So.... what really goes on when you put that innocent include Mongoid::Versioning in your model? I'm glad you asked:

A field named version of type Integer is added.
An embeds_many association named versions is created with the :class_name option set as your current class. Using our example above the generated code would look like this: embeds_many :versions, :class => Post.
A before_save callback is created that takes care of versioning the document.

Now this stuff came straight from the documentation.

Now that we've got the basics established lets go through some use cases that will better illustrate how versioning works.

Let's say I have an idea for a new post and I create one with just a title. Then at some point in the future I finally get around to writing the content and save it to MongoDB.

Since we've completed 2 save operations, there will be two versions in the database and the document look something like this:
Notice in the versions array that the content field is actually not there. If a default is not defined on your field and you do not create anything the field will not be created in the document.
Now we've got our newly finished post in MongoDB and it's available for the world to see. At some point, a user decides to comment on our post and we need to save that to our document.

The document in the database will now look like this:
Notice that in this instance that a new version was not created when we saved an embedded document. This is great when you've got a setup like this and you don't want to create multiple versions.

Gotchas & Workarounds

Take into consideration how you're going to create your embedded documents - By this I mean if you were to do something like this:
Mongoid will not only create a new comment but will also create a new version of the entire document
The entire contents of the document is saved in the version - When embedding something like comments on a post, depending on how many comments your had, that array of comments could get pretty big. Now if you were to update a field on the post and it had a large amount of comments, all of the comments would get saved in the old and new versions. Again depending on the size of the comments, this could quickly start taking up large amounts of disk space
You can skip creating a version if need be - I wouldn't recommend this if you're trying to keep track of documents, but it can be done.

Note that :revise is the name of the method in Mongoid::Versioning and that I did this on the class Post and not an instance. I'm not sure if you can skip a callback on an instance. I'll have to look into that.

That's all I've got for now. As I continue to learn more about mongoid, I'll continue to post on this blog. Maybe I'll even turn this into a larger series on the ins and outs of using mongoid.

File Uploads with Rails 3, Plupload, and CarrierWave

2010-06-01T09:00:00.000-07:00

You may know that I've been working on Be The Builders for a while now. When it came time to build in some functionality for file uploads, I just started to cringe. I searched for a while on ways to get it done, but I was hoping for something a little easier without the need for middleware. I came across Plupload and figured I'd give it go for integrating into Rails 3. Ultimately, it was pretty easy and painless. I'll walk you through the process. Just to note, I was using CarrierWave to handle the file uploads on the Rails side and Mongoid as my ORM.

Step 1 - Download Plupload and copy the js folder into your javascripts directory. I'm not going to go through all of the options for Plupload, just what it takes to get you going. You can figure out all of the options in Plupload's documentation.
Step 2 - Add the appropriate javascript_include_tag to your view. Since I only had one action that was going to handle the creation of the images, I just put it in my new.html.erb view.
Step 3 - Now comes the part where we need to add some of the javascript to get the Plupload controls to show up on the page. I followed the custom example that was posted on their website. The important points that you need to know about the javascript below is the multipart and multipart_params options. Without those, CarrierWave is going to throw all sorts of errors when you try to upload the file. Also important to note is the use of the authenticity token in the multipart_params. It's just something that needs to be done to keep the session legit The html in the view is the same as the Plupload documentation example.
Step 4 - Get your CarrierWave uploader mounted to your model. In this example, this is what I have: Now if you're using Rails 3 like I am, make sure you do a require 'carrierwave' and require 'carrierwave/orm/mongoid' at the top of your application.rb file. Also, you'll have to pull the latest code from github. At the time of this writing, the carrierwave gem does not support Rails 3.
Step 5 - Code up the create action in your controller. It could be something as simple as this: Mine doesn't look that that, it's a little messy and I didn't want to paste all of it here.
Step 6 - Relax and have a much deserved drink. You're done!

I found this process to be extremely easy and with Plupload I now have the option to use many different methods (flash, silverlight, html5, etc) to get files uploaded to Rails. Hope this quick tutorial helps you guys.

The Mythical Man-Month

2010-05-25T09:00:00.000-07:00

What is the mythical man-month? If you're familiar with the analogy that you can't add more people to a pregnancy to make it go faster (I think that's how it goes), then you're already familiar with the mythical man-month. It's a book about software development and project management written by Frederick Brooks, Jr. during his time at IBM. I know this gets quoted all of the time, but just bear with me.

While the book is a little old, I think every project manager, developer, or individual associated with software development should read. While some of the non-tech people may get lost in the acronyms, just encourage them to keep reading past it to get a gist of the processes. You can even tell them to skip to the back of the book with the outline. With many of the buzz words being thrown around today like Agile, eXtreme Programming, etc., I think this book provides a solid base for anyone to understand the onus behind the other management frameworks.

My takeaways from the book:

People and months are not interchangeable.
Communicate early and often.
Scope, time, and budget. Pick two. (Ok this one was more of my own, but it's well represented in the book.)

Wikipedia has much more on this book than I wanted to cover here. Don't let that summary fool you into thinking that you know everything the book has to offer. The real value is in the stories and knowledge of Brooks as he has learned them over the course of his career. But if you're that impatient, as I've mentioned before, then you can skip to the back of the book and everything is summarized in an outline format.

I've encouraged all of the non-tech people in my startup, Be The Builders, to read this. So go pick up a copy of The Mythical Man-Month. You and your team won't be disappointed.

Upgrading to Mongoid Beta 6

2010-05-18T16:22:00.000-07:00

Just as a quick note, I had upgraded from Mongoid 2.0 Beta 4 to Mongoid 2.0 Beta 6 when I ran into some weird errors like:
ActionView::Template::Error (Database should be a Mongo::DB, not NilClass)
and
Mongoid::Errors::InvalidDatabase: Mongoid::Errors::InvalidDatabase

Essentially, you can keep your Mongoid code the way it is, but you need to put this line of code at the top of your config/application.rb:

require 'mongoid/railtie'

I hope that helps someone out there.

Tips for working with MongoDB & Mongoid

2010-05-13T10:39:00.000-07:00

As with anything I do on this blog, it's to make sure that I don't forget what I just learned. Below are some of the things that I've learned while using Mongoid on my current project, Be The Builders. Just to note, I'm working off the current beta version of Mongoid 2.0.0.beta4 and MongoDB 1.4.0.

Document != Embedded Document - This seem pretty obvious, but it didn't to me at first. Be The Builders is my first attempt at using a document oriented db, and since I came from a very heavy relational db world (read: business intelligence), this kinda screwed me up. Once you write embedded_in in your model, that model then becomes an embedded document and some things from documents don't apply any more:
- Mongoid::Versioning
- indexes - Well sorta, you'll see later that you can get around this

Don't create indexes on Embedded Docs! (sorta) - When I initially started to create my Mongoid models, I thought I could do something like this:

class Post 
  include Mongoid::Document
  field :author
  field :text
  field :publish_date, :type => Date
  embeds_many :comments
  index [:author, :asc]
end

class Comment
  include Mongoid::Document
  field :author
  field :text
  field :publish_date, :type => Date
  embedded_in :post, :inverse_of => :comments
  index [:publish_date, :desc]
end

Unfortunately, I kept getting an InvalidCollection Error from Mongoid and for the life of me couldn't figure it out. I knew MongoDB allowed its users to create indexes on an embedded document. After a little testing, I came up with this instead:

class Post 
  include Mongoid::Document
  field :author
  field :text
  field :publish_date, :type => Date
  embeds_many :comments
  index [:author, :asc]
  index ["comments.publish_date", :desc]
end

class Comment
  include Mongoid::Document
  field :author
  field :text
  field :publish_date, :type => Date
  embedded_in :post, :inverse_of => :comments
end

Lesson: If you want to create indexes on embedded documents, create the index in the parent document. Then use a string to denote what you want indexed.

Don't hesitate to drop into the lower level 'mongo' driver - There's a lot that Mongoid already does that's not even listed on its website, like cursors. Yet, I find that I want to do things like map reduce on a collection. Fortunately for us, the mongo driver is a requirement for Mongoid. So when you call Post.collection you actually get a collection object from the mongo driver which then we can run map_reduce. Chaining this example together, it would look something like this:
```
m = 'function(){ this.tags.forEach( function(z){ emit( z , { count : 1 } ); } ); };'
r = <<-REDUCE
function( key , values ){
  var total = 0;
  for ( var i=0; i < values.length; i++ )
    total += values[i].count;
  return { count : total };
};
REDUCE
Post.collection.map_reduce(m, r)
```
There are plenty more functions that can be found in the mongo driver, but if you're looking for map_reduce, you can find it in the main docs and the RDoc.

That's all I've got for now. When I figure out more you can bet that I will be posting it here. MongoDB / Mongoid have been a joy to work with.

Templates in Rails 3

2010-04-19T08:31:00.000-07:00

Templates have been around since version 2.3.0 of Rails. They're great at getting an application started quickly with many of the gems, plugins, and code you need. Pratik has an excellent high level overview on the feature when it was first released. Now that Rails 3 is on the horizon, I just wanted to point out some of the new things that you can do with your templates.

Overview

As I'm sure you know, Rails 3 brings with it a number of rewrites to many areas of the codebase. The code that handles the templates is no different. What I found interesting is that the code that handles many of the actions within the templates comes from both Railties and Thor.

What's the same since 2.3

Many of the commands are the same from the previous iteration of the template feature. They include:

gem
plugin
initializer
rakefile
generate
run
route
inside
ask
yes?
no?
git
environment
run_ruby_script
gsub_file
append_file

If you want more detail on the specifics of these commands check out Pratik's post on the matter.

What's new

Most of the new stuff comes from Thor and is not Rails specific:

add_source - Adds a source line to the top of your Gemfile
apply - Loads an external file and then executes it.
thor - You can run any thor commands that you want within your template.
copy_file - copies a file to a directory that you name
get - will download files and place in a directory/file you name
template - ERB template that you specify and will get executed
chmod - Changes mode
prepend_file - Prepends text to file
inject_into_class - injects text into a given class
inject_into_file - injects text into file
remove_file - deletes files and entire directories
create_file - creates an empty file
empty_directory - creates an empty directory
directory - copies directories. It's also recursive.

That's it! If you need more detailed information on how these functions work, check out the rdoc. The rdoc for the Thor commands be found here and the Rails template commands can be found here I'm pretty excited about these changes and they will definitely help me clean up my templates.

Setting up a Slice with Wordpress

2010-04-02T10:24:00.000-07:00

The whole point of this post is to get Wordpress installed on your slice at Slicehost as quickly as possible. This was my first time doing so and I figured I should create a shell script to complete the process if my slice ever crashed and burned. Skip to the bottom if you want to get right to it.

I love Slicehost. The plans are relatively inexpensive and you know what you're going to get. Recently, my wife wanted to setup a blog and I really didn't feel comfortable setting it up for her on a shared host. She likes it when websites are responsive, who doesn't.

I wanted to keep my costs to a minimum on slicehost, so I opted for the 256 slice. That being said, I needed to keep the memory usage as low as possible. That meant that Apache was out the door and in came nginx. It's been a while since I've had to install php. Let's face it, I'm a ruby guy. Anyway, I'll review the setup and then tell you what the install script does and the assumptions made.

The Setup

PHP 5.3.1
PHP-fpm 0.6
Wordpress 2.9.2
Nginx

Assumptions

Youre logged in as root
Using Ubuntu 9.10
Starting from a newly created slice
You have a hostname to use for the install

What are steps in the bash script?

Collect new passwords for root account, a new user account, and the mysql database
Collect the hostname from the terminal
Reset the root password
Create the new user and it to the sudo list
Update sshd configuration based on this configuration
Update iptables
Upgrade the system
Install needed libraries to compile php
Download PHP 5.3.1, PHP-FPM, and libevent
Patch PHP with PHP-FPM (more info on php-fpm here, here and here)
Compile PHP and install
Create config file for nginx
Install Wordpress

Get the bash script

Enjoy the script!

How to hire the right people

2009-10-15T11:40:00.000-07:00

Being in business school is great. We get to learn about all sorts of topics like economics, marketing, and even organizational development. The one thing that not all students get to learn about is how to hire the right person for the job.

Since I love being a part of startups, especially software startups, I get a lot of people in the business school asking me for advice when it comes to hiring programmers and developers for their businesses. Outside of telling them that they should really have a technical co-founder on the team, I try to give them some advice based on my past experiences.

In one of my past experiences, I was brought on to a team to manage a couple of developers. Unfortunately, I was brought on to the process a little too late. When I was finally debriefed on what was going on, I found out that all of the developers had been hired, and without my input. The process for hiring developers was just a standard 1 hour interview. At first I didn't think anything of it but as the project progressed, cracks started to show in the team. For example, I had one individual who would always promise to do a set of tasks by the end of the day, but then at every status meeting the next day, without fail, some issue came up that prevented him from completing it. His excuses were alway very long winded and convoluted. After a week of this, I found out from other developers on the team that the individual in question kept pestering the other developers to help them solve very simple issues. Not only was this not productive for the other developers but for the project as a whole. Since the project was so short, I didn't have the time to let the person go and hire someone new. I had to just manage around the problem individual. It was at that point, I wished I was part of the hiring process.

Take aways:

Don't think that an hour interview is going to tell you if the interviewee is a good candidate.
You're gonna have to take the time to get to know someone.
Have the person work on small tasks or projects before actually bringing them on. (On Interviewing Programmers)

How do you know when your software meets your needs?

2009-07-07T10:15:00.000-07:00

As a Senior Project Manager at Sparkseed, my job encompasses managing our staff of interns, developing the company's long-term strategy, as well as mentoring any of the Sparkseed funded ventures. Recently, I was asked to mentor one venture, Capital Good Fund (CGF), on what software package to use to implement their back end systems. The team was struggling to determine what package could meet their needs now and into the future. It seemed like a very large task, but I was definitely up for it.

After a few initial emails to get a background on CGF and its mission, I jumped onto a call with one of the founders. One of the very first things that we did was talk about all of the software platforms that they were considering, like MIFOS, Salesforce.com, and Highrise, just to name a few. Almost immediately after identifying all of the software platforms that were up for consideration, the conversation turned into listing all sorts of features that CGF might need.

The key word to point out is "might." Just like with any software development project, I was finding feature creep. We'd discuss one feature and that would certainly lead into another. There were so many things that the team at CGF wanted the software to do that the platform was beginning to look over-complicated.

In terms of providing a solution for CGF, I recommended that we stop talking about technology all together and focus on the core process for the business. In this case, the core business function was the loan process. We then drilled into the loan cycle, how the loans are made, who has authority to make loans, and how the loan gets paid back. At this point, we had a clear understanding of how the business worked and who was involved.

Taking the current staff capabilities into consideration, I was able to recommended a system that had the bare minimum to meet the need of tracking the loan and the various touch points the loan officers would have with their customers.

Side Note: As with any software package, it should be robust enough that if you outgrow the software or want to manipulate it in other ways, the software should let you access the raw data.

Key Lessons learned:

Always pull back. Ask the question "what operational problem am I trying to solve?"
Only use as much software is needed to get the job done.

References:

The Right Mind-set for Managing Information Technology
Avoiding the Icebergs - Talks about software architecture but still pertinent
Avoiding the Icebergs #2
Scope Creep

Getting Up To Speed in Objective-C / Cocoa

2009-06-23T16:05:00.000-07:00

Since playing around with MacRuby, I figured, to keep myself relevant in the programming space I should learn Objective-C / Cocoa. That and it seems every person that has touched Ruby is now hopping on the Objective-C bandwagon. EVERYONE WANTS TO BE AN IPHONE DEVELOPER! Just kidding.

My requirements for learning materials:

I really didn't want to pay for anything, but I'll have to if I must.
I wanted all my materials to be online or pdf.

Basically my requirements cut out things like classes and most screencasts.

This is what I ended up with (its also in order of operation):

Peepcode Objective C Screencast: I have purchased a number of his screencasts in the past and have been pleased, so I figured why not get this one, it was only $9. It was a well put together screencast but I felt like something was lacking. That, or I was being lazy. Yet I found that I wanted more of an introduction to some of the major frameworks on Mac OS X and the iPhone. Uh oh, there's that word again, 'iPhone'. I think if you're just looking for an introduction into the semantics of the language, start here. Otherwise, like me you'll move on to the Stanford Class.
CS193P - iPhone Application Programming: This class was great, and on top of it, it was free! Now I wasn't specifically looking for iPhone application development, but the topics covered were still very pertinent to developing on Mac OS X. I found the instruction to be be very good. I would recommend this course for anyone that has some programming background but finds the classroom setting a little easier to work with.
Cocoa Dev Central: I wish I would have known about this before I started down the path of the Peepcode screencast. Yet since I purchased the Peepcode screencast, a reference was made to this site. I guess I wouldn't have known about it had I not gotten the screencast. Sounds like a paradox to me. Anyway, the site was great and you could find anything you needed to if you're learning about the intricacies of Objective-C and Cocoa.

After thoroughly going through all of the material I've mentioned above, I actually did decide to write an iPhone app. I decided against releasing it to the App Store because, well, I think it wasn't the best quality. Anyway, I don't claim to be a master at Objective-C, but I know enough to be dangerous. I believe that going through the above references will make you just as dangerous!

If you're an absolute beginner, I would recommend starting with the Stanford class first, and then going over to the Cocoa Dev Central site and skipping the Peepcode screencast all together. Now if that's still too much for you, well you need to start all the way at the beginning by learning C. At least you haven't burned any money on education.

If you do decide to write an app for the iPhone here are some posts that I found useful:

Your web app sucks because it doesn't give your users good metrics

2009-03-04T09:06:00.000-08:00

Plain and simple, your web app sucks because it doesn't have good metrics. Now I'm not talking about code coverage or anything internal to your operation. I'm talking about summary statistics for the end users of the application relative to how they interact with it. My case is this, developers hold themselves up to a very high standard when developing their apps using frameworks like Test Driven Development (TDD) and Extreme Programming (XP). If you can take some of the philosophies from those frameworks, TDD and XP, and provide measures for your users, we'll see much happier users in the end who will reap large benefits from your application.

Why Metrics Make Sense

Your Users' Standpoint:Why wouldn't users of your site want the same procedures that developers use to build quality sites?! Continuous feedback, communication, and simplicity. All three are values within XP and are targeted toward increasing productivity. By having a measure to gauge how you're doing, as a user, I can get more value out of using your application and react to any changing conditions.
The web app owner's standpoint: Let's face it, the whole reason you're writing web apps is to get something back. Whether its money or a learning experience. Let's take the money side of that argument. Analytics and metrics can be a boon for your bottom line. Take Twitter for example. With recent articles floating on the web about it's new BizDev guy, Kevin Thau has said that they'll sell analytics to enterprises to make money. That's a lot of money to make to cover the cost of running Twitter.

General steps that you can take to generate good analytics for your data:

Define your core user experience. The important part of defining the user experience is understanding the reason why they're using your service in the first place. Are they trying to manage projects, track their finances, or connect with friends? This part may take some research on your end because despite all of our best efforts sometimes the users of our web apps have totally different goals than what we expect. That being said, consider sending out surveys to a sample population of your users or analyze your Apache web logs for usage patterns. If you're still not sure where to start, try holding a focus group on Skype to at least get you going in the right direction for the surveys.
When trying to summarize multiple observations, consider using nonaveraging metrics (maximums, variance, counts). According to recent research from Shugan and Mitra @ the University of Florida, they found that although averaging metrics may sound great theoretically (like sales per user, or cost per user), most uses of metrics (benchmarking, monitoring, allocating resources, diagnosing problems, etc) require that the metrics provide information (explained variance) about all of the outcomes that you're measuring. For example, a simple count of the number of new products from a company might convey information about a firm's ability to innovate.
Keep the number of metrics relevant and to a minimum. Although I'm advocating for including some type of metric within your application, I also want to caution against providing too many. A danger exists whereby you can give your users too much information. Users can spend too much time trying to absorb everything that you're throwing at them instead of getting what they need and then reacting to it. Another part of that argument is that the information needs to be relevant, which goes back to defining your core user experience. For example, if I had a project management application, a metric to track how many times all of the people on the team logged into the application is not really a useful statistic when you're trying to finish a project. For further reading on this point, check out Russell Ackoff's article about "Management Information Systems." It's an old article, but still relevant to this day.

If you have an opinion on this subject, I'd love to hear about it! Leave a comment.

Want more control over your database tables in DataMapper?

2009-02-24T07:23:00.007-08:00

I had originally submitted a request to the DataMapper team to include an option to allow MySQL database users the ability to specify table options in their DataMapper models. My needs were around data warehousing and I wanted to use a different storage engine like Infobright. Unfortunately this request was denied because dm-core could have ended up looking like spaghetti if my request were approved. Mind you I'm oversimplifying the argument, but I understood where they were coming from. As a result, I bring you dm-mysql!

Overview

Plugin Name: dm-mysql
Summary: Plugin that allows MySQL DataMapper user the ability to specify MySQL table options when DataMapper creates database tables.
Requirements: Your DataMapper version must be > 0.9.10
Options: For available MySQL table options see the MySQL documentation. Note that this plugin was created against MySQL 5.1.
To Install: Make sure you have github listed as one of your sources.

sudo gem install neovintage-dm-mysql

Example

require 'dm-core'
require 'dm-mysql'

# Shows available options
class HasAllOption
  include DataMapper::Resource

  property :id, Serial
  property :name, String

  table_opts :engine => 'MyISAM',
    :avg_row_length => 7,
    :checksum => 1,
    :comment => "Best Comment EVER!!",
    :connection => 'another_connection',
    :data_directory => '/dude/wheres/my/dictionary',
    :delay_key_write => 0,
    :index_directory => '/index/this',
    :insert_method => 'FIRST',
    :key_block_size => 256,
    :max_rows => 5,
    :min_rows => 0,
    :pack_keys => 0,
    :password => 'secret',
    :row_format => 'COMPRESSED',
    :tablespace => 'DUDE',
    :union => 'has_its, not_includeds'

end

As always, I appreciate any feedback on the code or any features that you'd like to see implemented as part of this.

Top 10 Themes to Help Your Start-up from Startonomics (#STNX)

2009-02-06T11:04:00.016-08:00

Startonomics was setup by Dealmaker Media and took place on the wonderful campus of UCLA at the Anderson School of Management. As the website says, the purpose of the conference was to help entrepreneurs create strategies and actionable metrics to help them build better products and make better decisions. I personally was very keen on this event because I have a passion for measuring and managing performance and innovation through metrics.

I recommend that you check out the Startonomics blog for details on the conference as well as all of the video. I learned quite a bit and met many interesting people who were at various stages within their ventures. Yet if you want to get the quick gist of the conference, I bring you my top 10 themes of Startonomics:

Web 1.0 was about things, and Web 2.0 is about people. As a result, your value as a startup is proportional to the number of users / viewers of your site.
If you've got partners in your venture, please, draw up a legal agreement that spells out how equity will be split up. Very important if a partner decides to leave the start-up.
Fail Fast! Admit your failures and move on.
Decisions made in the first 90 days of your venture will have lasting effects.
The most important part of your start-up is passion.
What is your ultimate objective? Is it to make money? Get more users? Pick one and work toward it!
Create a blog and create compelling content. Make sure the design is clean and you cross link all of your content.
When hiring or getting money for your venture, can you see yourself hanging out with this person at a bar?
Hire good athletes - quick learners over those with deep experience.
Trends are more important than milestones.

Spending my career figuring out the best ways to track project / company performance, the last point really hits home. Sometimes managers and even some entrepreneurs get hung up on the idea that progress and processes need to be tracked within the business and tracked to great detail. That's simply not the case. As one of my mentors stated to me, it's more important to be directionally correct versus precisely inaccurate.

Running Rake Tasks in the Background with Monit

2009-01-24T14:49:00.020-08:00

In my last post, I had briefly talked about how I ran a rake task in the background to accomplish the twitter worker functionality. After the course of a couple of weeks, I've got a couple of updates to that process, so that hopefully, you'll have an easier time of running a rake task continuously in the background.

Use Monit instead of god for monitoring

I started with god to monitor my rake task. It was very easy to setup, almost too easy. After a couple of weeks, I started to notice that the god process was starting to leak memory all over the place. I'll qualify this by saying that I use slicehost for hosting tweetrics. I've seen some messages on the slicehost boards that talk about how god doesn't work well on the VMs. That being said, I had to switch to something else. Enter Monit. I didn't have that much experience using Monit so, I had a learning experience in front me. I found some instructions on the web on how to install Monit, which were fairly painless. Within a few minutes, I had everything installed. Now onto getting the rake tasks setup.

Setup the monitrc file

The next step in setting up monit to handle my background task was setting up the monit control file. This is analogous to setting up the configuration file that you'd use with god to handle the monitoring of your processes. I have to admit that the syntax for the monit control file was relatively straight forward. To make things even easier, I found this post from Ilya Grigorik. I copied the monitrc file and put it in my config directory of my merb project. Although Ilya's post was directed toward using mongrel with monit, all I needed to do was change the start and stop program lines to something like this:

start program = "/path/to/bash/script start"
stop program  = "/path/to/bash/script stop"

The purpose of these lines are to start an stop the rake tasks, which leads us to the next step.

Create the bash script to handle the startup of your rake task

Since rake tasks are not daemonized and do not create their own pid files, we'll have to set that up. Time to setup a bash script. I couldn't do this part without looking through the FAQs on the Monit site, more specifically what to do to set up a script for processes that don't create pid files. I'm not the best person when it comes to creating bash scripts, so finding this information in the FAQ was a big help. Now came the part for creating the rake specific parts:

#!/bin/bash
export PATH=$PATH:/usr/local/lib/site_ruby/1.8/rubygems

case $1 in
   start)
      cd /path/to/merb/root/directory;
      echo $$ > /path/to/log/metrics_gen_master.pid
      /path/to/rake twitter:metrics_gen MERB_ENV=production
      ;;
    stop)
      kill `cat /path/to/file/metrics_gen.ppid`
      kill `cat /path/to/file/metrics_gen.pid`
      kill `cat /path/to/file/metrics_gen_master.pid`
      rm -rf /path/to/pid/files
      ;;
    *)
      echo "usage: twitter_worker {start|stop}" ;;
esac
exit 0

Important notes:

In your rake task, make sure that you save the pid and the ppid to a file.
```
`echo "#{Process.pid}" > #{Merb.root}/log/metrics_gen.pid`
`echo "#{Process.ppid}" > #{Merb.root}/log/metrics_gen.ppid`
```
This is important because we'll need to stop the rake processes.
If you're running monit using init be careful of permissions.
Since my rake task loaded the merb environment, it would be running as root and would write to merb.main.pid. If I restarted apache, passenger could not write to the file merb.main.pid and would throw errors instead of running the app.
Don't use & when invoking your rake task!
By using & in the bash script, the rake task will get forked and and the incorrect pid will get written to file. Trust me, I spent a lot of time on this one.

Run it!

Not much I can say here other than take the time to really extensively test your monit script. The log output can be found at /var/log/monit.log. Also, make sure you run it with sudo if you are going to use init.

Tweetrics - Don't do what I did post

2009-01-08T12:21:00.023-08:00

I recently put together tweetrics. It's a site dedicated to tracking public twitter accounts, graphing daily and cumulative activity. I had a lot of fun putting the site together, unfortunately it wasn't without its problems. Before I get into that, lets get into how the site is setup. Backend Setup For the backend setup, I used Passenger and Merb. I know Merb is going to merge with Rails, but I found building tweetrics as an opportunity to get more familiar with the nuancies of Merb. I wanted to use the run_later method in merb to offload the processing of tweets because it's a very processor intensive task, but I decided against it. The main reason being run_later creates a queue per merb process on the server and not a global queue. The global queue was important to me because I did not want multiple instances of the method that builds the twitter activity because my server does not have very much memory. Also, from what I've read on the web, I couldn't use run_later even if I wanted to because it does work behind Passenger. Instead of run_later, I used beanstalk. I found instructions that met my needs at Nuby on Rails. The instructions were pretty clear and the setup of the software to get beanstalk up and running was pretty painless. Like the Nuby on Rails article, I also had a database that recorded the state of jobs in the queue should the beanstalk instance crash. Frontend Setup To get the front end working, I used a combination of JQuery, the Google Visualization API and a little bit of Ajax. Working with the Google Visualization API was a complete breeze. All you need is a couple of lines of javascript in your page and POW! The visualization shows up. I was even suprised the charts could be rendered after issuing some JQuery UI effects. Don't Make These Mistakes After writing this app, I learned a few things. Hopefully by posting this, you won't have the same problems that I did.

When using Beanstalk and the Ruby gem beanstalk-client, make sure that you use the yput method and not put.
yput will serialize the object before putting it into the message queue and then your worker can pull it out and deserialize it. I tried to get around the serialization step by using the put method but for me, I kept getting errors and the job would not get inserted into the queue. I struggled with this for a while before giving up and using yput.
When using Google Visualization API, make sure the user's url and the hostname in Json Requests actually match.
This was quite frankly a stupid mistake on my part. The charts page actually pulls in a static JSON file that has the chart data unfortunately the hostname of the url to request the JSON file was hardcoded as http://tweetrics.info/. If a user accessed the site using http://www.tweetrics.info/ instead, the Google Visualization would not pull the data into the charts.
If you're using the Google Visualization API, load JQuery from Google instead of hosting it on your own site.
Pulling JQuery from Google can save you oodles of bandwidth. Just do it like so.....

Is Rails a Competitive Advantage?

2008-11-18T15:29:00.011-08:00

My short answer.... Not anymore. A competitive advantage can be defined as an advantage a firm has over its competitors. Yet two types of competitive advantages exist, cost advantage and differentiation advantage. Rails currently provides a cost advantage but it’s starting to erode. As for a differentiation advantage, I have a hard time justifying Rails as one in and of itself anymore. In this part one of a two part series, I’ll explore the cost advantages Rails has in the marketplace. Cost Advantage It's defined as the ability of a firm to deliver the same benefits as competitors at a lower cost. I don't want to spend too much time in this area, because countless others (1, 2, 3, 4, 5) have talked about how using Rails to develop web applications has saved, time, money and energy while increasing the speed to market. What I do want to talk about is competing on cost and how that will be difficult to do so with Rails in the near future. Most of the stories that we've heard have been related to how Rails has saved teams so much money in development time. Yet these stories have pitted Rails against Java, .NET, or (insert other framework here). When Rails goes up against these other frameworks it has a distinct cost advantage. But what happens when the Rails is pitted against Rails? Here’s the problem:

Rails community is getting bigger
More businesses have been looking to use Ruby for their projects
Rails is open-source

I’m not saying that because Rails is open-source that’s a bad thing. What I am saying is that the barriers to entry for developers and businesses looking to get into developing with Rails are zero because it’s open-source and freely available for use. Also, that means that more and more people are looking for work. On top of that, because the ecosystem around Rails is very community driven, once a developer sets a best practice with Rails that results in some kind of efficiency, everyone else in the community will copy that almost immediately. That will therefore reduce any cost benefit attained by the original developers. Once everyone starts competing on cost in the Rails community, everyone starts fighting for business based upon their price for getting the application up and running. Customers will then start to make their choices based upon price and the lowest price will always win. The moral of the story is that if you’re gunning for a project where the technology has not been decided upon like should we use Java vs. Rails, Rails can provide a cost advantage. Yet if the customer is already set on using Rails, you’d better come up with another strategy for getting that business other than using cost. That’s something I’ll explore in my next post. For more detail on using cost to compete, read the section on 'Operational Effectiveness' from Michael Porter’s article in Harvard Business, Strategy and the Internet.

MIA - MBA in Action

2008-11-11T17:44:00.004-08:00

There are two truths to the title of this post. I did go MIA(Missing in Action) for a number of months and wasn't able to update the blog. Secondly, the MBA is one of the reasons for my disappearance. I made the decision to get my MBA approximately one year ago from this date and at the time I was living in Chicago. I took my GMATs and submitted all of my paperwork to various programs across the country. Little did I know that the submission was the easy part. Upon acceptance to Pepperdine, I was greeted with a lot of information from the school and not much time to get my life in order before I had to move to California. I will also note that during this time I got engaged and had to plan my wedding! All of this went down approximately in May of this year and these life altering tasks had to be finished before August 16th, the move date. Four months sounds like a long time, but trust me, my wife and I barely got through it. Anyway.... I'm writing this post to inform everyone that the dust has settled and I'm getting back into blogging. This blog will still have a focus on Ruby but I'll be intersecting that with the business aspects of technology. Don't get me wrong, I still love to write code and keep up with the happenings in the Ruby ecosystem. I've redesigned the blog and I'm looking forward to future posts.

Is it possible to do XY Scatter Charts in Gruff?

2008-05-21T11:23:00.008-07:00

YES! You Can! I've forked Gruff on github and written some code to give gruff the ability to plot X and Y points on a graph. Although, I can't take all of the credit for this. Topfunky started the ball rolling and I just took it a little further. I'm hoping to get this committed to the main branch soon.

Get the Code:

As of this point the code is only in my forked github repository. You'll have to download the code from github and create a gem if you'd like to use it.

git clone git://github.com/neovintage/gruff.git
rake gem

Example:

Let's run through an example of how to create the above chart. The code will look very similar to the sample provided on nubyonrails.com

scatter_chart = Gruff::Scatter.new(400)
g.title = "Basic Small Scatter Chart Test"

# Parameter 1 is the name of the dataset
# Parameter 2 is the values for the X-Axis
# Parameter 3 is the values for the Y-Axis
g.data('Chuck', [20,10,5], [5,10,20])
g.data('Brown', [10,12,14,8], [12,6,10,8] )
g.data('Lucy', [8,16,8], [8,10,6] )

g.write('basic_scatter.png')

The only thing that you can't do with the Scatter graph is set labels for the X-Axis. Maybe that could be a future enhancement.

Email me or throw up a comment if you happen to find any bugs in this new graph.

Connecting to Private Repositories on Github using Windows

2008-04-21T14:33:00.008-07:00

I'm going to be honest; I spent a little too much time trying to setup my Windows machine to work with my private repositories on Github. Ok, well it wasn't that much time, but enough for me to warrant a blog posting. For me, the situation really boiled down to step #4 and needing a config file to get Git Bash to work. 1. Install Cygwin. This is actually a requirement for MsysGit. If this doesn't happen you'll receive a fatal error when you try to perform any action against your repository because git won't be able to fork. The error message should read, fatal: Unable to Fork This is forking in the scope of operating systems not forking in the git sense of the term. Don't forget to add the install location of Cygwin to your PATH variable. 2. Install MsysGit. When asked how you would like to use Git at the command line, I selected the Use Git Bash only option. It's the most conservative option and will be less likely to mess anything up on your computer. 3. Generate your SSH keys and post them to Github. Follow the guide provided by the ninjas at Github. 4. Create a configuration file in your .ssh directory. In the directory where you found your public key (id_rsa.pub), you'll need to create a file called config. This will tell Git Bash where to look for your key along with a few other options for connecting. This is what mine looks like:

  Host github.com
  User git
  Port 22
  Hostname github.com
  IdentityFile ~/.ssh/id_rsa
  TCPKeepAlive yes
  IdentitiesOnly yes

Couple of important notes. Make sure that the User line says git. Also the IdentityFile option should point your private key. If in step #3, when you setup your ssh keys, you gave your key file another name, make sure you put that name in the IdentityFile. There you go! That should be it. Troubleshooting Notes: If you run into a some problems getting ssh to work, I would recommend running ssh -v github.com at the Git Bash command line. If you get an error Permission denied (public key). Make sure that your IdentityFile is setup correctly in your config file. For any other errors, I would recommend checking that the other options within the config file are correct.

Automigrating the Test Database (Merb .9 / Rspec / Datamapper)

2008-02-29T13:20:00.005-08:00

In a followup to Marcus' note about migrating the test database, I'd like to point out that when you try to auto migrate in spec/spec_helper.rb using DataMapper::Base.auto_migrate!, it fails. Instead I would use DataMapper::Persistence.auto_migrate!. This command will pick up all of the models that you setup in your app/models folder. Enjoy!

Cloning Merb 0.9

2008-01-28T08:20:00.001-08:00

Merb 0.9 is going to be officially announced in a couple of weeks at acts_as_conference but for those looking to get a jump start on the action (myself included), you can pull down the latest version from github. If you don't already have git installed, I recommend following these instructions on Ramanan's blog for those that use Mac OS X: Building Git on Mac OS X The next step is to clone the source from github. Now its important to note that merb has now been split into two parts, merb-core and merb-more. I'm only going to follow the instructions for merb-core but the same process applies to merb-more.

Clone the merb-core source to your local machine
```
git clone git://github.com/wycats/merb-core.git
```
This will create a new folder in your current working directory named 'merb-core'
Build the gem
```
cd merb-core/
rake gem
```
Install
```
cd pkg/
gem install merb-core-0.9.0.gem
```

And there you have it. You're ready to start playing around with the new merb-core! One quick note, make sure you use merb.9 at the command line instead of merb.

Merb + Namespace + Subdomain

2008-01-20T14:46:00.001-08:00

I'd like to post a clarification on getting namespaces to work within merb. I'm still very new to the merb world and its semantics regarding namespaces. I'll be honest, it tripped me up a little bit. In this example, I'm going to run through getting a subdomain and namespace setup in your router file and how your controller should be structured to take advantage of that namespace. Assumptions:

You know how to generate controllers in merb
Using merb 0.5.2

Example Prep:

I'll be creating a namespace Admin, because I'd like to keep all of my actions associated with administration separate from the rest of my application
Within the Admin namespace, I'll be creating a controller called User.
I want to keep all of the admin controllers on a separate subdomain on my site, something like, http://admin.mysite.com.

Setup:

Create the appropriate routes in the Router file

r.match(:domain => /^admin./).to(:namespace => 'admin') do |admin|
    admin.match('/').to(:controller => 'admin', :action => 'index')
end

Make sure your merb file structure jives with your new Namespace
Merb requires that if you do create a namespace that the resulting controller code be in its own subfolder in the controller folder. The subfolder needs to have the same name as the namespace. In our example it should look something like this:
```
app/
  controllers/
    admin/
      user.rb
```
Setup the Controller based upon the Namespace Note the use of the module declaration. At this point you need to make sure the structure in the code looks something like this:
```
module Admin
  class User < Application
    
    def index
      render
    end
  
  end
end
```

And there you have it, we've successfully setup your controller and routes to take advantage of the use of a namespace in concert with a subdomain.

ActiveRecord Table Name Prefix and Suffix Quirks

2007-10-18T10:50:00.002-07:00

I've just started working with a legacy database schema over the last week when I noticed some quirks in ActiveRecord table naming. Essentially, when you set the prefix in ActiveRecord::Base and then try to set a table name in the model, the model does not pick up the table_name_prefix when your classes are created. Lets walk through an example. I've setup two different classes in a test file that both inherit ActiveRecord. Keep in mind that when we create these classes the default behavior of ActiveRecord is to pluralize the table name. The Project class has its table name overridden with the set_table_name method and the Task class was left to the default behavior. Initial Setup in test.rb:

ActiveRecord::Base.table_name_prefix = 'myprefix.'

class Project < ActiveRecord::Base
  set_table_name 'project'
end

class Task < ActiveRecord::Base
end

In IRB:

>> require 'test'
=> true
>> Project.table_name
=> "project"
>> Task.table_name
=> "myprefix.tasks"

Very interesting behavior indeed. This caused alot of problems for me because I required the use of a table name prefix in my database, but all of my table names really don't follow the ActiveRecord mantra. I had a couple ways of fixing this predicament. 1. Pass a block to set_table_name in the model

class Project < ActiveRecord::Base
  set_table_name do 
    "#{table_name_prefix}project"
  end

This method would work, but in the event that I have to do this for multiple tables, it can get quite cumbersome. Lets look for another option.... 2. Patch ActiveRecord

line 765 in activerecord/lib/active_record/base.rb....
Original:
  def set_table_name(value = nil, &block)
    define_attr_method :table_name, value, &block
  end
New:
  def set_table_name(value = nil, &block)
    define_attr_method :table_name, "#{table_name_prefix}#{value}#{table_name_suffix}", &block
  end

Now this approach works a little bit better. Now I can set the table names in the models without worrying about setting up blocks or doing some other magic. Yet, I'm a little uncomfortable patching base.rb in my ActiveRecord installation for the typical reasons. 3. Override set_table_name in my test.rb file

module ActiveRecord
  class Base
    def self.set_table_name(value = nil, &block)
      define_attr_method :table_name, "#{table_name_prefix}#{value}#{table_name_suffix}", &block
    end
  end
end

This approach is probably the best. At least this way if there are any changes to activerecord in the future, I wont forget what I did to get my table prefixing to work. SIDEBAR: define_attr_method creates an attribute method. A new class method will be created with the given name. In this post, the class method name is table_name.

Errors with Better Nested Set and MySQL

2007-04-10T09:13:00.001-07:00

It looks like theres an error when trying to create nodes for your model that uses the Better Nested Set. You should get something that looks like this:

Mysql::Error: #42000You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ....

Whats happening is that the :scope attribute is not getting valued properly. Instead of one statement coming back, the entire ternary operation statement is getting returned.

Not sure if its been patched yet, if not, we'll have to write something to fix that. I haven't seen anything come across in the mailing lists. In the meantime, when specifying your :scope attributes make sure that you use a string and not a symbol. That seems to work.

Inserting NULL values when trying to create mulitple model objects

2007-04-08T08:43:00.001-07:00

When creating a new object in Rails and you want the value to result in a NULL value in the database, you typically just have to set the value of the field to nil and you're on your way. In my situation, I needed to default the value of the Boolean field on the table to NULL if the user did not select a value. I used a collection select method to allow the user to select "Yes", "No" or to leave the field blank. Then in the controller I thought that I could just refer to the params[:new_object] and I would be good. Unfortunately, when creating the new object the value of the field was always getting set to false. Lets go over how I initially setup my code. In the form:

<% form_tag(save_review_reviews_path(@project, @phase), :method => :put, :id => params[:phase_id]) do %>
  <% @review_points.each_with_index do |rp, index| %>
    <% fields_for "rp[#{index}]", rp do |f| %>
      <%= f.collection_select(:compliant, [{"1" => "Yes"}, {"0" => "No"}], :keys, :values, :include_blank => true) -%>
    <% end %>
  <% end %>
<% end %>

In the controller code:

params[:rp].each_value do |rp|
  @review.review_points.build(:compliant => rp[:compliant])
end

Every time I ran through the tests the value was always getting set to false when the user had left the select box blank. I tried removing the :include_blank in the collection select and putting another hash for nil, {nil => ""}. That didn't work either, the value was still getting set to false. In try number two, I used the code as posted above, and then I tried to default the value of record to NULL through the migration. That failed as well. Finally, I just decided to add an extra step before actually building the record in the controller. As a result, the action worked as I expected, I was able to insert NULL values. This is how the controller looks now:

params[:rp].each_value do |rp|
  rp[:compliant] = nil if rp[:compliant] == ""
  @review.review_points.build(:compliant => rp[:compliant])
end

Too bad we could just use the params object and automatically default the field to null when creating the object.

New Job!!! G'yeah!!!

2007-01-20T12:14:00.000-08:00

It's been a long time coming. I've decided to change jobs. This is my first job change out of college. I've been at that company for 5.5 years. Thats quite some time. I start at the new place on January 29th. My new company is CCC Information Services located within the Merchandise Mart. I'm still downtown which is very nice but now I have a longer walk to get to the office from the train station. Maybe that walk will motivate me to start working out more. At my last job, I led a team of programmers and analysts that performed conversions from 3rd party systems onto the proprietary application. I did a lot of work on a Mainframe using SAS, for the most part. Also, I did do some scripting in the windows shell and using Ruby(very nice! (c) borat), but by and large SAS was the language of choice at my last job. Note that SAS was not the language of my choice. It kinda feels weird saying "last job". At CCC, I expect to be in a lead role as well, but I'll be doing a number of tasks. More of my time will be spent doing Business Intelligence type work, but I will also get to do some of the prototype work in Rails. So right now I'm leading a pilot project to develop a new internal app for the PMO that will go on the intranet. All in all, I'm very pleased with making this new job change and I'm confident it will lead to bigger and better things.

Back from Vail!!!

2006-12-19T09:19:00.000-08:00

Well I got my first ski of the season in. The gf and I made it out to Vail around the beginning of this month, more precisely we were there from 12/6 through 12/10. It happened to be just the two of us and we had a great time. Unfortunately the conditions on the mountain weren't as favorable as we would have liked them to be and I managed to crack one of my skis.... The entire time that we were there it didn't snow at all. That kinda sucked especially since the last snow Vail had was 7 days before we showed up at the resort. Here are a couple of things that I learned on this trip:

Never try to fly in on the early morning flight and try to get to the slopes on the same day, it will only lead to heartbreak.
Always get a 4x4 when driving around the mountains in CO
EZ Car Rental kicks ass
East Vail is too far from the resort
West Vail is where its at.
The back bowls on a warm day in Vail are not a lot of fun
Skiing + Extended Warm Weather = Exposed Rocks = Bad News

I definately recommend going to Vail, but doing so at the beginning of Dec is a real gamble. Some of the folks we talked to at the resort had told us that in years prior they didn't get any snow.

Pics from the trip coming soon

Let it snow!!!

2006-12-01T11:07:00.000-08:00

First snow of the season!!!!! I'm definitely hitting the meager slopes that we have here in Illinois tomorrow. I need the little warm up before I head out to Vail next week. Too bad this is the wet, heavy snow... See how the tree on the left is not holding up too well.

DWL-520e1 working in Ubuntu

2006-12-01T09:12:00.000-08:00

Theres a lot of info out on web concerning how to get the D-Link DWL-520 E1 working in ubuntu. Unfortunately I could not find one complete source for all of the information to properly get this wireless card working. The wiki article on the Ubuntu site is fairly complete, but it was just missing a few critical details. I'll attempt to walk you through the process I followed of finding the actual error on the system, finding the firmware and finally updating the startup scripts. My box is a fairly generic setup that doesn't have any cutting edge hardware installed. I proceeded to install ubuntu v6.10 onto this machine with all the relative ease that is ubuntu. Once installed, everything worked except for the wireless lan card, the afore mentioned DWL-520. So.. I started by looking up a breakdown of the wireless cards that would work upon installation. Just my luck, the DWL-520 doesn't work upon installation, but on this doc, I found the link to the wiki article that explained what needed to be done to install it. I then followed the directions starting with the "Prepare the Firmware" section and proceeded to reboot upon completion. Upon the restart, the card still didn't work. Crap. Now it was time for a little investigation. I did a quick ifconfig at the command line and found that the device wasn't showing up at all. I also found that the device was showing up when examining the devices on the pci bus. So, I did some digging on the web and found that this particular card requires the firmware to be loaded to memory. Unfortunately, the wiki article only goes as far as setting up the /etc/network/interfaces file, but you really need to take it to the next step to actually actively load the firmware. Heres a break down of the extra steps that are needed to get the DWL-520 to work. This assumes that you followed the directions from the wiki article. 1. Make sure you have orinoco_pci, hermes, and orinoco all blacklisted in /etc/modprobe.d/blacklist


# causes the dlink dwl-520vE1 card not to be recognized properly
# at boottime
blacklist orinoco_pci
blacklist orinoco
blacklist hermes

2. Install hostap-utils.


apt-get install hostap-utils


wget http://hostap.epitest.fi/releases/hostap-utils-0.4.7.tar.gz

I actually would recommend the not using apt-get because there is a script that comes with the utils package that makes the setup much easier. Although, if you download the package you have to compile it and install. 3. Move the hostap_fw_load script to /usr/local/sbin, and properly configure for your system.


#!/bin/sh

# Example script for automatically loading firmware images when needed. This
# can be run, e.g., from /etc/pcmcia/wireless.

# Firmware images for the card
# TODO: could try to select correct firmware type automatically

PRI=/etc/firmware/pm010102.hex
STA=/etc/firmware/rf010804.hex
PRISM2_SREC=/usr/sbin/prism2_srec

set -e

if [ -z "$1" ]; then
    echo "usage: $0 "
    exit 1
fi

IFNAME=$1
DIR=/proc/net/hostap/$IFNAME

if [ ! -d $DIR ]; then
    echo "Host AP driver data for device $IFNAME not found in procfs."
    exit 1
fi

if grep -q no_pri=1 $DIR/debug; then
    # no primary image - load primary in two steps, first without PDA and then
    # with full PDA plugging
    echo "Downloading primary firmware $PRI"
    $PRISM2_SREC -gs $IFNAME $PRI
    $PRISM2_SREC -gp $IFNAME $PRI
fi

if grep -q pri_only=1 $DIR/debug; then
    echo "Downloading secondary (station) firmware $STA"
    $PRISM2_SREC -rp $IFNAME $STA
fi

echo "Card is ready with both PRI and STA firmware images"

Please note the top options, PRI, STA, and PRISM2_SREC. This is the complete script, so if you originally did an apt-get of the hostap-utils, you can copy this and make it an executable file. 4. Tweak the /etc/network/interface file Add this line to the wlan0 configuration:


pre-up /usr/local/sbin/hostap_fw_load wlan0

It should look something like this now:


auto wlan0

iface wlan0 inet dhcp
pre-up /usr/local/sbin/hostap_fw_load wlan0
wireless_nick DWL-520
fw_primary /etc/firmware/pm010102.hex
fw_secondary /etc/firmware/rf010804.hex
wireless_mode managed

5. Restart your networking


/etc/init.d/networking restart

Now your card should be working and ready to go. You can now go to System -> Administration -> Networking to setup your credentials to attach to a wireless network.

The Neovintage Collection (new mix)

2006-10-16T21:09:00.000-07:00

Its been quite a while since I've made a mix. Its been even longer since I got new music. Regardless I spend some time today to pick some new tunes out for an hour of listening enjoyment. Please note that its been a while since I last made a mix and my "skillz" weren't ninja-like in the past, so some of the parts are a little choppy, not to mention my levels were off. Tracklist:

Aspect - Soul Lady (blu saphir)
Paul B & Yana Kay - Waiting for your Love (Euphoria)
Logistics - Call Me Back (Now More Than Ever EP - Hospital)
Calibre - Derranged (Derranged EP - Soul:R)
Sonic - Saigon Traffic (Bingo)
Seba - Heaven Sent feat Robert Manos (Soul:R)
Kubicks and Lomax - Edge City (Liquid V)
Logistics - City Life (Now More Than Ever EP - Hospital)
Mathematics and Tactile - Espirit (Social Studies)
Synopix - Veteran (Fokuz)
Total Science - Soul Patrol (Sunny Side Up Mix) (CIA)
Locuss - Foundation (Cymbalism Liquified)
Junior Cartel - Havana (Renegade Recordings)

Download: ~~The link is coming soon.... I'm probably going to redo the mix tomorrow cause I'm not really content with it in its current form.~~ Go and get it at sendspace! Enjoy!

On the side

2006-10-02T21:26:00.000-07:00

I've been wanting to post something over the last couple of days. Just about anything really. I've been working on that SAS testing framework for the mainframe for the last couple of weeks in the office, but really I wanted to talk about whats been going on, on the side. Recently myia has been taking sewing classes. Really, really cool. Maybe she'll get to making me clothes... ;) I have yet to see any of the stuff that shes actually sewn for her first class but I have seen a lot of the materials from the second, pattern design. The pattern design class definately seemed like a lot of work but I'm sure by the end of the semester she'll be able to design clothing. I doubt she'll be the next ralph lauren after this semester, but if she gives it some time, I'm sure she'll be able to make some pretty rad things. As for me, I've been busy doing things around the house and for my little side project. I got this wyld idea that I would make my own hard cider. Where I came up with that idea is still a mystery to me. So, I ended up purchasing a home brewing kit from leeners, namely one specific for hard cider. When I originally purchased it, I had no idea what I was doing. Quite frankly I still don't. The reading material that came with the equipment was pretty inadiquate. So I turned to my trusty friend Google to get the answers. I really didn't find a ton of information out there, but enough to at least get started. Anyway, the kit only took care of the fermentation process, I still needed a way to get the juice from the apples. As a result, over the last weekend, I managed to build my own apple press. It took a little while only because I had other things to do, but I was able to finish. Hopefully, in the coming days, I'll be able to post a little more about my cider making process. Just if you were wondering, I did take pictures the entire time.

TDD and SAS, part II

2006-09-18T06:24:00.000-07:00

My jump into the world of TDD on a mainframe with SAS has been a little frutsrating. I can see why the team that came up with FUTS did not touch the mainframe world. One thing that I have realized is that the macros associated with FUTS will work on a mainframe as is, but we dont have the luxury of having a perl script that will summarize all of the test cases and run all of the test suites which provides a nice little output. My goal is to provide a better means of reporting on the tests that do occur. Since we dont have some of those luxuries on a mainframe, I tried to develop a pure SAS solution for developing, running, and summarizing all of the unit tests. Typically in most other languages, you have seperate object that is setup to perform tests. Then this is run by some sort of test runner that will summarize all of the output. In my case, I've realized that I have to include all of the unit test code within the SAS itself. By that I mean, if I invoke the SAS interpreter I need to include the tests within the same instance as the program that is currently running. Thats pretty straight forward, just include whatever unit tests you want within the code that you want to test. I recommend encapsulating all of your unit tests within another macro so that you can turn the tests on an off. This will be helpful when you move your code to production. That takes care of getting our tests completed. My second concern is with the fancy reporting. Since the output of the FUTS macros go to the SASLOG, we need another SAS program to summarize the results from the log. This is where the recommendation for an encapsulation macro becomes even more useful. Once the encapsulation macro has been invoked, I typically indicate in the current SASLOG that the testing will begin and I save off the path name. Then I change the path of the current SASLOG to something temporary, I'll elaborate on this in a little bit. Next, you should include the code for all of your test cases. I typically create the code in another file, that way that allows for two people to work on the same code. Finally, read in the temporary file and run it through a set of SAS code that will generate our results and outputting it to our destination of choice. This may not be the most awesome way of dealing with testing on the mainframe but its my attempt of trying to deal with writing better code and being more confident of changes that I may put into my code. I'll try to include a copy of my macro code, or at least pseudo code for how this process was set up.

Submitting Jobs within SAS

2006-09-17T15:14:00.000-07:00

I ran across this in the mailing list online. I found it to be pretty helpful. Since all of my work is done on a mainframe I can attest to it working there but for the other environment, windows, unix, etc, you may need to help me out on that one.


FILENAME INJCL 'PATH.TO.JOB' DISP=SHR;
FILENAME OUTRDR SYSOUT=A PGM=INTRDR RECFM=FB LRECL=80;
DATA _NULL_;
INFILE INJCL(JOBNAME);
FILE OUTRDR NOPRINT NOTITLES;
INPUT;
PUT _INFILE_;
RUN;

One thing to note. We are not really "submitting" the job, but we basically are adding it to the internal reader which then adds the job to the queue. So be careful when referring to this section of code as "submitting" a job. For the site that I work at, I've created a macro that I've compiled that will perform this function.

TDD and SAS

2006-09-14T20:14:00.000-07:00

Unlike most other programming languages, SAS does not have its own framework for Testing. This can be somewhat frustrating when developing processes to convert data or to evaluate large datasets. Well that time is over. I found the FUTS framework on the web. It was incredibly easy to find, I just typed "testing" and "SAS" into google and viola! This framework was only released in March so its only been on the block for a little while. It's such a great development because now like JUnit and NUnit, I can preform unit tests alot easier against my code. This framework only works on unix/linux and windows system. That kinda stinks since most of the work that I do is done on a z/OS box. I'm going to attempt to port this to our mainframes at work, because I think it will be able to increase the quality of all of the work that we do. I'll keep everyone updated on the results.

New Site

2006-09-12T08:34:00.000-07:00

Finally.... It's complete. I found a template on the open source web design site that I really liked so I figured I would give it a shot with the new version of blogger thats in beta. I must say that the process was a little annoying in terms of creating the site design. The people at Blogger came up with a new set of tags that can be used to create your entire site. Unfortunately, this tag structure is very rigid. Let me run through an example and then afterward I'll list a few of my annoyances. When encapsulating any kind of function within the template you need to create whats call a widget. Then within that widget you have to create multiple code blocks called includeables. The includables function like subroutines analogous to the subroutines you'd have in Visual Basic. All of the includeable tags need to have an id attribute set to identify it between the others in the widget. Also, you need to set at least one of the includeables with an id of main. Thats the main entry point for the widget. Then you can reference the other includeables from within the "main includeable". One more thing, for each widget that you create on the page, you must encapuslate that within a section. You can have multiple widgets within a section. Now when the page is actually rendered, the widget and section tags will actually get transformed into <div> tags. Now lets run down the annoyances.....

When creating a widget and setting the class variable, Blogger overrides anything that you set in this attribute when rendering the page. This removes the ability to set the class variable to something of your choosing. You could probably use the rendered name in the class attribute in your css, but since class is unique for each widget, it kinda defeats the purpose.
The only tags that can be in the Blogger template section tags are widgets, and I mean only. You're not allow to have any comment tags either. I think it makes the template a little messy.
Forcing widgets to be within section tags. I found this a little annoying because it created another layer of div tags in the markup. It made it a little difficult when dealing with the css. I'm still trying to figure out what use the section tag has other than containing widgets.
The help file is still a little sparse, but since the new service is still in beta, we'll let this one slide.

The new features of Blogger are nice, but I wouldn't call them earth shattering. If the template markup was a little more flexible, I'd probably feel a little better about Blogger as a service. I still would recommend you check them out for you site.

Being Around

2006-05-21T19:52:00.000-07:00

I spent most of this weekend contemplating and thinking. No..... I wasn't alone most of the time. I actually had things to do and I was relatively busy the entire time. I one thought seemed to come up more than a couple of times. Making plans. Do some people expect everyone else to make all of the plans? That question was a little extreme, but I'm referring to something more subtle than that. The rhtorical question of the day..... Do you feel like nothing happens unless you explicitly say that youre free?

Xterra!

2006-05-10T14:28:00.000-07:00

Look who enjoys work

2006-04-28T14:11:00.000-07:00

I guess the cat likes staring at the computer while working at home. At least one of us does......

Necessary Evil

2006-03-07T21:56:00.000-08:00

I recently purchased a new car. Ahh... cars. Something like like to deem a necessary evil. Really, if you dont live in a major metro area, you definately need to have that car. Theres just no other way to get around town. Unfortunate at best, but oh well. You must be thinking, well what kind of car did you get? If cars are such a necessary evil, you must have purchased something thats modest. So... I ended up purchasing a Nissan Xterra. I know, I know, if cars are such an evil, why did I get something that really isn't so modest. I figured I really wanted something with utility and that wasn't so cushy. I really trashed my last car, and I didn't want anything that wouldn't be able to handle a little beating up. Anyway, I'm pleased with my purchase, although the gas prices piss me off sometimes, but I guess thats what I get for buying an suv. Maybe I'll get some pics up here of the new wheels.

2004-08-06T14:14:00.000-07:00

something goes here.... who knows what?