An experiment on finding a way to create a base controller to inherit from when dealing mostly with CRUD operations.

Have you noticed that if you are working on an app, and need to create an admin interface for it, most of the work to be performed is your typical create, update, destroy and list actions?

In my “day-to-day work” in Rails-based applications, this is typically the case. I bet the same case presents to you as well, and though I am aware of sound solutions like the ActiveAdmin and RailsAdmin gems, in some edge cases the queries returned by their default implementations always leave something to be desired.

Now, let’s not confuse this with an attachment to the NIH (Not Invented Here) syndrome, but sometimes, having an in-house and flexible implementation is more than sufficient to achieve “Just the Right Feature^TM”.

Getting back to day-to-day cases in controllers, specially in a Rails application, you can easily find a pattern:

• you need to retrieve a record to display it
• you need to retrieve a record to edit/update it
• you need to delete a record
• you need to create a new record
• you need to list all existing records

Though one of Rails tenets is DRY (Don’t Repeat Yourself), you find yourself adding similar actions across multiple controller files, only changing model and routes specific details.

Let’s see an example

Suppose you’re working in a Rails app that deals with a Books and Movies catalog (among other things). You will obviously need an interface to perform CRUD operations on these resources, so you start working your way through the first controller.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

# app/controllers/admin/books_controller.rb
class Admin::BooksController < ApplicationController
  def index
    # code to list all books
    # could include logic for ordering, pagination, etc.
  end

  def show
    # code to retrieve a single book
  end

  def new
    # code to instantiate a new book
  end

  def create
    # code to instantiate and persist a new book
  end

  def edit
    # code to retrieve a single book
  end

  def update
    # code to retrieve a single book
    # and persist updated info
  end

  def destroy
    # code to delete a single book
  end

  private

  def books_params
    # rules for accepted params to pass to a book instance
  end
end

Then, comes the time to work on the next controller and you come up with this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

# app/controllers/admin/movies_controller.rb
class Admin::MoviesController < ApplicationController
  def index
    # code to list all movies
    # could include logic for ordering, pagination, etc.
  end

  def show
    # code to retrieve a single movie
  end

  def new
    # code to instantiate a new movie
  end

  def create
    # code to instantiate and persist a new movie
  end

  def edit
    # code to retrieve a single movie
  end

  def update
    # code to retrieve a single movie
    # and persist updated info
  end

  def destroy
    # code to delete a single movie
  end

  private

  def movies_params
    # rules for accepted params to pass to a movie instance
  end
end

Then comes the time to work on another controller, and another, and another, and yet another controller… and if you pause for a moment and look at the big picture… you basically have only one controller repeated N times your models count.

How could we refactor these controllers into a flexible base use case?
Well, I recently had some time to read Growing Rails Applications in Practice and pretty much liked the suggestion on the “Beautiful controllers” chapter and decided to take it as a base to build upon… after practicing in an app I was developing at that moment, I came up with:

A small step in refactoring

Going back to our example… Let’s declare a base controller and a mixable methods module:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118

# app/controllers/admin/crud_methods.rb
module Admin
  module CRUDMethods
    def list_resources
      @resources = model.load
    end

    def build_resource(attrs = {})
      @resource = model.new(attrs)
    end

    def load_resource(id)
      @resource = model.find(id)
    end

    def save_resource
      if @resource.save
        success_action('Successfully created resource.')
      else
        error_action(@resource, :new)
      end
    end

    def update_resource(attrs)
      if @resource.update_attributes(attrs)
        success_action('Successfully updated resource.')
      else
        error_action(@resource, :edit)
      end
    end

    def delete_resource(id)
      load_resource(id)
      @resource.destroy
      destroy_success_action
    end
  end
end

# app/controllers/admin/base.rb
class Admin::BaseController
  include Admin::CRUDMethods

  class NotImplementedError < StandardError; end

  def index
    list_resources
  end

  def new
    build_resource
  end

  def create
    build_resource(accepted_params)
    save_resource
  end

  def show
    load_resource(id_key)
  end

  def edit
    load_resource(id_key)
  end

  def update
    load_resource(id_key)
    update_resource(accepted_params)
  end

  def destroy
    delete_resource(id_key)
  end

  private

  def model
    fail(
      NotImplementedError,
      'Configure main ActiveRecord class to manage in this controller.'
    )
  end

  def id_key
    fail(
      NotImplementedError,
      'Configure params[:key] needed to retrieve a record for this controller.'
    )
  end

  def accepted_params
    fail(
      NotImplementedError,
      'Configure the required key and accepted params for a record in this controller'
    )
  end

  def success_action
    fail(
      NotImplementedError,
      'Configure what to do when saving/updating a record succeeds.'
    )
  end

  def error_action(resource, view)
    @resource = resource
    flash[:error] = 'Something went wrong.'
    render view
  end

  def destroy_success_action
    fail(
      NotImplementedError,
      'Configure what to do when deleting a record succeeds.'
    )
  end
end

I know, I know… this looks like an awful lot of boilerplate… but bear with me for a second while I show you how the Books and Movies controllers will look like now that we have this base class in place.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

# app/controller/admin/books_controller.rb
class Admin::BooksController < Admin::BaseController
  private

  def model
    Book
  end

  def id_key
    params[:id]
  end

  def accepted_params
    params.require(:book).permit(
      :title, :author, :isbn, :page_count, :price, :category, :etc
    )
  end

  def success_action(message)
    redirect_to admin_book_path(@resource), notice: message
  end

  def destroy_success_action(message)
    redirect_to admin_books_path, notice: message
  end
end

# app/controllers/admin/movies_controller.rb
class Admin::MoviesController < Admin::BaseController
  private

  def model
    Movie
  end

  def id_key
    params[:id]
  end

  def accepted_params
    params.require(:movie).permit(
      :title, :director, :upc_code, :duration, :price, :category, :etc
    )
  end

  def success_action(message)
    redirect_to admin_movie_path(@resource), notice: message
  end

  def destroy_success_action(message)
    redirect_to admin_movies_path, notice: message
  end
end

If you take a look at the resulting controllers, you’ll notice we have separated the things that change (params, model, routes, etc. on each derivative controller), from those that don’t (the base CRUD methods). I bet there’s even a pattern name for this ; )

Now, suppose you need to work on building CRUD actions for a MusicAlbum controller… guess what that would look like? And the next controller you need to implement the same actions all over again?

Exactly… now that we have ourselves a base platform, we can re-use and override at will; do you need a more complicated logic in any of the actions? do you need to ensure your queries are database optimized when retrieving a collection or a single record and its associations?
Not a problem, simply override the method you need, be it the main CRUD-based action in the controller (index, show, update, etc.), or the abstract methods provided by the base controller or the mixable module (load_resource, update_resource, etc.) and voilá! you can have just the flexibility you need while keeping your eyes in the big picture.

While this is not a silver bullet design, I pretty much like what it provides so far. Let’s not forget that we don’t need all our controllers to inherit from this base controller; we need to evaluate the corresponding use case, but if CRUD operations is what you need repeated ad-infinitum in a uniform approach, why not take this approach for a spin?

Do you like what we’ve done here? Do you have a different opinion or approach? I’d love to know your comments on this topic. You can tweet or email me and start a conversation.

Weeks ago, Marvel Comics announced the release of its Comics API. I’ve always been fond of comics, and in the spirit of learning how to consume APIs, I thought this would be a great pet-project to work on: A gem that interacts with this API. You can access the end product here.

So, I began working on an initial draft (paper mostly) after reading Marvel’s Developer portal instructions about API requests.

My first steps into pulling data from Marvel’s API was through the browser just passing query params in the url bar, and watching a JSON response appear in the window (I highly recommend the JSON View browser extension to enhance the experience).

My first gem release was nothing fancy or to be really proud of, merely two endpoints available (/characters and /characters/:id) and a rough sketch of how to add more endpoints in time… but hey, I released something into the public and just shortly after the API was available.

After one week of iterating over the gem design, working merely at night for a couple of hours each time, here are the lessons I learned from this:

1. Agree on a public API for (potential) users.
2. Metaprogram for fun and profit.
3. Do you want people to contribute? Add tests!
4. Provide detailed documentation to users.
5. Add tests. Seriously, add tests!

Let’s elaborate on each of these points.

Agree on a public API for users

Perhaps it’s because my programming experience is heavily influenced by Rails, when I started drafting how I wanted to expose methods in this gem, I decided to use sort of Rails-y conventions:

1
2
3
4
5
6

comics(123)
characters(234)

# and why not, the option to pass a character's name to retrieve it,
# because we don't know ids (not yet at least).
character('Spider-Man')

When I realized I needed to implement a small router that should provide an API client with available routes, I thought it would be a good idea to implement them like this:

1
2
3
4
5

comics_path
comic_path(123)
character_comics_path(234)

# yes, a Rails like router ;)

These decisions influenced every line of code that came after them.

Metaprogram for fun and profit

When I was working on the gem, I had the opportunity to pair-program with Ismael Marín, a friend from the Bajío on Rails community. During our session, he pointed out that my (at the moment) current codebase could benefit from using metaprogramming tricks, which in the end will result in reducing the maintenance burden of adding new endpoints and functionality as the Marvel API makes them available.

And so, I went from manually-defined methods like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

## WARNING!: The following is fugly code. It has been refactored FTW :)

# code from the Marvelite::API::Router class

def characters_path
  "/#{api_version}/public/characters"
end

def character_path(id)
  "/#{api_version}/public/characters/#{id}"
end

def character_comics_path(id)
  "/#{api_version}/public/characters/#{id}/comics"
end

# code from the Marvelite::API::Client class

def characters(query_params = {})
  response = self.class.get("/#{api_version}/#{router.characters_path}",
    :query => params(query_params))
  build_response_object(response)
end

def character_comics(id, query_params = {})
  if id.is_a?(String)
    character = find_character_by_name(id)
    return false unless character
    id = character.id
  end

  response = self.class.get("/#{api_version}/#{router.character_comics_path(id)}",
    :query => params(query_params))
  build_response_object(response)
end

You can already start discussing the burden this will create for maintenance, just by manually adding new methods to tackle access to all available API endpoints.

With just cause, you will be talking some sense to me to fix this ASAP. Just want to point out in my defense, that this was only exploratory code to have something out-the-door.

After some thinking, tweaking and refactoring, I ended up with something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

# code from the Marvelite::API::Router class

def add_route(name, endpoint)
  routes["#{name}_path".to_sym] = { :name => name, :endpoint => endpoint }
end

def method_missing(method, *args, &block)
  if routes.keys.include?(method)
    endpoint = "#{routes[method][:endpoint]}"
    params = *args
    if params.any?
      params[0].each do |p_key, p_value|
        endpoint.gsub!(":#{p_key.to_s}", p_value.to_s)
      end
    end
    "/#{api_version}#{endpoint}"
  else
    super
  end
end

# code from the Marvelite::API::Client class

def build_methods
  @router.routes.each do |_, hash|
    name = hash[:name]
    self.class.send(:define_method, name) do |*args|
      params = process_arguments(*args)
      response = fetch_response(name, params)
      build_response_object(response)
    end
  end
end
# just call build_methods on initialization, and voilá, our methods are there.

You can see the finished files here: Marvelite::API::Router and Marvelite::API::Client.

The benefit of it all? I would only need to have a file where I would map the original API endpoints to a route-like structure my gem would understand.

There’s still work to do, I must confess, but I think this is a good first step in the right direction. This way, I reduced the maintenance burden adding new API endpoints, just add a new line and it’s done, how cool is that?

Do you want people to contribute? Add tests!

Since day one I started working on this, I invited my friends from the Mexicali Open Source community to join in and contribute if they thought like it.

Question was, how to be sure things work while having contributors get up to speed understanding the pieces of the gem? Easy, add tests and be descriptive about the functionality in place.

I think this would be better explained through an image, from one of the pull requests I received.

Thinking back, someone commenting you have done adding endpoints an easy task, is the best compliment to decisions made when building the project… and definitively, that made my day.

Provide detailed documentation to users

As a developer, I spend most of the time making things that work and then refactor them… and most of the time, only me or another developer understands what I ended up with, either because you assume they’ll consult specs or read the source.

So, I set as a personal goal, that as I would have no control over who may end up using my gem (maybe a seasoned developer that lives and breathes RTFM or a newbie developer, who knows), to write human, non-geek, non-developer exclusively, understandable documentation on how to use the gem.

I’d like to say that I made it right, but I can’t be sure. Although if I judge by the numbers in this image, I like to think, documentation has made a difference.

Add tests. Seriously, add tests!

You would think that in the Ruby/Rails community, where in every conference you attend to, everyone spreads the Gospel of Testing… truth be told, not everyone practices this.

I could count with the fingers in one hand, how many projects that I’ve worked on, really abide by test-driven development. Yes, that few. And have been working on development projects for nearly 5 years now.

I will not argue about this point, and I will not say whoever doesn’t practice TDD is wrong and even less I will not say I am strict practitioner of it, because truth be told, I am not.

But hey, not being a strict practitioner does not mean I don’t want to become one. So, I set as a skill-building exercise to add tests to prove me (and potential contributors) that things worked as expected.

Was it hard? was it fun? did it help?

Yes, yes and yes.

Remember when I was discussing refactoring to make use of metaprogramming? Well, that wouldn’t be possible nor an enjoyable task if it wasn’t because of the tests in place to let me know when things broke in the course of refactoring.

Also, remember the pull request from above? It would have not been possible without, yes you guessed, the tests in place that explained another person what endpoint was being tested and how to replicate it to test another endpoint.

Overall, I must confess this has been an interesting journey and experience. I believe I have learned more in the course of a week working on a pet-project, than in months of contract work.

I’m really looking forward for Marvel to add new endpoints or release a newer version of its API, to get back to the drawing board and address whatever issues may arise from the change.

The main topic after all this, I liked how pair-programming went out. And honestly, would like to experience more of it. So, if you have a pet project yourself and think maybe pairing could help you, let me know. I for sure, know it will be great for me.