How to get from they’re to they’re

In last week’s article, you learned a short process that solves most encoding problems. But there’s one encoding problem that’s much harder to solve.

I know you’ve seen it. (Or maybe you’ve seen it?) It’s when a curly quote turns into ’, or an em-dash turns into —. It’ll make you think you’ve gone crazy. It should just work!

You could create a giant table, so you could find bad characters and replace them with good ones:

1
2
3
4
5
6
[{broken: '–',    fixed: "—"}
 {broken: "—",    fixed: "–"}
 {broken: "‘",    fixed: "‘"}
 {broken: "’",    fixed: "’"}
 {broken: "“",    fixed: "“"}
 {broken: "”", fixed: "”"}, ...]

But there’s an easier, more reliable way to fix those broken characters.

Why does good typography always break?

Last week, you learned that an encoding is just a way to turn groups of meaningless bytes into displayable characters. Not every character can be represented in a single byte, because there are more than 256 possible characters. So some characters, like the curly quote , are represented with more than one byte:

1
2
irb(main):001:0> "they’re".bytes
=> [116, 104, 101, 121, 226, 128, 153, 114, 101]

Even though the string only has 7 characters, they’re represented by 9 bytes!

When you focus on just the curly quote:

1
2
irb(main):002:0> "’".bytes
=> [226, 128, 153]

You’ll see it uses 3 bytes. And our messed up string, they’re, has three characters where it should just have one. That seems like more than a coincidence, right?

It seems like those three bytes should be read as UTF-8, where they’d represent a curly quote. Instead, each byte is showing up as a different character. So, which encoding would represent [226, 128, 153] as ’? If you look at a few tables of popular encodings, you’ll see it’s Windows-1252.

You can check this in irb:

1
2
irb(main):003:0> "they’re".force_encoding("Windows-1252").encode("UTF-8")
=> "they’re"

(We need that last .encode("UTF-8") to display the string in the console.)

Yep! That’s the problem. But it gets worse.

The data is supposed to be UTF-8, but is being misread as Windows-1252. But you’ll probably save that data to a database, or a file, as UTF-8. Ruby will helpfully convert it to UTF-8 for you, so you’ll end up with:

1
2
3
4
irb(main):004:0> "they’re".force_encoding("Windows-1252").encode("UTF-8")
=> "they’re"
irb(main):005:0> "they’re".force_encoding("Windows-1252").encode("UTF-8").bytes
=> [116, 104, 101, 121, 195, 162, 226, 130, 172, 226, 132, 162, 114, 101]

Your string has been badly-encoded twice. Those broken characters now look like they’re supposed to be there. And if you didn’t know how it happened, it’d be almost impossible to untangle it.

How do you fix it?

How do you get things back to normal? Let’s think about the problem backwards:

  • You have a UTF-8 string, (they’re)

  • converted from a Windows-1252 string, (they’re)

  • whose bytes should have been read as UTF-8 (they’re)

To fix it, you just have to follow those backwards steps. Use encode to convert the UTF-8 string back into a Windows-1252 string. Then, use force_encoding to force that mis-encoded Windows-1252 string to be read as UTF-8:

1
2
irb(main):006:0> "they’re".encode("Windows-1252").force_encoding("UTF-8")
=> "they’re"

Fixed!

There’s one small problem…

Unfortunately, you probably found this problem because a bunch of files or database records had badly encoded data in it. And not every file or record is necessarily badly encoded – you might have a mix of good and bad data. Especially if that data came from the people visiting your site.

If that’s the case, you can’t blindly run that code on every string:

1
2
3
4
irb(main):007:0> "they’re".encode("Windows-1252").force_encoding("UTF-8")
=> "they’re"
irb(main):008:0> "they’re".encode("Windows-1252").force_encoding("UTF-8")
=> "they\x92re"

If you run it on good data, you’ll just turn it into bad data. So what can you do?

You can use a heuristic: only change strings that have one of the bad characters in them, like â. This works well if a character like â won’t ever appear in a valid string.

The last time I fixed this kind of bug, though, I wanted to play it safe. I used another useful tool to help: my eyes.

Whenever I found a badly encoded string, I printed it out, along with its replacement:

1
Changing title with ID 6 from "They’re over there!" to "They’re over there!"

That way, I could spot-check the small number of strings that changed, and make sure they didn’t break any further.

I think I have a headache

Like I said last week, keeping different interpretations of the same data straight in your head is hard! But if you’re confused, exploring in an irb console will help. So try it out! Open one up, and see if you can go back and forth between and —, or and “.

Practicing complicated ideas like these is the fastest way to feel confident when you need them. And in the free sample chapter of Practicing Rails, you’ll learn the best techniques and processes to do just that.


3 steps to fix encoding problems in Ruby

You only really think about a string’s encoding when it breaks. When you check your exception tracker and see

1
Encoding::InvalidByteSequenceError: "\xFE" on UTF-8

staring you in the face. Or maybe “they’re” starts showing up as “they’re”.

So, when you have a bad encoding, how do you figure out what broke? And how can you fix it?

What is an encoding?

If you can imagine what encoding does to a string, these bugs are easier to fix.

You can think of a string as an array of bytes, or small numbers:

1
2
irb(main):001:0> "hello!".bytes
=> [104, 101, 108, 108, 111, 33]

In this encoding, 104 means h, 33 means !, and so on.

It gets trickier when you use characters that are less common in English:

1
2
irb(main):002:0> "hellṏ!".bytes
=> [104, 101, 108, 108, 225, 185, 143, 33]

Now it’s harder to tell which number represents which character. Instead of one byte, is represented by the group of bytes [225, 185, 143]. But there’s still a relationship between bytes and characters. And a string’s encoding defines that relationship.

Take a look at what a single set of bytes looks like when you try different encodings:

1
2
3
4
5
6
7
8
9
10
11
12
13
# Try an ISO-8859-1 string with a special character!
irb(main):003:0> str = "hellÔ!".encode("ISO-8859-1"); str.encode("UTF-8")
=> "hellÔ!"

irb(main):004:0> str.bytes
=> [104, 101, 108, 108, 212, 33]

# What would that string look like interpreted as ISO-8859-5 instead?
irb(main):005:0> str.force_encoding("ISO-8859-5"); str.encode("UTF-8")
=> "hellд!"

irb(main):006:0> str.bytes
=> [104, 101, 108, 108, 212, 33]

The bytes didn’t change. But that doesn’t look right at all. Changing the encoding changed how the string printed, without changing the bytes.

And not all strings can be represented in all encodings:

1
2
3
4
5
irb(main):006:0> "hi∑".encode("Windows-1252")
Encoding::UndefinedConversionError: U+2211 to WINDOWS-1252 in conversion from UTF-8 to WINDOWS-1252
 from (irb):61:in `encode'
 from (irb):61
 from /usr/local/bin/irb:11:in `<main>'

Most encodings are small, and can’t handle every possible character. You’ll see that error when a character in one encoding doesn’t exist in another, or when Ruby can’t figure out how to translate a character between two encodings.

You can work around this error if you pass extra options into encode:

1
2
irb(main):064:0> "hi∑".encode("Windows-1252", invalid: :replace, undef: :replace)
=> "hi?"

The invalid and undef options replace characters that can’t be translated with a different character. By default, that replacement character is ?. (When you convert to Unicode, it’s �).

Unfortunately, when you replace characters with encode, you might lose information. You have no idea which bytes were replaced by ?. But if you need your data to be in that new encoding, losing data can be better than things being broken.


So far, you’ve seen three key string methods to help you understand encodings:

  • encode, which translates a string to another encoding (converting characters to their equivalent in the new encoding)

  • bytes, which will show you the bytes that make up a string

  • force_encoding, which will show you what those bytes would look like interpreted by a different encoding

The major difference between encode and force_encoding is that encode might change bytes, and force_encoding won’t.

A three-step process for fixing encoding bugs

You can fix most encoding issues with three steps:

1. Discover which encoding your string is actually in.

This sounds easy. But just because a string says it’s some encoding, doesn’t mean it actually is:

1
2
irb(main):078:0> "hi\x99!".encoding
=> #<Encoding:UTF-8>

That’s not right – if it was really UTF-8, it wouldn’t have that weird backslashed number in it. So how do you figure out the right encoding for your string?

A lot of older software will stick to a single default encoding, so you can research where the input came from. Did someone paste it in from Word? It could be Windows-1252. Did it come from a file or did you pull it from an older website? It might be ISO-8859-1.

I’ve also found it helpful to search for encoding tables, like the ones on those linked Wikipedia pages. On those tables, you can look up the characters referenced by the unknown numbers, and see if they make sense in context.

In this example, the Windows-1252 chart shows that the byte 99 represents the “™” character. Byte 99 doesn’t exist under ISO-8859-1. If ™ makes sense here, you could assume the input was in Windows-1252 and move on. Otherwise, you could keep researching until you found a character that seems more reasonable.

2. Decide which encoding you want the string to be.

This one’s easy. Unless you have a really good reason, you want your strings to be UTF-8 encoded.

There’s one other common encoding you might use in Ruby: ASCII-8BIT. In ASCII-8BIT, every character is represented by a single byte. That is, str.chars.length == str.bytes.length. So, if you want a lot of control over the specific bytes in your string, ASCII-8BIT might be a good option.

3. Re-encode your string from the encoding in step 1 to the encoding in step 2.

You can do this with the encode method. In this example, our string was in the Windows-1252 encoding, and we want it to become UTF-8. Pretty straightforward:

1
2
irb(main):088:0> "hi\x99!".encode("UTF-8", "Windows-1252")
=> "hi™!"

Much better. (Even though the order of the encodings in that call always seemed backwards to me).


It can be brain-bending to imagine different interpretations of the same array of bytes. Especially when one of those interpretations is broken. But there’s a great way to become a lot more comfortable with encodings: Play with them.

Open an irb console, and mess around with encode, bytes, and force_encoding. Watch how encode changes the bytes making up the string. Build intuition about what different encodings look like. When you’ve grown more comfortable with encodings and use these steps, you’ll fix in minutes what would have taken you hours before.

Finally, if you want to learn how to make a habit out of learning these kinds of things by doing, grab the free sample chapter of my book. Breaking things in the console is a really fun way to study ideas like this.


Keeping your logs from becoming an unreadable mess

When you run into a strange, seemingly unsolvable bug, improving your logging can be the best step you can take. Great logging is the easiest way to detect and fix entire classes of bugs. When you log enough information, you can see how your data changes during a request. You can track the calls you make to other services, and investigate the response. In fact, when debuggers failed, logging helped me fix the toughest bug I’ve ever run into.

But log too much, and your log files will quickly turn into a jumble of unreadable, unhelpful messages. How can you slice just the information you care about out of that pile of data? Can you print messages in a way that’s easy to filter later?

Marking your log messages

Rails includes TaggedLogging, which can help you quickly categorize related log messages. When you tag a logger, you’ll get a marker at the beginning of your message. So instead of:

1
2
3
Finding people...
  Person Load (0.3ms)  SELECT "people".* FROM "people"
Found 0 people!

You could tag the Rails logger:

1
2
3
4
5
logger.tagged("People") do
  logger.debug "Finding people..."
  @people = Person.all
  logger.debug "Found #{@people.length} people!"
end

And you’d see something like this:

1
2
3
[People] Finding people...
[People]   Person Load (0.3ms)  SELECT "people".* FROM "people"
[People] Found 0 people!

Now, log messages that care about different things can look different.

Some tagged logger examples

As you log more often, and log more complicated things, you’ll naturally notice areas where those tags will make your messages clearer. But there are a few places I’ve found tagged logging particularly helpful. I’ll usually tag those right away.

You can log requests you make to other APIs:

1
2
3
4
5
6
7
logger.tagged("GitHub API") do
  uri = URI("https://api.github.com/repos/rails/rails/tags")

  logger.info { "Fetching #{uri}" }
  tags = JSON.parse(Net::HTTP.get(uri))
  logger.info { "First tag: #{tags.first["name"]}" }
end
1
2
[GitHub API] Fetching https://api.github.com/repos/rails/rails/tags
[GitHub API] First tag: v4.2.4.rc1

That way, you can easily see how and when your app is talking to that API.

(This works particularly well with Faraday middleware, or if you only communicate with a server through a Gateway).

Background jobs also work well with tagged logging:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
require "active_support/tagged_logging"

Resque.logger = ActiveSupport::TaggedLogging.new(Resque.logger)

module LoggedJob
  def around_perform_log_job(*args)
    logger.tagged(name) do
      logger.info { "Performing #{name} with #{args.inspect}" }
      yield
    end
  end
end

class MyJob
  extend LoggedJob

  def self.perform(*args)
    ...
  end
end

Now, any job that extends LoggedJob will have all of its log messages tagged with the class name of the job.

And if you have a logged-in user, you could tag messages with their user ID:

1
2
3
4
5
logger.tagged(current_user_id ? "user-#{current_user_id}" : "user-anonymous") do
  logger.debug "Finding people..."
  @people = Person.all
  logger.debug "Found #{@people.length} people!"
end
1
2
3
[user-123] Finding people...
[user-123]   Person Load (0.3ms)  SELECT "people".* FROM "people"
[user-123] Found 0 people!

Finally, if you add a line to your config/environments/production.rb (or development.rb), you can have Rails automatically tag your messages:

1
config.log_tags = [ :subdomain, :uuid ]

log_tags lists the tags you want to appear at the beginning of every Rails log entry. Each symbol refers to a method on ActionDispatch::Request, so :uuid means request.uuid.

You can also pass a Proc that takes a request object:

1
config.log_tags = [ :subdomain, :uuid, lambda { |request| request.headers["User-Agent"] } ]

But I don’t see that very often.

These default tags are nice: uuid can tie together all the log entries that happen in one request, and if you’re keeping sessions on the server, the session ID is also helpful. With those tags, and enough messages, you can trace some really complicated paths through your app. And usually, that’s what it takes to figure out how a nasty bug happened.

How much do you use the Rails logger in your apps? Have you tried tagged logging? If you haven’t, try finding a place for it. Tagging actions taken by your users is a good start. It’ll help you out the next time you have to debug a crazy multi-step bug.

If you want to learn more about logging and other debugging techniques, I’ve dedicated an entire chapter of Practicing Rails to finding and fixing the errors you’ll run into as you create your apps. Grab the first chapter for free here.


Should you use scopes or class methods?

Scopes are a great way to grab the right objects out of your database:

app/models/review.rb
1
2
3
class Review < ActiveRecord::Base
  scope :most_recent, -> (limit) { order("created_at desc").limit(limit) }
end

You’d use the scope like this:

app/models/homepage_controller.rb
1
@recent_reviews = Review.most_recent(5)

Calling that scope, though, looks exactly like calling a class method on Review. And it’s easy to build it as a class method, instead:

app/models/review.rb
1
2
3
def self.most_recent(limit)
  order("created_at desc").limit(limit)
end
app/controllers/homepage_controller.rb
1
@recent_reviews = Review.most_recent(5)

So why would you use a scope when you could use regular Ruby class methods? Is it worth keeping these totally separate, but equivalent, concepts in your head? What if you run into weird bugs? Isn’t all this extra stuff the kind of thing that makes Rails harder to learn?

When would it make sense to use a scope instead of a class method?

Why use scopes when we already have class methods?

What if you wanted to grab all the reviews written after a specific date? But if no date was specified, you wanted all the reviews returned instead?

As a scope, that looks like this:

app/models/review.rb
1
scope :created_since, ->(time) { where("reviews.created_at > ?", time) if time.present? }

Easy enough, right? What about the class method?

app/models/review.rb
1
2
3
4
5
6
7
def self.created_since(time)
  if time.present?
    where("reviews.created_at > ?", time)
  else
    all
  end
end

It takes a little bit of extra work. Scopes prefer to return scopes, so they’re easy to chain together:

1
Review.positive.created_since(5.days.ago)

But to get the class method to work the same way, you have to specifically handle the case where time is nil. Otherwise, the caller would have to figure out whether it has a valid, chainable scope.

Methods that always return the same kind of object are really useful. You don’t have to worry as much about edge cases or errors. You can assume you’ll always be handed back an object you can use.

Here, it means you can chain scopes together, without having to worry about nil values coming back.

There are still ways you can break the assumption that you’d always get a scope back:

app/models/review.rb
1
scope :broken, -> { "Hello!!!" }
1
2
irb(main):001:0> Review.broken.most_recent(5)
NoMethodError: undefined method `most_recent' for "Hello!!!":String

But I’ve never had that happen in real code.

The thing I love most about scopes is that they express intent. You’re telling the next person who reads your code, “This method can be chained, will eventually turn into a list of objects, and will help you select the right set of objects.” That’s a whole lot more than a generic class method says.

When should you use a class method instead of a scope?

Because scopes express intent, I use them whenever I’m chaining simple, built-in scopes (like where and limit) into more complicated scopes. Finding the right bunch of objects is what scopes were designed for.

There are two exceptions:

  1. When I need to preload scopes, I turn them into associations instead.
  2. When I do more than chain built-in scopes into larger scopes, I use class methods.

When your scope logic gets complicated, a class method feels like the right place to put it.

Inside a class method, you can easily mix Ruby code with database code. If you have sorting code that’s easier to write in Ruby, you could grab your objects in their default order, and use sort_by to put them in the right order.

Or, if you’re feeling particularly tricky, a class method could grab data from a few different places: your database, Redis, or an external API or service. Then, it could assemble it all into a collection of objects that feels like a scope that’s been turned into an array.

Even then, it’s still good to put your selecting, sorting, joining, and filtering code inside scopes. Then, use your scopes inside your class method. You’ll end up with a clearer class method, and scopes you can use throughout your app.


Scopes are one of my favorite Rails features. You can do some powerful stuff – read my article on sorting and filtering Rails models to see an especially useful scope example.

And there’s a really simple way to master using scopes: play with them inside tiny, focused apps. The free sample chapter of Practicing Rails will show you how. Check it out!


How to predict the future of programming

Why did Rails become so popular, so quickly?

The simplicity helped, especially if you came from the Java, XML, Enterprise world. It was also marketed incredibly well. But that’s not everything.

A lot of Rails’ success in the startup world came from a simple fact: The problems businesses have aren’t that unique. Rails was great at creating CRUD sites, while remaining flexible. And that’s really all a ton of businesses need. Especially at the beginning.

But this isn’t just true for businesses. A lot of the problems we face as software developers don’t change. Sure, our solutions evolve. They cycle. We get better. But the same solutions discovered by the last generation of developers can still help us today.

So, do you want to know the answers to the problems you’ll face in the future? The best thing you can do is look to the past.

Look to the past

On his website, Martin Fowler has an unbelievable collection of good solutions to common problems. Have you been hearing devs talk about Event Sourcing? He wrote the definitive article on it 10 years ago. Chasing performance and reliability problems with your new REST API or Service-Oriented Architecture? It’s his first law of distributed objects, from close to 15 years ago.

Avdi Grimm told me that “If you want to lead the technology curve, start investigating whatever Martin Fowler was writing about a decade ago.” That’s totally true. The time you spend reading through the patterns on his website will be a great investment in your programming future. And that’s not even mentioning the refactoring patterns.

Going further, just about any book or article written by the authors of the Agile Manifesto is worth reading – 15 years ago, they were working through the same software architecture problems we run into today.

You can find a lot of their discussions on the C2 Wiki. The debates we have about when TDD makes the most sense? They’re all there. And they’ve been there. The wiki has been around for a while, and it’s still a fantastic resource.

Books from the late 90’s to the early 2000’s are also helpful. I smiled all the way through Smalltalk Best Practice Patterns and Patterns of Enterprise Application Architecture (which was a huge influence on Rails), because they described problems I’ve had so well.


Just like design fashions, software development practices cycle. From distributed to centralized, from client side to server side, from dynamic to static.

Do you want to get ahead, understand what’s coming, and maybe even drive what comes next? Look to the past. Research the solutions to the problems that our current solutions will cause. And help bring the good practices of the last generation of software developers to the next.


Turn Ruby conference videos into your own personal podcast

Ruby conferences are awesome. There are so many people sharing so much knowledge, and you’ll take something away from almost every talk. And even if you can’t be there, new conferences mean lots of new talk videos.

But there’s a problem. Videos take time. Even at 1.5x, they’ll still last 20 or 30 minutes each. And that’s focused time that’s hard to find as often as I’d like.

Podcasts, though, have already found a place in my life. I start almost every commute by firing up Overcast and listening to a few episodes. And besides the commute, you can listen to them while you walk the dog, or do the dishes. You can hear them as you go off to sleep.

So, after conference videos are posted, I’d love to take some of the non-code-heavy talks and put them where I’ll naturally hear them: inside my podcast player. And it turns out there’s a pretty easy way to do just that.

Creating a personal podcast with Huffduffer

Huffduffer is a website that turns audio files you find anywhere on the web into your own personal podcast station. This is really cool! But there’s a problem when you try it with conference talks:

Most conference videos are conference videos.

So, before you can add a talk to Huffduffer, you have to grab just the audio out of it. And to do that, you have to find a way to download the video.

Fetching a video with youtube-dl

When you see a video on a site like Youtube, Vimeo, or Confreaks, and you’d rather have it on your machine, you can use a little tool called youtube-dl. On a Mac, it’s easy to install using Homebrew:

1
brew install youtube-dl

(You can install it on other systems using the instructions on the site).

Once it’s installed, if you wanted to download Kylie Stradley’s great RailsConf 2015 talk so you could watch it offline, run:

1
youtube-dl http://confreaks.tv/videos/railsconf2015-amelia-bedelia-learns-to-code

This is close to what we want. But to put it in a podcast, you only need the audio. youtube-dl supports that with the -x flag, but you have to install ffmpeg first:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ brew install ffmpeg
$ youtube-dl -x http://confreaks.tv/videos/railsconf2015-amelia-bedelia-learns-to-code

[generic] railsconf2015-amelia-bedelia-learns-to-code: Requesting header
WARNING: Falling back on generic information extractor.
[generic] railsconf2015-amelia-bedelia-learns-to-code: Downloading webpage
[generic] railsconf2015-amelia-bedelia-learns-to-code: Extracting information
[download] Downloading playlist: Confreaks TV | Amelia Bedelia Learns to Code - RailsConf 2015
[generic] playlist Confreaks TV | Amelia Bedelia Learns to Code - RailsConf 2015: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[youtube] bSbla50tqZE: Downloading webpage
[youtube] bSbla50tqZE: Extracting video information
[download] Destination: RailsConf 2015 - Amelia Bedelia Learns to Code-bSbla50tqZE.m4a
[download] 100% of 31.88MiB in 00:05
[ffmpeg] Correcting container in "RailsConf 2015 - Amelia Bedelia Learns to Code-bSbla50tqZE.m4a"
[youtube] Post-process file RailsConf 2015 - Amelia Bedelia Learns to Code-bSbla50tqZE.m4a exists, skipping

Then, in Huffduffer, click the “Huffduff it” link in the header, and you’ll run into the last hoop you have to jump through.

Uploading the audio with Dropbox

You can’t upload your new audio file to Huffduffer, because Huffduffer wants a link to the file, not the file itself.

If you’re a Dropbox user, you can easily get the file onto the internet if you drop it into Dropbox’s Public/ folder. Then, you can right click and “Copy Public Link” to get the link you’ll use on Huffduffer.

Finally, use the “RSS” link on Huffduffer to get your podcast into your favorite podcast player.

Sadly, as a podcast, you miss out on the (frankly awesome) pictures in this talk. And it doesn’t work well with code- or demo-heavy talks (though you might still be able to get the gist of it).

But sometimes, it’s not a choice between listening to a talk or watching a talk. It’s between listening to a talk and not experiencing it at all. If those are your options, audio-only seems like a decent compromise.

Putting it all together

So, here’s that process again:

  1. Sign up for a free Huffduffer account.
  2. Install youtube-dl.
  3. Install ffmpeg.
  4. Download the talk’s audio with youtube-dl -x <url_to_talk_video>.
  5. Toss the audio file into ~/Dropbox/Public, or somewhere else that’s publicly accessible.
  6. Click “Huffduff it,” and paste the link to your new audio file.
  7. Listen to the new episode of your very own conference podcast!

If you’re a fan of podcasts, give this a try. It’s really nice to have conference talks pop up where you’re already listening.


A web server vs. an app server

When you research how to deploy your Rails app, you’ll see a lot of names: Apache, Unicorn, Puma, Phusion Passenger, Nginx, Rainbows, and many more. They all seem to fit under the “deploying Rails” category of software, but there’s a key difference between them. Some are “web servers,” and others are “app servers.”

Once you understand which is which, and where each category fits in your system, deployment will make a lot more sense. But the categories aren’t always clear.

What’s a web server, and how is it different than an app server? Can you use one without the other? And where does Rack fit in?

What’s a web server?

A web server is a program that takes a request to your website from a user and does some processing on it. Then, it might give the request to your Rails app. Nginx and Apache are the two big web servers you’ll run into.

If the request is for something that doesn’t change often, like CSS, JavaScript, or images, your Rails app probably doesn’t need to see it. The web server can handle the request itself, without even talking to your app. It’ll usually be faster that way.

Web servers can handle SSL requests, serve static files and assets, compress requests, and do lots of other things that almost every website needs. And if your Rails app does need to handle a request, the web server will pass it on to your app server.

What’s an app server?

An app server is the thing that actually runs your Rails app. Your app server loads your code and keeps your app in memory. When your app server gets a request from your web server, it tells your Rails app about it. After your app is done handling the request, the app server sends the response back to the web server (and eventually to the user).

You can run most app servers by themselves, without a web server in front of it. That’s probably what you do in development mode! In production, though, you’ll usually have a web server in front. It’ll handle multiple apps at once, render your assets faster, and deal with a lot of the processing you’ll do on every request.

There are a ton of app servers for Rails apps, including Mongrel (which isn’t used much anymore), Unicorn, Thin, Rainbows, and Puma. Each has different advantages and different philosophies. But in the end, they all accomplish the same thing – keeping your Rails app running and handling requests.

What about Passenger?

Phusion Passenger is a little unique. In “standalone mode,” it can act just like an app server. But it can also be built right into a web server, so you don’t need a separate app server to run your Rails apps.

This can be really convenient. Especially if you’re planning to run a bunch of apps and don’t want to spend time setting up an app server for each one. After installing Passenger, you just point the web server directly at your Rails app (instead of an app server), and your Rails app will start handling requests!

Passenger is a nice option, but having a separate app server can be still be good. Keeping the app server separate gives you the flexibility to choose an app server that best fits your needs, and you can run and scale it on its own. Still, I’m going to try it again the next time I deploy a new small app. I’m hoping it’ll make it easier to deploy future apps to the same server.

What about Rack?

Rack is the magic that lets any of these app servers run your Rails app. (Or Sinatra app, or Padrino app, or…)

You can think of Rack as a common language that Ruby web frameworks (like Rails) and app servers both speak. Because each side knows the same language, it means Rails can talk to Unicorn and Unicorn to Rails, without having either Rails or Unicorn know anything about the other.

How do they relate?

So, how does this all fit together?

Out of these pieces, a web request will hit your web server first. If the request is something Rails can handle, the web server will do some processing on the request, and hand it off to the app server. The app server uses Rack to talk to your Rails app. When your app is done with the request, your Rails app sends the response back through the app server and the web server to the person using your app.

More specifically, Nginx might pass a request to Unicorn. Unicorn gives the request to Rack, which gives it to the Rails router, which gives it to the right controller. Then, your response just goes back through the other way.


This overview might be simplified. But even just knowing these categories will help you put the software you run into into the right mental buckets.

After you understand how app servers and web servers fit together, it’ll be a lot easier to debug server problems when you have them. You’ll know all the different places you could look, and how they interact. And once the next interesting app server arrives, it’ll be even easier for you to swap it in!


The easiest way to get into open source

Thom Parkin made a great point in the comments of an earlier article of mine:

Great advice. But you missed one very important [final] point. Since this is Open Source, once you have figured out the details of that feature/function where the documentation is a bit light, YOU SHOULD UPDATE THE DOCS AND SUBMIT A PULL REQUEST. In that way the entire community benefits, and you can even gain some “coder cred” for your participation!

I’m happy Thom mentioned this, because it’s so important. Fixing documentation is the easiest way to start contributing back to the projects you use and love.

My first contributions to projects like Rails, Rubinius, and Elixir have all been doc fixes. I’ve made small tweaks to make things clearer, explained some things that you could only discover by reading the code, even just fixed broken formatting. These have all been quick, easy ways to help out some big open source projects. Even when they’re my only contributions to a project, they’ve still helped future users, and Future Me. And that’s what open source is all about.

Why documentation fixes are such a great way to get started

Doc fixes are the least intimidating way to contribute to a big project like Rails:

  • You don’t have to set up the project in order to fix the bug. Since you’re just updating the documentation, you don’t have to get the tests or the app running. Sometimes, you won’t even have to clone the project to your machine – you can make your change right on GitHub!

  • If the maintainer asks you to make changes to your pull request, they’re usually a matter of wording or taste. Those kind of changes can be easier to stomach than criticism of your code. And it’s easier for you to make those changes, because you don’t have to update tests or code, just words.

  • Documentation is hard for a project maintainer, so updates are appreciated. Often, authors are too close to the code to understand where the confusing parts are. They need other, newer developers to tell them where the docs need help. It takes practice to see your project as a beginner would, and not everyone has built that skill.

  • Finally, you’re starting to build a relationship with the maintainer, with a low-impact change. You’re not changing the direction of the project, like you would if you were contributing an entire feature. So your change is easier for a maintainer to review, and they’ll usually respond to you more quickly. Your merge request won’t get stuck in the “Is this a good idea?” phase.

As you keep building that relationship, you’ll start to be seen as a reliable contributor. Your pull requests will get reviewed faster, and it’ll be easier for both of you to talk through more complicated feature requests and bug fixes.

They’re easier to start, they’re easier to do, and they tend to get merged more quickly. So why wouldn’t your first contribution be a doc fix?

How to start contributing back updated documentation

There’s an important way contributing doc updates is like fixing bugs: They both rely on being sensitive to things that feel wrong. You have to pay attention.

When you run into behavior you didn’t expect, it might be time to update the docs. If you have to dive into the code to solve a problem, you might also want to tell other people about it. You should even be sensitive to broken formatting and typos in the documentation you read. If you’re not going to fix it, who will?

Once you have a good idea of where to make the change and how you want to word it, make your change and send a pull request through GitHub.

If you’re still trying to decide on the best way to update the docs, open an issue on GitHub. It can be something like this:

“Hey, this was confusing to me. I was thinking of updating it to look something like this: … What do you think? Anything else I should mention?” Together, you can come up with wording that satisfies everyone.

Finally, don’t be discouraged if you don’t get a response. Big projects have a lot going on, so it’s easy for your contribution to fall through the cracks. In a week or so, if you still don’t hear from anyone, ask the maintainer again.

Documentation is often the first thing you encounter when you work with a library, so it’s important that it’s detailed and clear.

So when you’re confused about the code you use, or have to dive into the source, make it easier for the next person. Write a quick update, and contribute it back. It’s the easiest way I know of to become an open source contributor.

This article was originally sent to the people on my list. To read more like it, sign up here!


Creating easy, readable attributes with ActiveRecord enums

After you pick up the Rails basics, you still have a lot to learn. You have to understand gems, DSLs, refactoring, testing, and the deeper parts of Rails itself.

With the Ruby Book Bundle, launching on Monday, July 6, you can kickstart your Rails education with Practicing Rails and 5 other great Ruby books at a huge discount. Sign up to get a reminder when the bundle sale starts!


Imagine a question that can be either “pending”, “approved”, or “flagged”. Or a phone number that’s a “home”, “office”, “mobile”, or “fax” (if it’s 1982).

Some models call for this kind of data. An attribute that can have only one of a few different values. And that set of values almost never changes.

It’s a situation where, if it were plain Ruby, you’d just use a symbol.

You could create a PhoneNumberType or QuestionStatus model and a belongs_to relationship to hold these values, but that doesn’t seem worth it. You could stuff them in a yaml file, but now you have to look in a totally different place to figure out what your object can do.

In 4.1, Rails took a stab at solving this problem with ActiveRecord enums.

A few values, in the model

ActiveRecord enums are pretty easy. You give your model an integer column:

1
bin/rails g model phone number:string phone_number_type:integer

List the values that attribute can take:

app/models/phone.rb
1
2
3
class Phone < ActiveRecord::Base
  enum phone_number_type: [:home, :office, :mobile, :fax]
end

And now you can deal with strings instead of numbers.

Instead of this:

1
2
irb(main):001:0> Phone.first.phone_number_type
=> 3

You’ll see this:

1
2
irb(main):002:0> Phone.first.phone_number_type
=> "fax"

You can change that attribute using either strings or ints:

1
2
3
4
irb(main):003:0> phone.phone_number_type = 1; phone.phone_number_type
=> "office"
irb(main):004:0> phone.phone_number_type = "mobile"; phone.phone_number_type
=> "mobile"

Or even using a bang method:

1
2
3
4
irb(main):005:0> phone.office!
=> true
irb(main):006:0> phone.phone_number_type
=> "office"

You get methods for asking if your attribute has some specific value:

1
2
irb(main):007:0> phone.office?
=> true

And you can find all objects with the value you’re looking for:

1
2
irb(main):008:0> Phone.office
  Phone Load (0.3ms)  SELECT "phones".* FROM "phones" WHERE "phones"."phone_number_type" = ?  [["phone_number_type", 1]]

If you want to see all the different values you can use, along with the numbers they’re associated with, use the phone_number_types class method:

1
2
irb(main):009:0> Phone.phone_number_types
=> {"home"=>0, "office"=>1, "mobile"=>2, "fax"=>3}

Which makes them easy to put into an HTML form:

app/views/phones/_form.html.erb
1
2
3
4
<div class="field">
  <%= f.label :phone_number_type %><br>
  <%= f.select :phone_number_type, Phone.phone_number_types.keys %>
</div>

An enum in a form

A few things to watch for

Enums aren’t without their problems, though. You have to keep a few things in mind if you don’t want to run into trouble later on.

When you define an enum, order matters. So if you go back to your code and decide that those values should really be in alphabetical order:

app/models/phone.rb
1
2
3
class Phone < ActiveRecord::Base
  enum phone_number_type: [:fax, :home, :mobile, :office]
end

Your phones won’t have the right types anymore. You can get around this by telling enum which number goes with which value:

app/models/phone.rb
1
2
3
class Phone < ActiveRecord::Base
  enum phone_number_type: {fax: 3, home: 0, mobile: 2, office: 1}
end

But really, your best option is to keep the order consistent.

A bigger problem is what to do outside the Rails world. Even though Rails sees these enum values as strings, they’re just numbers inside your database. So someone looking at your raw data will have no idea what those numbers mean. This also means that every app that reads that database will have to know that enum mapping.

You could dump your enum mapping to the database or a yaml file if you really needed other people to see them. But that’s not DRY, because now you’re defining your enum in two places. And if you’re going that far, it might be better to do what we were avoiding in the beginning: create a totally separate model and association, so that a Phone would belong_to a PhoneNumberType.

But if you’re keeping it simple, enums are a great way to start.

P.S. In case you missed it, Practicing Rails is going to be included in the Ruby Book Bundle, launching on Monday, July 6. Get it and 5 other great Ruby books at a huge discount!


How to preload Rails scopes

This article is also available in Korean, thanks to Soonsang Hong!

Rails’ scopes make it easy to find the records you want:

app/models/review.rb
1
2
3
4
5
class Review < ActiveRecord::Base
  belongs_to :restaurant

  scope :positive, -> { where("rating > 3.0") }
end
1
2
3
4
irb(main):001:0> Restaurant.first.reviews.positive.count
  Restaurant Load (0.4ms)  SELECT  `restaurants`.* FROM `restaurants`  ORDER BY `restaurants`.`id` ASC LIMIT 1
   (0.6ms)  SELECT COUNT(*) FROM `reviews` WHERE `reviews`.`restaurant_id` = 1 AND (rating > 3.0)
=> 5

But if you’re not careful with them, you’ll seriously hurt your app’s performance.

Why? You can’t really preload a scope. So if you tried to show a few restaurants with their positive reviews:

1
2
3
4
5
6
7
8
9
10
irb(main):001:0> restauraunts = Restaurant.first(5)
irb(main):002:0> restauraunts.map do |restaurant|
irb(main):003:1*   "#{restaurant.name}: #{restaurant.reviews.positive.length} positive reviews."
irb(main):004:1> end
  Review Load (0.6ms)  SELECT `reviews`.* FROM `reviews` WHERE `reviews`.`restaurant_id` = 1 AND (rating > 3.0)
  Review Load (0.5ms)  SELECT `reviews`.* FROM `reviews` WHERE `reviews`.`restaurant_id` = 2 AND (rating > 3.0)
  Review Load (0.7ms)  SELECT `reviews`.* FROM `reviews` WHERE `reviews`.`restaurant_id` = 3 AND (rating > 3.0)
  Review Load (0.7ms)  SELECT `reviews`.* FROM `reviews` WHERE `reviews`.`restaurant_id` = 4 AND (rating > 3.0)
  Review Load (0.7ms)  SELECT `reviews`.* FROM `reviews` WHERE `reviews`.`restaurant_id` = 5 AND (rating > 3.0)
=> ["Judd's Pub: 5 positive reviews.", "Felix's Nightclub: 6 positive reviews.", "Mabel's Burrito Shack: 7 positive reviews.", "Kendall's Burrito Shack: 2 positive reviews.", "Elisabeth's Deli: 15 positive reviews."]

Yep, that’s an N+1 query. The biggest cause of slow Rails apps.

You can fix this pretty easily, though, if you think about the relationship in a different way.

Convert scopes to associations

When you use the Rails association methods, like belongs_to and has_many, your model usually looks like this:

app/models/restaurant.rb
1
2
3
class Restaurant < ActiveRecord::Base
  has_many :reviews
end

But if you check out the documentation, you’ll see that they can do more. You can pass other parameters to those methods and change how they work.

scope is one of the most useful. It works just like the scope from earlier:

app/models/restaurant.rb
1
2
3
4
class Restaurant < ActiveRecord::Base
  has_many :reviews
  has_many :positive_reviews, -> { where("rating > 3.0") }, class_name: "Review"
end
1
2
3
4
irb(main):001:0> Restaurant.first.positive_reviews.count
  Restaurant Load (0.2ms)  SELECT  `restaurants`.* FROM `restaurants`  ORDER BY `restaurants`.`id` ASC LIMIT 1
   (0.4ms)  SELECT COUNT(*) FROM `reviews` WHERE `reviews`.`restaurant_id` = 1 AND (rating > 3.0)
=> 5

Now, you can preload your new association with includes:

1
2
3
4
5
6
7
irb(main):001:0> restauraunts = Restaurant.includes(:positive_reviews).first(5)
  Restaurant Load (0.3ms)  SELECT  `restaurants`.* FROM `restaurants`  ORDER BY `restaurants`.`id` ASC LIMIT 5
  Review Load (1.2ms)  SELECT `reviews`.* FROM `reviews` WHERE (rating > 3.0) AND `reviews`.`restaurant_id` IN (1, 2, 3, 4, 5)
irb(main):002:0> restauraunts.map do |restaurant|
irb(main):003:1*   "#{restaurant.name}: #{restaurant.positive_reviews.length} positive reviews."
irb(main):004:1> end
=> ["Judd's Pub: 5 positive reviews.", "Felix's Nightclub: 6 positive reviews.", "Mabel's Burrito Shack: 7 positive reviews.", "Kendall's Burrito Shack: 2 positive reviews.", "Elisabeth's Deli: 15 positive reviews."]

Instead of 6 SQL calls, we only did two.

(Using class_name, you can have multiple associations to the same object. This comes in handy pretty often.)

What about duplication?

There still might be a problem here. The where("rating > 3.0") is now on your Restaurant class. If you later changed positive reviews to rating > 3.5, you’d have to update it twice!

It gets worse: If you also wanted to grab all the positive reviews a person has ever left, you’d have to duplicate that scope over on the User class, too:

app/models/user.rb
1
2
3
4
class User < ActiveRecord::Base
  has_many :reviews
  has_many :positive_reviews, -> { where("rating > 3.0") }, class_name: "Review"
end

It’s not very DRY.

There’s an easy way around this, though. Inside of where, you can use the positive scope you added to the Review class:

app/models/restaurant.rb
1
2
3
4
class Restaurant < ActiveRecord::Base
  has_many :reviews
  has_many :positive_reviews, -> { positive }, class_name: "Review"
end

That way, the idea of what makes a review a positive review is still only in one place.


Scopes are great. In the right place, they can make querying your data easy and fun. But if you want to avoid N+1 queries, you have to be careful with them.

So, if a scope starts to cause you trouble, wrap it in an association and preload it. It’s not much more work, and it’ll save you a bunch of SQL calls.


A guide to the best beginning Rails resources

There are a ton of books, videos, podcasts, and courses for learning Rails. There’s no way you’d have time to go through them all! So what’s the best way for an absolute beginner to learn Ruby and Rails? Which resources should you start with, and when?

Books and websites

If you’re totally new to programming, the best place to start is Learn to Program, by Chris Pine. It’s an intro to the core programming ideas you’ll need to know. If you’re planning to learn Ruby and Rails, it’s especially great, because it uses Ruby for all of the examples.

After that, Daniel Kehoe’s Learn Ruby on Rails is a gentle introduction to Rails. It teaches you a small part of Rails that will prepare you to take on the harder resources.

If you already know a few other languages or frameworks, check out the free Getting Started with Rails guide. It’s a good, short intro to Rails, that will teach you Rails’ concepts and core ideas.

Once you know the basics, there are two bigger books that will fill out your Rails knowledge.

Agile Web Development with Rails is my favorite general Rails book. It does a good job of teaching first by example, and then by reference. We use it at work to teach devs without Rails experience, and like most of the rest of the Pragmatic Bookshelf books, it’s very good.

The Ruby on Rails Tutorial is the other big Rails resource. It walks you through most of what you need to know to build a fully functional example app. I know a lot of great Rails developers who got started with the Rails Tutorial. And the web version is free, so you can see if it’s your style before you commit to it. If you put in the effort, you’ll get a lot out of it.

Once you’ve gone through one or two of these books, it’s pretty normal to feel confused and frustrated. Especially when you try to put everything together and build your own apps. My book, Practicing Rails, will help you solve the most painful problems you’ll run into as you start your programming career. In Practicing Rails, you’ll learn how to debug your code when it breaks, pick up some processes you can follow to turn the ideas in your head into real features, and discover how to write tests without getting stuck.

While you build your own apps, there are two resources you’ll use more than any others:

The Rails Guides will teach you the most important parts of Rails with documentation and examples. I go back to these all the time. And they’re always up to date.

When you want to know how to call a Rails method, or even whether a method exists to do what you want to do, you’ll need the official Rails API documentation.

(There are much better ways of browsing the API documentation, though, and I talk through a few of them in one of the lessons in my free email course).

You can start building simple apps without knowing a whole lot of Ruby, but spending more time learning Ruby will become important, quickly. And Programming Ruby is the best book I’ve found to get comfortable with the language.

Videos and guided courses

Books and websites are my favorite way to learn new things about Ruby and Rails. But if you prefer watching to reading, there are lots of great screencasts and courses you can check out, too.

If you want a video course to teach you Ruby and Rails, I’ve heard lots of praise for the Pragmatic Studio courses. They sound like a great place to start.

The RailsCasts haven’t been updated in a few years, but they’ll still show you great answers to common web problems. The APIs might have changed, but the ideas have stayed pretty much the same. They’re definitely worth watching.

Avdi Grimm’s Ruby Tapas screencasts will show you fun Ruby code in 5-10 minute videos. They’re Ruby-focused (rather than Rails-focused), but I always learn a lot from them. You can find a few free sample videos on the site, but they’re all great. It’s really worth subscribing to.

The Destroy All Software screencasts aren’t specifically about Ruby and Rails, but watching them will make you a better developer, whatever your language.

Finally, bloc.io is an online bootcamp some readers have recommended. They pair you with a mentor who can help you with your specific problems when you get stuck.

One-on-one help is great – it can be exactly what you need while you’re learning. If you can’t find a friend or mentor to help you out, I wrote an email to my list about where you should look. (By the way, you can sign up here to get helpful emails like that every Friday).

What do I recommend?

I know, that’s still a whole lot of stuff! My recommendation, if you like reading and already know the programming basics, is to start with Programming Ruby and Agile Web Development with Rails. While you read, build some tiny sample apps to try out the things you learn. (You’ll learn more about how to do that in the free sample chapter of Practicing Rails).

Do you have any other recommendations for resources that helped you out? Anything you can’t believe I missed? Comment and tell us all about them!

This article was inspired by a question from James on my advice page. If you’re stuck on questions about Ruby and Rails, and need some help or advice, ask me there!


You’ve got the Rails basics. So why do you feel so slow?

You’re confident about the core ideas behind Rails. You can write working code, no problem. And you’re learning more about code quality, refactoring, writing great tests, and object-oriented design.

By this point, you’re starting to feel like you’re getting it, that you’re on the path to becoming an expert. When you look backwards, you see just how far you’ve come, and you’re pretty happy with your progress.

So why do you feel so slow? Now that you care about testing, maintainability, and design, it feels like it takes you way more time to ship anything!

Is it even possible to ship high quality code quickly?

It’s all part of the process

This feeling is incredibly common, no matter what you’re learning.

Now that you’re no longer a beginner, you’re starting to see all the different shapes that your code could have. You have more alternatives to think through whenever you put down a line of code. You have to test edge cases you never recognized before.

You’ve learned lots of helpful skills. But right now, they still take a lot of thought. You have to weigh every decision you make, so you feel comfortable that you’re making the right decision based on the things you’ve learned.

It will get faster, though. The skills you’ve learned will become more automatic. You’ll build intuition. And you’ll be able to make better decisions more quickly.

Which is nice to know, but it doesn’t help you right now. So what can you do now, to finish things faster?

Take it in stages

If you’re obsessed with writing perfect, high-quality, highly-maintainable code every time you put your fingers on the keyboard, you’ll never get anything done.

When I get stuck, I write code the same way I write articles. You’d start with a rough draft. Maybe sketch out some tests, code, or comments. Or even write some ideas out on paper. At this point, you wouldn’t worry about structure, you’re just using code to clear up the vague ideas you have in your head.

Then, I turn those ideas into a straightforward implementation. What you might call “The simplest thing that could possibly work.” It’s not perfect, and not even close. But don’t worry about it. Because once the code works, you’ll do a tidying pass. TDD edge cases, refactor obviously bad code, or make names clearer.

These “refined drafts” are usually good enough to ship. But I’ll usually do a few more passes. Not too many, though – you’ll soon start to see diminishing returns. You’ll spend more time cleaning up the code than it’s worth.

Then, if you really want to end up with the cleanest possible code, let it settle for a while. Come back to it in a few weeks or months, and do another pass at it. By that time, you’ll know more about your system, and you’ll have learned more about how to write great, highly-maintainable code. So you’ll do an even better job.

Just like writing, that process is:

  1. Sketch out a rough outline, draft, or prototype.
  2. Write a simple, unedited, straightforward implementation (often guided by TDD, or written along with tests).
  3. Refine, refactor, and clean up that implementation a little bit.
  4. Let it settle.
  5. Come back to it, and do one more pass.

It sounds like a lot more work. But when you go in stages like this, you’ll move faster, without always second-guessing yourself. And you won’t end up overthinking decisions between a few just-as-good options.

This article was inspired by a question from Topher on my advice page. If you’re stuck on questions about Ruby and Rails, and need some help or advice, ask me there!


3 ways to configure your Ruby API wrappers

When you use Ruby to wrap an API, you have to have a way to configure it. Maybe the wrapper needs a username and secret key, or maybe just a host.

There are a few different ways to handle this. So which one should you choose?

The easy, global way

You might want your service to act like it’s always around. No matter where you are in your app, you’d have it ready to use. Otherwise, you’ll spend three lines of configuring it for every line of using it!

You could make the configuration global, using constants or class attributes:

config/initializers/product_api.rb
1
2
3
ProductApi.root = "https://staging-host.example.com/"
ProductApi.user = "justin"
ProductApi.secret = "mysecret123"
app/controllers/products_controller.rb
1
2
3
def show
  @product = ProductApi.find(params[:id])
end

Lots of gems use this pattern. It’s pretty easy to write, and really easy to use. But it has some big problems:

  • You can only have one ProductApi.

    If you want to use the Product API as two different users, or hit different servers from a single app, you’re out of luck.

  • ProductApi has global data that’s easy to accidentally change.

    If a thread or a part of your app changed ProductApi.user, everything else using ProductApi would break. And those are painful bugs to track down.

So, class variables have some problems. What if you configured instances of your Product API class, instead?

What would it look like with #initialize?

If you used instances, you’d create and configure your API wrapper when you need it:

app/controllers/products_controller.rb
1
2
3
4
5
6
7
def show
  product_api = ProductApi.new(
    root: "https://staging-host.example.com/",
    user: "justin",
    secret: "mysecret123")
  @product = product_api.find(params[:id])
end

Now, you can pass different details to your API whenever you use it. No other methods or threads are using your instance, so you don’t have to worry about it changing without you knowing it.

This seems better. But it’s still not as easy as it should be. Because you have to configure your API every time you use it.

Most of the time you don’t care how the API is set up, you just want to use it with sane options. But when you’re working with instances, every part of your app that uses the API has to know how to configure it.

But there’s a way to get the convenience of global access, using good defaults, while still being able to change it if you need to.

And this pattern shows up all the time in an interesting place: OS X and iOS development.

How do you get good defaults and flexibility?

What if you could configure each instance of your API wrapper, but you also had a global “default” instance when you just didn’t care?

You’ll see this “defaultSomething” or “sharedWhatever” pattern all over the iOS and Mac OS SDKs:

1
2
3
[[NSURLSession sharedSession] downloadTaskWithURL:@"http://www.google.com"];

[[NSFileManager defaultManager] removeItemAtPath:...];

And you can still ask for instances of these classes if you need more than what the default gives you:

1
2
3
NSURLSession *session = [NSURLSession sessionWithConfiguration:...];

NSFileManager fileManager = [[NSFileManager alloc] init];

You could build something like that in Ruby, with a default_api class method:

app/controllers/products_controller.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
def show
  @product = ProductApi.default_product_api.find(params[:id])
end

...

def show_special
  special_product_api = ProductApi.new(
    root: "https://special-product-host.example.com/"
    user: "justin"
    secret: "mysecret123")
  @special_product = special_product_api.find(params[:id])
end

And the implementation might look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class ProductApi
  def initialize(root:, user:, secret:)
    @root, @user, @secret = root, user, secret
  end

  def self.default_api
    @default_api ||= new(
      root: ENV['PRODUCT_API_ROOT'],
      user: ENV['PRODUCT_API_USER'],
      secret: ENV['PRODUCT_API_SECRET'])
  end

  def find(product_id)
    ...
  end
end

Here, I used environment variables in default_api, but you could also use config files. And you could switch the ||= to use thread- or request-local storage instead.

But this is a decent start.


Most gems I’ve seen, like the Twitter gem, will have you configure and create each API object when you need them. This is an OK solution (though I usually see people assigning these to globals anyway).

But if you go one step further, and also use a pre-configured default object, you’ll have a much more comfortable time.


With so much Rails to learn, where do you start?

Have you seen the Rails Competency Chart?

Brook Riggio of CodeFellows put it together to show all of the concepts a modern Rails developer should know. Take a look:

Frightening, isn’t it? It looks like a two-hundred-tentacled monster that’s preparing to attack you.

It’s no wonder learning Rails is intimidating. Some of the branches, like SQL and Deployment, could be entire career paths. You could spend years on Application Architecture and still not feel like an expert.

But it’s accurate. If you show this chart to a professional Rails dev, you’ll probably hear, “Yep, that sounds about right.” If anything, you’ll hear about what it’s missing.

So how can you deal with all this? How do you learn all this stuff without having started 5 years ago?

How do you handle all these topics?

Yeah, most professional Rails developers know a lot about a lot of those concepts. But you don’t have to know all these things to start building your Rails apps. You don’t need to study deployment until you’re ready to deploy, and you can look up how to group things in SQL the first time you have to do it.

One thing this chart doesn’t illustrate well (even though the blog post talks about it) is how the different areas reinforce each other. For instance, unit tests, integration tests, acceptance tests, and all the rest use similar skills and knowledge. Yes, there are some differences between the different types of tests. But once you start to understand the fundamentals of testing by writing a bunch of unit tests, you’ll pick up the other types much more quickly.

The more you learn, the faster you’ll learn. Functional programming is a lot easier to learn when you know Object-Oriented Programming. Service-Oriented Architectures can “feel” a little like Object-Oriented Programming. Learning CSS selectors will make using jQuery much easier. Many of the principles you learn will translate across different branches.

Where do you start?

If you try to pick up every one of these skills at the same time, you won’t take advantage of the extra speed you’ll get as you master them one-by-one. So focus on a few at a time, and learn them well.

Ask yourself this question:

“What do I need to know to make progress on the problem I’m facing?”

Use your answer to narrow in on which competencies to start with. As you pick those up, you can check out the skills right around it, and take advantage of what you just learned to learn similar things faster. For instance, if you’re feeling comfortable with the command line, you could pretty easily branch out into things like package management and permissions.

By building these skills as you need them, you’ll have a little extra kick of motivation. You’ll be spending time on the stuff that matters to you.

For instance, if I was learning multitenancy because I felt like it’s something I should just know, I’d be asleep halfway through the first blog post. If I was learning it because it was the only way I could get my app to work, I’d be glued to every tutorial and reference guide I could find.


It’s crushing to see a few hundred skills, and know you need to learn them all. Especially when those first few competencies take you weeks or months to pick up. You’ll feel like you’ll never become a professional Rails dev.

The chart’s not wrong. As a Rails dev, you’ll eventually know a lot about most of those things. But we didn’t all start there.

So, start somewhere. Prioritize, and move along the path that leads to your app being built. Branch out to fill in the gaps. And recognize that you’ll get faster as time goes on.

And if you want to learn those skills faster, and remember how to use them when it counts, grab this free sample chapter of Practicing Rails. You’ll learn the method I use to learn new Rails ideas quickly and thoroughly, without getting overwhelmed.


How to dispel Ruby magic and understand your gems

When you build Rails apps, you’ll use piles of gems. Some of them seem totally magical! But how does that magic happen?

In most gems, there’s nothing they’re doing that you couldn’t. It’s just Ruby code. Sometimes it’s complicated Ruby code. But if you explore that code, you’ll begin to understand where that magic comes from.

Finding the source code

To understand how a gem works, you have to find its code.

If there’s a method that you want to know more about, you have an easy way to find the source: method and source_location. I wrote a little bit about these earlier. Here’s an example:

1
2
3
irb(main):001:0> ActiveRecord::Base.method(:find).source_location
=> ["/usr/local/lib/ruby/gems/2.1.0/gems/
  activerecord-4.2.0.beta4/lib/active_record/core.rb", 127]

But what if you’re interested in more than one method?

If you have a console open inside your Rails app’s directory, you can go right to a gem’s code:

1
~/Source/gem_example[master] jweiss$ bundle open rack

If it’s in your Gemfile, bundle open rack will open up the entire rack gem in your editor. You can comfortably browse all of the files inside it.

Where do you start?

Now that you know where the gem’s code is, how do you begin to understand it?

If you try to learn how activerecord works by reading lib/active_record.rb, you’re not going to get anywhere. You’re just going to find a lot of autoloads and requires.

It’s easiest to understand a gem after your app uses it a little bit. Once you know more about the kind of work that the gem is doing for you. That way, you’ll already have an idea about which interesting classes and methods you should start with.

After you have the names of some interesting methods, you can use source_location, Find in Project in your editor, or ag on the command line to see where those methods are defined. And that’s when the fun starts.

The gem’s code is on your machine, right? That means you can change it however you want! You could even break it, and nobody else has to know.

When I’m trying to understand how a gem works, I don’t just read the code. I add puts statements into the gem, I throw exceptions to figure out how my app got to certain lines, and I mess with the code to understand why the author wrote it that way.

Once you know how the trick’s done, it’s a lot less magical. And you won’t have to guess how that code will act in strange situations, because you’ll be able to see it for yourself.

Cleaning up after yourself

After you mess with gem code, your app could be in pretty bad shape. It might not even run anymore! And even if it does, it’s going to spam all those puts statements you added into your console.

But RubyGems has a quick way to bring things back to normal:

1
2
3
~ jweiss$ gem pristine activerecord -v 4.2.0
Restoring gems to pristine condition...
Restored activerecord-4.2.0

Or, if you don’t remember which gems you messed with, and you’re really patient:

1
~ jweiss$ gem pristine --all

After that, all your gems will be back to the way they were when you installed them.

What are you going to explore?

When you find, read, and explore the code inside your gems, you’ll understand the code you depend on at a deeper level. And you won’t have to rely on assumptions and guesses anymore.

This post was originally sent exclusively to my list. To get more posts like it in your inbox every Friday, sign up here!


A new way to understand your Rails app’s performance

Is your Rails app slow?

When it takes seconds to load what should be a simple view, you have a problem worth digging into.

You could have too many database calls or some slow methods. Or maybe it’s that speedup loop someone put in your code and forgot about.

You can find lots of tools to help you find out what’s slowing your app down. A few weeks ago, I talked about rbtrace. New Relic’s rpm gem has also helped me speed up my apps.

But my favorite tool for exploring performance problems does a lot more. Out of the box, it shows you what your code is doing. But when you add a plugin to it, it becomes even more powerful. It helps you see your app’s performance problems, visually. And that can help you find and fix slow apps, faster.

My favorite Rails profiler

My favorite Rails performance tool is called rack-mini-profiler. When you add that gem into your app, you get a little indicator on each of your pages. It looks like this:

A MiniProfiler indicator.

If you click on that box, it expands and you can see all kinds of great stuff. Which SQL statements were run, how long it took to render partials, and more:

An expanded MiniProfiler indicator.

MiniProfiler gives you a constant reminder of how long each page takes to load. That helps you learn more about how your app is performing. You’ll build an intuitive sense of which pages are slow, and which ones are fast. You’ll start to notice when a page takes surprisingly long to render. And you can start fixing it right away, while it’s still on your mind.

MiniProfiler can do more. But first, you’ll have to install the flamegraph gem.

When you do that, you’ll unlock a new way to see your app’s performance.

Flamegraphs: exactly as fun as they sound

A flamegraph looks like this:

Flamegraph!

Pretty clear where the name comes from, right?

After you install the rack-mini-profiler gem and the flamegraph gem, you can see a flamegraph for any of your requests. Just add pp=flamegraph as an HTTP parameter, which would look like this:

1
http://www.example.com/restaurants?pp=flamegraph

The flamegraph will pop up, and you can zoom in and out, scroll around, and try to find interesting things to explore.

Each “layer” in the flamegraph is one line in a stack trace:

Flamegraphs. Identified.

And the horizontal axis is time. So the far left side of the graph is when your request started, and the far right side is when the request finished.

So, it certainly looks cool. But what can you do with a flamegraph?

How to use a flamegraph

Because the X axis represents time, you can really get a clear picture of where your app’s getting bogged down. The widest layers take the longest to run. They’re the first areas you should look into, because speeding them up could have the biggest impact.

How much time is your app spending rendering your view? In the controller action? Hitting the database? Rendering partials?

All those are really easy to see, visually:

Parts of a request.

There’s another useful thing that a flamegraph can show you:

Do you see a bunch of spikes that are all about the same height, like this?

Makes me uncomfortable just looking at it.

That often means you have some kind of N+1 query. You’re missing an includes somewhere, or making a bunch of calls to an API. If you added an includes, you’d get a flamegraph that looks more like this:

Much better.

N+1 SQL queries are pretty easy to see with most performance tools: you just look for SQL calls that look similar. But non-SQL N+1 issues, like hitting an API too many times, are a lot harder to notice. Especially if your logging isn’t that great.

With flamegraphs, though, those problems are a lot more visible.

What not to pay attention to

Flamegraphs can be overwhelming. They show you a lot of information, and you’re pretty much forced to take it all in at once. So what can you ignore?

Usually, you can skip the bottom and top layers of the graph. Instead, I start exploring around the middle of the graph, or maybe ¾ of the way to the top. That’s where my code tends to hang out.

The top of the graph is usually ActiveRecord- or IO-related, and the bottom is Rails framework code, so it makes sense that your code would be somewhere toward the middle.


Have you ever used a flamegraph? They’re a great way to find the best places to optimize. So give it a try! Add the rack-mini-profiler and flamegraph gems to your Gemfile. You’ll be surprised how much more insight you’ll get into your code.


How much testing is too much?

You know how painful it is to work with badly tested code. Every time you fix a bug, you create five more. And when things do work, you never really know if it was designed that way, or just worked coincidentally.

On the other hand, you just wrote what seems like 200 tests to ship one tiny feature. You constantly have to redesign already-working code to hit 100% test coverage. You can’t shake the feeling that your best-tested code is somehow getting less readable. And worst of all, you’re starting to get burned out on your app.

There must be a middle ground. So how much testing is the right amount?

It’d be great if there was a nice round number you could use as a rule: twice as many lines of test code as app code, maybe, or 95% test coverage. But even saying “95% test coverage” is ambiguous.

Coverage can be an indicator of well-tested code, but it’s not a guarantee of well-tested code. I’ve had 100%-covered apps that had more bugs than apps with 85% coverage.

So, the right amount of testing can’t be about a number. Instead, it’s about something fuzzier, and harder to define. It’s about testing efficiently.

Efficient testing

Testing efficiently is all about getting the most benefit for the least amount of work. Sounds great, doesn’t it?

But there’s a lot that goes into testing more efficiently. So it helps to think about three things in particular: size, isolation, and focus.

Size

Integration tests are awesome. They mirror a path an actual person takes through your app. They test all of your code, working together, the same way it’s used in the real world.

But integration tests are slow. They can be long and messy. And if you want to thoroughly test one small part of your system, they add a lot of overhead.

Unit tests are smaller. They run faster. They’re easy to think about, since you only need to keep one tiny part of your system in your head while you write them.

But they can also be fake. Just because something works inside a unit test doesn’t mean it’ll also work in the real world. (Especially if you’re doing a lot of mocking).

So how do you balance those?

Since unit tests are fast and easy to write, it doesn’t cost much to have a lot of them. So they’re a great place to test things like edge cases and complicated logic.

Once you have a bunch of well-tested pieces of your system, you still have to fill in the gaps. You have to test how those parts interact, and the full journeys someone could take through your app. But because most of your edge cases and logic are tested by your unit tests, you only need a few of these more complicated, slower integration tests.

You’ll hear this idea called the “Test Pyramid.” It’s a few integration tests, sitting on top of a base of many unit tests. And if you want to learn more about it, take a look at the third chapter of my book, Practicing Rails.

Isolation

Still, if your system is complicated, it might take what feels like an infinite number of tests to cover every situation you might run into. This can be a sign that you need to rethink your app’s design. It means that parts of your system depend too closely on each other.

Say you have an object that could be in one of a few different states:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
case user.type
when :admin
  message = admin_message
when :user
  message = user_message
when :author
  message = author_message
else
  message = anonymous_message
end

if user.preferred_notification_method = :email
  send_email(message)
elsif user.preferred_notification_method = :text
  send_text_message(message)
else
  queue_notification(message)
end

If you wanted to test every possible path here, you’d have 12 different situations to test:

  1. User is an admin, preferred_notification_method is email
  2. User is an admin, preferred_notification_method is text
  3. User is an admin, preferred_notification_method is neither
  4. User is a user, preferred_notification_method is email
  5. User is a user, preferred_notification_method is text
  6. User is a user, preferred_notification_method is neither
  7. User is an author, preferred_notification_method is email
  8. User is an author, preferred_notification_method is text
  9. User is an author, preferred_notification_method is neither
  10. User is anonymous, preferred_notification_method is email
  11. User is anonymous, preferred_notification_method is text
  12. User is anonymous, preferred_notification_method is neither

There are so many cases to test because “sending a message based on a notification method” and “generating a message based on the type of a user” are tied together. You might be able to squeeze by with fewer, but it’s not obvious – and it’s just asking for bugs.

But what if you broke them apart?

1
2
3
message = get_message_based_on_user_type(user.type)

send_notification(message, user.preferred_notification_method)

Now you can test each part separately.

For the first part, you can test that the right message is returned for each type of user.

For the second part, you can test that a given message is sent correctly based on the value of preferred_notification_method.

And finally, you can test that the parent method will pass the message returned from do_stuff_based_on_user_type along to send_email_or_text. So now, you have 8 states to test:

  1. User is an admin
  2. User is a user
  3. User is an author
  4. User is anonymous
  5. preferred_notification_method is email
  6. preferred_notification_method is text
  7. preferred_notification_method is neither
  8. and one test for the parent method

Here, you save four tests by breaking code apart so you can test it separately. In the second example, it’s a lot more obvious that you can get by with fewer tests. And you can imagine how as you add more states, splitting your code up becomes an even better idea.

It takes time and practice before you’ll find the best balance between isolation and readability. But if you break your dependencies in the right place, you can get by with a lot fewer tests.

Focus

Your app should be well-tested. But that doesn’t mean every part of your app deserves the same amount of attention on its tests.

Even if you do aim for 100% test coverage, you still won’t test everything. You probably won’t test every line of text in your views, for instance, or that you’re polling for updates every five seconds instead of ten.

That’s where focus comes in. Writing fewer, more useful tests. And making a conscious decision where you can best spend the time you have.

Focus is another thing that’s hard to get right. These are a few questions I ask myself that help me concentrate on the most important tests:

  • How interconnected is this with the rest of my app? If it breaks, how many other pieces will go down with it?

  • How likely is it that this will change naturally? If my tests fail, will it be because of a bug, or because someone updated some text in the UI?

  • What’s the impact of this breaking? Am I going to charge someone’s credit card twice, or is it just going to end up with some missing text?

  • How often is this part used? Is it critical to the app’s behavior, or is it an about page buried somewhere in the footer?

You shouldn’t only test the important parts. But you’ll have an app that feels higher quality if you spend your testing time well.


If you try to test every single possible path someone could take through your app, you’ll never ship. TDD helps, but it won’t solve all of your testing problems.

Of course, that doesn’t mean you shouldn’t test at all.

You can use the test pyramid to keep your tests small. You can isolate and break dependencies to turn m * n test cases into m + n. And you can prioritize, so you can spend more time testing the most important parts of your app.

So, how much do you test? Do you consider any of these ideas as you build out your app? And how do you know which parts of your app to focus on? Leave a comment and tell me all about it!


3 quick gem tricks

I recently updated Ruby and upgraded a few projects. And while I did, I found some pretty cool RubyGems features that I didn’t know about before:

When your executables get out-of-date

I used to use rvm to manage Ruby versions. But the last time I set up my machine, I decided to try going without it. You don’t need Gemsets when you have Bundler, and you can use Homebrew to keep Ruby up to date.

This works great, until you update Ruby. And rails new, bundle install and all those other commands break. They’ll point to the old Ruby version, not the one you just installed.

You could fix this by uninstalling and reinstalling each gem one by one. But that’s just crazy. Instead, try gem pristine:

1
gem pristine --all --only-executables

gem pristine takes a gem, and resets it to the version you originally downloaded. It’s like uninstalling and reinstalling the gem. (This is also helpful if you decided to edit a gem while debugging, and forgot to change it back.)

--all means “all gems”, and --only-executables means “only reset files like /usr/local/bin/rails and /usr/local/bin/bundle”. That is, only fix the scripts you use to run a gem from the command line.

So this command resets files like /usr/local/bin/rails to what they would have been if you uninstalled and reinstalled the gem.

A minute later, you’ll be back to working on your app.

When you need an older version

When I wrote my post on respond_to, I created some tiny apps to learn more about it. I used a few different Rails versions to see how each version dealt with respond_to and respond_with.

How do you generate each Rails app with the right Rails version? You don’t have to do this:

1
2
3
4
5
6
7
gem install rails -v 4.0.0
rails new respond_to_4.0
gem uninstall rails

gem install rails -v 4.1.0
rails new respond_to_4.1
gem uninstall rails

There’s an easier way. You can tell RubyGems which version you want to run, right on the command line, with underscores:

1
2
rails _4.0.0_ new respond_to_4.0
rails _4.1.0_ new respond_to_4.1

Fuzzy gem versions

But in that last section, there’s still a problem. You probably don’t actually want to install 4.0.0. You want the newest version of 4.0, with all the minor updates.

But do you remember what the newest minor version of Rails 4.0 is?

There are lots of ways to look it up. But why look it up, when RubyGems can just do what you want?

1
gem install rails -v "~>4.0.0"

You can use all the version strings you know from Bundler:

1
gem install rails -v ">3.1, <4.1"

Useful! Especially if, like me, you hate doing stuff that a computer is better at.

But you don’t have to know all this

When I ran into these problems, I didn’t know there was an easy answer. But you could guess there would probably be one.

These were all situations where you can solve the problem by yourself, but it’d be repetitive and annoying.

And when you find a repetitive, annoying task like this, especially in a well-used project, it means one of two things:

  1. Someone has already automated it, or
  2. Lots of people are hoping you’ll automate it.

So before you do the busywork, dig a little deeper. Investigate a little bit. It’ll be worth your time.

This post was originally sent exclusively to my list. To get more posts like it in your inbox every Friday, sign up here!


How to select database records in an arbitrary order

In Rails, it’s easy to get a bunch of records from your database if you have their IDs:

1
Person.where(id: [1, 2, 3]).map(&:id) => [1, 2, 3]

But what if you wanted to get the records back in a different order? Maybe your search engine returns the most relevant IDs first. How do you keep your records in that order?

You could try where again:

1
Person.where(id: [2, 1, 3]).map(&:id) => [1, 2, 3]

But that doesn’t work at all. So how do you get your records back in the right order?

The compatible way: case statements

Just like Ruby, SQL supports case...when statements.

You don’t see it too often, but case...when statements can almost act like hashes. You can map one value to another:

1
2
3
4
5
case :b
when :a then 1
when :b then 2
when :c then 3
end # => 2

That case statement kind of looks like a hash:

1
2
3
4
5
{
  :a => 1,
  :b => 2,
  :c => 3
}[:b] # => 2

So, you have a way to map keys to order they should appear in. And your database can sort and return your results by that arbitrary order.

Knowing that, you could put your IDs and their position into a case statement, and use it in a SQL order clause.

So if you wanted your objects returned in the order [2, 1, 3], your SQL could look like this:

1
2
3
4
5
6
7
SELECT * FROM people
  WHERE id IN (1, 2, 3)
  ORDER BY CASE id
    WHEN 2 THEN 0
    WHEN 1 THEN 1
    WHEN 3 THEN 2
    ELSE 3 END;

That way, your records are returned in the right order. The CASE transforms each ID into the order it should be returned in.

Of course, that looks ridiculous. And you could imagine how annoying a clause like that would be to build by hand.

But you don’t have to build it by hand. That’s what Ruby’s for:

lib/extensions/active_record/find_by_ordered_ids.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
module Extensions::ActiveRecord::FindByOrderedIds
  extend ActiveSupport::Concern
  module ClassMethods
    def find_ordered(ids)
      order_clause = "CASE id "
      ids.each_with_index do |id, index|
        order_clause << sanitize_sql_array(["WHEN ? THEN ? ", id, index])
      end
      order_clause << sanitize_sql_array(["ELSE ? END", ids.length])
      where(id: ids).order(order_clause)
    end
  end
end

ActiveRecord::Base.include(Extensions::ActiveRecord::FindByOrderedIds)

Person.find_ordered([2, 1, 3]) # => [2, 1, 3]

Exactly how we wanted it!

A cleaner, MySQL-specific way

If you use MySQL, there’s a cleaner way to do this. MySQL has special ORDER BY FIELD syntax:

1
2
3
SELECT * FROM people
WHERE id IN (1, 2, 3)
ORDER BY FIELD(id, 2, 1, 3);

You could also generate that from Ruby:

lib/extensions/active_record/find_by_ordered_ids.rb
1
2
3
4
5
6
7
8
9
10
11
module Extensions::ActiveRecord::FindByOrderedIds
  extend ActiveSupport::Concern
  module ClassMethods
    def find_ordered(ids)
      sanitized_id_string = ids.map {|id| connection.quote(id)}.join(",")
      where(id: ids).order("FIELD(id, #{sanitized_id_string})")
    end
  end
end

ActiveRecord::Base.include(Extensions::ActiveRecord::FindByOrderedIds)

So, if you’re using MySQL, and not too worried about compatibility, this is a good way to go. It’s a lot easier to read as those statements fly through your logs.


When you want to display records in a specific, arbitrary order, you don’t need to sort them in Ruby. With a little code snippet, you can let the database do what it’s good at: finding, sorting, and returning data to your app.


A shortcut to see your Minitest failures instantly

You’re about to check in your next small feature, so you kick off a full integration test run. You wait, and wait, as the dots fill your screen, until…

......FF....

:-(

You still have a few minutes before your tests finish running. But if you quit the test run early, you’ll have no idea which tests failed.

Do you really have to wait for the entire run to finish before you can see those failures?

Ctrl-T to the rescue!

If you’re using a Mac, there’s a way to see your test failures early:

Hit Ctrl-T while your tests are running.

When you do, you’ll see which test case is currently running, and how long it’s been running for. If any tests have failed so far, you’ll also see those failures, so you can get a head start on fixing them before your next run!

This is also really handy for debugging tests that just hang. Ctrl-T will tell you which test is trying to run, so you can isolate just that one test and fix it.

Finally, I’ve built a habit of hitting Ctrl-T anytime a test takes a noticeably long time (say, a second or longer) to finish. It’s pointed me to plenty of slow tests that I need to make faster.

How does Ctrl-T work?

On a Mac, Ctrl-T sends a message, or signal, called INFO, to whichever program is running:

signal_test.rb
1
2
3
4
puts "Starting..."
trap("INFO") { puts "INFO triggered!" }

loop { print "."; sleep 0.1}
1
2
3
4
5
6
7
8
9
10
11
12
~/Source jweiss$ ruby signal_test.rb
Starting...
........^Tload: 7.14  cmd: ruby 6121 running 0.10u 0.08s
INFO triggered!
.......^Tload: 7.14  cmd: ruby 6121 running 0.10u 0.08s
INFO triggered!
................^Tload: 11.77  cmd: ruby 6121 running 0.10u 0.08s
.INFO triggered!
......^Csignal_test.rb:5:in `sleep': Interrupt
 from signal_test.rb:5:in `block in <main>'
  from signal_test.rb:5:in `loop'
 from signal_test.rb:5:in `<main>'

Minitest knows about INFO, and responds to it by printing information about the test run:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
~/Source/rails/activesupport[master] jweiss$ be rake
/usr/local/Cellar/ruby/2.2.0/bin/ruby -w -I"lib:test"  "/usr/local/Cellar/ruby/2.2.0/lib/ruby/2.2.0/rake/rake_test_loader.rb" "test/**/*_test.rb"
Run options: --seed 33445

# Running:

.................F........^Tload: 1.62  cmd: ruby 29646 running 4.37u 1.40s
Current results:


  1) Failure:
CleanLoggerTest#test_format_message [/Users/jweiss/Source/rails/activesupport/test/clean_logger_test.rb:13]:
Expected "error\n" to not be equal to "error\n".



Current: DigestUUIDExt#test_invalid_hash_class 0.02s
............................

Pretty nice!

Knowing that this is possible, you might think of ways other apps could handle INFO:

  • Rails could display the currently running controller action or some performance stats.
  • Sidekiq could tell you what each worker is doing, so you could see where they get stuck.

And Sidekiq actually used to use INFO to print a backtrace of each thread it ran. But because INFO isn’t supported on Linux, Sidekiq switched to a different signal. Unfortunately, that signal can’t be triggered by a keyboard shortcut the way INFO can.

Because INFO isn’t available on Linux (and some might say that using INFO this way isn’t totally right, anyway), this behavior isn’t as widespread as it could be.

Still, it’s a little bit of extra help that could be useful in a wide range of situations. If you’re building an app, it’s worth thinking about what kind of status messages you could display on-demand to people who are interested.


How to debug Ruby performance problems in production

You know that performance is a feature. And a lot of performance problems can be found and fixed during development.

But what about those slowdowns that only show up in production? Do you have to add log messages to every single line of code? That would just slow things down even more! Or do you ship tons of tiny “maybe this fixes it” commits to see what sticks?

You don’t have to ruin your code to analyze it. Instead, try rbtrace-ing it.

Trace your running Ruby app

With rbtrace, you can detect performance problems, run code inside another Ruby process, and log method calls without having to add any code. Just add gem "rbtrace" to your Gemfile.

I learned about rbtrace from Sam Saffron’s amazing post about debugging memory leaks in Ruby (which you should really check out, if you haven’t already).

In that post, Sam used rbtrace to see all of the objects a process used:

1
bundle exec rbtrace -p $SIDEKIQ_PID -e 'Thread.new{GC.start;require "objspace";io=File.open("/tmp/ruby-heap.dump", "w"); ObjectSpace.dump_all(output: io); io.close}'

This is awesome. But there’s a whole lot more you can do.

What can you do with rbtrace?

Ever wanted to see the SQL statements you’re running in production (and how long they took)?

1
2
3
~/Source/testapps/rbtrace jweiss$ rbtrace -p $RAILS_PID --methods "ActiveRecord::ConnectionAdapters::PostgreSQLAdapter#execute_and_clear(sql)"
*** attached to process 7897
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter#execute_and_clear(sql="SELECT  \"articles\".* FROM \"articles\" WHERE \"articles\".\"id\" = $1 LIMIT 1") <0.002631>

All method calls that take longer than 2 seconds?

1
2
3
4
~/Source/testapps/rbtrace jweiss$ rbtrace -p $RAILS_PID --slow 2000
*** attached to process 8154
    Integer#times <2.463761>
        ArticlesController#create <2.558673>

Do you want to know every time a certain method gets called?

1
2
3
~/Source/testapps/rbtrace jweiss$ rbtrace -p $RAILS_PID --methods "ActiveRecord::Persistence#save"
*** attached to process 8154
ActiveRecord::Persistence#save <0.010964>

See which threads your app is running?

1
2
3
4
~/Source/testapps/rbtrace jweiss$ rbtrace -p $RAILS_PID -e "Thread.list"
*** attached to process 8154
>> Thread.list
=> [#<Thread:0x007ff4fcc9a8a8@/usr/local/lib/ruby/gems/2.2.0/gems/puma-2.6.0/lib/puma/server.rb:269 sleep>, #<Thread:0x007ff4fcc9aa10@/usr/local/lib/ruby/gems/2.2.0/gems/puma-2.6.0/lib/puma/thread_pool.rb:148 sleep>, #<Thread:0x007ff4fcc9ab50@/usr/local/lib/ruby/gems/2.2.0/gems/puma-2.6.0/lib/puma/reactor.rb:104 sleep>, #<Thread:0x007ff4f98c0410 sleep>]

Yep, with -e you can run Ruby code inside your server:

1
2
3
4
~/Source/testapps/rbtrace jweiss$ rbtrace -p $RAILS_PID -e "ActiveRecord::Base.connection_config"
*** attached to process 8154
>> ActiveRecord::Base.connection_config
=> {:adapter=>"postgresql", :pool=>5, :timeout=>5000, :database=>"rbtrace_test"}

Yeah, OK, now I’m a little scared. But that’s still very cool. (And only users with permission to mess with the process can rbtrace it, so it’s probably OK).


rbtrace gives you a ton of tools to inspect your Ruby processes in staging and production. You can see how your processes are using (or abusing) memory, trace slow function calls, and even execute Ruby code.

You don’t have to create tons of test commits and log messages to fix problems. You can just hop on to the server, get some data, and hop back out. And even if I’m not totally comfortable using it in production yet, I’m sure it’ll even help out in our test environments.

How about you? What could you use rbtrace for?


Fun with keyword arguments, hashes, and splats

You’ve probably seen this pattern before. A method has an options hash as its last argument, which holds extra parameters:

1
2
3
4
5
6
def hello_message(name_parts = {})
  first_name = name_parts.fetch(:first_name)
  last_name = name_parts.fetch(:last_name)

  "Hello, #{first_name} #{last_name}"
end

Unfortunately, you need to extract those parameters out of the hash. And that means there’s a lot of setup to wade through before you get to the good part.

But if you changed this method to use keyword arguments in Ruby 2.0+, you wouldn’t have to pull :first_name and :last_name out of your hash. Ruby does it for you:

1
2
3
def hello_message(first_name:, last_name:)
  "Hello, #{first_name} #{last_name}"
end

Even better, if your app uses Ruby 1.9+ hash syntax, your methods can use keyword arguments without changing those methods’ callers:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def hello_message_with_an_options_hash(name_parts = {})
  first_name = name_parts.fetch(:first_name)
  last_name = name_parts.fetch(:last_name)

  "Hello, #{first_name} #{last_name}"
end

def hello_message_with_keyword_arguments(first_name:, last_name:)
  "Hello, #{first_name} #{last_name}"
end

hello_message_with_an_options_hash(first_name: "Justin", last_name: "Weiss")

hello_message_with_keyword_arguments(first_name: "Justin", last_name: "Weiss")

See? Those arguments are identical!

Pushing keyword argument syntax one step too far

What if you haven’t switched to the new Hash syntax, though? You could convert all your code. But, at least in Ruby 2.2.1, the old hash syntax works just fine with keyword arguments:

1
2
irb(main):007:0> hello_message_with_keyword_arguments(:first_name => "Justin", :last_name => "Weiss")
=> "Hello, Justin Weiss"

Nice! What about passing a hash object, instead of arguments?

1
2
3
irb(main):008:0> options = {:first_name => "Justin", :last_name => "Weiss"}
irb(main):009:0> hello_message_with_keyword_arguments(options)
=> "Hello, Justin Weiss"

Whoa. What if we want to mix a hash and keyword arguments?

1
2
3
4
irb(main):010:0> options = {last_name: "Weiss"}
irb(main):011:0> hello_message_with_keyword_arguments(first_name: "Justin", options)
SyntaxError: (irb):11: syntax error, unexpected ')', expecting =>
 from /usr/local/bin/irb:11:in `<main>'

OK. I guess we took that one step too far. To fix this, you could use Hash#merge to build a hash you could pass in on its own. But there’s a better way.

If you were using regular arguments instead of keyword arguments, you could splat arguments from an Array, using *:

1
2
3
4
5
6
def generate_thumbnail(name, width, height)
  # ...
end

dimensions = [240, 320]
generate_thumbnail("headshot.jpg", *dimensions)

But is there a way to splat keyword arguments into an argument list?

It turns out there is: **. Here’s how you’d fix that broken example with **:

1
2
3
irb(main):010:0> options = {last_name: "Weiss"}
irb(main):011:0> hello_message_with_keyword_arguments(first_name: "Justin", **options)
=> "Hello, Justin Weiss"

And if you’re really crazy, you can mix regular arguments, keyword arguments, and splats:

1
2
3
4
5
6
7
8
def hello_message(greeting, time_of_day, first_name:, last_name:)
  "#{greeting} #{time_of_day}, #{first_name} #{last_name}!"
end

args = ["Morning"]
keyword_args = {last_name: "Weiss"}

hello_message("Good", *args, first_name: "Justin", **keyword_args) # => "Good Morning, Justin Weiss!"

Of course, if you find yourself in the situation where that’s necessary, you probably made a mistake a lot earlier!

Capture keyword arguments the easy way

Do you know how you can turn all your method arguments into an array using *?

1
2
3
4
5
def argument_capturing_method(*args)
  args
end

argument_capturing_method(1, 2, 3) # => [1, 2, 3]

This also works with keyword arguments. They’re converted to a hash, and show up as the last argument of your args array:

1
argument_capturing_method(1, 2, 3, key: "value") # => [1, 2, 3, {:key=>"value"}]

But args.last[:key] isn’t the best way to read keyword arguments grabbed this way. Instead, you can use the new ** syntax to get the keyword arguments by themselves:

1
2
3
4
5
def dual_argument_capturing_method(*args, **keyword_args)
  {args: args, keyword_args: keyword_args}
end

dual_argument_capturing_method(1, 2, 3, key: "value") # => {:args=>[1, 2, 3], :keyword_args=>{:key=>"value"}}

With this syntax, you can access the first regular argument with args[0] and the :key keyword argument with keyword_args[:key].

… Of course, now we’re back to options hashes.


Keyword arguments are great for removing a ton of parameter extraction boilerplate from your code. And you might not even have to change any of your code to take advantage of them.

But when you write more generic methods, there are some new techniques you’ll have to learn to handle keyword arguments well. You might not have to use those techniques very often. But when you do, this power and flexibility will be there, waiting for you.


Go beyond the easy fix with code archaeology

When you go bugfixing, the quick, obvious change isn’t always the best one. And the code in front of you is never the whole story. To go beyond the easy fix, you have to know why certain decisions were made. You have to understand the history behind the code. And there are three great ways to learn what you need to know to confidently change code.

git blame

With the help of git blame, you can trace through every version of every line of code in a project, all the way back to when it was written.

For example, say you were looking at ActiveJob’s queue_name.rb file, and you wanted to know what this queue_name_delimiter attribute was all about:

activejob/lib/active_job/queue_name.rb
1
2
3
4
5
6
7
included do
  class_attribute :queue_name, instance_accessor: false
  class_attribute :queue_name_delimiter, instance_accessor: false

  self.queue_name = default_queue_name
  self.queue_name_delimiter = '_' # set default delimiter to '_'
end

You could run git blame on it:

1
2
3
4
5
6
7
8
$ git blame queue_name.rb

...
da6a86f8 lib/active_job/queue_name.rb           (Douwe Maan               2014-06-09 18:49:14 +0200 34)     included do
1e237b4e activejob/lib/active_job/queue_name.rb (Cristian Bica            2014-08-25 17:34:50 +0300 35)       class_attribute :queue_name, instance_accessor: false
11ab04b1 activejob/lib/active_job/queue_name.rb (Terry Meacham            2014-09-23 15:51:44 -0500 36)       class_attribute :queue_name_delimiter, instance_accessor: false
11ab04b1 activejob/lib/active_job/queue_name.rb (Terry Meacham            2014-09-23 15:51:44 -0500 37)
...

And for each line, in order, you’ll see:

  • The revision that changed that line most recently (11ab04b1, for example),
  • The name of the author of that commit,
  • And the date the change was made.

To learn more about that line of code, you’ll need the revision number. Pass the id (that 11ab04b1 part) to git show or git log:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ git show 11ab04b1

commit 11ab04b11170253e96515c3ada6f2566b092533a
Author: Terry Meacham <zv1n.fire@gmail.com>
Date:   Tue Sep 23 15:51:44 2014 -0500

    Added queue_name_delimiter attribute.

    - Added ActiveJob::Base#queue_name_delimiter to allow for
      developers using ActiveJob to change the delimiter from the default
      ('_') to whatever else they may be using (e.g., '.', '-', ...).

    - Updated source guide to include a blurb about the delimiter.

diff --git a/activejob/lib/active_job/queue_name.rb b/activejob/lib/active_job/queue_name.rb
index d167617..6ee7142 100644
...

Cool! You get to learn a little more about the change, why it’s useful, and see the part of the Rails Guide about it that you might have missed before.

Here, we got pretty lucky. We found the information we were looking for right away. But git blame only shows you the most recent time that line changed. And sometimes, you won’t find what you’re looking for until you go two or three commits back.

To see earlier commits, you can call git blame again. But this time, pass the revision before the commit git blame found. (In git, you can say “the commit before this other commit” by putting a ^ after the revision, like 11ab04b1^):

1
2
3
4
5
6
7
8
9
10
$ git blame 11ab04b1^ queue_name.rb

...
da6a86f8 lib/active_job/queue_name.rb           (Douwe Maan               2014-06-09 18:49:14 +0200 33)     included do
1e237b4e activejob/lib/active_job/queue_name.rb (Cristian Bica            2014-08-25 17:34:50 +0300 34)       class_attribute :queue_name, instance_accessor: false
94ae25ec activejob/lib/active_job/queue_name.rb (Cristian Bica            2014-08-15 23:32:08 +0300 35)       self.queue_name = default_queue_name
...

$ git blame 1e237b4e^ queue_name.rb
... and so on ...

That’s pretty mind-numbing, though.

Instead, explore your text editor. Most editors make tracing through history with git blame easy. For example, in Emacs, after git blame-ing your code, place your cursor on a line. Then, you can press a to step back through each change that affected that line, and use l and D to see more detail about a commit.

Does your team refer to issue numbers and pull requests in your commit messages? If so, git blame can easily take you from a line of code, to a commit, to the discussion about that commit. And that discussion is where you’ll find all the really good stuff.

Github Issues

A little while ago, I noticed that Rails 4.2 removed respond_with. In the docs, it’s clear that it was removed, but I didn’t understand why.

There’s a ton of great knowledge hidden behind GitHub’s issue search box. If you want to know why a feature was removed, there’s no better place to learn than the discussion the team had while they decided to remove it.

So, if you search Rails’ GitHub repo for respond_with, you’ll find some interesting threads about respond_with. If you’re trying to find out why it was removed, you’ll probably land on this thread. Unfortunately, it describes how it was removed, but not why.

Later on in that thread, though, you’ll find a comment that points to the actual discussion about removing respond_with. That’s where the good stuff is!

As with git blame, you might not find exactly what you’re looking for right away. You’ll have to follow references, read comments, and click on links. But GitHub’s issue search will get you started in the right place. And with a little curiosity and a sense of exploration, you’ll learn what you came for.

Ask questions

Unfortunately, not all knowledge about a project can be found in its history, issues, and pull requests. Not everything is written down.

So if you can find the person that originally wrote the code, ask about it. You’ll discover the dev culture and ideas that led to a decision. You’ll learn some history about the project that might never have been recorded. And you’ll hear about paths that were never taken, which might actually make sense to try now.

The easy way can be dangerous

Sometimes, a fix just seems so easy. All you have to do is rescue this one exception in this one place, and then you can go home!

But it’s dangerous to change code you don’t understand.

So when a line of code seems off, or a decision seems weird, or a call seems useless, put on your archeologist’s hat. Learn as much as you can, however you can, about that code. That’s how you’ll make your code change intentionally, and fix the problem at its root.


How Rails sessions work

What if your Rails app couldn’t tell who was visiting it? If you had no idea that the same person requested two different pages? If all the data you stored vanished as soon as you returned a response?

That might be fine for a mostly static site. But most apps need to be able to store some data about a user. Maybe it’s a user id, or a preferred language, or whether they always want to see the desktop version of your site on their iPad.

session is the perfect place to put this kind of data. Little bits of data you want to keep around for more than one request.

Sessions are easy to use:

1
session[:current_user_id] = @user.id

But they can be a little magical. What is a session? How does Rails know to show the right data to the right person? And how do you decide where you keep your session data?

What is a session?

A session is just a place to store data during one request that you can read during later requests.

You can set some data in a controller action:

app/controllers/sessions_controller.rb
1
2
3
4
5
def create
  # ...
  session[:current_user_id] = @user.id
  # ...
end

And read it in another:

app/controllers/users_controller.rb
1
2
3
4
def index
  current_user = User.find_by_id(session[:current_user_id])
  # ...
end

It might not seem that interesting. But it takes coordination between your user’s browser and your Rails app to make everything connect up. And it all starts with cookies.

When you request a webpage, the server can set a cookie when it responds back:

1
2
3
~ jweiss$ curl -I http://www.google.com | grep Set-Cookie

Set-Cookie: NID=67=J2xeyegolV0SSneukSOANOCoeuDQs7G1FDAK2j-nVyaoejz-4K6aouUQtyp5B_rK3Z7G-EwTIzDm7XQ3_ZUVNnFmlGfIHMAnZQNd4kM89VLzCsM0fZnr_N8-idASAfBEdS; expires=Wed, 16-Sep-2015 05:44:42 GMT; path=/; domain=.google.com; HttpOnly

Your browser will store those cookies. And until the cookie expires, every time you make a request, your browser will send the cookies back to the server:

1
2
3
4
5
6
7
...
> GET / HTTP/1.1
> User-Agent: curl/7.37.1
> Host: www.google.com
> Accept: */*
> Cookie: NID=67=J2xeyegolV0SSneukSOANOCoeuDQs7G1FDAK2j-nVyaoejz-4K6aouUQtyp5B_rK3Z7G-EwTIzDm7XQ3_ZUVNnFmlGfIHMAnZQNd4kM89VLzCsM0fZnr_N8-idASAfBEdS; expires=Wed, 16-Sep-2015 05:44:42 GMT; path=/; domain=.google.com; HttpOnly
...

Many cookies just look like gibberish. And they’re supposed to. Because the information inside the cookie isn’t meant for the user. Your Rails app is in charge of figuring out what a cookie means. Your app set it, so your app can read it.

What does this have to do with a session?

So, you have a cookie. You put data in during one request, and you get that same data in the next. What’s the difference between that and a session?

By default, in Rails, there isn’t much of a difference. Rails does some work with the cookie to make it more secure. But besides that, it works the way you’d expect. Your Rails app puts some data into the cookie, the same data comes out of the cookie. If this was all there was, there’d be no reason to distinguish sessions from cookies.

But cookies aren’t always the right answer for session data:

  • You can only store about 4kb of data in a cookie.

    This is usually enough, but sometimes it’s not.

  • Cookies are sent along with every request you make.

    Big cookies mean bigger requests and responses, which mean slower websites.

  • If you accidentally expose your secret_key_base, your users can change the data you’ve put inside your cookie.

    When this includes things like current_user_id, anyone can become whichever user they want!

  • Storing the wrong kind of data inside a cookie can be insecure.

If you’re careful, these aren’t big problems.

But when you can’t store your session data inside a cookie for one of these reasons, Rails has a few other places to keep your sessions:

Alternative session stores

All of the session stores that aren’t the cookie session store work in pretty much the same way. But it’s easiest to think about using a real example.

If you were keeping track of your sessions with ActiveRecord:

  1. When you call session[:current_user_id] = 1 in your app, and a session doesn’t already exist:

  2. Rails will create a new record in your sessions table with a random session ID (say, 09497d46978bf6f32265fefb5cc52264).

  3. It’ll store {current_user_id: 1} (Base64-encoded) in the data attribute of that record.

  4. And it’ll return the generated session ID, 09497d46978bf6f32265fefb5cc52264, to the browser using Set-Cookie.

The next time you request a page,

  1. The browser sends that same cookie to your app, using the Cookie: header.

    (like this: Cookie: _my_app_session=09497d46978bf6f32265fefb5cc52264;
    path=/; HttpOnly)

  2. When you call session[:current_user_id]:

  3. Your app grabs the session ID out of your cookie, and finds its record in the sessions table.

  4. Then, it returns current_user_id out of the data attribute of that record.

Whether you’re storing sessions in the database, in Memcached, in Redis, or wherever else, they mostly follow this same process. Your cookie only contains a session ID, and your Rails app looks up the data in your session store using that ID.

Cookie store, cache store, or database store?

When it works, storing your sessions in cookies is by far the easiest way to go. It doesn’t need any extra infrastructure or setup.

But if you need to move beyond the cookie session store, you have two options:

Store sessions in a database, or store them in your cache.

Storing sessions in the cache

You might already be using something like Memcache to cache your partials or data. If so, the cache store is the second-easiest place to store session data, since it’s already set up.

You don’t have to worry about your session store growing out of control, because older sessions will automatically get kicked out of the cache if it gets too big. And it’s fast, because your cache will most likely be kept in memory.

But it’s not perfect:

  • If you actually care about keeping old sessions around, you probably don’t want them to get kicked out of the cache.

  • Your sessions and your cached data will be fighting for space. If you don’t have enough memory, you could be facing a ton of cache misses and early expired sessions.

  • If you ever need to reset your cache (say you upgraded Rails and your old cached data is no longer accurate), there’s no way to do that without expiring everyone’s sessions.

Still, this is how we store session data at Avvo, and it’s worked well for us so far.

Storing sessions in the database

If you want to keep your session data around until it legitimately expires, you probably want to keep it in some kind of database. Whether that’s Redis, ActiveRecord, or something else.

But database session storage also has drawbacks:

  • With some database stores, your sessions won’t get cleaned up automatically.

    So you’ll have to go through and clean expired sessions on your own.

  • You have to know how your database will behave when it’s full of session data.

    Are you using Redis as your session store? Will it try to keep all your session data in memory? Does your server have enough memory for that, or will it start swapping so badly you won’t be able to ssh in to fix it?

  • You have to be more careful about when you create session data, or you’ll fill your database with useless sessions.

    For example, if you accidentally touch the session on every request, googlebot could create hundreds of thousands of useless sessions. And that would be a bad time.

Most of these problems are pretty rare. But you should still be aware of them.

So how should you store your sessions?

If you’re pretty sure you won’t run into any of the cookie store’s limitations, use it. It doesn’t need much setup, and isn’t a headache to maintain.

Storing sessions in the cache vs. the database is more a judgement call about how bad it would be to expire a session early. I treat session data as pretty temporary, so the cache store works well for me. So I usually try cookie first, then cache, then database.

But how about you? How do you store your sessions? Leave a comment and let me know!


A couple of callback gotchas (and a Rails 5 fix)

ActiveRecord callbacks are an easy way to run code during the different stages of your model’s life.

For example, say you have a Q&A site, and you want to be able to search through all the questions. Every time you make a change to a question, you’ll want to index it in something like ElasticSearch. Indexing takes a while and isn’t urgent, so you’ll do it in the background with Sidekiq.

This seems like the perfect time to use an after_save callback! So in your model, you’ll write something like:

app/models/question.rb
1
2
3
4
5
6
7
8
9
10
11
class Question < ActiveRecord::Base
  after_save :index_for_search

  # ...

  private

  def index_for_search
    QuestionIndexerJob.perform_later(self)
  end
end
app/jobs/question_indexer_job.rb
1
2
3
4
5
6
7
class QuestionIndexerJob < ActiveJob::Base
  queue_as :default

  def perform(question)
    # ... index the question ...
  end
end

This works great! Or, at least, it seems to. Until you queue a lot more jobs and see these errors show up:

1
2015-03-10T05:29:02.881Z 52530 TID-oupf889w4 WARN: Error while trying to deserialize arguments: Couldn't find Question with 'id'=3

Sure, Sidekiq will retry the job and it’ll probably work next time. But it’s still a little weird. Why can’t Sidekiq find the question you just saved?

A race condition between processes

Rails calls after_save callbacks immediately after the record saves. But that record can’t be seen by other database connections, like the one Sidekiq is using, until the database transaction is committed, which happens a little later. This means there’s a chance that Sidekiq will try to find your question after you save it, but before you commit it. It can’t find your record, and it explodes.

This problem is so common that Sidekiq has an FAQ entry about it. And there’s an easy fix.

Instead of after_save:

app/models/question.rb
1
2
3
4
5
class Question < ActiveRecord::Base
  after_save :index_for_search

  # ...
end

use after_commit:

app/models/question.rb
1
2
3
4
5
class Question < ActiveRecord::Base
  after_commit :index_for_search

  # ...
end

And your job won’t get queued until Sidekiq can see your model.

So, when you queue a background job or tell another process about a change you just made, use after_commit. If you don’t, they might not be able to find the record you just touched.

But there’s one more problem…

OK, you switched a bunch of your after_save hooks to use after_commit instead. Everything seems to work. Time to check it all in and go home, right?

First, you’ll want to run your tests:

test/models/question_test.rb
1
2
3
4
5
6
7
8
9
require 'test_helper'

class QuestionTest < ActiveSupport::TestCase
  test "A saved question is queued for indexing" do
    assert_enqueued_with(job: QuestionIndexerJob) do
      Question.create(title: "Is it legal to kill a zombie?")
    end
  end
end
1
2
3
  1) Failure:
QuestionTest#test_A_saved_question_is_queued_for_indexing [/Users/jweiss/Source/testapps/after_commit/test/models/question_test.rb:7]:
No enqueued job found with {:job=>QuestionIndexerJob}

Whoops! Shouldn’t the test have queued the job? What just happened there?

By default, Rails wraps each test case in its own database transaction. This can really speed things up. It takes just one database command to undo all the changes you made during the test.

But this also means your after_commit callback won’t run. Because after_commit callbacks only run when the outermost transaction has been committed.

When you call save inside a test case, it still commits a transaction (more or less), but that’s the second-most-outermost transaction now. So your after_commit callbacks won’t run when you expect them to. And you can’t test what happens inside them.

This problem also has an easy fix. Include the test_after_commit gem in your Gemfile:

Gemfile
1
2
3
group :test do
  gem "test_after_commit"
end

And your after_commit hooks will run after your second-to-last transaction commits. Which is what you were expecting to happen.

You might be thinking, “That’s weird. Why do I have to use a whole separate gem to test a callback that comes with Rails? Shouldn’t it just happen automatically?”

You’re right. It is weird. But it won’t stay weird for long.

Once Rails 5 ships, you won’t have to worry about test_after_commit. Because this problem was fixed in Rails about a month ago.


In my own code, I use after_commit a lot. I probably use it more than I use after_save! But it hasn’t come without its problems and strange edge cases.

Version by version, though, it’s getting better. And when you use after_commit in the right places, a lot of weird, random exceptions just won’t happen anymore.


A faster way to cache complicated data models

When your data model gets complicated, and your APIs hit that sad 1 second response time, there’s usually an easy fix: :includes. When you preload your model’s associations, you won’t make as many SQL calls. And that can save you a ton of time.

But then your site slows down again, and you think about caching responses. And now you have a problem. Because if you want to get responses from the cache:

1
2
3
4
results = {lawyer_1: 1, lawyer_2: 2, lawyer_3: 3}
cached_objects = Rails.cache.fetch_multi(results.keys) do |key|
  Lawyer.find(results[key]).as_json
end

You’ve now lost all your :includes. Can you have both? How do you get a fast response for your cached objects, and still load the objects that aren’t in the cache, quickly?

There’s a lot to do, so thinking about it is tough. It’s easier when you break the problem apart into smaller pieces, and come up with a simple next step.

So what’s the first thing you can do? To do much of anything, you need to know which objects are in your cache, and which ones you still need to find.

Separate the cached from the uncached

So, say you have a bunch of cache keys:

1
cache_keys = [:key_1, :key_2, :key_3]

How can you tell which of these are in a cache?

ActiveSupport::Cache has a handy method called read_multi:

1
2
3
4
# When only lawyer_1 is cached

cache_keys = [:lawyer_1, :lawyer_2, :lawyer_3]
Rails.cache.read_multi(cache_keys) # => {:lawyer_1 => {"id": 1, "name": "Bob the Lawyer"} }

read_multi returns a hash of {key: value} for each key found in the cache. But how do you find all the keys that aren’t in the cache? You can do it the straightforward way: Loop through all the cache keys and find out which ones aren’t in the hash that read_multi returns:

1
2
3
4
5
6
7
8
cache_keys = [:lawyer_1, :lawyer_2, :lawyer_3]
uncached_keys = []

cached_keys_with_values = Rails.cache.read_multi(cache_keys)

cache_keys.each do |key|
  uncached_keys << key unless cached_keys_with_values.has_key?(key)
end

So, what do you have now?

  • An array of all the cache keys you wanted objects for.
  • A hash of {key: value} pairs for each object you found in the cache.
  • A list of the keys that weren’t in the cache.

And what do you need next?

  • The values for the keys that weren’t in the cache. Preferably fetched all at once.

That’s your next step.

Preload the uncached values

Soon, you’ll have to find an object using a cache key. To make things easier, you can change the code to something like:

1
2
3
4
5
6
7
8
9
cache_identifiers = {lawyer_1: 1, lawyer_2: 2, lawyer_3: 3}
cache_keys = cache_identifiers.keys
uncached_keys = []

cached_keys_with_values = Rails.cache.read_multi(cache_keys)

cache_keys.each do |key|
  uncached_keys << key unless cached_keys_with_values.has_key?(key)
end

So cache_identifiers now keeps track of the cache key and the object id to fetch.

Now, with your uncached keys:

1
uncached_keys # => [:lawyer_2, :lawyer_3]

And your cache_identifiers hash:

1
cache_identifiers # => {lawyer_1: 1, lawyer_2: 2, lawyer_3: 3}

You can fetch, preload, and serialize all those objects at once:

1
2
3
4
uncached_ids = uncached_keys.map { |key| cache_identifiers[key] }
uncached_lawyers = Lawyer.where(id: uncached_ids)
                         .includes([:address, :practice_areas, :awards, ...])
                         .map(&:as_json))

So what do you have now?

  • An array of all the cache keys you wanted objects for to begin with.
  • A hash of {key: value} pairs for each object found in the cache.
  • A list of the keys that weren’t in the cache.
  • All the values that weren’t found in the cache.

And what do you need next?

  • To cache all the values you just fetched, so you don’t have to go through this whole process next time.
  • The final list of all your objects, whether they came from the cache or not.

Cache the uncached values

You have two lists: one list of uncached keys and another of uncached values. But to cache them, it’d be easier if you had one list of [key, value] pairs, so that your value is right next to its key. This is an excuse to use one of my favorite methods, zip:

1
[1, 2, 3].zip(["a", "b", "c"]) # => [[1, "a"], [2, "b"], [3, "c"]]

With zip, you can cache your fetched values easily:

1
2
3
uncached_keys.zip(uncached_lawyers).each do |key, value|
  Rails.cache.write(key, value)
end

What do you have now?

  • An array of all the cache keys you wanted objects for to begin with.
  • A hash of {key: value} pairs for each object found in the cache.
  • A list of formerly-uncached values that you just cached.

And what do you still need?

  • One big list of all your objects, whether they came from the cache or not.

Bring it all together

Now, you have an ordered list of cache keys:

1
cache_keys = cache_identifiers.keys

Your list of the objects you fetched from the cache:

1
cached_keys_with_values = Rails.cache.read_multi(cache_keys)

And your list of objects you just now grabbed from the database:

1
2
3
4
uncached_ids = uncached_keys.map { |key| cache_identifiers[key] }
uncached_lawyers = Lawyer.where(id: uncached_ids)
                         .includes([:address, :practice_areas, :awards, ...])
                         .map(&:as_json))

Now you just need one last loop to put everything together:

1
2
3
4
results = []
cache_keys.each do |key|
  results << cache_keys_with_values[key] || uncached_lawyers.shift
end

That is, for each cache key, you grab the object you found in the cache for that key. If that key wasn’t originally in the cache, you grab the next object you pulled from the database.

After that, you’re done!

Here’s what the whole thing looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
cache_identifiers = {lawyer_1: 1, lawyer_2: 2, lawyer_3: 3}
cache_keys = cache_identifiers.keys
uncached_keys = []

# Fetch the cached values from the cache
cached_keys_with_values = Rails.cache.read_multi(cache_keys)

# Create the list of keys that weren't in the cache
cache_keys.each do |key|
  uncached_keys << key unless cached_keys_with_values.has_key?(key)
end

# Fetch all the uncached values, in bulk
uncached_ids = uncached_keys.map { |key| cache_identifiers[key] }
uncached_lawyers = Lawyer.where(id: uncached_ids)
                         .includes([:address, :practice_areas, :awards, ...])
                         .map(&:as_json))

# Write the uncached values back to the cache
uncached_keys.zip(uncached_lawyers).each do |key, value|
  Rails.cache.write(key, value)
end

# Create our final result set from the cached and uncached values
results = []
cache_keys.each do |key|
  results << cache_keys_with_values[key] || uncached_lawyers.shift
end
results

Was it worth it? Maybe. It’s a lot of code. But if you’re caching objects with lots of associations, it could save you dozens or hundreds of SQL calls. And that can shave a ton of time off of your API responses.

At Avvo, this pattern has been incredibly useful: a lot of our JSON APIs use it to return cached responses incredibly quickly.

The pattern has been so useful that I wrote a gem to encapsulate it called bulk_cache_fetcher. So if you ever find yourself trying to cache big, complicated data models, give it a try!


How to configure your Rails app to ship

You’re ready to launch your first production app, and it’s time to get it talking to some external services. You still have to get everything hooked up. So what’s the best way to configure your services in production, without making things more complicated on your dev machine?

Set up your Environment

To configure production apps, today’s best practice is to use environment variables (those ENV["REDIS_HOST"]-looking things).

But why?

  • It’s harder to accidentally commit your production keys.

    If you’re not paying attention, you might git push a file with important secret keys in it. And that could be an expensive mistake.

  • Configuration is what environment variables are there for.

    Environment variables are a common way to configure apps on almost every kind of system. Many other programs (like Ruby) use environment variables for configuration, so it only makes sense to try in your own app.

  • Environment variables are easy to set up in production.

    Heroku has a a web UI and a command line tool for easily setting environment variables. And if you’re building your own server, server management tools like Chef and Docker make setting environment variables easy.

What does it look like on the Rails side?

This is how an app that depends on environment variables could configure itself:

config/my_service.yml
1
2
3
production:
  host: <%= ENV["MY_SERVICE_HOST"] %>
  port: <%= ENV["MY_SERVICE_PORT"] %>
config/initializers/my_service.rb
1
2
3
my_service_config = Rails.application.config_for(:my_service)

my_service = MyService.new(my_service_config["host"], my_service_config["port"])

The initializer uses Rails 4.2’s config_for method to find the right .yml file and pick the right environment.

Then, config_for runs the ERB code inside my_service.yml, and grabs MY_SERVICE_HOST and MY_SERVICE_PORT out of the environment. It passes those values along to MyService.

You could also just have the initializer read from ENV["MY_SERVICE_HOST"] directly. But I prefer to keep them in .yml files, for reasons you’ll see in a minute.

Your app’s configuration in development

Environment variables are fine for production. But once you set up your production config, how do you handle development and test mode?

You have a few options. But I usually follow the convention in Rails’ config/secrets.yml: use environment variables in production, and hardcode non-secret values in development and test.

With the development and test environments, config/my_service.yml could look like this:

config/my_service.yml
1
2
3
4
5
6
7
8
9
10
11
production:
  host: <%= ENV["MY_SERVICE_HOST"] %>
  port: <%= ENV["MY_SERVICE_PORT"] %>
  
development:
  host: localhost
  port: 8081
  
test:
  host: localhost
  port: 8081

Awesomely enough, the initializer can stay exactly the same. The values in this file will be used in the development and test environments, and the production environment will get its values from the environment variables.

But why would you hardcode these values?

  • The configuration values are easier to see and change.

    You can tweak your config as you experiment with new features, which is something you want in development, but not so much in production.

  • It’s easier for someone new to get started.

    If all the sample config you need is checked into your git repository, a new dev just has to clone and run your app. They won’t need to muck around with setting just the right values to get the app working.

  • You don’t have to worry about conflicting environment variables.

    You’ll probably work on more apps on your dev machine than you’ll ever deploy to a single production machine. If you used system-wide environment variables to configure all those apps, there’s a good chance two of them will stomp on each other.

So, try using environment variables in production, and hardcoded .yml config in development. It’s easy, it’s readable, and Rails has built-in support for dealing with exactly those kinds of config files.

Another option for development

There’s another way to handle configuration in development mode: dotenv. It looks neat, but I haven’t tried it in an app of my own yet.

With dotenv, you can put environment variables in a file named .env in your Rails app’s root directory, and those values will get picked up by your app. This is nice, because your development environment acts more like your production environment. That’s a good way to avoid bugs that only ever happen in production.

It’s something I’ll try someday. But for now, I haven’t found anything more convenient than .yml and config_for.


Most production apps need some kind of configuration. So when you deploy your next app, try using .yml files, populated by environment variables in production, to configure it. You’ll get the flexibility, the simplicity, and the reliability you’re hoping for.

Do you have a different way you like to configure your production apps? Leave a comment, I’d love to hear about it!


A guide to extracting your first Ruby Gem

Is your GitHub contribution chart solid gray? You could use an open source project to work on. But you don’t have to start it from scratch. The easiest way to create a useful side project is to pull it out of the app you’re already building. That’s how Rails was born!

But how do you know what to extract? And how do you turn it into a gem, without destroying your workflow?

Find the code you want to extract.

Somewhere, deep inside your app, is some code that doesn’t belong there. Code that doesn’t need your app to do its job. Where is it?

Sometimes, you’ll just have to guess. But I often find extractable code in the same few places:

  • Validations

    Have you written any custom validations for your attributes? Those can make great gems.

  • Changes you’ve made to Rails

    Every once in a while, you’ll have to mess with Rails to get commit hooks working in tests or blank attributes to turn into NULL in the database. Whenever I’ve moved this kind of logic into a gem, it’s become more stable and easier to understand.

  • Non-activerecord models

    Do you do so much email address or phone number parsing that you moved it into its own class? These classes are often useful in other apps, and they’re pretty easy to turn into gems.

  • Mock objects and custom assertions

    You can write more readable tests by using custom assertions. And once you have some good custom assertions written for a library or pattern, they’re helpful to anyone else who uses that library or pattern.

You don’t have to think big. Some of my favorite gems are just one file!

And if you still can’t decide what to extract, browse the category list on RubyToolbox for some inspiration.

Put your code in a gem-like directory

Once you know which code you’ll turn into a gem, move that code around so it fits a gem-like structure inside your app.

In Rails apps, I use lib/ as a gem staging area. lib/ is where I put code that has the potential to turn into its own gem. So, if you’re creating a gem called “json_api_client”, your Rails app’s lib/ directory might look like:

1
2
3
4
5
...
my_rails_app/lib/json_api_client.rb
my_rails_app/lib/json_api_client/associations.rb
my_rails_app/lib/json_api_client/connection.rb
...

Most gems will have a file under lib/ named after the gem (lib/json_api_client.rb), and a bunch of files in a directory named after that gem (everything under lib/json_api_client/). So, take that same structure and match it inside your Rails app. That will make it much easier to move code into the gem later on.

If you’re confused about what a gem layout looks like, take a look at some of your favorite gems’ source on GitHub. You’ll pick up the pattern pretty quickly.

What about tests?

I used to follow lib/’s structure inside test/unit/:

1
2
3
4
5
...
my_rails_app/test/unit/json_api_client_test.rb
my_rails_app/test/unit/json_api_client/associations_test.rb
my_rails_app/test/unit/json_api_client/connection_test.rb
...

It worked OK, even if putting models and libraries in the same folder got a little messy.

Now, though, Rails uses test/models/ instead of test/unit/. And storing your lib/ tests inside test/models/ doesn’t make a whole lot of sense. I haven’t really decided on a convention for this yet. Do you have any suggestions?

Break the dependencies

Once your code is inside a gem, it won’t be able to depend on your app. This means you’ll have to go through the code you put in lib/, and look for places where it depends on classes, objects, or behavior specific to your app.

If you do find any of these dependencies, you’ll have to break them. There’s a lot of great writing about how to break (or inject) dependencies, so I’m not really going to go into that now.

Create a gem

I use bundle gem to create my gems. Specifically, to create a gem called bulk_cache_fetcher:

1
bundle gem bulk_cache_fetcher -t minitest

The -t adds some test helper files and the test tasks to the gem’s Rakefile.

You’ll have to do some housekeeping next, like filling out the .gemspec, writing the README, picking a LICENSE, all that stuff.

And then, since you already have your gem’s code in your Rails app’s lib/ folder, you can just move that code and its tests into the lib/ and test/ folders in your new gem.

A lot of the time, there’ll be things you missed or forgot about, or code that assumes things about your app that aren’t true inside a gem. So, before you move on, run the tests you moved into your gem, and make sure they all pass.

Use your new gem in your app

Now that you have a gem, you want to use it, right? But testing changes to your gem inside your Rails app can get annoying, quickly. You have to:

  1. Make the change in your gem
  2. Build the gem
  3. Remove all traces of the gem from your system, or update the version
  4. Install the gem
  5. Restart your server

This is pretty awful. Luckily, bundler gives you an easier way. Say you’ve created your gem in ~/Source/bulk_cache_fetcher. While you’re testing gem changes, you can write this inside your Rails app’s Gemfile:

Gemfile
1
gem "bulk_cache_fetcher", path: "~/Source/bulk_cache_fetcher"

Next, run bundle install, and you’ll be able to make changes to your gem as if that code still lived in your app’s lib/ folder.

One last thing: make sure you remove the path: before you check in your code! That path may not point to the right place on other systems, so chances are it won’t work anywhere except your machine.

Build, ship, and enjoy!

Once your gem is ready, you can send it out to the world.

So, sign up for an account on RubyGems if you haven’t already, check in your changes, and run rake release. Congratulations, you’re now a gem author! And once you push that gem to github, you’ll get your nice green square for the day.

Do you have any parts of your own apps that seem extractable? Anything that you think might make a good gem? I’d love to hear about it – just leave a comment below.

And if you want to learn a ton more about creating, managing, and maintaining Ruby gems, I highly recommend Brandon Hilkert’s Build a Ruby Gem. The process his book follows is very close to this one, and it covers a whole lot more.


Practicing Rails is now available!

Practicing Rails, my book on learning Rails without being overwhelmed, is now available. And you can still get 25% off until 11:59 PM Thursday night! (Pacific time)

Pick it up here: https://www.justinweiss.com/practicing-rails

A little background:

While I’ve written here, I’ve heard from so many people who have gone through introductory Rails books and videos, have learned the different parts of Rails, but still can’t get beyond the tutorial Rails apps.

They heard it was supposed to be fun, but it was actually just frustrating. They didn’t know where to start. The things they learned didn’t stick with them. They couldn’t spend time learning consistently enough to grow as a Rails developer. Most of all, they were overwhelmed, because the space of possibilities in front of them seemed too big.

Tutorials seem so easy when they’re words on a page. But when you try to transform your ideas into code, all you feel is stress, frustration, and fear. You don’t have the time to learn it all over again, so did you spend that time reading for nothing? Did you waste all that money you spent on bootcamps and video courses?

Practicing Rails will give you the processes, the structure, and the help you need to finally become the Rails developer you know you should be.

If you want to build a new Rails app from the ideas you have in your head…

Learn step-by-step ways to test your code easily and efficiently

Know what to do when your apps blow up, and you have no idea how to fix them…

Know which parts of Rails to learn now , and what you can put off until later…

Get just the important information out of the noisy Rails community…

And keep your motivation fired up, so you can turn mastering Rails into a habit you’ll actually keep…

This book is written for you.

Start Practicing Rails here.

P.S. At 11:59 PM on February 12th, the price will go up. So if you want to transform from a tutorial-follower to an app-builder, now’s the best time to buy.


How to keep Rails questions from killing your productivity

Practicing Rails is out! Get it for 25% off until Friday, February 13th.

While you write code, questions pop into your head constantly: How do I call this method? What options does it take? What happens if I pass an object instead of a number?

These questions destroy your productivity. When you don’t know the answer right away, you forget what you were doing. You get yanked out of your flow, and it takes a half hour to get back to where you left off.

So how do you find the answers you need without slowing down?

Fast lookup

Speed is key when you look up API documentation. You want to get your answer and get out as quickly as possible. Otherwise, it’s way too easy to get off-track. I can’t count how often I’ve needed to know the parameters to a method and somehow ended up looking at embarrassing text messages on reddit for an hour.

For speedy documentation, you’ll want Dash, for Mac, or Zeal, for everything else. With these tools, you can hit two keys on your keyboard and bring up the docs you need without having to think about what you’re doing.

But they need a tiny bit of setup first.

Once you download either of these apps, do these two things:

  1. Download the docsets for the libraries you use.

    I grab docs for Ruby, Rails, CSS, HTML, and Javascript, and that’s about it. If there’s a complicated gem you use a lot, like Nokogiri, grab those docs too. But don’t go overboard – if you have too many docs, you’ll slow down as you try to figure out which concat method you actually mean.

  2. Set up the Global Search Shortcut. On the Mac, I usually use option-Space.

    This is what transforms Dash from “just another doc site” into a totally essential tool. Once you set up the global search shortcut, you can hit option-Space, start typing, see the docs you need, hit option-Space again, and have your answer without even realizing what you just did. You can answer your API questions before you knew you had them.

Details, questions, and comments

Unfortunately, sometimes the API docs don’t have all the details you want to know. Or you might need to know how a feature works on the inside.

You’ll have to sacrifice speed to answer these questions. But there are two documentation sites that replace that speed with detail and community.

APIDock

APIDock only covers Ruby, Rails, and RSpec. But it’s still helpful:

  • APIDock will tell you when Rails methods were deprecated, and what you should be using instead. It keeps track of version-to-version changes in Rails all the way back to 1.0. So if you’re working with legacy Rails code, it’s an incredible resource.

  • Lots of API methods have comments, contributed by Rails developers. The comments are really good. With APIDock, you’ll hear about edge cases and common problems before you run into them.

Omniref

Omniref is new, and has some neat ideas. It has API docs, just like Dash and APIDock. But it also has StackOverflow-like Q&A on the methods, classes, or gems you’re looking at. And when that doesn’t solve your problem, you can see the source code right next to the docs.

This is a great way to get familiar with Rails and gem internals. If the documentation doesn’t answer your question, you can go straight to the source. And once you figure it out, you can leave a note right in the code for the next person to come along.

Your new documentation workflow

So, how do you put all this together?

  1. When you have a question about an API, use a hotkey to quickly bring up Dash or Zeal.
  2. If you need more detail, go to the web. Try Omniref first, except:
  3. If you’re searching for an API in an older Rails app, use APIDock.

When you can instantly find documentation, it’ll transform your Rails productivity. You’ll not only stay more focused, you’ll learn even more about the libraries and frameworks you depend on.

You can’t know everything about every tool you use. You’ll always rely on documentation at some level. So spend a little time now to save time when you look things up. It’ll pay off massively through your entire programming career.


Finding your way around a new Rails project

This article has been translated to Russian, thanks to Leonid Suschev.

You just changed teams at work, or just started a new job. Or you found a bug in your favorite open-source app, and want to write your first pull request. But after you git clone and open app/models, you’re totally lost. Even though the Rails structure is the same as what you’re used to, it’s impossible to find your way around the code. So what’s the fastest way to learn this new, unfamiliar Rails app?

Build a project vocabulary

What is a Player, and what’s a Session? How about an Account vs a User vs a Provider?

Every app uses different terms and different metaphors. And if you don’t understand the vocabulary of an app, you’ll waste time browsing app/models every time you see a new class name. Or worse, you’ll make bad assumptions about how features work.

So, the first thing you have to do is build that vocabulary. Learn what this app is made of. And if you want to do that quickly, start by reading db/schema.rb. You’ll learn the terms the app uses, and how all its data fits together. Pay special attention to the *_id columns, so you can see how the models connect to one another.

If you’re more visual, try the rails-erd gem. It generates a diagram from your Rails models, showing all your tables and columns and how they interact. Keep a printout on your desk, and you’ll have an easier time understanding the rest of your app.

Continue with the models and plain Ruby objects

In a well-designed Rails app, not everything will be backed by a database table. To continue to build your project vocabulary, look inside app/models for extra classes and attributes that you didn’t see inside db/schema.rb.

Don’t spend too much time here yet. It’s easy to get lost in how things work. You’re just here to fill out your project vocabulary.

As you learn more about the parts your app is built from, you might wonder why certain objects exist, or why they fit together the way they do. And that’s the point of the next stage.

Try the app, and find out “why”

Once you feel comfortable with the objects the app uses, start the app up. Click around and explore. If there’s documentation, it can help guide you. Otherwise, just see those terms you just learned come to life.

As you use the app, you’ll start to see it from a higher level. You’ll notice that there are reasons why models have the associations they do. Why they’re grouped together. Why those class names were chosen. Maybe they all appear on pages together, or are built from the same form.

And when you start to understand the parts the app is built from and why they exist, you’re ready to figure out how it all works.

Figure out the “how”

By now, you know enough about the app to learn it in detail. You won’t get distracted by new terms and new interactions. You can pick one thing you want to learn, and follow it all the way through.

There are lots of ways you can go from here. These are a few of my favorites:

  • Browse the UI, and think: How could this feature be built, knowing what you know about the pieces? Can you guess? Use a debugger to confirm your guess, or discover how it actually works.
  • Pick a test case and follow it all the way through the app. Either read through the code, or explore it with a debugger.
  • Pick a route or controller action. Can you figure out how to reach that controller action from the UI?

This is also a good time to go back through your models and plain Ruby objects. And this time, you can go into the details of how the models work.

Become curious about the app. Find gaps in your knowledge and close them with a debugger and the code.

Why not start with the tests?

I’ve always heard people say, “If you want to understand a codebase, start with the tests! Tests are executable documentation!”

But this has never worked for me. Test cases are too narrow, too detailed. If I learn from the tests, I feel like I’m missing the bigger point of the app.

In order to understand something as big as a Rails app, or even most gems, I’ve always needed an overview first. I have to know which components make up the app, and why they exist.

But absolutely, make sure the tests run. If they don’t, you’re working with a broken system. And you can’t rely on the things you discover from a broken system.

What, to why, to how

To understand a new Rails app, go through these three phases, in this order:

  • What parts make up this app?
  • Why do those parts exist? Why are they put together that way?
  • How does each piece work?

That will give you both the broad point of view and the narrow knowledge you need in order to make that new app yours.

P.S. I’m releasing Practicing Rails next week. So if you want to get the 25% early access discount, you’ll want to order it soon!


How to upgrade to Rails 4.2

Upgrading some Rails apps is as easy as bundle update rails.

But what if you have one of the other kind of apps? The ones that are still on Rails 4.0, or even 3.2, because you didn’t feel like dragging them kicking and screaming into the future?

Whether you’re facing a quick upgrade or a painful one, these steps will help you get your app to Rails 4.2 as smoothly as possible. And in the process, you’ll learn how to take full advantage of Rails 4.2’s new features.

Read the upgrade guide

You should always start your Rails upgrade by reading the Rails Upgrade Guide.

The upgrade guide will walk you through most of the major changes between Rails versions. It’ll tell you exactly how to change your app to fix the biggest problems you’ll run into. (For example, this change in Rails 4.1 would have caused us hours of pain if we hadn’t known to watch for it.)

But more than that, the upgrade guide will explain why you should make those changes. And that’s important, because it helps you understand which of its suggestions you should follow, and which you can ignore.

And if you want to understand the Rails 4.2 changes more deeply, read the release notes. They’ll introduce you to all the cool new Rails 4.2 features you’ll soon be able to use. If nothing else, it’ll motivate you to get that upgrade done.

Upgrade in steps

When you’re moving to a new Rails version, always upgrade to the newest point version first. And don’t skip minor versions.

So if you want to upgrade from 4.1.6 to 4.2, go 4.1.6 to 4.1.9, and 4.1.9 to 4.2. If you’re starting at 4.0.12, go from 4.0.12 to 4.0.13, then 4.0.13 to 4.1.9, and then from 4.1.9 to 4.2. It might seem like a lot more work, but it’ll save you time in the long run.

Why?

The newest point version of a Rails release has the best deprecation warnings.

Don’t underestimate how useful deprecation warnings can be. If you upgrade in steps, they’ll tell you exactly what will break and how to prevent it. If you jump too far, your app will just plain break, and you’ll have no idea why.

Finally, if a point version Rails upgrade (like from 4.0.9 to 4.0.13) breaks a test, it’ll usually also break during a major or minor upgrade. But those problems are much easier to debug and fix when less has changed.

So here’s what to do:

  1. Upgrade to the newest point release of whichever Rails version your app is on.
  2. Upgrade any other gems that keep you from bundle updateing.
  3. Fix the tests, committing as needed.
  4. Fix the deprecation warnings, committing as needed.
  5. Upgrade to the next minor or major version of Rails, and repeat.

Lean on your tests

When you upgrade Rails, your tests will be your first sign that something’s not right. They should be the first thing you run, and the first thing you fix. Don’t even bother trying to get your Rails app to boot before your tests mostly pass.

Upgrade your gems

When a test fails, and it doesn’t look like it’s your fault, it usually means one of your gems needs to be updated.

A Rails upgrade is just about the worst time to upgrade a gem. Since so much of your app is changing, it’s hard to tell if the Rails upgrade broke something, or a gem upgrade broke something, or if you broke something.

But because some gems rely so heavily on Rails internals, this is something you’ll just have to do. So if something inside sass fails, upgrade the sass-rails gem. And hope it doesn’t make things worse.

Take lots of notes

If you’re having a particularly rough Rails upgrade, you’ll have to experiment. You’ll probably do some debugging, tweak some code, or change some tests around.

As you do this, you will forget the changes you’ve made. Some of the things you tried that got you mostly, but not quite, there. Which tests you were going to look at next. Other places you need to change code. Tips to share with coworkers that are upgrading their own apps to Rails 4.2.

So, take lots of notes. Write everything down. Things you tried, and files you changed. Hypotheses you have about why things were failing, and whether they were correct. Which gems you had to update. Behavior that seems weird, and you’ll want to check out more closely later.

You don’t want to make a change, roll it back, realize it was actually the right thing to do, and not remember what you did. So write it down.

A few other resources

In the comments, Miha Rekar pointed to RailsDiff, which shows you the difference between new Rails apps generated under different Rails versions. This is a fantastic way to learn about configuration parameters you don’t need anymore, or new ones you should add to your app.

And Daniel Kehoe wrote a comprehensive guide to updating Ruby and Rails using RVM and RVM gemsets.

The best possible Rails upgrade

Some Rails upgrades are smooth. Things just work. Others are challenging, multi-day or multi-week projects. It depends on your app, your dependencies, and the specific things that Rails changed. But if you follow these steps, you’ll have the best possible upgrade experience.

Are you on Rails 4.2 yet? How did your upgrade go? Did you just have to update a bunch of gems, or are you still chasing down that one last failing test?

(A quick note: I’m going to release Practicing Rails on February 11th! So you only have a few weeks left to get the 25% early access discount.)


3 ways to monkey-patch without making a mess

Monkey Patching. When you first try Ruby, it’s amazing. You can add methods right to core classes! You don’t have to call Time.now.advance(days: -1), you can write 1.day.ago! It makes Ruby a joy to read and write. Until…

You hit weird bugs because a patch changed Hash.

You get confused about which code actually ran, so you can’t debug it when it breaks.

And you finally figure out that all your problems were caused six months ago, when you monkey-patched Enumerable to make one line of code five characters shorter.

But what’s the alternative? No convenience at all? Code that looks like GroupableArray.new([1, 2, 3, 4]).in_groups_of(2)? Waiting for blank? to make it into core Ruby before you’re allowed to have nice user input handling?

You don’t want to give up on monkey patches entirely. But how can you write monkey patches that won’t make you want to fire yourself for incompetence the next time you see them?

Put them in a module

When you monkey patch a class, don’t just reopen the class and shove your patch into it:

1
2
3
4
5
class DateTime
  def weekday?
    !sunday? && !saturday?
  end
end

Why not?

  • If two libraries monkey-patch the same method, you won’t be able to tell.

    The first monkey-patch will get overwritten and disappear forever.

  • If there’s an error, it’ll look like the error happened inside DateTime.

    While technically true, it’s not that helpful.

  • It’s harder to turn off your monkey patches.

    You have to either comment out your entire patch, or skip requiring your monkey patch file if you want to run code without it.

  • If you, say, forgot to require 'date' before running this monkey patch, you’ll accidentally redefine DateTime instead of patching it.

Instead, put your monkey patches in a module:

1
2
3
4
5
6
7
8
9
module CoreExtensions
  module DateTime
    module BusinessDays
      def weekday?
        !sunday? && !saturday?
      end
    end
  end
end

This way, you can organize related monkey patches together. When there’s an error, it’s clear exactly where the problem code came from. And you can include them one group at a time:

1
2
# Actually monkey-patch DateTime
DateTime.include CoreExtensions::DateTime::BusinessDays

If you don’t want the patch anymore, just comment out that line.

Keep them together

When you monkey patch core classes, you add to the core Ruby APIs. Every app with core patches feels a little bit different. So you have to have a way to quickly learn those changes when you jump into a new codebase. You have to know where your monkey patches live.

I mostly follow Rails’ monkey-patching convention. Patches go into lib/core_extensions/class_name/group.rb. So this patch:

1
2
3
4
5
6
7
8
9
module CoreExtensions
  module DateTime
    module BusinessDays
      def weekday?
        !sunday? && !saturday?
      end
    end
  end
end

would go into lib/core_extensions/date_time/business_days.rb.

Any new developer could browse through the Ruby files in lib/core_extensions and learn what you added to Ruby. And they’ll actually use those convenient new methods you wrote, instead of those methods just getting in the way.

Think through the edge cases

I don’t know why Enumerable doesn’t have a sum method. So often, I wish I could write [1, 2, 3].sum, or ["a", "b", "c"].sum, or [Article.new, Article.new, Article.new].sum… Oh.

When you monkey patch a class, you’re usually thinking about one thing you want to make easier. You want to calculate a sum of numbers, but forget that Arrays can hold other things.

Right now, it makes sense. You’d never try to calculate the average of a bunch of hashes. But when you attach methods to an object that just plain fail when you call them sometimes, you’ll confuse yourself later on.

You can deal with this in a few ways. From best to worst:

  • Handle unexpected input reasonably.

    This works well if your patch works with strings. You can get something reasonable from almost anything if you call to_s on it first. And Confident Ruby will teach you a ton of ways to deal with different kinds of input.

  • Handle the error in a clearer way.

    This could be as easy as throwing an ArgumentError with a good message when you see input you’re not expecting. Don’t depend on someone else understanding random NoMethodErrors.

  • Document the kind of input you expect in a comment.

    The other two options are better, if you can use them. But if you can’t check for edge cases inside your patch, at least document them. That way, once your caller figures out it’s your patch that’s causing their problem, they’ll have an idea of what you were trying to do.

My all-time favorite monkey patch

Finally, I want to leave you with my all-time favorite monkey patch: Hash#string_merge:

lib/core_extensions/hash/merging.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
module CoreExtensions
  module Hash
    module Merging
      def string_merge(other_hash, separator = " ")
        merge(other_hash) {|key, old, new| old.to_s + separator + new.to_s}
      end
    end
  end
end

{}.string_merge({:class => "btn"}) # => {:class=>"btn"}

h = {:class => "btn"} # => {:class=>"btn"}
h.string_merge({:class => "btn-primary"}) # => {:class=>"btn btn-primary"}

It makes attaching CSS classes to HTML elements so much nicer.

Sensible monkey patching

Monkey-patching core classes isn’t all bad. When you do it well, it makes your code feel more like Ruby. But just like any of Ruby’s sharp edges, you have to take extra care when you use them.

If you keep your patches together, group them into modules, and handle the unexpected, your monkey patches will be as safe as they can be.

What’s the best monkey patch you’ve ever written (or seen)? Leave a comment and tell me about it!


When edge cases poke holes in your perfect solution

You found the perfect solution to your crazy testing problem. All you have to do is override the DEFAULT_HOST constant, and you’ll be in business.

Except that you have to turn off warnings to get that ugly message to go away. But now all your tests pass, and you only had to change a few lines of code!

Except for that one test where you don’t want to override the host. But you could just re-override the constant, turn off the warnings again, and make sure it gets reset at the end of the test. You’re so close to being done, you can almost taste it!

Except… Except… Except…

And a few days later, when you get stuck for the twenty-seventh time and your app has become one giant ball of hacks, you’ll sit back and wonder: Why am I doing all this? Isn’t the solution worse than the problem?

How to solve the too-clever problem

It’s clear your original idea won’t solve your entire problem. So how do you think up a better idea, that solves all the edge cases your original idea couldn’t?

You can’t. You can’t fight too much cleverness with more cleverness. At least, not directly. Instead, go the other way. Go simple. Go straightforward.

What does that mean?

Inline the code you were trying to abstract away. Do repeat yourself. Keep your code explicit.

If you were trying to override a DEFAULT_HOST constant with a different default host, forget about the whole idea of a default. Just specify it every time.

So, instead of:

test/integration/welcome_test.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
require 'test_helper'

silence_warnings do
  Rack::Test::DEFAULT_HOST = "www.justinweiss.com"
end

class WelcomeTest < ActionDispatch::IntegrationTest
  include Rack::Test::Methods

  test "can visit the homepage" do
    get "/"
    # ...
  end

  # ...
end

Do something like:

test/integration/welcome_test.rb
1
2
3
4
5
6
7
8
9
require 'test_helper'

class WelcomeTest < ActionDispatch::IntegrationTest
  test "can visit the homepage" do
    get "http://www.justinweiss.com/"
    # ...
  end
  # ...
end

Whenever my seemingly perfect solutions break down, it’s because I didn’t imagine the edge cases I’d eventually have to handle.

It’s OK. We can’t predict the future. But when you notice it happening, stop digging. Don’t just apply patch after patch after patch. Instead, unwind your original solution and extract a better one.

How to extract a better solution

When you have all your code written out in an explicit, straightforward way, you’ll start to think of ways to reorganize it.

Usually, it’s enough to apply Extract Method or Extract Class in the right place. The trick is deciding what that right place is. But figuring that out is a lot easier when you see lots of repetition right in front of you.

And lean on inheritance and delegation. Those are the simple building blocks that will help you clean up your code without getting too clever.

One more thing

Don’t forget to read the documentation:

test/integration/welcome_test.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
require 'test_helper'

class WelcomeTest < ActionDispatch::IntegrationTest

  setup do
    # This already exists:
    host! "www.justinweiss.com"
  end

  test "can visit the homepage" do
    get "/"
    # ...
  end
  # ...
end

The answer won’t always be that obvious. But there’s nothing more humbling than realizing there’s a one-method built-in solution to the exact problem you wrote three classes and a gem to solve.

A better solution, in the end

Your second solution will usually be better in every way than your original one.

Why is that?

  • You have more experience as a developer.

    So you’ll have a better idea of what makes good code.

  • You know more about the system you’ve built.

    So you can make better decisions about how the code you’re writing fits into it.

  • You know which of your assumptions were wrong

    So your solutions can better fit the actual problems that exist, not the ones you imagined might exist.

And in the end, there might be a place for the best of your cleverness to fit back in. This time, without the hacks.

You have to stop digging

Clever code is fun to write. In Ruby, it’s also easy to write. And it’s especially easy to keep going down a path, even when you know it’s taking you to the wrong place.

But once you get that nagging sense that something’s not quite right, stop for a minute. Un-factor your code. Read the documentation. Make your code straightforward and explicit. And find a better way.

Can you remember the last time you kept on digging yourself a hole that just wouldn’t end? How did you get yourself out of it? And what did the code you ended up with look like?


Lessons learned in 2014

It’s so easy to get lost in the small bits of daily work you do. When you’re that focused, you can totally miss what you actually accomplished with all those small steps.

So here’s a look back at the writing and speaking I did last year, along with the best things I learned along the way.

What went well

I started to write here on January 10, with Estimates Are Not a Goal, They’re a Communication Tool. In 2014, I posted 53 articles, more than one per week! Starting in April, I’ve posted an article every Tuesday. Creating that kind of schedule makes writing and planning so much easier.

I tried to start writing a few times before, and always gave up for two reasons: I ran out of topics, and it didn’t seem like anyone cared. Two things made it different this time:

  • I started solving problems instead of just writing. I learned this from Amy Hoy and Alex Hillman’s (completely awesome) 30x500 course: If you focus on helping people with the problems they have, you’ll never run out of things to write about.

  • I started telling people when I posted. I’ve always been nervous about sharing my own stuff. But when you just start out, how are people going to know about what you write?

    If you’re posting helpful things and participating in the discussion around it, people are happy to learn from you. So tell them about it!

These were my most popular articles from last year:

Take a look if you missed them the first time around!

A few articles in, I started an email list. Through the end of 2014, it’s grown to 1,670 subscribers! I send emails to the list every Friday, with 41 sent so far. They range from deeper looks on some of the articles I posted, to answering questions, to exclusive articles about the problems I hear about from my subscribers. If you haven’t joined already, sign up here. I’d love to hear from you.

One problem with the email list is that once you miss an email, you miss it. So this year, I’m going to find a way to get the best emails you’ve missed to you.

I also wrote a book about learning Rails without getting overwhelmed, and started to pre-sell it. You can get early access here. It’s 25% off until the final release, and you’ll get the final update once it ships. More than 300 people have received early access, and it’s already helped a lot of them get past the tutorial stage so they could start building their own Rails apps.

Finally, here are a few other things I did in 2014:

What didn’t go so well

I create schedules and due dates for myself that are, in retrospect, totally insane.

Once it’s clear that you’re not going to hit a goal, you start beating yourself up. You start thinking, maybe if you worked a little harder, maybe if you stayed up a little later, maybe if you were just plain better at what you were doing, you wouldn’t be in the mess you’re in. But it’s a self-imposed mess! It makes no sense.

With aggressive goals, I’ve done a lot of stuff. But too-aggressive goals and too-high expectations have hurt more than they’ve helped. I have to find more of a balance.

Lessons learned

The best things I’ve learned this year about creating things:

  • Schedules and habits are better than motivation.

    I credit all of the work I’ve done here to setting up the right habits and schedules.

    You still need motivation to get those habits started. But that motivation is better spent setting up good systems and habits, instead of spending the motivation on the work itself. Because motivation eventually fades, but a good habit sticks around for a lot longer.

  • Rough first drafts make the creation process faster.

    The first drafts of my articles are unreadable. But rough drafts are easy to improve. I don’t have to keep what I’m thinking about in my head, I can put something down and improve it piece by piece.

  • Sometimes you have to close your eyes and hit submit.

    Some of the articles I’m most proud of are articles I came close to not posting. It’s nerve-wracking to put something you created out in public, especially as you’re learning. When that happens, you just have to git push and walk away from the computer for a little while. What you made is never as bad as it seems right before you publish.

What about you? What were your biggest accomplishments last year? What did you learn? And what are your plans for 2015?


How do you take an app from no tests to TDD?

You’re excited about the app you’ve made. There’s just one problem – you don’t have any tests. You wanted to write it using Test-Driven Development, but you didn’t quite know where to start. So you’re stuck. Where do you go from here? How do you get from an app with no tests, to writing your apps using TDD?

Test the code you already have

You have a bunch of code with no tests. But that doesn’t mean you can’t write your tests now, against the existing code. So start testing the code you already have, to make sure your code works the way you expect it to work.

It’s not TDD. But testing existing code will help you learn TDD:

  • You practice thinking about edge cases and error conditions.

    To write tests without spending years testing every single possible input, you have to think about where the code is most likely to break. If the method you’re testing takes a string, what happens if you pass it a symbol? What happens if you pass it nil? And if you’re testing a function that divides numbers, you’d better test it with 0. But you probably don’t need to test it with 1 and 2.

    After you write enough tests, you’ll start to predict where your methods are most likely to break. And once you start TDDing, you can use this skill to write robust tests that force your code to better handle your edge cases.

  • You practice writing well-structured tests.

    When you write tests after-the-fact, you can try different patterns for structuring those tests. The code you’re testing is already there, so you can focus on your test, and how it’s written. And once you learn some good patterns, you’ll write better tests when you don’t have the code to lean on.

  • You discover the things that make code hard to test.

    As you write more tests, you’ll begin to sense which areas of your system will be the hardest to test. When you notice those areas, you can highlight them as places that need refactoring. Even better, you’ll start to write more testable code the first time.

    Once you know what easy-to-test code looks like, you can TDD easy-to-test APIs with that knowledge. And that will help you build your apps faster.

Ease into TDD

You can use test-after to build skills that help you learn TDD. But you still want to TDD pieces of your app eventually. And there’s a simple way to ease into TDD, and still lean on existing code: write regression tests.

Regression tests keep you from breaking code you’ve already fixed. The idea is pretty simple. Whenever you hear about a bug, instead of clicking around in the browser to reproduce it:

  • Write a failing test to reproduce the bug.
  • Run the tests, and make sure they fail (because that bug still exists).
  • Fix the bug in the simplest way possible.
  • Run the tests, and make sure they pass.
  • Refactor your fix, if necessary.

This is a lot easier than TDDing a new system from scratch, because you’re just test-driving a change to code that’s already there. You build the habit of “Red, Green, Refactor” that is the essential loop of TDD. And from here, TDD is a shorter step away than trying to go straight to TDD from no tests.

From nothing to TDD

An app without tests isn’t a bad place to start. When you test existing code, you’ll learn a lot of what you need to write good TDD tests. Test-after is easier than TDD at first, because you don’t have to imagine APIs that you don’t know how to design yet. And once you decide to bring TDD into your app, you can ease into it with regression tests.

So, if you don’t know how to TDD the system you’re imagining, keep writing tests. Even if you have to write the code first.


How to choose from conflicting Rails advice

The Rails community is full of conflicting advice. Do you use Minitest or RSpec? Do your plain Ruby objects go in app/models, app/classes, or lib/?

If more experienced devs constantly argue about these things, what hope do you have of deciding which advice to follow?

What’s the secret?

Here’s the secret: just pick one side, and follow it.

It seems like a cop-out. But these are arguments over small differences in philosophy. The stronger the argument, the less important the choice actually is. Especially in small apps, like the ones you’ll start with.

Experienced devs got that experience somehow. They learned, they wrote code, they shipped apps. Whichever choice they made didn’t sabotage their progress. So following that choice won’t sabotage yours.

I’m a big fan of Minitest, and don’t use RSpec when I can avoid it. I’ve written a ton about why I prefer Minitest over RSpec. But if you decide that RSpec is the framework you should be using, you’re not going to end up with an unusable app. If you end up switching to Minitest later, you won’t be starting all over again. Most of the testing skills you learned will carry right over. So just pick one. Flip a coin if you have to. Because either choice is better than none.

It’s easier to change than you think

The right choice for you depends on your personality, your environment, your team, and your past experiences. These things will change over the course of your career. So if you spent days learning all the background and the nuances of each side and finally made the perfect choice, that choice may eventually change.

You’ll make some bad choices. Some of the best developers I know wrote entire gems based on philosophies that they now argue against. In code, few opinions last forever.

So don’t agonize over picking the right side in a Rails argument that you don’t totally understand yet. If it doesn’t work out, you can always change your mind.


Where does your code go?

After you finish the Rails tutorials and start your own app, things get confusing. Like where does your non-CRUD, general logic go? How does grabbing followers from Twitter fit into MVC? Ask two people, and you get four answers. Or your thread devolves into a bunch of smart people insulting each other for hours, while you beat your head against the desk. Either way, you’re left with a headache.

You can’t build the app you’ve been dreaming of without some general, non-Rails logic. So where do you put your code, and still keep things simple?

The easy place to start

When I have logic that feels related to an existing ActiveRecord model, I’ll start by putting it into that model. For example, if I had a Game model and I wanted to import a bunch of games from CSV files, I’d put that method right onto the Game class:

app/models/game.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
class Game < ActiveRecord::Base
  def self.parse_from_csv(csv_string)
    games = []
    CSV.parse(csv_string, quote_char: "'") do |row|
      games << Game.from_csv_row(row) if (row[0] == 'G')
    end
    games
  end

  def self.from_csv_row(row)
    Game.new({
      dgs_game_id: row[1],
      opponent_name: row[2],
      created_at: row[4],
      updated_at: row[4],
    })
  end
end

You have all the information your methods need right at hand. It’s easily testable. And it’s probably where a new contributor would look for that logic first.

But if you keep adding and changing that model, it’ll get big and complicated. Different parts of the model will interact in strange ways. The more you change it, the harder it will be to change.

In that case, you’d probably want to refactor that code out to a non-ActiveRecord model.

Non-ActiveRecord models

Just because it’s in a Rails app, doesn’t mean it has to inherit from Active/Action-whatever.

You can write your own Ruby code, in plain Ruby objects, and use them in your Rails app. These objects can still be called models, because they’re modeling part of your problem. They just don’t have an ActiveRecord database storing their data.

The next time I worked on that game CSV parser, the Game class was getting a little too big. So I moved the parser logic into its own GameCSVParser class.

The whole commit is here, but this is what the non-ActiveRecord class looks like:

app/models/game_csv_parser.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
class GameCSVParser

  def initialize(csv_string)
    @rows = CSV.parse(csv_string, quote_char: "'")
  end

  def games
    game_rows.map { |row| Game.new(game_attributes(row)) }
  end

  private

  def game_rows
    @rows.select { |row| is_game_row?(row) }
  end

  def game_attributes(row)
    {
      dgs_game_id: row[1],
      opponent_name: row[2],
      created_at: row[4],
      updated_at: row[4],
    }
  end

  def is_game_row?(row)
    row[0] == 'G'
  end
end

I’ll go right to creating a new plain Ruby object if the logic I’m adding doesn’t feel related to any specific ActiveRecord model. Or if the code seems like it should be a part of a thing that doesn’t exist yet in the app. Otherwise, they mostly pop up through refactoring.

With plain Ruby objects, you can write anything. But knowing you can write anything doesn’t help you with direction. What methods do you need? How will all your new objects interact?

A lot of Rails apps use the same categories of plain Ruby objects. These categories are patterns you can follow to write code that other developers would recognize. You might have heard of a few of them already.

Service objects, presenters, and jobs

There’s nothing special about service objects, presenters, and jobs. They’re just plain Ruby objects that act in a particular recognizable way.

For example, a Resque job is a plain Ruby class that has a perform method and a @queue:

app/workers/fetch_games_for_player.rb
1
2
3
4
5
6
7
8
class FetchGamesForPlayer
  @queue = :default

  def self.perform(player_id)
    player = Player.scoped_by_id(player_id).ready_for_fetching.first
    player && player.fetch_new_games!
  end
end

perform is called when the job is run.

A presenter is a plain Ruby object with code that only makes sense inside a view:

app/presenters/user_presenter.rb
1
2
3
4
5
class UserPresenter
  def show_related_users?
    @user.related.count > 3
  end
end

It might also include Rails’ view helpers, or take a few different objects and treat them as one unified object for the view’s convenience.

A service object is a plain Ruby object that represents a process you want to perform. For example, writing a comment on a post could:

  1. Leave the comment.
  2. Send a notification mail to the author of the post.

A service object could do both, and keep that logic out of your controller.

There’s a great take on service objects here. It’s full of examples.

For simple processes, I don’t bother with service objects. But if the controller starts getting too heavy, they’re a good place to put that extra logic.

You can use these patterns to organize your own business logic. They’re just plain Ruby objects, but they’re Ruby objects that share a certain flavor, that have a name, and that you can talk to other developers about.

Where do you start?

There are a lot of different places your non-Rails business logic could go. It might be hard to choose. So here’s what I do:

  1. If the logic is mostly related to an existing class, even if it’s an ActiveRecord model, I put it in that class.
  2. If it doesn’t fit an existing class, I create a new plain Ruby class to hold the logic.
  3. If it feels like the logic should be part of something that doesn’t exist yet, I create a new plain Ruby class for it.
  4. If I come back to the code later, and the model is getting too complicated, or the code doesn’t make sense in that model anymore, I’ll refactor it into its own plain Ruby class.
  5. If the code only makes sense in a view, I’ll add it to a helper or create a presenter.
  6. If the code doesn’t need to run during an HTTP request, or has to run in the background, it goes in a job.
  7. If I’m juggling several different models or stages of a process, and it’s making the controller too hard to understand, I’ll put it into a service object.

How about you? Where does your code go? And do you have any patterns besides these that you’ve found helpful? Leave a comment and let me know.

And if you don’t have a process yet, try mine. See how it fits you. There’s no perfect way to write code, but when you get stuck, a process like this will help you get started.


How to keep bad abstractions from handcuffing your code

(This post is based off of one I sent to my list a few months ago. To read more like it every week, join up here!)

In Using a Little Bit of Convenience to Save Your Programming Flow, I talked about how the simplest abstractions can make your codebase more fun to work with.

But what about when you have a bad abstraction?

You dig out the API reference every time to understand what that method that clearly should have been named leaderboard is actually called. You try to refactor around it, but one part of the code hands you structured data and the other part gives you raw lines of text. And every time you touch the code, something breaks, and you blow the rest of your day trying to debug it. Because you tweaked position, but didn’t update score at the same time.

So how do you build good abstractions, while staying far away from the bad?

Using good metaphors to write clearer code

Good abstractions come from good metaphors.

A metaphor describes something in your system by relating it to a concept that’s already in your head. For example:

  • Many command lines use the metaphor of a pipe.

    You put data in one end of the pipe, and it comes out the other end. Easy to imagine and easy to use.

  • In concurrent programming, a lock is a good metaphor.

    Without knowing anything about how locks work internally, you know that when a resource is locked up, you can’t use it. You have to wait until it’s unlocked.

  • In Object-Oriented Programming, message passing triggers a specific mental image: An object passes another object a message. The second object can do something with the message or pass it to someone else.

    You don’t need to know anything about the implementation in order to get the idea (and it makes things like the proxy pattern much easier to understand).

A good metaphor is critical. If you see an abstraction based on a good metaphor, you’ll rarely have to read the documentation or go source diving to understand the code. The better the metaphors you come up with, the easier it will be to describe your system to someone else, and the easier it’ll be to work on your system later on.

How do you come up with good metaphors?

A good metaphor is specific, self-contained, and creates a vivid mental image of the thing it describes. It doesn’t leave room for misunderstanding, and it matches the code’s behavior as closely as possible. If someone makes assumptions about the code based on their understanding of the metaphor, those assumptions should be valid.

Coming up with good metaphors is hard. You’ll often go through a few leaky metaphors before you can find the right one, and that’s OK! It will get easier with practice. But here’s a process that’s helped me:

Think about what a bit of code does, without thinking about how it does it. What image comes up in your mind? How does other code use it? If you had to describe it in a word or two, which words would you use? Write that down.

Then, think about that image. Does it fit how that code works? Are there edge cases that leak through the metaphor? Is there anything besides the metaphor a new contributor would have to understand before they could use the code? When you think about your metaphor, do you get a clear picture in your mind, or is it fuzzy?

These are indicators that your metaphor doesn’t match your code closely enough. But the closer you are to a good metaphor, the less of a leap you’ll have to make to get there.

Finally, use it in a sentence. “This code is a (something), because…” Could you explain your metaphor to someone without a technical background, and have them understand what it does? The less background someone needs to understand the metaphor, the more helpful your metaphor will be.

If you’re close, but not quite there, take a look in a thesaurus. Subtle differences between similar words can mean the difference between an OK metaphor and a perfect one.

Metaphors and OOP

Object-Oriented Programming is fundamentally based on metaphor. That’s where it gets its power! Well-designed classes don’t just hide their implementations from other classes, they hide their implementations from you.

With a great metaphor, you should be able to tell just from the class name how to use your objects. If your classes are named well, and they cause the right images to appear in your mind, you shouldn’t have to dive into their implementation to understand them.

So now, I’d like you to think about the classes in the last code you worked with. Can you think of any that have particularly clear metaphors? Which classes do you feel like you work with, rather than work against? How often do you go digging into them to figure out how to use them?

I’d love to hear what you find out. Leave a comment, and let me know.

To become a great software developer quickly, you have to consciously think about the code you’re writing and the processes you’re following. And I can help you.

Download my free 32-page sample of Practicing Rails, and learn how to try ideas like this on a small scale, without getting overwhelmed.


Better globals with a tiny ActiveSupport module

Generally, global variables are bad. But sometimes, a global in the right place can make your code a lot simpler.

In Rails, you might set data once during a request, but use it in every layer of your app. Which user is making the request? What permissions do they have? Which database connection should be used?

This information should be available throughout your code, so passing it everywhere would just be a bunch of noise. But using a Ruby global variable, or setting a class variable, will cause its own problems. Multiple threads could overwrite it, and you’d end up with a huge mess, and maybe unrecoverably bad data.

You need something that’s global, but for just your request.

The plain Ruby way

Usually, you’ll see this handled with Thread.current[]. It looks like this:

1
Thread.current[:current_user] = user

This is simple enough. It’s a decent solution. But it has two big drawbacks:

  1. Someone could accidentally overwrite your data.

    What if two people picked the same key? You’d stomp on each other’s data. And if you’re storing something like users, that would be seriously bad. In your own apps, this probably won’t be a problem. But if you’re writing gems, or your Rails app is big and messy, it’s something you need to think about.

  2. It’s not well-structured.

    Thread.current[] is just a big bag of data. You can’t easily document it. If you want to know what you can take out of it, you have to search for what you put into it.

    Of course, if you’re putting away enough global data that this is a problem, you have more to worry about than Thread.current[]’s lack of structure. But it’s still a point to keep in mind.

So, is there a better solution than Thread.current[]? As you might imagine, Rails itself manages a lot of request-local data. And the box of goodies that is ActiveSupport has a different pattern to follow.

The Rails way

If you dig through ActiveSupport, you might run into ActiveSupport::PerThreadRegistry. You can probably tell from the name that this is exactly what we’re looking for.

With ActiveSupport::PerThreadRegistry, the earlier example would look like this:

1
2
3
4
5
6
7
8
class RequestRegistry
  extend ActiveSupport::PerThreadRegistry

  attr_accessor :current_user, :current_permissions
end

RequestRegistry.current_user = user
RequestRegistry.current_user # => user

It’s a tiny bit more work. But with ActiveSupport::PerThreadRegistry:

  • You have a place to put documentation. Anyone looking at your class will know exactly what global data you’re expecting, and what global data they can use.

  • You get an API that looks good. In fact, it looks like you’re just setting class variables. That’s probably what you’d be doing if you didn’t have to worry about threads.

  • Any data you set on the class will be namespaced. You don’t have to worry about RequestRegistry.current_user colliding with PhoneNumberApiRegistry.current_user, they’re treated totally separately.

Rails uses ActiveSupport::PerThreadRegistry internally, for things like ActiveRecord connection handling, storing EXPLAIN queries, and ActiveSupport::Notifications. So if you’re looking for more good examples of the power of PerThreadRegistry, Rails itself is a great place to start.

When threads and requests don’t match

For some Rails servers, a single thread handles multiple requests. This means anything you put into a PerThreadRegistry will stick around for the next request. This is probably not what you want.

You could do what Rails does, and clean up the things you put into the PerThreadRegistry after you’re done with them. Depending on what you put in there, you might be doing this anyway.

In the comments, MattB points to an easier solution: request_store. It acts like Thread.current[], but clears itself after each request.

This leaves the door wide open for someone to create a PerRequestRegistry, which combines the safety of request_store with the API of PerThreadRegistry. If someone’s done this already, I’d love to hear about it!

Try it out

Global data can be bad. Too many globals can quickly lead to unmaintainable code.

But if it makes sense to keep data in a central place for an entire request, go beyond Thread.current[]. Give ActiveSupport::PerThreadRegistry or request_store a shot. If nothing else, it’ll make dealing with global data a little less risky.


Each… or else

I love each, but I have a problem with it. What happens when you have an empty collection?

If you call [].each, nothing will happen and [] will be returned. Sometimes, that’s what you want. But more often, especially when you’re building UI, you’ll want to handle empty lists in a special way. For example, I usually want to show a different message when I don’t have any data.

But since [].each returns [], not nil, you’re stuck writing something like this:

app/views/users/index.html.erb
1
2
3
4
5
6
7
<% if @users.empty? -%>
  <p>You don't have any users. <%= link_to "Add one!", new_user_path %></p>
<% else -%>
  <% @users.each do |user| -%>
      <p><%= user.name %></p>
  <% end -%>
<% end -%>

That works, but it feels awkward. I’d rather say, “Do this thing to each item in this collection, unless there’s nothing in it, in which case do this other thing entirely.” I want something like each, with an “or else.”

Rendering Rails Collections

Inside your views, Rails can help. It’s great at rendering collections of things:

app/views/users/index.html.erb
1
<%= render @users %>
app/views/users/_user.html.erb
1
<p><%= user.name %></p>

When render is passed @users, it renders _user.html.erb once for each user inside @users. You can skip the each entirely.

As a bonus, if @users is empty, render returns nil, just like we want! So you can write this, and get the same output as the original version:

app/views/users/index.html.erb
1
<%= render(@users) || render('empty') %>
app/views/users/_user.html.erb
1
<p><%= user.name %></p>
app/views/users/_empty.html.erb
1
<p>You don't have any users. <%= link_to "Add one!", new_user_path %></p>

It’s a lot more direct, once you understand Rails’ conventions.

What about outside of a view?

If you follow Rails, render with a collection is a fast, powerful way to render collections of objects.

But sometimes you won’t want to deal with an extra partial to render each item. Or render won’t support your design. Or you won’t even be inside a view.

The best solution I’ve found so far is presence.

list.presence is the same as:

1
2
3
4
5
if list.present?
  list
else
  nil
end

That is, you’ll get the list back if it has anything in it, and nil if it’s empty.

With presence, you could write:

1
2
3
@users.each do |user|
  puts user.name
end.presence || puts("You don't have any users.")

This prints each name if there are users in @users, or You don't have any users otherwise.

Still, I could do without the presence. It feels like a hack, because it is one. If it was supported, something like this might be better:

1
2
3
@users.each do |user|
  puts user.name
end || puts("You don't have any users.")

or the Smalltalk-ish:

1
2
3
@users.each(if_empty: lambda { puts "You don't have any users." }) do |user|
  puts user.name
end

or even (gasp!):

1
2
3
4
5
for user in @users
  puts user.name
else
  puts "You don't have any users."
end

For now, though, I usually just go with presence or the basic if blank?; else each pattern.

Which do you think is better? How do you handle empty lists? Have you found a better way? If so, tell me about it!


What’s the difference between `irb`, `bundle exec irb`, `bundle console`, and `rails console`?

There are a lot of ways to launch a Ruby or Rails console: irb, bundle exec irb, bundle console, and rails console are some of the most common. They seem similar, but they each work a little bit differently.

If you don’t know what those differences are, you’ll have some problems. Maybe you won’t be able to get your ActiveRecord models connected to the database. Or you’ll require a file and get the wrong version. Or a library you thought would be avaliable, wasn’t.

How do you make sure you’re using the right console at the right time?

Bundler vs non-Bundler

irb is just a plain Ruby console. It doesn’t care about your Gemfile. It doesn’t load anything but the core Ruby libraries. Anything else you want, you have to require.

If you install a gem using gem install, you can require it inside irb. If you used bundle install, you might be able to require it, depending on where Bundler put it. (Bundler will sometimes put gems outside of Ruby’s gem path, if you run something like bundle install --path or bundle install --deployment).

Since irb ignores your Gemfile, the versions inside your Gemfile.lock don’t matter. irb will load the newest version of a gem it can find:

1
2
3
4
5
6
7
8
9
10
11
12
13
~/Source/testapps/consoles[master *] jweiss$ gem list rails

*** LOCAL GEMS ***
rails (4.2.0.beta2, 4.2.0.beta1, 4.1.5, 4.1.1)

~/Source/testapps/consoles jweiss$ cat Gemfile | grep rails
gem 'rails', '4.1.5'

~/Source/testapps/consoles jweiss$ irb
irb(main):001:0> require 'rails'
=> true
irb(main):002:0> Rails.version
=> "4.2.0.beta2"

This can cause really weird problems with your code, if you’re not expecting it.

irb is great if you’re messing with core Ruby files. It’s fast, and doesn’t need any setup.

But if you want to use your Gemfile when you run a console, run bundle exec irb instead. The bundle exec allows irb to load the gems that Bundler knows about, and only the gems Bundler knows about:

1
2
3
4
5
~/Source/testapps/consoles jweiss$ bundle exec irb
irb(main):001:0> require 'rails'
=> true
irb(main):002:0> Rails.version
=> "4.1.5"

We got exactly the Rails version we were looking for.

bundler/setup vs Bundler.require

When would you run bundle console instead of bundle exec irb?

bundle exec irb sets things up so you can only require the gems in your Gemfile.lock.

bundle console goes one step further. When you run bundle console, you don’t even need to require the gems in your Gemfile. They’re already required for you:

1
2
3
4
5
6
7
8
9
~/Source/testapps/consoles jweiss$ bundle exec irb
irb(main):001:0> Rails.version
NameError: uninitialized constant Rails
  from (irb):1
  from /usr/local/bin/irb:11:in `<main>'

~/Source/testapps/consoles jweiss$ bundle console
irb(main):001:0> Rails.version
=> "4.1.5"

You could also get this behavior if you called Bundler.require inside your bundle exec irb console. Any gem in your Gemfile that isn’t marked require: false will be automatically required, and you’ll be able to use it without any extra work. When you’re working on projects with a Gemfile, that’s incredibly convenient.

Accessing Rails

There’s still one difference to think about: bundle console and rails console.

1
2
3
4
5
6
7
8
~/Source/testapps/consoles jweiss$ bundle console
irb(main):001:0> Rails.application
=> nil

~/Source/testapps/consoles jweiss$ rails console
Loading development environment (Rails 4.1.5)
irb(main):001:0> Rails.application
=> #<Consoles::Application:0x007f8db4d5ab30 @_all_autoload_paths=["/Users/jweiss...

bundle console just requires a bunch of gems. rails console requires those gems, but it will also load your entire Rails environment, set up autoloading, initialize your application, and give you a full Rails environment to play around in.

You can get something like the Rails console from bundle console if you require config/environment.rb:

1
2
3
4
5
6
7
~/Source/testapps/consoles jweiss$ bundle console
irb(main):001:0> Rails.application
=> nil
irb(main):002:0> require_relative 'config/environment.rb'
=> true
irb(main):003:0> Rails.application
=> #<Consoles::Application:0x007fd264f0b7c8 @_all_autoload_paths=["/Users/jweiss...

Each one, just a little more complicated

So, to recap:

  • irb is the basic Ruby console. It ignores your Gemfile, and only core Ruby classes are accessible without require-ing them. It can’t easily load gems that Bundler installs outside of RubyGems’ load path.

  • bundle exec irb is like irb, if you also required bundler/setup. You can only easily require gems that are in your Gemfile.lock, but you can load those gems no matter where Bundler put them.

  • bundle console is like bundle exec irb, if you also called Bundler.require. All of the gems in your Gemfile, except the ones marked require: false, can be used without requiring them. It’s really convenient when you’re writing your own gems, or working on non-Rails code.

  • rails console is like running bundle console inside a Rails app, if you also required config/environment.rb. You can play with your entire Rails app, autoloads and database connections work, and everything’s hooked up the way you’d expect. If you’re working in a Rails app, this is the most helpful kind of console.

There aren’t too many differences between these consoles. And while most of those differences won’t be too big (Oh, this file isn’t required? Better require it!), others will be totally infuriating if you don’t know what’s going on. (WHY IS THIS LOADING THE WRONG VERSION OF RAKE AGAIN!?)

But if you know the idea behind each of these consoles, you’ll be able to use the right kind of console at the right time. And all the libraries you need will be close by when you need them.


Fun with the `method` method

Have you ever wanted to pass a method to a function that only takes a block? Or figure out which of your object’s superclasses broke the method you’re trying to call?

Those things are easy to do with the method method. Use it well, and you can learn about your dependencies, save yourself hours of debugging, and get your code to the places it needs to be.

Methods as easy as lambdas

Lots of Ruby methods take blocks or lambdas. But you can’t directly pass a method to another method, the way you would with a lambda. You have to use method first:

1
2
3
4
irb(main):001:0> sin_method = Math.method(:sin)
=> #<Method: Math.sin>
irb(main):002:0> (1..10).map(&sin_method)
=> [0.8414709848078965, 0.9092974268256817, 0.1411200080598672, -0.7568024953079282, -0.9589242746631385, -0.27941549819892586, 0.6569865987187891, 0.9893582466233818, 0.4121184852417566, -0.5440211108893699]

So, what’s happening here?

The first line turns the method Math.sin into a Method object. Just like lambdas, Method objects respond to call. Method objects respond to to_proc (Thanks, Benoit). So they can be used in the same places you’d use a lambda:

1
2
3
4
irb(main):004:0> sin_method = -> (x) { Math.sin(x) }
=> #<Proc:0x007fe9f90a9bd8@(irb):4 (lambda)>
irb(main):005:0> (1..10).map(&sin_method)
=> [0.8414709848078965, 0.9092974268256817, 0.1411200080598672, -0.7568024953079282, -0.9589242746631385, -0.27941549819892586, 0.6569865987187891, 0.9893582466233818, 0.4121184852417566, -0.5440211108893699]

Take a look at the first line again. This code, using a lambda, works the same way as the earlier code, using method.

Depending on how you’ve written your code, getting ahold of a Method object can be a lot easier than wrapping it in a lambda. So when you need a lambda and all you have is a method, remember the method method.

Where did that method come from?

You just called a method on your object, and it does something you didn’t expect. Maybe someone overrode it or monkey-patched it. How do you figure this out?

You could dig through the source code for a few hours. But Method has two methods that can speed things up.

To figure out which class defined the method you’re calling, use owner:

1
2
irb(main):003:0> Task.new.method(:valid?).owner
=> ActiveRecord::Validations

When you want to go a little deeper and figure out exactly where the method was defined, use source_location:

1
2
irb(main):004:0> Task.new.method(:valid?).source_location
=> ["/usr/local/lib/ruby/gems/2.1.0/gems/activerecord-4.2.0.beta2/lib/active_record/validations.rb", 55]

source_location returns an array. The first element is the path to the file where the method was defined, and the second element is the line number. With those, you know exactly where to look next.

Reading the source (without having to dig)

Once you can figure out where a method was defined, you might want to see how it’s defined.

Method can’t do that by itself. But if you install the method_source gem, you can see the source code for many methods right from your console:

1
2
3
4
5
6
7
irb(main):002:0> puts Task.new.method(:valid?).source
    def valid?(context = nil)
      context ||= (new_record? ? :create : :update)
      output = super(context)
      errors.empty? && output
    end
=> nil

You can even see the comments:

1
2
3
4
5
6
7
8
9
10
11
12
irb(main):003:0> puts Task.new.method(:valid?).comment
# Runs all the validations within the specified context. Returns +true+ if
# no errors are found, +false+ otherwise.
#
# Aliased as validate.
#
# If the argument is +false+ (default is +nil+), the context is set to <tt>:create</tt> if
# <tt>new_record?</tt> is +true+, and to <tt>:update</tt> if it is not.
#
# Validations with no <tt>:on</tt> option will run no matter the context. Validations with
# some <tt>:on</tt> option will only run in the specified context.
=> nil

Pretty awesome, right? Your documentation is right in the console!

Introspection is awesome

Some of my favorite Ruby classes are those that let you inspect your code from within your code. Classes like Class, Module, and Method. With these, you can learn a ton about your code as it runs, and modify it on the fly. Learn these classes well, study their API, and you’ll be able to do amazing things with Ruby.


`respond_to` without all the pain

When you generate a scaffold in Rails, you’ll see the usual respond_to blocks:

app/controllers/tasks_controller.rb
1
2
3
4
5
6
7
  def destroy
    @task.destroy
    respond_to do |format|
      format.html { redirect_to tasks_url, notice: 'Task was successfully destroyed.' }
      format.json { head :no_content }
    end
  end

But some of your actions, like index, don’t have them!

app/controllers/tasks_controller.rb
1
2
3
4
5
  # GET /tasks
  # GET /tasks.json
  def index
    @tasks = Task.all
  end

This is bad. Why? If you hit /tasks.txt, and txt isn’t supported by your app, you’ll get the wrong error:

1
ActionView::MissingTemplate (Missing template tasks/index, application/index with {:locale=>[:en], :formats=>[:text], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :coffee, :jbuilder]}

This isn’t quite right. You should be telling the client that they’re requesting a format you don’t support, not that you can’t find the right file.

If this was a UnknownFormat error, you could return a better response code. Instead, these errors will get mixed together with other, unrelated errors, and it’ll be really hard to handle them.

You could add a respond_to block to your index action:

app/controllers/tasks_controller.rb
1
2
3
4
5
6
7
8
9
  # GET /tasks
  # GET /tasks.json
  def index
    @tasks = Task.all
    respond_to do |format|
      format.html
      format.json
    end
  end

Then, you’d get the exception and error code you’d expect:

1
2
3
4
5
6
Started GET "/tasks.txt" for 127.0.0.1 at 2014-11-03 22:05:12 -0800
Processing by TasksController#index as TEXT
Completed 406 Not Acceptable in 21ms

ActionController::UnknownFormat (ActionController::UnknownFormat):
  app/controllers/tasks_controller.rb:8:in `index'

Much better. But littering all your controllers with respond_to is crazy. It feels un-Rails-ish. It violates DRY. And it distracts you from the work your controller is actually doing.

You still want to handle bad formats correctly. So what do you do?

A respond_to shortcut

If you’re not doing anything special to render your objects, you can take a shortcut. If you write:

app/controllers/tasks_controller.rb
1
2
3
4
def index
  @tasks = Task.all
  respond_to :html, :json
end

it works the same way as writing the full respond_to block in index. It’s a short way to tell Rails about all the formats your action knows about. And if different actions support different formats, this is a good way to handle those differences without much code.

Handle formats at the controller level

Usually, though, each action in your controller will work with the same formats. If index responds to json, so will new, and create, and everything else. So it’d be nice if you could have a respond_to that would affect the entire controller:

app/controllers/tasks_controller.rb
1
2
3
4
5
6
7
8
9
10
class TasksController < ApplicationController
  before_action :set_task, only: [:show, :edit, :update, :destroy]
  respond_to :html, :json

  # GET /tasks
  # GET /tasks.json
  def index
    @tasks = Task.all
    respond_with(@tasks)
  end

And this actually works:

app/controllers/tasks_controller.rb
1
2
3
4
5
6
Started GET "/tasks.txt" for 127.0.0.1 at 2014-11-03 22:17:37 -0800
Processing by TasksController#index as TEXT
Completed 406 Not Acceptable in 7ms

ActionController::UnknownFormat (ActionController::UnknownFormat):
  app/controllers/tasks_controller.rb:8:in `index'

Exactly the kind of error we were hoping to get! And you didn’t have to mess with each action to do it.

Sometimes you’ll want to do different things depending on the state of a model. For instance, for create, you’d either redirect or re-render the form, depending on whether or not the model is valid.

Rails can handle this. But you still have to tell it which object you want it to check, with respond_with. So instead of:

app/controllers/tasks_controller.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
  def create
    @task = Task.new(task_params)

    respond_to do |format|
      if @task.save
        format.html { redirect_to @task, notice: 'Task was successfully created.' }
        format.json { render :show, status: :created, location: @task }
      else
        format.html { render :new }
        format.json { render json: @task.errors, status: :unprocessable_entity }
      end
    end
  end

you can write:

app/controllers/tasks_controller.rb
1
2
3
4
5
  def create
    @task = Task.new(task_params)
    flash[:notice] = "Task was successfully created." if @task.save
    respond_with(@task)
  end

This way, you separate your code from the formats you respond to. You can tell Rails once which formats you want to handle. You don’t have to repeat them in every action.

The responders gem

In Rails 4.2, there’s a catch: respond_with is no longer included. But you can get it back if you install the responders gem. And the responders gem brings some other nice features with it.

You can set flash messages in respond_with by including responders :flash at the top of your controller:

app/controllers/tasks_controller.rb
1
2
class TasksController < ApplicationController
  responders :flash

Conveniently, you can set defaults for these flash messages in your locale files.

Also, if you have the responders gem in your Gemfile and you generate a Rails scaffold, the generator will create controllers using respond_with instead of respond_to:

app/controllers/tasks_controller.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
class TasksController < ApplicationController
  before_action :set_task, only: [:show, :edit, :update, :destroy]
  respond_to :html, :json

  def index
    @tasks = Task.all
    respond_with(@tasks)
  end

  def show
    respond_with(@task)
  end

  # ...

This is a lot cleaner than the scaffolds Rails comes with.

respond_with or respond_to?

If you want to return different information for different formats, you have a few options. The controller-level respond_to combined with respond_with is a great way to get short controllers. But it tends to help the most when all of your controller actions respond to the same format, and act in the way Rails expects them to.

Sometimes, though, you want to be able to have a few actions that act differently. The one-liner respond_to is great for handling that situation.

If you need more control, use the full respond_to with a block, and you can handle each format however you want.

With any of these, requests for formats you don’t support will get the right error. And both your app and its clients will be a lot less confused.


How to go beyond documentation to learn a new library

You’re working with a brand new library or feature, and you just can’t understand the documentation. Maybe it’s hard to get set up, so you have to bounce between RDocs until you learn how everything fits together. Maybe the documentation doesn’t even exist. Or maybe you just learn best by seeing lots of examples.

You need some help. But where do you learn how to use a feature, if not its documentation?

One of my favorite GitHub features

When you want to go beyond the documentation and see real-world uses of a library or feature, use GitHub’s code search.

For example, say I just read the Rails guide on Conditional GET Support. I feel like I understand it, but I want to see some more examples before I use it for real.

I could hop onto GitHub search, and look for "if stale?". Then, in the sidebar, pick code and Ruby:

And now I have tons of great examples of how stale? could be used.

Not all of the examples are interesting. But in one listing, I notice the author using Post.maximum:

app/controllers/archives_controller.rb
1
2
3
4
5
6
7
class ArchivesController < ApplicationController
  def index
    if stale?(last_modified: (Post.maximum(:updated_at) || EPOCH).utc, public: true)
      @months = Post.find_all_grouped_by_month
    end
  end
end

That seems like a great way to use stale?. And if I hadn’t seen this example, I might not have thought to try it.

How should you use what you find?

When you search open-source code, you get hundreds of examples for nearly any feature. You’ll notice where the documentation makes sense, and where it’s over-simplified. And you’ll see objects working together in real code. That will teach you more about how classes are connected than any RDoc can.

Still, use the examples to learn new things, not to cargo cult into your own project. Using code without understanding it is the fastest way to a poorly designed system.

In my example, maximum might not be the right thing to do in my project. I shouldn’t copy the line without learning more about it. Without understanding when it makes sense to use. But examples can teach you things you didn’t know before. And they’re a great way to decide what to learn about next.

Like any search, it’ll take some practice to get good results. But I’ve learned a lot from the code I’ve found, and I know you will too.


Learn Rails without being overwhelmed

When I was first learning the board game Go, I bought and read a few introductory books. They taught me the rules, and some basic strategy. I knew that there was a fun game there. But I just wasn’t seeing it.

When it came time to play my own games, I was confused and stressed out. There were 361 possible moves I could make, but which of those was the best one? Not even that, which of those were even acceptable? Using the information I got from the first books, I could maybe narrow decent moves down to about a hundred, but I still had trouble knowing where to start, and what to do. And it just wasn’t fun to me.

I felt like I couldn’t even play the game unless I studied everything first.

So, on a friend’s recommendation, I picked up another book. The book was called “The Second Book of Go.” It had short chapters, each on a specific topic that beginners tend to struggle with. And it gave me some processes I could follow, so I could know that I wasn’t making insane moves.

It taught me the reason behind crazy-looking strategies, and it taught me tricks to make better decisions and evaluate my own moves. And it showed me that even wrong moves had value.

When I started playing again, I began to have fun.

As long as I’ve written here, I’ve heard from people learning Rails that were in the same boat as I was learning Go. They heard it was supposed to be fun, but it was actually just frustrating. They didn’t know where to start. The things they learned didn’t stick with them. They couldn’t spend time learning consistently enough to grow as a Rails developer. Most of all, they were overwhelmed, because the space of possibilities in front of them seemed too big.

If this sounds familiar, I wrote a book for you. It’s meant to bring the joy of Ruby and Rails to those who were about to give up. To show that with a little structure, a little practice, and a little curiosity, you can write your own Rails applications, and have fun doing it.

It’s available for pre-sale, starting today. It’s 25% off until it’s finished. And when you buy it, you’ll get today’s draft immediately, and updates to it from here on out.

I hope you enjoy it.


Write better tests with the three-phase pattern

(This is a short excerpt from Practicing Rails. Sign up here to get the first chapter free!)

So, you’re working on a new app, and Rails just generated a test for you:

test/models/bug_test.rb
1
2
3
4
5
6
7
require 'test_helper'

class BugTest < ActiveSupport::TestCase
  # test "the truth" do
  #   assert true
  # end
end

You uncomment it, come up with a name, and you’re ready to write your test, right? But then what? What do you write first? What should your test code look like?

If you follow a simple pattern, you’ll turn those stubbed out lines of code into clear, well-structured test cases.

The three-phase test pattern

Your test cases should work in three phases:

  1. First, you set some stuff up (“Arrange”)
  2. Then, you do something (“Act”)
  3. Then, you make sure that what you expected to happen, actually happened. (“Assert”)

For instance, imagine you were testing a method on an array in Ruby. Following this pattern, your test could look like:

1
2
3
4
5
6
7
8
9
10
test "Array#sort will sort an array of numbers" do
  # arrange
  unsorted_array = [7, 4, 2, 3]

  # act
  sorted_array = unsorted_array.sort

  # assert
  assert_equal [2, 3, 4, 7], sorted_array
end

Simple enough. But every part of the test has a place to go, and each stage of the test almost tells you how to write it.

Sometimes, you won’t need an Arrange phase, or the Act and Assert phases will be combined. But it still helps to think about all three phases as you write your tests.

The Assert phase gotcha

There’s a trick to the Assert phase: you shouldn’t use the same logic that the Act phase used in the Assert phase. You should always take two paths to the same answer. Otherwise, you won’t notice bugs in the code you’re calling, because it’s just getting called again in the Assert phase.

For example, if you’re doing some math:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
test "average returns the average of a set of numbers" do
  # arrange
  numbers = [1, 2, 3, 4]

  # act
  average = numbers.average

  # assert

  # this is bad
  assert_equal [1, 2, 3, 4].average, average

  # this is better
  assert_equal 2.5, average
end

Calling [1, 2, 3, 4].average again in the Assert phase is bad, because average could return almost anything and that assertion would still pass.

Here, that’s pretty clear. But even when things get more complicated, make sure you’re not just running the same code twice. Otherwise you’re only verifying that your method was called, not that it worked the way you expect it to.

Usually, the easiest way to take a second path to the answer is to find the answer by hand and hardcode it. It can be brittle, but it’s better than your tests breaking without you realizing it.

Why three phases?

If you split your tests into those three phases, you have simpler questions to answer. Instead of “How should I write this test?”, you can focus on each phase: “How should I set this test up?”, “What am I testing?”, “What should the answer look like?”

These questions still might not have easy answers, but the answers will be a lot easier than thinking about the entire test at once. And if you’re lucky, you can even share phases between related tests, making your next test much less painful to write.


How does Rails handle gems?

A few weeks ago, I wrote about how RubyGems manages Ruby’s load path. But Rails doesn’t use RubyGems directly – it uses Bundler to manage its gems.

If you don’t know how Bundler works, the way gems get pulled into Rails might seem a little too magical. How does adding a line to a Gemfile get code into your app? How do Bundler, Rails, and RubyGems work together to make handling dependencies easy?

Why Bundler?

I think of Bundler as a strict gem manager. That is, Bundler helps you install the right versions of the gems you need, and forces your app to only use those versions.

This turns out to be really helpful. To understand why, you have to go back to the world before Bundler.

Before Bundler, it was still pretty easy to install the right versions of your gems with some kind of setup script:

bin/setup
1
2
3
gem install rails -v 4.1.0
gem install rake -v 10.3.2
...

(That is, as long as Rails 4.1’s dependencies don’t conflict with Rake 10.3.2’s dependencies!)

But what happens when you’re working on a few different Rails apps, each depending on different versions of gems? Unless you’re really careful, you’ll run into the terrible gem activation error:

gem_error
1
2
Gem::Exception: can't activate hpricot (= 0.6.161, runtime),
already activated hpricot-0.8.3

Ugh. That message still gives me nightmares. It usually meant you’re in for a day of installing and uninstalling gems, so you can get just the right versions on that machine. And all it takes is an accidental gem install rake to completely mess up all of your careful planning.

rvm gemsets helped with this problem for a while. But they needed some time to set up, and if you accidentally installed into the wrong gemset, you’d be back to the same problem. With Bundler, you rarely have to think about your dependencies. Your apps usually just work. And Bundler takes a lot less setup than gemsets did.

So, Bundler does two important things for you. It installs all the gems you need, and it locks RubyGems down, so those gems are the only ones you can require inside that Rails app.

How does Rails use Bundler?

At its core, Bundler installs and isolates your gems. But that’s not all it does. How does the code from the gems in your Gemfile make it into your Rails app?

If you look at bin/rails:

bin/rails
1
2
3
4
5
6
7
8
#!/usr/bin/env ruby
begin
  load File.expand_path("../spring", __FILE__)
rescue LoadError
end
APP_PATH = File.expand_path('../../config/application',  __FILE__)
require_relative '../config/boot'
require 'rails/commands'

You’ll see that it loads Rails by requiring ../config/boot. Let’s look at that file:

config/boot
1
2
3
ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__)

require 'bundler/setup' # Set up gems listed in the Gemfile.

Hey, it’s Bundler! (Also, I just learned you can choose a different Gemfile to use by setting the environment variable BUNDLE_GEMFILE. That’s pretty cool.)

bundler/setup does a few things:

  • It removes all paths to gems from the $LOAD_PATH (which reverses any load path work that RubyGems did).
  • Then, it adds the load paths of just the gems in your Gemfile.lock back to the $LOAD_PATH.

Now, the only gems you can require files from are the ones in your Gemfile.

So all the gems you need are on your load path. But when you use RubyGems by itself, you still have to require the files you need. Why don’t you have to require your gems when you use Rails with Bundler?

Take a quick look at config/application.rb, which runs after Rails boots:

config/application.rb
1
2
3
# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)

It’s Bundler again! Bundler.require requires all the gems in all the groups you pass to it. (By “groups”, I mean the groups you specify in your Gemfile.)

Which groups are in Rails.groups, though?

railties/lib/rails.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
   # Returns all rails groups for loading based on:
    #
    # * The Rails environment;
    # * The environment variable RAILS_GROUPS;
    # * The optional envs given as argument and the hash with group dependencies;
    #
    #   groups assets: [:development, :test]
    #
    #   # Returns
    #   # => [:default, :development, :assets] for Rails.env == "development"
    #   # => [:default, :production]           for Rails.env == "production"
    def groups(*groups)
      hash = groups.extract_options!
      env = Rails.env
      groups.unshift(:default, env)
      groups.concat ENV["RAILS_GROUPS"].to_s.split(",")
      groups.concat hash.map { |k, v| k if v.map(&:to_s).include?(env) }
      groups.compact!
      groups.uniq!
      groups
    end

Well, that explains that. Rails.groups is going to be [:default, :development] when you’re running Rails in development mode, [:default, :production] in production mode, and so on.

So, Bundler will look in your Gemfile for gems belonging to each of those groups, and call require on each of the gems it finds. If you have a gem nokogiri, it’ll call require "nokogiri" for you. And that’s why your gems usually just work in Rails, without any extra code on your part.

Know your tools

If you understand the tools you use well, it’ll be easier to work with them. So if you find yourself using something all the time, it’s worth taking a few minutes to dig into it a little more.

If you’re working in Ruby and Rails, you’ll use gems every day. Take the time to learn them well!


Three great ways to learn Ruby faster

There are lots of good places to learn Ruby. But learning isn’t just reading books or watching videos. It’s running head-first into a problem, getting stuck, struggling, getting frustrated, looking things up, having it click, playing around with it, and finally (finally!) getting something working.

You have to use the things you learn, or they won’t stick with you. And there are a few great ways I’ve found to do just that.

Ruby Quiz

Ruby Quiz is a group of over 150 short, interesting problems to solve with Ruby. Things from converting Roman Numerals to generating ASCII-art dungeons. Each problem has solutions, too – you can see different approaches to the same question. Ruby Quiz has been around forever, and it’s still a lot of fun.

exercism.io

exercism.io starts like Ruby Quiz – you’ll build solutions to small programming problems. But in exercism, after you submit your solution, you share it with other people. You’ll review your code, and refactor it to make it even better.

Your goal with exercism isn’t just working code. It’s refactoring toward small, simple code. You’ll practice your refactoring and object-oriented design skills. And those will stick with you through your entire programming career.

Try it in a tiny app

Programming challenges are great for building your general Ruby knowledge. But sometimes you’ll want to try a feature you just learned about, so you can understand it better.

So, try it in an app. Generate a new Rails app with a scaffold or two. Dedicate it to playing with the feature you want to learn. Even if you’re not doing something Rails-specific, Rails’ code generators are great for trying new things without much setup. You don’t have to worry about setting up tests, requiring the right files, getting Rake set up, or anything like that.

This is how I’ve been playing with the new Rails 4.2 features, and it’s the way I try most of the things I’ve written about here. (It’s also the topic of the first chapter of my book).

What have you forgotten?

I have a long, long list of things I’ve forgotten because I never used them. And that’s just not a good use of my time.

So, balance your reading and viewing with some practice and play. Do some challenges, or build some tiny apps. You’ll be surprised how much more quickly you’ll pick stuff up.


How do gems work?

Most of the time, gems in Ruby Just Work. But there’s a big problem with Ruby magic: when things go wrong, it’s hard to find out why.

You won’t often run into problems with your gems. But when you do, Google is surprisingly unhelpful. The error messages are generic, so they could have one of many different causes. And if you don’t understand how gems actually work with Ruby, you’re going to have a tough time debugging these problems on your own.

Gems might seem magical. But with a little investigation, they’re pretty easy to understand.

What does gem install do?

A Ruby gem is just some code zipped up with a little extra data. You can see the code inside a gem with gem unpack:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
~/Source/playground jweiss$ gem unpack resque_unit
Fetching: resque_unit-0.4.8.gem (100%)
Unpacked gem: '/Users/jweiss/Source/playground/resque_unit-0.4.8'
~/Source/playground jweiss$ cd resque_unit-0.4.8
~/Source/playground/resque_unit-0.4.8 jweiss$ find .
.
./lib
./lib/resque_unit
./lib/resque_unit/assertions.rb
./lib/resque_unit/errors.rb
./lib/resque_unit/helpers.rb
./lib/resque_unit/plugin.rb
./lib/resque_unit/resque.rb
./lib/resque_unit/scheduler.rb
./lib/resque_unit/scheduler_assertions.rb
./lib/resque_unit.rb
./lib/resque_unit_scheduler.rb
./README.md
./test
./test/resque_test.rb
./test/resque_unit_scheduler_test.rb
./test/resque_unit_test.rb
./test/sample_jobs.rb
./test/test_helper.rb
~/Source/playground/resque_unit-0.4.8 jweiss$

gem install, in its simplest form, does something kind of like this. It grabs the gem and puts its files into a special directory on your system. You can see where gem install will install your gems if you run gem environment (look for the INSTALLATION DIRECTORY: line):

1
2
3
4
5
6
~ jweiss$ gem environment
RubyGems Environment:
  - RUBYGEMS VERSION: 2.2.2
  - RUBY VERSION: 2.1.2 (2014-05-08 patchlevel 95) [x86_64-darwin14.0]
  - INSTALLATION DIRECTORY: /usr/local/Cellar/ruby/2.1.2/lib/ruby/gems/2.1.0
  ...
1
2
3
~ jweiss$ ls /usr/local/Cellar/ruby/2.1.2/lib/ruby/gems/2.1.0
bin           bundler     doc         gems
build_info    cache       extensions  specifications

All of your installed gem code will be there, under the gems directory.

These paths vary from system to system, and they also depend on how you installed Ruby (rvm is different from Homebrew, which is different from rbenv, and so on). So gem environment will be helpful when you want to know where your gems’ code lives.

How does gem code get required?

To help you use the code inside of your gems, RubyGems overrides Ruby’s require method. (It does this in core_ext/kernel_require.rb). The comment is pretty clear:

core_ext/kernel_require.rb
1
2
3
4
5
6
7
8
9
10
11
  ##
  # When RubyGems is required, Kernel#require is replaced with our own which
  # is capable of loading gems on demand.
  #
  # When you call <tt>require 'x'</tt>, this is what happens:
  # * If the file can be loaded from the existing Ruby loadpath, it
  #   is.
  # * Otherwise, installed gems are searched for a file that matches.
  #   If it's found in gem 'y', that gem is activated (added to the
  #   loadpath).
  #

For example, say you wanted to load active_support. RubyGems will try to require it using Ruby’s require method. That gives you this error:

core_ext/kernel_require.rb
1
2
3
4
LoadError: cannot load such file -- active_support
  from (irb):17:in `require'
 from (irb):17
 from /usr/local/bin/irb:11:in `<main>'

RubyGems looks at that error message, and sees that it needs to find active_support.rb inside of a gem, instead. To do that, it’ll scan through its gems’ metadata, looking for a gem that contains active_support.rb:

1
2
irb(main):001:0> spec = Gem::Specification.find_by_path('active_support')
=> #<Gem::Specification:0x3fe366874324 activesupport-4.2.0.beta1>

Then, it activates the gem, which adds the code inside the gem to Ruby’s load path (the directories you can require files from):

1
2
3
4
5
6
irb(main):002:0> $LOAD_PATH
=> ["/usr/local/Cellar/ruby/2.1.2/lib/ruby/site_ruby/2.1.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/site_ruby/2.1.0/x86_64-darwin14.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/site_ruby", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/vendor_ruby/2.1.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/vendor_ruby/2.1.0/x86_64-darwin14.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/vendor_ruby", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/2.1.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/2.1.0/x86_64-darwin14.0"]
irb(main):003:0> spec.activate
=> true
irb(main):004:0> $LOAD_PATH
=> ["/usr/local/Cellar/ruby/2.1.2/lib/ruby/gems/2.1.0/gems/i18n-0.7.0.beta1/lib", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/gems/2.1.0/gems/thread_safe-0.3.4/lib", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/gems/2.1.0/gems/activesupport-4.2.0.beta1/lib", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/site_ruby/2.1.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/site_ruby/2.1.0/x86_64-darwin14.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/site_ruby", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/vendor_ruby/2.1.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/vendor_ruby/2.1.0/x86_64-darwin14.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/vendor_ruby", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/2.1.0", "/usr/local/Cellar/ruby/2.1.2/lib/ruby/2.1.0/x86_64-darwin14.0"]

Now that active_support is on the load path, you can require the files in the gem just like any other piece of Ruby code. You can even use the original version of require, the one that was overwritten by RubyGems:

1
2
irb(main):005:0> gem_original_require 'active_support'
=> true

Cool!

A little knowledge goes a long way

RubyGems might seem complicated. But at its most basic level, it’s just managing Ruby’s load path for you. That’s not to say it’s all easy. I didn’t cover how RubyGems manages version conflicts between gems, gem binaries (like rails and rake), C extensions, and lots of other stuff.

But knowing RubyGems even at this surface level will help out a lot. With a little code reading and playing in irb, you’ll be able to dive into the source of your gems. You can understand where your gems live, so you can make sure that RubyGems knows about them. And once you know how gems are loaded, you can dig into some crazy-looking loading problems.


Can you learn Rails before learning Ruby?

Can you learn Rails before learning Ruby? You can, for a little while. But as your app grows past the REST, generated-scaffold, twitter-clone stage, you’re going to run into some walls. Sure, you’ll learn some Ruby through osmosis. But that’ll take forever.

If you want to learn enough Ruby to master Rails, study Ruby on its own. But there are tons of Ruby books out there. I counted four shelves the last time I was in a bookstore. A lot of them are good. But there are a few that are in a class of their own. And they’ll help you through the entire process of learning Ruby.

From the beginning

Rails might be your first framework, and Ruby your first language. If you’re new to programming in general, you’ll want to start with Learn to Program. It’ll teach you to write your own programs in Ruby, even if you’ve never written code before. It’ll start you off with good habits. And I know lots of people that had a lot of fun with this as their first-ever programming book.

Next, Eloquent Ruby will teach you to write Ruby that looks like Ruby. (Which is really important to Rubyists). You’ll learn how to define classes, how to create methods on the fly, and why you’d choose one way of writing Ruby code over another.

After reading Eloquent Ruby, you’ll know idiomatic Ruby. And you’ll have an easier time reading and understanding the code you encounter.

When things get tougher

Once you read those books, you’ll know how to write Ruby code, but you’ll still have trouble organizing it. When does a method go into one class instead of another? When are classes too big? What should you do when you wrote a method so complicated that you can’t understand it?

Confident Ruby and Practical Object-Oriented Design in Ruby are the gems of intermediate Ruby development. Read these, and your Ruby code will be forever changed for the better.

How’s that?

Confident Ruby will teach you to write readable Ruby code by following some simple, general patterns. It’s about writing code that’s clear about what it does. Code that pushes decisions, duplication, and edge cases away from itself, so you don’t have to think about the complicated parts when you don’t need to. Code that Just Works.

Practical Object-Oriented Design in Ruby is about applying good design to your code. You’ll learn how to structure your objects, methods, and communication between them, so your code is easier to use and test. It’s about making your code flexible, without applying too much structure. And writing tests that you won’t hate fixing when your code changes.

Ongoing education

Next, you’ll learn more by seeking out lots of ideas, and learning more about the ones that seem interesting. Blogs are good, but there’s been an explosion of other great resources, released regularly, on a variety of Ruby topics.

Some of my favorites are:

RubyTapas and Ruby Steps are both paid services. But they each have some episodes / newsletters available for free, so you can see if they’re your style before you sign up.

I’ve learned a ton about Ruby from years of random articles and screencasts. It’s not the most organized way to get information, but it works.

Bringing it back to Rails

Rails is Ruby. The time you spend learning Ruby will teach you about Rails, and vice versa.

So learning Ruby first isn’t a requirement for using Rails. You can do a lot with Rails even if you have to muddle through the Ruby.

But it’s not the best way to do it. You’ll have to learn Ruby eventually. And when you do, you can use these books to build a solid foundation.


A lightweight way to handle different validation situations

(I sent this post to my list a few months back. If you like it, and want to read more like it, you should sign up!)

I ran into a weird situation the other day with one of my apps.

Say you have an Article model, and these articles are first created as drafts. These drafts should be lightweight. You want to be able to create and edit them right away, without needing a body, or even a title.

But when you publish these articles, you need to have some extra validation. They should have a title, a body, and all the rest of that stuff.

You could write up a bunch of if statements to do the checking yourself. But you wouldn’t have all the nice form handling you get from Rails. And I’ve found that going with Rails for the things Rails handles well will save you a lot of headache later on. Is there still a way to use validations for this?

if and unless?

Validations support :if and :unless parameters, but this isn’t always so clean:

app/models/article.rb
1
2
3
4
5
6
7
8
9
class Article < ActiveRecord::Base
  # validate in draft mode
  validates_presence_of :author_id

  # validate only when published
  validates_presence_of :body, unless: lambda { |o| o.draft? }

  ...
end

Besides, this isn’t the way I want to publish an article. I want to think about publishing an article as an action, not a state the article is in, so setting and checking an attribute isn’t the right answer.

A little-known, lightly documented Rails feature

This problem is a perfect fit for Rails’ validation contexts. I first learned about custom validation contexts from a gist DHH wrote. But it was hard to find a good explanation of what these were, and how to use them.

The documentation for custom contexts is pretty thin. The only mention of it I could find in the API docs was here. And the API docs for these validations don’t mention custom contexts at all! This makes it especially hard – if you don’t know what something’s called, how do you find more documentation on it?

I finally found a good overview here: http://blog.arkency.com/2014/04/mastering-rails-validations-contexts/. It’s worth a read. Let’s see what it looks like for our example:

app/models/article.rb
1
2
3
4
5
6
7
8
class Article < ActiveRecord::Base
  # validate in draft mode
  validates_presence_of :author_id

  # validate only when published
  validates_presence_of :body, on: :publish
  ...
end
app/models/article_controller.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
class ArticleController < ApplicationController
  respond_to :html

  def create
    @article = Article.create(article_params)
    respond_with @article
  end

  def publish
    @article = Article.find(params[:id])
    @article.attributes = article_params
    @article.save(context: :publish)
    respond_with @article
  end

  private
  def article_params
    params.
      require(:article).
      permit(:body, :title, :author_id)
  end
end

Not too bad! The only weird thing is having to use save instead of update in the publish method. That happens because update can’t take a validation context as a parameter. (Maybe that’s an opportunity for a pull request?)

Beyond saving

Custom validation contexts are useful for more than just saving a record in different ways. Contexts work with valid?:

1
@article.valid?(:publish)

so you can use validation contexts anywhere you want to answer the question: “Does this object have this property? And if not, why not?”

DHH creates a method ending in -able? that delegates to valid?(:context)). I really like the look of that:

app/models/article.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
class Article
  validate :has_been_published, on: :view
  validate :is_not_spam, on: :view

  def viewable?
    valid? :view
  end

  private

  def has_been_published
    if published_at.future?
      errors.add(:published_at, "is in the future")
    end
  end

  def is_not_spam
    if is_spam?(body)
      errors.add(:body, "has been detected as spam")
    end
  end
end

This way, when you check it, you get more information than just true or false:

1
2
3
unless @article.viewable?
  # @article.errors is now filled out
end

Is the article viewable? Well, why not?

Just lightweight enough

There are a few other ways you could get this kind of validation behavior. You could build custom ActiveModel service objects with their own validations. You could use :if or :unless on your validations. But like a lot of the stuff in Rails, custom validation contexts are a good compromise between readability and flexibility.


Rails 5, Module#prepend, and the end of `alias_method_chain`

The Rails 4.2 announcement had some interesting news about the upcoming Rails 5: It’s probably going to require Ruby 2.2. Which will make it the first Rails version to take advantage of all the good stuff from Ruby 2.

The post mentioned garbage collected symbols and keyword arguments. But to me, one of the most interesting Ruby 2 features is Module#prepend.

The good (and bad) of alias_method_chain

When I first learned Rails, alias_method_chain totally blew my mind. It really showed off how flexible Ruby could be.

With just a single line of code, you could completely change the way a method worked. You no longer needed to hack around libraries to add the code you wanted, you could just add it on the fly. alias_method_chain led to my first patches to gems, which led to my first pull requests, which led to my first open source contributions.

But just like monkey patching, alias_method_chain got overused, and its problems started to become obvious:

  • The method names it generates are confusing, which makes errors hard to find and debug. For example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class Person
  def greeting
    "Hello"
  end
end

module GreetingWithExcitement
  def self.included(base)
    base.class_eval do
      alias_method_chain :greeting, :excitement
    end
  end

  def greeting_with_excitement
    "#{greeting_without_excitement}!!!!!"
  end
end

Person.send(:include, GreetingWithExcitement)

If you had an error in Person#greeting, the backtrace would tell you that an error actually happened in Person#greeting_without_excitement. But where is that method even defined? I don’t see it anywhere. How do you know which greeting method is the one with the bug in it? And the method names get even more confusing the more you chain.

  • If you call alias_method_chain twice with the same parameters on the same class, you could cause a stack overflow. (Can you see why?) This doesn’t normally happen, as long as your require statements are consistent about which paths they use. But it’s super annoying if you frequently paste code into a Rails console.

  • And the rest of the things described by Yehuda Katz’s blog post. This post convinced a lot of Rails devs to start abandoning alias_method_chain in favor of module inheritance.

So, why is it still used?

You can replace most alias_method_chains by overriding those methods in a module, and including those modules into your child classes. But that only works if you want to override your superclass, not your class itself. That is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class ParentClass
  def log
    puts "In parent"
  end
end

class ChildClass < ParentClass
  def log
    puts "In child"
    super
  end

  def log_with_extra_message
    puts "In child, with extra message"
    log_without_extra_message
  end

  alias_method_chain :log, :extra_message
end

If you ran ChildClass.new.log, you’d see:

1
2
3
In child, with extra message
In child
In parent

if you tried to use modules instead of alias_method_chain, you could get the output to be:

1
2
3
In child
In child, with extra message
In parent

But you cannot match the original output without changing the log method in ChildClass. Ruby inheritance doesn’t work that way. Well, it didn’t.

What changed in Ruby 2.0?

Until Ruby 2.0, there was no way to add code below a class, only above it. But with prepend, you can override a method in a class with a method from a module, and still access the class’s implementation with super. So, using our last example, we could get the original output with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class ParentClass
  def log
    puts "In parent"
  end
end

module ExtraMessageLogging
  def log
    puts "In child, with extra message"
    super
  end
end

class ChildClass < ParentClass
  prepend ExtraMessageLogging
  def log
    puts "In child"
    super
  end
end
1
2
3
In child, with extra message
In child
In parent

Perfect.

If prepend is still hard to wrap your head around, think of it as doing something like this:

1
2
3
4
5
class NewChildClass < ChildClass
  include ExtraMessageLogging
end

ChildClass = NewChildClass

Except it won’t mess with your class names, and it affects objects that already exist.

(Yes, you can reassign class names in Ruby. No, it’s probably not a great idea.)

What does this mean for Rails?

So, the last excuse for using alias_method_chain is gone in Ruby 2.0. We could take one of the few remaining examples of alias_method_chain in Rails:

rails/activesupport/lib/active_support/core_ext/range/each.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
require 'active_support/core_ext/module/aliasing'

class Range #:nodoc:

  def each_with_time_with_zone(&block)
    ensure_iteration_allowed
    each_without_time_with_zone(&block)
  end
  alias_method_chain :each, :time_with_zone

  def step_with_time_with_zone(n = 1, &block)
    ensure_iteration_allowed
    step_without_time_with_zone(n, &block)
  end
  alias_method_chain :step, :time_with_zone

  private
  def ensure_iteration_allowed
    if first.is_a?(Time)
      raise TypeError, "can't iterate from #{first.class}"
    end
  end
end

and replace it with a module, instead:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
require 'active_support/core_ext/module/aliasing'

module RangeWithTimeWithZoneSupport #:nodoc:

  def each(&block)
    ensure_iteration_allowed
    super(&block)
  end

  def step(n = 1, &block)
    ensure_iteration_allowed
    super(n, &block)
  end

  private
  def ensure_iteration_allowed
    if first.is_a?(Time)
      raise TypeError, "can't iterate from #{first.class}"
    end
  end
end

Range.send(:prepend, RangeSupportingTimeWithZone)

It’s cleaner, Range#each doesn’t get renamed, and ensure_iteration_allowed isn’t monkey-patched in.

Use inheritance, not patches

Ruby gives you a ton of flexibility, and that’s one of the reasons I love it. But it also has a powerful object model. So when you want to inject your own code, try leaning on modules and inheritance before you hack it in. Your code will be much easier to understand and debug, and you’ll avoid some of the harder-to-detect side effects of something like alias_method_chain.

alias_method_chain was one of the coolest methods I was introduced to in Rails. But its days are numbered. We’ve outgrown it. And I’m not going to miss it when it’s gone.


A few tips for cutting down exception noise

No app survives first contact with actual users. Once people start to use it, they’re going to run into errors.

So, once in production, most apps will have a way of tracking and reporting errors. You could go simple with exception_notification or use a webapp like Honeybadger or Raygun.

But soon, you’ll see the same few exceptions over and over. Maybe a web service you depend on isn’t totally stable. Or the people using your site typo’d their email, so none of your messages are going through. Exceptions should be exceptional, they should be unexpected. But how unexpected can an error be if you see it thirty times a day?

There are better ways of solving these problems than reporting and ignoring. Most noisy exceptions fall into a few basic categories. And for each of these categories, you can use patterns to cut down the noise and make your users happier at the same time.

The network is down!

Few apps work alone. Most communicate with other apps. But when your geolocation API goes down or ec2 has a hiccup, you don’t want to get spammed with thousands of exceptions that you can’t do anything about.

When you deal with unreliable services, try the Circuit Breaker pattern, from Michael Nygard’s Release It:

The basic idea behind the circuit breaker is very simple. You wrap a protected function call in a circuit breaker object, which monitors for failures. Once the failures reach a certain threshold, the circuit breaker trips, and all further calls to the circuit breaker return with an error, without the protected call being made at all.

So, when a service goes down, you’ll automatically stop trying to connect to it. You’ll just go on without that functionality. You can even make it self-healing, so your app will automatically check the service again after a certain amount of time.

The circuit breaker pattern is designed to prevent cascading failures, but you can also use it to limit exception notifications. With this pattern, you really only need to get notified when it trips and when it fails to recover. It can turn thousands of exceptions into a few. If that’s still too many, you can tell the breaker to only report after it’s failed a few retries in a row. (Or find a different, more reliable service!)

Using this pattern takes work, but it also makes the experience better for your users. Instead of a hard error page, they’ll see a message saying that this feature isn’t working right now, and they should try again later. It’s better information for them, delivered at the right time.

Turns out gmaaaail.com isn’t what you meant

Another kind of exception I see a lot comes from bad user data.

For example, say someone typo’d their email when they signed up. They said, “justinweiss@gmaill.com”, but meant “justinweiss@gmail.com”. The first could theoretically be a valid email address, but all the emails you send bounce. And you get notified about those bounces by your email provider.

These notifications are just noise.

Instead, take a two-sided approach. Prevent the bad data up-front, and disable the feature and notify the user if it fails later on.

For email, I’ve used the mailcheck-js gem to spellcheck things like “gmail.com” and “yahoo.com” when new users register:

Then, if an email still bounces later on, turn off email to that user.

Once you turn the feature off for someone, you also need to tell them that it’s disabled and how to fix it. A banner on the top of the site is usually a good answer. Something like “We weren’t able to send your last few emails, so we’ve turned off sending emails to you. Click here to update your email address, and we’ll turn them right back on.”

You’ll get better data, and the user’s emails won’t just go into the void. Way better than those errors you’ve just been ignoring.

404s and RoutingErrors

You probably want to know about broken links or assets on your site. But those things don’t belong in your exception tracker.

For these, and other “half-expected errors”, batch them up and handle them all at once. You don’t need to get notified about them as they happen. You want pull, not push.

Things like RoutingErrors and 404s can be handled with something like Google Webmaster Tools, which will show you the pages Google knows about that are throwing 404s. Or you could run something like link-checker to check the links on your site as part of your pre-release process.

Exceptions should be actionable

It should be rare to get an exception email. Too much noise in your error tracker will keep you from seeing and fixing real problems right away.

If you’re more annoyed than embarrassed about the exceptions you see, you have a noise problem. Use the patterns here to cut down on that noise and give your users a better experience at the same time.

I’ve talked about a few of the noisy exception categories I’ve seen most often. But I’m sure I haven’t seen them all. Which exceptions annoy you the most in your apps? Do they fit into any of these categories, or do they define a new one? How do you keep them from bothering you a few hundred times a day?


The lesser-known features in Rails 4.2

The first beta of Rails 4.2 was announced last week, and it already looks amazing. I’m really excited to use ActiveJob, Web Console, Adequate Record, and Foreign Key support in my own apps.

But the beauty of Rails is in the details. And if you do a little digging, you’ll find some less advertised features that will totally improve your daily work with Rails.

Easily load config files

OK, I might be biased. But having a built-in way to load config files is going to be awesome.

config_for, new in Rails 4.2, works exactly like you’d expect:

config/redis.yml
1
2
3
4
5
6
7
8
9
development:
  host: localhost
  port: 6379
test:
  host: localhost
  port: 6379
production:
  host: redis-production
  port: 6379
1
2
irb(main):001:0> Rails.application.config_for(:redis)
=> {"host"=>"localhost", "port"=>6379}

That is, if you call config_for(:redis), it’ll look for config/redis.yml in your Rails app, parse it, and returns the right configuration for your RAILS_ENV.

It even lets you put ERB in your yaml:

config/redis.yml
1
2
3
4
5
6
7
8
9
development:
  host: localhost
  port: <%= ENV['REDIS_PORT'] %>
test:
  host: localhost
  port: <%= ENV['REDIS_PORT'] %>
production:
  host: redis-production
  port: <%= ENV['REDIS_PORT'] %>
1
2
3
4
$ REDIS_PORT=6380 bin/rails c
Loading development environment (Rails 4.2.0.beta1)
irb(main):001:0>  Rails.application.config_for(:redis)
=> {"host"=>"localhost", "port"=>6380}

If you configure lots of services in your app, this’ll make your initializers much easier to read.

Bootstrapping your apps

Most Rails apps need you to run some commands before you can try them out. Even mostly-empty Rails apps still need the database to be set up before they’ll boot.

So, Rails, once again going with convention over configuration, created a place specifically for your setup code: bin/setup.

The default is good. But bin/setup is also the place to put any other code you need to get your app started up.

So now that this is the convention, if you’ve already written a bootstrap script for your app, rename it to bin/setup, so people using your apps can get started easily.

bin/setup gives you one less decision to make when you generate new Rails apps. And once you get in the habit of running bin/setup after you git pull, you’ll never have to remember to run rake db:setup when you generate a new app again.

Transform your hash values

This is another thing I need often enough that I wish it was built-in to Ruby. When you call transform_values on a hash, it acts like calling map on the hash values, and returns a new hash with the original keys associated with the replaced values:

1
2
3
h = {a: 1, b: 2, c: 3}

h.transform_values { |v| v * 2 } # => {a: 2, b: 4, c: 6}

Simple enough. But you’d be surprised how often it comes in handy.

Bonus: More configuration!

Rails 4.2 offers a simple way to set your own global configuration:

1
2
3
4
Rails.application.config.x.some_configuration_key = "Some Value"

Rails.application.config.x.some_configuration_key # => "Some Value"
Rails.configuration.x.some_configuration_key # => "Some Value"

This works especially well when you combine it with config_for:

1
2
app_config = Rails.application.config_for(:app)
Rails.application.config.x.block_phone_calls = app_config["block_phone_calls"]

And there’s more!

It looks like Rails 4.2 is set up to be an amazing release. But it’s the little refinements that make it so great to work in Rails every day. So, if you haven’t yet, take a quick look through the 4.2 Release Notes and see which other helpful improvements you can find.

(And if you do find some cool stuff, don’t keep it to yourself. Share it here, so we can all learn something new!)


Rails i18n: 3 quick tips and 1 crazy abuse

Rails’ i18n library is more powerful than it looks. You don’t need to only use it for translation. The i18n library is useful anywhere you want to decouple the text you display from the place you display it.

While I’ve been at Avvo, we’ve used i18n to do some really cool stuff. I’m going to share a few of the things that we learned that have come in handy, along with a crazy abuse of the library that ended up working out incredibly well for us.

Automatic HTML escaping

Do you have lines that look like:

1
<%= raw(t('form.required_field_header')) %>

in your views? You don’t need that raw, because if your translation key ends with _html, you get escaping for free:

1
<%= t('form.required_field_header_html') %>

Simple, and more consistent with your other t() calls.

Accessing locales more conveniently

When you build your locale files, you’ll probably have some of your keys grouped under the same parent:

1
2
3
4
5
6
en:
  bugs:
    index:
      new_label: "File a new bug"
      edit_label: "edit"
      delete_label: "delete"

To reference a bunch of these all together, you could reference them by their full key:

1
<%= t('bugs.index.edit_label') %> | <%= t('bugs.index.delete_label') %>

That’s pretty annoying and repetitive. But t() helpfully takes a scope, so you can reference keys more conveniently:

1
2
<% bugs_scope = 'bugs.index' -%>
<%= t('edit_label', scope: bugs_scope) %> | <%= t('delete_label', scope: bugs_scope) %>

If you name your partials well, you don’t even need to specify the scope:

app/views/bugs/index.html.erb
1
<%= t('.edit_label') %> | <%= t('.delete_label') %>

That is, .edit_label references bugs.index.edit_label, because you’re in bugs/index.html.erb.

ActiveRecord backends

Sometimes, static translations in a yaml file just won’t work for your project. If you use the ActiveRecord i18n backend instead:

config/initializers/locale.rb
1
2
require 'i18n/backend/active_record'
I18n.backend = I18n::Backend::ActiveRecord.new

you can look up translations in a translations table in your database. This way, you don’t have to define all your translations up-front. Instead, you can add new translations on the fly.

Setting up the translations table requires a specific migration. The README on the i18n-active_record repo has all the information you need to get it working.

Sharing partials between object types

At Avvo, we have a lawyer directory, attorney reviews, and legal advice. But a few years ago, we didn’t just have lawyers on our site. We had doctors and dentists, too!

A lot of our UI was the same between these professions. But enough was different (for example, what lawyers call “practice areas”, doctors call “specialties”), that it would be hard to share the same views or partials without some pretty ugly code.

Eventually, we thought about trying to lean on the i18n system for this. After all, English lawyerese is kind of a dialect of English, and doctorese is the same way. We didn’t want to block ourselves from eventually using other languages, so we decided to create two new locales: en_jd and en_md.

This turned out to be easy to set up as a custom i18n backend:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
class AvvoI18nStore < I18n::Backend::Simple
  def translate(locale, key, options = {})
    begin
      default = options.delete(:default)
      super(locale, key, options)
    rescue I18n::MissingTranslationData => e
      # fall back to "en" if we can't find anything under "en_jd", etc.
      fallback_locale = locale.to_s.split("_").first
      super(fallback_locale, key, options.merge(:default => default))
    end
  end
end

I18n.backend = AvvoI18nStore.new

And defining translations was just as easy:

1
2
3
4
5
en_jd:
  practice_area: "practice area"

en_md:
  practice_area: "specialty"

It worked great for translating entire partials, too (note the filename):

app/views/questions/_answer_badge.en_jd.html.erb
1
<!-- Lawyer badge HTML -->
app/views/questions/_answer_badge.en_md.html.erb
1
<!-- Doctor badge HTML -->

And it even fell back to the en locale, for shared translations:

1
2
3
en:
  leaderboard:
    title: "Leaderboard"

When you were in the “Doctors” section of the site, we changed the default locale to :en_md, and vice versa:

1
I18n.locale = :en_md

And everything just worked!

I’m not sure I’d recommend this. It’s a little nuts, thinking of different professions having different languages. But it worked amazingly well while we did it. And it reveals just how much power simple tools like i18n can have.

Where do you pick up this stuff?

Last week, I talked about taking deep dives into specific technologies as a way to move from a beginner to an expert. This is another example of that: most of what we learned about i18n we learned from reading the Rails guides and API docs.

So, take that extra step. Learn the tools you rely on well. You’ll find that you’ll run into conveniences and tricks that will not only make you more productive, but might unlock brand new categories of solutions you’d never even have thought of otherwise.



Where are all the intermediate Rails resources?

So, you’ve finished a few Rails tutorials. You might have taken a class or two or watched some screencasts. You’ve followed along and built a copy of some tutorial apps. It’s pretty clear that it’s time to move to the next level in your Rails development.

Somehow, though, you’re stuck. There are tons of books, classes, and videos for people just starting out. But where are all the tutorials for intermediate Rails devs?

It’s not like it was before

Once you pass the beginner, “building baseline knowledge” stage of being a Rails developer, the resources dry up. Why is that?

Becoming an intermediate Rails developer is nothing like being a beginner. It might seem like it’s just a little different, with some more complicated stuff to learn. But passing through the intermediate stage of learning is a totally separate process.

Don’t think of it as, “A beginner knows this stuff. An intermediate developer knows all that stuff a little better, plus some extra things off to the sides.” For an intermediate developer, learning is more focused. Instead of knowing a little about a lot, you’ll learn a lot about a little.

Find a few areas to focus on. Learn more details about testing and TDD. Learn a few of the design patterns that Rails encourages, in depth. Learn about how you should design a great data model. But don’t learn it all at once. Learn one thing at a time, and learn it really well.

Once you understand the Rails basics, you’ll know enough to read API documentation, even if you don’t understand all of it yet. You could read the best books on TDD, even if they’re not written specifically for Rails. After you’ve seen enough examples of Ruby code, you can go source-diving and understand parts of Rails at a level that few others do.

When you concentrate on learning one thing well, there are many more resources to learn from. And each thing you learn will make every other thing easier to learn.

(If it seems easy for people to pick up their third or fourth language, this is one reason for it. A lot of the core concepts stay the same, which means there’s a lot less you have to learn to travel through the intermediate stage in a new language).

How do you choose what to play with next?

If you’re going to learn one thing at a time, you have to choose what to focus on first. But how do you pick, when there’s so much to learn?

Doesn’t it always seem easier to learn something when you’re interested in it? It definitely does for me. So, what’s the best way to force yourself to be interested in learning something new?

Write your own apps! Lots and lots of apps. Write apps, modify apps, add features to existing apps.

When you write an app, you will get stuck. You’ll get stuck a lot. But that’s a great thing! You should learn to love the stuckness, because it means you’re about to learn something new. Being stuck is awesome, and the more you can enjoy it, and even seek it out, the faster you’ll become an expert.

Once you research, learn, and write a solution, try to do it again in another app. Try to write it without looking at the documentation. Really learn it. Then, continue building your app, and watch for the next place you get stuck.

I love learning this way, because you get to focus on the stuff you’re going to run into most often. You’ll learn faster, because you’re not just studying to study. You’re trying to figure out how to solve a problem. And isn’t that why you fell in love with software development to begin with?

But you can’t get them all this way

After you understand the basics of Rails, you can learn serendipitously. If you understand what intermediate and expert Rails devs are talking about, you can accidentally run into conversations that will take your studying down really interesting paths.

So, when I get past the beginning, tutorial stage of learning something new, I subscribe to a bunch of blogs. I follow some of the leaders of the community on twitter. I subscribe to the WhateverWeekly newsletter for the language (RubyWeekly, iOS Dev Weekly, JavascriptWeekly, etc.).

You don’t have to read it all. You shouldn’t read it all. But let good stuff hit your radar, and try to read and follow the things that seem interesting. I’ve learned some really great tips this way, and they’ve led me into some useful topics I might not have encountered until much later.

So, what are you going to learn next?

If you’re feeling stuck because you can’t find great intermediate resources, take control of your learning. Start building an app, and pay attention to how you build it. Try something you’re only halfway sure will actually work, and seek out places you can get stuck. Use that stuckness, and find API docs, guides, or tutorials on just that small part. Go source-diving if you need to. (It might feel weird at first, but the more you look at Rails code, the more comfortable you’ll feel with it).

Really understand a small part, become an expert in it. And then, move on to the next thing.


Shaking off your TDD fatigue with a few quick tips

You want to test-drive some code, but you’re stuck. Maybe you’re not totally sure what your object’s interface should look like. Maybe you’re distracted by another sunny summer day. You might not be sure you can test what you’re thinking of building. Or you could be procrastinating, because you just don’t feel like writing tests right now. How do you get the benefits of TDD when you don’t feel like TDDing?

Prototyping, or building one to throw away

“Where a new system concept or new technology is used, one has to build a system to throw away, for even the best planning is not so omniscient as to get it right the first time. Hence plan to throw one away; you will, anyhow.” – Fred Brooks

Sometimes, when I start a new feature, I’ll feel paralyzed when I think about writing my first tests. There are so many decisions to make about how the API should look, which patterns and practices will be most appropriate, and how it should all fit together.

I have a few ways I get started when I’m stuck like that. Building a prototype is another.

When you write a prototype, you should think of it like a sketch. Brainstorm and try things out. Don’t bother with tests or TDD yet. Instead, explore the decisions you’re having a hard time making, with code. Try out a few patterns, and see which of them fit the feature you’re trying to design. Figure out where the shaky parts of your app are, what needs more thought, and what you were worrying about for no real reason.

Then, throw it away and rebuild it. This time, using TDD and your newfound knowledge about how you can best build the feature.

You know when you accidentally run git checkout -f after you wrote something, and you get that feeling like you just got kicked in the stomach? But then you figure, this has to be built, so I guess I’ll do it again, and it turns out much better than it did the first time? This is also true with rebuilding prototypes. Now that you have some solid ideas of how the feature could look, TDDing that feature will go a lot more easily.

Reverse TDD

Have you ever run into a situation where you know how to build something, but you don’t know how to write the tests for it first? Or maybe it’s a one-line change to a method, but it might need some tweaking, so you don’t want to lock it down with a test before you know what the change will actually look like?

There’s a simple process that helps a lot with these situations:

  1. Write the code, without tests.
  2. Write a test for the code. Make sure it passes.
  3. Comment out the code you wrote, or revert the file you made the code change in.
  4. Run the test again. Make sure it fails.
  5. Write the code again. Run your tests. Make sure they pass.
  6. Refactor the code, taking advantage of your new tests.

I think of this as Reverse TDD or Green-Red-Green-Refactor. You won’t get the “test-driven design” part of TDD, but you’ll still get to see your tests fail when your code is broken. That’s important, because you can’t trust tests that don’t fail at least once.

Reverse TDD usually works best with tiny changes that don’t need much design. Think bugfixes, or changes that are localized to a single line, method, or class. But I use this technique a lot, especially for small changes that I’m still playing with.

The Pairing Game

The Pairing Game is a great way to break out of your Red-Green-Refactor routine when you have another developer around. It works like this:

  1. Find a partner.
  2. You write a test that will break.
  3. Your partner tries to fix that test with the simplest code possible.
  4. You write another test that will fail on that code.
  5. Your partner writes code that will pass that test.
  6. Repeat…
  7. At some point when the tests are green, you can choose to refactor some code instead of writing a test.
  8. After refactoring, swap tester and coder positions.

I’ll let you in on a secret: playing the Pairing Game on a bowling score calculator is how I originally learned TDD. It’ll help you practice writing simple code based on failing tests, and writing those tests to begin with. Both developers will learn a lot about each other’s coding style and favorite techniques. And you’ll hit flow almost immediately, which means you’ll feel great when you finish.

You might not produce code as quickly, but it’s a lot of fun, and a great learning experience.

Stay playful and experiment

TDD shouldn’t be dogma. There are ways you can play with the core TDD concepts that can lead you to some really interesting places. And you might end up with even better code.

So experiment. Try new approaches to writing code and TDD. See how they make you feel, and what kind of code you end up with. And keep rediscovering the fun and the flow in the work you do each day.


4 simple memoization patterns in Ruby (and one gem)

Memoization is a technique you can use to speed up your accessor methods. It caches the results of methods that do time-consuming work, work that only needs to be done once. In Rails, you see memoization used so often that it even included a module that would memoize methods for you.

Later, this was controversially removed in favor of just using the really common memoization pattern I’ll talk about first. But as you’ll see, there are some places where this basic pattern just doesn’t work right. So we’ll also look at more advanced memoization patterns, and learn some neat things about Ruby in the process!

Super basic memoization

You’ll see this memoization pattern all the time in Ruby:

app/models/order.rb
1
2
3
4
5
6
class User < ActiveRecord::Base
  def twitter_followers
    # assuming twitter_user.followers makes a network call
    @twitter_followers ||= twitter_user.followers
  end
end

The ||= more or less translates to @twitter_followers = @twitter_followers || twitter_user.followers. That means that you’ll only make the network call the first time you call twitter_followers, and future calls will just return the value of the instance variable @twitter_followers.

Multi-line memoization

Sometimes, slow code won’t fit on one line without doing terrible things to it. There are a few ways to extend the basic pattern to work with multiple lines of code, but this is my favorite:

app/models/order.rb
1
2
3
4
5
6
7
8
9
class User < ActiveRecord::Base
  def main_address
    @main_address ||= begin
      maybe_main_address = home_address if prefers_home_address?
      maybe_main_address = work_address unless maybe_main_address
      maybe_main_address = addresses.first unless maybe_main_address
    end
  end
end

The begin...end creates a block of code in Ruby that can be treated as a single thing, kind of like {...} in C-style languages. That’s why ||= works just as well here as it did before.

What about nil?

But these memoization patterns have a nasty, hidden problem. In the first example, what if the user didn’t have a twitter account, and the twitter followers API returned nil? In the second, what if the user didn’t have any addresses, and the block returned nil?

Every single time we’d call the method, the instance variable would be nil, and we’d perform the expensive fetches again.

So, ||= is probably not the right way to go. Instead, we have to differentiate between nil and undefined:

app/models/order.rb
1
2
3
4
5
6
class User < ActiveRecord::Base
  def twitter_followers
    return @twitter_followers if defined? @twitter_followers
    @twitter_followers = twitter_user.followers
  end
end
app/models/order.rb
1
2
3
4
5
6
7
8
9
10
class User < ActiveRecord::Base
  def main_address
    return @main_address if defined? @main_address
    @main_address = begin
      main_address = home_address if prefers_home_address?
      main_address ||= work_address
      main_address ||= addresses.first # some semi-sensible default
    end
  end
end

Unfortunately, this is a little uglier, but it works with nil, false, and everything else. (To handle the nil case, you could also use Null Objects and empty arrays to avoid this problem. One more reason to avoid nil!)

And what about parameters?

We have some memoization patterns that work well for simple accessors. But what if you want to memoize a method that takes parameters, like this one?

app/models/city.rb
1
2
3
4
5
class City < ActiveRecord::Base
  def self.top_cities(order_by)
    where(top_city: true).order(order_by).to_a
  end
end

It turns out that Ruby’s Hash has an initalizer that works perfectly for this situation. You can call Hash.new with a block:

1
Hash.new {|h, key| h[key] = some_calculated_value }

Then, every time you try to access a key in the hash that hasn’t been set, it’ll execute the block. And it’ll pass the hash itself along with the key you tried to access into the block.

So, if you wanted to memoize this method, you could do something like:

app/models/city.rb
1
2
3
4
5
6
7
8
class City < ActiveRecord::Base
  def self.top_cities(order_by)
    @top_cities ||= Hash.new do |h, key|
      h[key] = where(top_city: true).order(key).to_a
    end
    @top_cities[order_by]
  end
end

And no matter what you pass into order_by, the correct result will get memoized. Since the block is only called when the key doesn’t exist, you don’t have to worry about the result of the block being nil or false.

Amazingly, Hash works just fine with keys that are actually arrays:

1
2
3
h = {}
h[["a", "b"]] = "c"
h[["a", "b"]] # => "c"

So you can use this pattern in methods with any number of parameters!

Why go through all this trouble?

Of course, if you start adding these memoization patterns to a lot of methods, your code will get pretty unreadable pretty quickly. Your methods will be all ceremony and no substance.

So if you’re working on an app that needs a lot of memoization, you might want to use a gem that handles memoization for you with a nice, friendly API. Memoist seems to be a good one, and pretty similar to what Rails used to have. (Or, with your newfound memoization knowledge, you could even try building one yourself).

But it’s always interesting to investigate patterns like this, see how they’re put together, where they work, and where the sharp edges are. And you can learn some neat things about some lesser-known Ruby features while you explore.


A guide to learning from outdated resources

(I sent this post to my list a while back. If you enjoy it, and want to read more like it, you should sign up!)

The Rails ecosystem moves fast, way too fast for print. If you’re like me, you want to learn the most recent version of your frameworks and gems. But the best resources are often a few versions behind.

These resources are still useful, though. There’s nothing like a well-edited book, screencast, or tutorial to learn a library’s API, philosophy, and structure.

For example, if the best Rails books only describe Rails 4.0, you should still start from there. When you’re learning a new gem or framework, it’s important to learn why it’s designed a certain way and how all the pieces fit together. That’s hard to get from reference documentation and blog posts, but easy to get from books.

But once you build experience with an older version of a gem, how do you get caught up?

Catching up with changes

Most popular gems have CHANGELOG files in their git repository, like bundler: https://github.com/bundler/bundler/blob/master/CHANGELOG.md. These are a great way to catch up on the big changes from version-to-version. Usually, they’re just a short summary of each major change. But they give you a starting point, so you can do more research on interesting changes.

Many changelogs reference bug numbers on GitHub. If you see an entry in a changelog that has a bug number attached, you can find the bug in the project’s Issues to understand what changed, how it changed, and why it changed.

So, how do you find changelogs? Usually, I just search google for bundler github (if I’m looking for the bundler changelog), and they’re usually on the first GitHub page you see.

If there isn’t a changelog, you can also look at the project’s README or the project’s wiki on GitHub. But since those aren’t designed to help you catch up from version-to-version, they usually take longer to go through.

Using up-to-date reference documentation

When you’re working with gems, you’ll also need to keep up-to-date reference documentation around. That way, you can look up API usage and see examples while you’re writing your own apps.

You can find API documentation for any version of any gem at rdoc.info. But for even faster doc lookup, you should check out Dash or Zeal.

I use Dash, so when I need to look up API docs, I hit option-Space, start typing, and all my gem documentation shows up instantly. It’s a change in my workflow that’s paid for itself many times over.

A few tips on Rails, specifically

Rails is a large project, and the Rails contributors do a great job of maintaining changelogs and documentation.

The Rails guides are good, and they’re also built from the same git repository as Rails, so they’re always up to date.

If you’re trying to catch up to the newest version of Rails, the release notes are the best place to start. For example, here are the release notes for Rails 4.1: http://guides.rubyonrails.org/4_1_release_notes.html

The process I usually take

Putting it all together, this is what I do when I want to get completely up to date on a new library:

  1. Read a book, tutorial, or documentation about the gem. I usually try to find the newest resource I can. While I’m reading, I build an app, practice, etc. on whichever version I’m learning.
  2. Find the project on GitHub.
  3. Read the project’s changelog, readme, or wiki to take me from the version I know to the most recent version.
  4. If I’m interested in a specific change, google around for it or go through the project’s issues to learn more.
  5. Upgrade to the newest version of the gem.
  6. Install the new gem’s documentation in Dash.
  7. Write code with it!

I don’t do all those steps all the time. But that’s generally the order I go in, and this kind of process is helpful to keep around in case I get stuck on a step or don’t know where to go next.

What if you can’t find a changelog or can’t make sense of the docs?

Sometimes you just won’t be able to get the information you need out of the gem’s GitHub repo or API documentation. When that happens, you’ll have to dive into the code and start reading it.

Surprisingly, reading code is not like reading a book. Instead of reading files from beginning to end, you have to explore the code. This is more of an art than a science, but it’s an important skill to learn. So, I’ll probably have more to say about it later!

What tricks have you learned for getting caught up on gem changes? How do you stay up to date?


When using an API goes terribly wrong

API docs will tell you how to use an API. But when things go wrong, you can be left on your own. Error messages are often incomplete, misleading, or unhelpful. And what are you supposed to do with NoMethodError: undefined method '[]' for nil:NilClass, anyway?

When you learn an API, framework, or library, you can’t just learn how to use it when things go well. You also need to figure out what to do when it hands an error back.

How do you break code?

There’s an easy way to learn what to do when an API you’re using breaks. Break it yourself!

For example, I’ll:

  • Pass data of the wrong type. You could pass a symbol instead of a String, a String instead of an Array, a Hash instead of an Array, things like that.

  • Pass incomplete data. You can pass nil, hashes without some fields filled out, and objects that have just been initialized.

  • If the API requires network access, disconnect from WiFi or pull the network cable. Does it just time out, or does it tell you which service it couldn’t reach?

  • If the API allows you to pass in a block, throw exceptions inside the block, or return data of the wrong type.

A great API will tell you what you did wrong. An excellent API will tell you how to fix it. But most often, you’ll run into a Ruby NoMethodError, unexpected nils, or, worse, getting totally bizarre return values back.

Why break code?

This isn’t all bad. If you’re playing with an open-source gem, you can take time to understand where that unexpected behavior came from and why it happened. You can debug and read a little bit of code to learn a lot about how the library or API works.

Once you discover where the NoMethodError is coming from, you can go a step further. You can fix the error message for the next person to run into this problem for real! Little error message fixes make great open-source contributions and easy pull requests. And they make the entire Ruby ecosystem a little better for everyone else.

Even if it’s a closed-source REST API, you can still get something out of this exercise. After you see the different errors you’ll get from the API, you’ll have an easier time fixing the problem when you run into an error for real.

Once you become more comfortable seeing and fixing errors in the code around you, you’ll see broken code as a puzzle to be solved. You won’t automatically recoil when you see an exception and backtrace dumped to the screen. Instead, you’ll see them as an opportunity to learn more about the system you’re working with.

Finally, you’ll know that when code breaks, you’ll be able to put it back together again. You’ll improve your confidence in trying new libraries and APIs. And that boldness will increase your rate of learning new things.


Tips for finding Rails blogs at your level

There’s a lot of good, free Rails information around. But as you improve your development skills, it can be hard to find knowledge that’s useful to you.

If it’s too basic, you’ll just read about things you already know. Too advanced, and your eyes glaze over and your brain shuts off. Three paragraphs in, and you don’t even remember what your name is anymore!

You can’t just type “intermediate-level Rails blogs” into Google and hope some good sites pop out. To get the information you’re looking for, you’re going to have to do some digging.

Finding & filtering

First, look wide and shallow. When you use social sites like reddit, twitter, and Stack Overflow, click on every link that looks interesting to you. Skim an article or two on each site, and if the site seems interesting, keep it around.

Next, you can filter and curate. You should build a place where posts find you. If you use an RSS reader (I use Feedbin), subscribe to the site. If you prefer email, sign up to receive email updates. You just want to be sure you see new articles as they come in.

If you skip more than a few articles from a site, unsubscribe. Eventually, you’ll find a small group of people that you enjoy hearing from, who post things you’re interested in, and that you can learn something from.

They’ll grow along with you, and the articles will get more advanced at a rate you can handle. Plus, they’ll often link to the people who they find insightful, which is a much faster way of finding people to learn from.

It’ll take a while to build this group. But you’ll get a little bit more of a benefit from every site you find.

Learn serendipitously

This doesn’t completely solve the “at your level” problem. But I have a little secret: Finding people at just the right level for you isn’t actually all that important. Many of my favorite writers write things that are way more basic or way more advanced than my current skill level.

How does this work?

When something is too basic, you can think of it as a way to review your fundamentals. The first time you learned something, you might have formed bad habits or internalized some ideas incorrectly. When you review basic information you get a second chance to think about and improve those things.

I had this happen when I read Eloquent Ruby as part of the RubyRogues’ book club.

The first few chapters were really basic–I almost stopped reading it, because I didn’t think I’d learn anything interesting. But I went through it, I focused on it as if I was learning Ruby for the first time. And I came out of it with some Ruby tricks and conventions that I had totally forgotten about.

What about articles that are too advanced? You can get a lot out of those, too:

  • You get introduced to more jargon and patterns. Even if you can’t understand what the terms mean, you can search for the ones that sound interesting. You might find some better descriptions, or other good sites to subscribe to!

  • The more often you see an advanced idea, the less you’ll be intimidated by it when you have to learn it for real. It’ll also be easier to learn, because you will have already learned a little about it from seeing it in context.

Just for fun, I subscribe to some CS and math blogs that assume my math skills are way beyond where they actually are. Sometimes I only get a paragraph or two into the article before I give up. But the paragraphs I do read are some of the most interesting and insightful things that come through my feed reader.

You do have to come into advanced blogs and resources with the right mindset. You can’t let them intimidate or overwhelm you. You have to remember that it’s not a reflection on you if you don’t get it. And getting anything out of an article that’s beyond your skill level is a big accomplishment.

So, find some interesting articles. Follow some blogs. If they stay interesting, keep them around. Even if they’re not quite at your current skill level.


Why does refactoring make code simpler?

Last week, I wrote about methods with consistent return values, and how they’ll make your code simpler. But how does that happen? How can the right refactoring make your code easier to work with? What makes good abstraction good, and bad abstraction bad? I mean, you can’t just blindly rearrange your code and end up with quality software.

If you can learn why these techniques work, you’ll be better able to understand where your code needs help. And you’ll be able to use the right tools where they matter most.

What’s the hardest problem in software development?

“There are only two hard problems in Computer Science: cache invalidation and naming things.”

– Phil Karlton

Sure, that quote is everywhere. But there’s actually a harder problem in real-world software development: managing complexity. Complexity in software development has a few different definitions. At its core, though, the complexity of a program is the amount of stuff you have to keep in the front of your mind while you’re working on it.


As you write more software, you’ll be able to keep more stuff in your head at once. But even the best devs have limits. If your project is too complex, you’ll overload what your mind can handle, and you’ll start forgetting about different areas of your program. And the worst bugs show up when you make a change in one part of your project without realizing how the change will affect a totally different part of the same project.

Simplifying through assumptions

If you had photographic memory, and could remember how all the different parts of your program fit together, you’d have a lot fewer bugs, right? Memory can be hard to build. But you can get a lot of the same benefits from being able to focus on one thing at a time.

Many best practices in software development are about reducing the number of things you have to keep in your mind at once. A few examples:

  • Consistency in your return values means you only have to think about how to deal with one kind of data, instead of different kinds of data in different situations.

  • Abstraction is a way to hide code behind a simpler interface. With good abstractions, you can just think about how the abstraction should act, and you don’t have to worry about how it works on the inside.

  • Testing can keep you from accidentally breaking code in one area while you’re working in another. You can assume that anything you accidentally break will be caught. This means you can focus just on the code you’re working on, and fix anything you break later.

  • Test-Driven Development will help you write code that will work the way you expected it to work. You can concentrate on writing the simplest possible implementation of your method that passes the tests. If it’s too simple and doesn’t actually work as you’d expect, your tests will catch it.

When you use these techniques in the right way, you can make some good assumptions. With these assumptions, you don’t have to keep anywhere near as much in the front of your mind. You can write better, more reliable code, because you don’t have to worry about the nearly infinite consequences your code could have on the system.

On the other hand, using these techniques in the wrong place will hide code that you actually need to see. By hiding the code, the assumptions you make will sometimes be wrong. This makes it even more likely that you’ll end up breaking it!

That’s why bad abstractions are worse than no abstraction at all, and why you can’t just throw “extract method” at code until it magically becomes good.

Right refactoring, right place

Clear code is code that’s up-front about what it’s doing. Showing what you’ll need to know is as important as hiding what you don’t need to know. When you perform a refactoring, or use a software best practice, you should think about the code you end up with, and how well it’s helped you reduce complexity. Are you helping the rest of the code make good, simplifying assumptions? Or are you burying essential knowledge behind a too-simple interface?


Simplify your Ruby code with the Robustness Principle

I originally learned a lot of these ideas from Confident Ruby, one of my favorite Ruby books. If you like this post, you should buy it and read the entire thing. There’s so much good stuff in there.

Your current_user method returns a User, except when there is no user and it returns nil. A search method returns an Array of results, unless there’s only one result, and it returns just that result instead. Seems reasonable, right? Maybe even convenient!

But soon, these decisions will bury your code under a mountain of if statements. Maybe it’s a bunch of if kind_of? sprinkled all over. Or maybe you feel like you have to check for nil everywhere. Or worse, NoMethodErrors start showing up whenever you ship a new feature. Guess it’s time for another hotfix!

There is a way to prevent this, though, and all it takes is a little thoughtfulness.

The Robustness Principle

There’s a principle in computing that says,

Be conservative in what you do, be liberal in what you accept from others.

You can apply this principle to your Ruby methods. The methods you write should accept reasonable input, and should return consistent output.

Focusing on the last part: When someone calls a method you wrote, they should know exactly what that method will return.

Be thoughtful about your output

Take a look at Rails 2.1’s implementation of ActiveRecord::Errors#on:

1
2
3
4
5
6
7
# File activerecord/lib/active_record/validations.rb, line 212
def on(attribute)
  attribute = attribute.to_s
  return nil unless @errors.has_key?(attribute)
  errors = @errors[attribute].map(&:to_s)
  errors.size == 1 ? errors.first : errors
end

When called, this could return either an Array of Strings, a String, or nil. It’s up to the caller to figure out which type of object it’s dealing with. This is a bad idea:

  • The caller has to muddy up its own code with obtrusive type-checking.

  • The caller has to know a lot about the method it’s calling. At a minimum, it needs to know every type of object the method could return and when each type could be returned.

  • You have more edge cases to test. If you want to be confident that your code is doing what it’s supposed to do, you have to try out all three scenarios.

Your methods should be consistent about what they return. If you usually return an Array, do what you need to do to always return an Array. If you usually return a User, but sometimes return nil, you could create a Null User object and return that instead of nil.

You could even be less strict: “I’m going to return something that includes the Taggable module”. You could go more generic: “I’m going to return something with id and name attributes.” The important thing is consistency and making sure your caller knows what to expect.

jQuery is an interesting example. Most jQuery methods return the same kind of Array-like object. Because of this, jQuery’s methods are incredibly composable, and you can do crazy amounts of work in a single line of code.

And in case you were wondering, Rails fixed that method in later versions:

1
2
3
4
# File activemodel/lib/active_model/errors.rb, line 133
def [](attribute)
  get(attribute.to_sym) || set(attribute.to_sym, [])
end

Now it always returns an Array. Simpler for them, and simpler for us.

Kill inconsistency

The next time you find yourself returning “An Array or nil”, just return an Array. Take a look through your codebase and see where you’re using kind_of? and respond_to?. See if you can refactor the methods called by that code to return a single type.

And watch as the assumptions you can make about your return values ripple through your project and simplify all of your nearby code.


Fast, consistent setup for your Ruby and Rails tools

After you build a few Rails apps, you’ll begin to have some preferred ways of working with them. Maybe you always want to use awesome_print in your Rails consoles. Or you might want rails new to create projects that use rspec instead of minitest.

Sure, it’s only a little annoying to have to specify these preferences each time you run a command. But they’re easy to forget. And it’s infuriating when a computer doesn’t do what you thought you told it to do:

1
2
3
4
$ rails new best_app_ever
^C (sigh...)
$ rm -r best_app_ever
$ rails new best_app_ever --template=my_template

There is an easier way. With a few small tweaks, you can have these commands remember your preferences, so you won’t have to.

Capture your preferences with .rc files

Lots of Ruby tools give you a simple way to store your preferences using .rc files.

.rc files live in your home directory:

1
2
3
$ ls ~/.*rc
.bashrc       .irbrc      .screenrc
.gemrc        .railsrc    .zshrc

and they give commands like gem, irb, and rails a place to find your preferences. Here are some .rc files you should try using:

.irbrc

When you run rails console, you might want to see all of your objects awesome_printed by default. You could type AwesomePrint.irb! at the beginning of each console session. Or, you can add the command to your ~/.irbrc file:

~/.irbrc
1
2
require "awesome_print"
AwesomePrint.irb!

And the next time you start a rails console, your objects will be printed more awesomely, automatically.

.irbrc is also a great place to put hacky convenience methods that help you play with your objects:

~/.irbrc
1
2
3
4
5
6
7
8
require "awesome_print"
AwesomePrint.irb!

# returns the instance methods on klass 
# that aren't already on Object
def m(klass)
  klass.public_instance_methods - Object.public_instance_methods
end
1
2
3
4
5
6
7
8
~/Source/rcfiles jweiss$ bin/rails c
Loading development environment (Rails 4.1.1)
irb(main):001:0> m ActiveRecord::Base
[
    [  0]                                        [](attr_name)                     ActiveRecord::Base (ActiveRecord::AttributeMethods) (unbound)
    [  1]                                       []=(attr_name, value)              ActiveRecord::Base (ActiveRecord::AttributeMethods) (unbound)
    [  2]                         _commit_callbacks()
... about 200 more methods...

When you customize your environment like this, you’ll begin to feel like the libraries you use are yours, instead of just libraries that you use.

.gemrc

Rubygems has its own .rc file to make dealing with gems more convenient. Lots of people use .gemrc to skip rdoc and ri generation, which makes gems install much faster:

~/.gemrc
1
gem: --no-document

But there’s a lot more you can do. There are a few other values you can tweak in your .gemrc, including :sources: (to look for gems on other gem servers) and :backtrace: (to see a full backtrace when errors occur):

~/.gemrc
1
2
3
4
5
gem: --no-document
:backtrace: true
:sources:
- https://my.internal.gem.server
- https://rubygems.org

For the rest of the settings you can place in this file, take a look at the gem environment command documentation.

.railsrc

Years ago, Rails 3 made it easy to generate apps that don’t use the default Rails stack. If you want an app that doesn’t use minitest (so you can include rspec later, for example), it’s as simple as:

1
$ rails new my_awesome_app --skip-test-unit

Or maybe the place you work has their own project template:

1
$ rails new my_awesome_corporate_app --template=http://example.com/my_app_template

Once you get familiar with a Rails stack, you’ll probably want to use it for all your new projects. But it’s easy to forget to add these parameters when you generate another app. Instead, you can put these parameters in a .railsrc file:

~/.railsrc
1
2
--template=http://example.com/my_app_template
--database=postgresql

And now you can run rails new with the right parameters every time. And if you ever want to override those parameters for a single project, you can do it with --no-skip-test-unit-style commands.

Make your environment yours

A little up-front configuration work can make your environment feel more like home. You can skip the pain of accidentally running a command without the parameters you want. You don’t always have to dig up documentation to find the right options. And you can get to the fun part of writing code right away.


How do you choose the right libraries for your Rails app?

Angular vs Ember. RSpec vs Minitest. Haml vs Slim vs ERB. You have so many choices to make when you start a new project. There are vocal defenders on each side. And soon you start to realize that you could have started your project in the time you’ve wasted reading that fourth tutorial or 30-comment argument about whether Sass is better than Less.

So how do you choose the right library, so you can start writing actual code?

You’re under some serious pressure

The development community moves really fast. New libraries appear every day, built to solve all kinds of problems. Many of those solve problems that have already been solved before. And many of those are built as reactions to the libraries that came before.

These debates get personal. People have spent part of their lives building experience using these libraries. Some have been burned by the problems their new favorite libraries were built to solve. So the arguments for using a particular library are passionate and emotional.

When you read all these passionate arguments, it’s easy to feel like you’ll be looked down on if you make the wrong decision.

How do you get comfortable picking a library?

It’s clear that not everyone will agree with the choice you make. The best you can do is be personally satisfied with your decision. You’ll have to be happy with the library you choose on a technical level, as well as a personal level.

On the technical side, I wrote a guide to picking the right gem for your projects, and most of that advice holds for any of these kind of choices.

Your personal preferences will also factor into your choice. Without building an app that uses the library, though, it can be hard to predict which library you’ll prefer to use.

GitHub’s code search can help you make those decisions. When you search for some class names from the libraries you’re thinking of using, you can find projects that use each library. (You can usually get those class names from the README or gem documentation).

From the code, you can see how the library is used, how well it integrates with the code and other libraries, and how simple it is to use. This helps a lot, especially when you don’t have enough context to make a decision based on technical arguments.

Does it even matter?

Making the wrong choice can cause you some pain later on. But when it comes to building software, Failure 0 is failing to start the project. Spending too much time picking the right library for your app does nothing to help you prevent that failure. Most of the time, making any choice will help you much more than making the wrong choice hurts you.

You shouldn’t feel bad about the choices you make. There’s nothing wrong with learning new libraries, trying them out on projects, and making personal and technical judgements about which libraries you prefer to use.

Even if you change your mind later, the things you learned from the library you chose will always stay with you. You can learn a new library much faster as a diff against your existing knowledge than studying it from scratch.

A long time ago, I could have just as easily become a Pythonista instead of a Rubyist. I may not have been quite as happy with it. (Although honestly, who knows?) But I’d still be writing code, and I’d still love it.


How to learn TDD without getting overwhelmed

(I originally wrote this article for my newsletter. If you want to read more stuff like this, you should sign up!)

The Rails community has an intense focus on testing, and, specifically, TDD. This culture of testing is one of the best things about Rails. But the pressure to test, and test right, can be overwhelming for newcomers.

If you’re coming from a place where you weren’t testing at all, TDD can be especially hard to learn. Think about all the things you need to know in order to do TDD right:

  • You have to know which features you want to implement
  • You have to be able to identify which parts of each feature should be tested
  • You have to learn a testing tool
  • You have to know how to write some tests for each of those parts
  • You have to know how to write a test without seeing the code first
  • You have to know how to write the ‘simplest thing that could possibly work’
  • You have to learn how to use your tests to grow the design of the code

When it’s written out like that, I’d be surprised if you didn’t struggle. It’s clear that you can’t really go from zero to TDD in a single step. So, what do you do? How can you learn TDD without being overwhelmed?

Break it apart!

When you feel like you’re facing an impossible task or you don’t know what to do next, see if there’s something easier you could learn that would get you closer to where you want to be.

For example, to learn TDD with RSpec, you could break it apart like this:

  • Write some exploratory code for a feature without tests. (So you can get some experience coming up with and writing features)

  • Study some code and write down some ways that the code might break on different kinds of input. (So you can learn about what to test and how to test)

  • Write tests for code that already exists, using the minitest built into Ruby (So you can learn minitest and basic testing, without having to learn TDD and RSpec, or fight with installing gems and getting your RSpec environment set up)

  • Test-drive some code using TDD and minitest (So you can focus on TDD without having to deal with gem dependencies or learning RSpec. Since you already know how to write tests, you just have to learn to write them first)

  • Test-drive some code using RSpec (once you’ve already practiced a bit with TDD, so you can focus on getting RSpec set up and learn its API and culture)

Each of these is a much smaller skill that you can practice separately. You can then focus on and practice the first step, then the next, and so on. Instead of jumping straight to TDD with RSpec, you can grow toward it.

It seems like a lot more work. But you’ll learn each of these skills faster than trying to learn TDD from scratch, and you’ll have a better foundation to build on when the next helpful testing style comes along.

Focus on one thing at a time

The key is to focus and practice on the thing just beyond your comfort zone, without trying to leap straight to the last step. Each of these steps has some value in itself. Even if you stopped after the first step, you’d still learn something valuable. But they also build on one another so you can focus on learning one thing really well, instead of having a bunch of things all thrown at you at once. You can learn without getting overwhelmed and giving up.

As you go through this process, give yourself permission to mess up. You’re learning, it’s totally OK! It’ll get a lot easier as you do it more often. Practice consistently and thoughtfully, and keep reaching one level beyond what you know already, and you’ll get there.


Using a little bit of convenience to save your programming flow

While you’re building software, there will be things that will frustrate you every time you have to write them. Bits of code that look ugly. Or those lines you can never quite remember how to write, so you find an example somewhere else in your codebase to copy and paste. These break your flow! So it’s important to recognize them, and make them more convenient.

For example, you’ll sometimes need custom configuration in your Rails app. Maybe you want some extra app-specific settings. Or maybe the libraries you’re using don’t want to look in config/ for their own configuration files. Normally, you’d do the YAML-ERB-read-Rails.env dance:

1
  YAML.load(ERB.new(File.read(Rails.root.join('config', 'google_analytics.yml'))).result)[Rails.env]

But that’s kind of ridiculous. And sometimes you’ll forget the ERB, and things’ll break when you’re not expecting it.

In our codebase, we have a simple Settings class to make this more convenient:

1
2
3
4
require 'settings'

GoogleAnalyticsSettings = Settings.new(:google_analytics)
GoogleAnalyticsSettings.google_analytics_id # => "UA-XXXXXXX-1"

Settings will automatically find the correct configuration file in the config/ directory. Then, it pipes it through YAML and ERB, and uses Rails.env to grab the configuration for the Rails environment. Finally, it wraps everything in an OpenStruct to make accessing the top-level configuration a little nicer. Here’s a gist of a basic Settings implementation.

It’s really simple. But it’s convenient. It’s much easier to remember Settings.new than all that file loading stuff. And these little conveniences add up, and will make your codebase so much more fun to work in.

In your code, can you think of something that annoys you, breaks your flow, or where you have to look up an example every time? Can you find a way to make it easier for you to use?


When is an ActiveRecord model too fat?

When you read Rails blogs and books, or watch conference talks, you’ll learn a lot about making your models skinnier.

These techniques are awesome, because your models will get too big or complex to handle. But do you really want to go as far as having your models only responsible for persistence, associations, and validations? How do you decide how much logic should stay in your ActiveRecord models, anyway?

Skinny. But not too skinny.

Active Record is a pattern that works best when your models closely match your database schema. That’s what it’s designed for! But what does that mean?

  • If some code does work from the point of view of an ActiveRecord model, it can go into the model.

  • If some code does work that spans multiple tables/objects, and doesn’t really have a clear owner, it could go into a Service Object.

  • Anything that’s attribute-like (like attributes calculated from associations or other attributes) should go into your ActiveRecord model.

  • If you have logic that has to orchestrate the saving or updating of multiple models at once, it should go into an ActiveModel Form Object.

  • If code is mostly meant for displaying or formatting models in a view, it should go into a Rails helper or a Presenter.

Beyond those guidelines, you should use the same rules you’d use to refactor any class that’s getting too big. But generally, you shouldn’t feel bad about leaving some logic in your ActiveRecord models. They’re meant to have it!

So, since Rails chose the Active Record pattern, it makes sense to have some logic in your models. But why did Rails pick that pattern instead of something cleaner?

What Would Ruby Do?

In Patterns of Enterprise Application Architecture, the Active Record pattern holds the middle ground between Row Data Gateway and Data Mapper. Row Data Gateway is a mostly dumb object-oriented wrapper around table rows, like the skinniest of all possible models. Data Mappers are more complex than Active Records, and are mostly used to convert between objects containing nothing but business logic and objects containing nothing but persistence logic.

So, in the context of Ruby, Active Record is absolutely the correct default pattern.

Why do I say that?

Ruby is designed to make programmers happy. When it’s forced to make tradeoffs between cleanliness and convenience, it almost always chooses convenience.

I mean, Array has over a hundred methods in its public API. It has a ton of methods that are just aliases of one another. Because some developers just prefer writing .detect to .find.

In that context, it makes a lot of sense that Rails would default to a pattern that’s convenient over one that’s more flexible, or has more object-oriented purity. It’s the Ruby way. And I love it.

You can always refactor to something more flexible when you need to. But then again, YAGNI.


How to learn Ruby while you walk the dog

There’s never enough time to learn everything you want. And it always seems like there’s too much other stuff taking up the time you do have. This makes it hard to keep up with your Ruby knowledge.

Books and screencasts are great, but they take time and focus. And doing the dishes, commuting to work, and walking the dog can get boring. It’s nice to have those moments to think or relax. But most of the time, I want to take advantage of those minutes of boredom and learn something!

For this, there’s nothing better than podcasts. Instead of playing iOS games or checking my email for the dozenth time, I can learn something new about my favorite language.

Ruby news podcasts like Ruby5 (and the sadly abandoned The Ruby Show) are great for keeping up with new gem releases and good blog posts. But the podcasts that go in depth on a single topic are even better. Here are a few of my favorites:

Ruby Rogues

I’ve listened to the Ruby Rogues Podcast since the first episode. (I mean, with panelists like Aaron Patterson, David Brady, James Edward Gray II, and Peter Cooper, how could I not?)

They cover a lot, and I’m not interested in every episode. But listening to this podcast taught me so much. Practical stuff, too. Things I learned from the Square episode, for example, really influnced our Service Oriented Architecture. The time you spend listening will pay off.

The regular panelists are some of the Rubyists I admire most, the picks are awesome, there are transcripts of each episode (which makes looking things up later really easy), and Parley is one of the best-kept secrets in the Ruby community. It’s just a great, well-put-together show.

The Ruby on Rails Podcast

The first Rails podcast I ever listened to was just relaunched (after almost 5 years!), and it’s off to a strong start.

So far, it’s been full of great guests and interesting stories, and it’s been a lot of fun. I just started listening to it, and I’m really interested in seeing how it grows from here.

So, two good podcasts. But that’s not all the knowledge you can squeeze out of the time you have.

Another way to get your Ruby knowledge fill

There’s so much good stuff locked away in conference talks, and not enough time to watch all the videos. But it turns out that a lot of conference talks don’t actually need video! And with the right tools, you can treat conference videos like a podcast, too.

To listen to conference talks, I use an iOS app called Swift Player. With Swift, I can download, speed up, and listen to these talks while I commute to and from work. These talks have taught me a lot, so I recommend trying this (or something like it)!

If you don’t want to use a dedicated app for this, you can download the videos, extract the audio track and play it using your phone’s music player. But that’s a little too much work for me.

Give them a listen!

Now you have a few ways to learn a little Ruby during a few of the spare minutes you can scrape together. So, which ones sound interesting to you? And which of your favorites did I miss?


Untangle spaghetti code with a simple change of perspective

That giant mess of if statements keeps staring you in the face. You feel like you should be able to simplify it, except for that Business Logic that keeps getting in the way.

For example, say you have a sales platform where you build Quotes, which have many LineItems. Except, you can have a quote with duplicate line items if they’re ads, but if you have multiple websites, you have to sum the prices together and have it show up as a single line item. Oh and also, if you buy a website and already have five ads in your quote, you have to give them a 20% discount on the website.

I can hear you throwing your laptop through the window from all the way over here.

You could write a bunch of if statements to handle these rules:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class Quote
  attr_accessor :line_items

  ...

  def add_line_item(line_item)
    if line_item.kind_of?(Ad)
      self.line_items << line_item
    elsif line_item.kind_of?(Website)
      if @line_items.select {|item| item.kind_of?(Ad) }.length >= 5
        # TODO: Put the fractions of a cent into a bank account
        # I have set up
        line_item.price *= 0.8
      end
      existing_website = self.line_items.detect { |item| item.kind_of?(Website) }
      if existing_website
        existing_website.price += line_item.price
      else
        self.line_items << line_item
      end
    end
  end
end

But I think we can agree that that’s just terrible. How can you possibly untangle something like that?

You could decompose the method into a bunch of smaller methods, but that’s like shoving all your toys in your closet so your mom thinks you cleaned your room. And those kind_of?s would still bother me a lot.

But what if you started seeing things from the line item’s perspective, instead of the quote’s? If instead of asking what kind of line item you’re dealing with and adding it to the quote, you just told the line item to add itself to the quote?

Reverse your methods!

One of my favorite ways to refactor code is to try reversing the caller and the callee. Here’s an example, using the code above:

app/models/quote.rb
1
2
3
4
5
6
class Quote
  ...
  def add_line_item(line_item)
    line_item.add_to_quote(self)
  end
end
app/models/line_item.rb
1
2
3
4
5
6
class Ad < LineItem
  ...
  def add_to_quote(quote)
    quote.line_items << self
  end
end
app/models/website.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class Website < LineItem
  def add_to_quote(quote)
    if quote.line_items.select {|item| item.kind_of?(Ad) }.length >= 5
      # TODO: Put the fractions of a cent into a bank account
      # I have set up
      self.price *= 0.8
    end
    existing_website = quote.line_items.detect { |item| item.kind_of?(Website) }
    if existing_website
      existing_website.price += self.price
    else
    quote.line_items << self
    end
  end
end

It’s not perfect. website.rb still needs a lot of refactoring help, and I’m not happy with how reversing the methods broke encapsulation of line_items.

But you’ve removed the first layer of complexity. You can now put code on the LineItem or the Quote, depending on where it makes the most sense. The LineItem objects can use inheritance and mixins to handle similarities and differences between each LineItem subclass. Plus, it’s now really easy to add new LineItem subclasses without bloating your add_line_item method.

Your code is a little cleaner, and a lot more flexible. So generally, I’d call it a win.

Where you might not want to use this pattern

As useful as Reversing Method is, there are some reasons you might not want to use this pattern:

  • It can break encapsulation. You might have to expose attributes on the Quote object that you didn’t want to expose publicly.

  • It can increase coupling. Both Quote and Ad now need to know about each other. And depending on how much they need to know about each other, it can make your code more complicated.

  • It can violate the Single Responsibility Principle on Ad, because now Ad has the responsibility of knowing how to add itself to a Quote.

You can usually work around these problems. But you should be aware of them, because you don’t want refactoring to make your code worse!

Why it’s one of my favorites

Even with those problems, this is one of my favorite refactorings. The code I write after using this pattern tends to be clearer and more confident.

But even when it isn’t, using this pattern makes me think about the relationships between my objects in a different way. When I get into the “this feature is awful, I can’t believe I have to write this terrible code to handle it” rut, it kicks my brain into seeing new ways I can solve those problems. It forces me to think about how I could structure my code differently, and that’s incredibly useful.

Give it a try in your own code

Like many of my favorite patterns, I first came across Reversing Method in Smalltalk Best Practice Patterns, and it’s been a valuable tool ever since.

Next time you have a hard time dealing with similar objects that have slightly different behavior, give it a try! If you like the new code better, keep it. Even if you don’t, though, it’ll take your mind down a path that will lead you to better code.


After RailsConf: Should I still learn TDD?

I’ve written a lot about TDD. So, it’s no surprise that when I published my last article at almost the exact minute David Heinemeier Hansson mentioned TDD during his RailsConf keynote, I got some questions:

Do you agree with DHH’s opinions on TDD? Do you still recommend TDD? If not, what should I do instead?

What DHH Said

If you missed the keynote, DHH’s essay captured the gist of it:

Maybe it was necessary to use test-first as the counterintuitive ram for breaking down the industry’s sorry lack of automated, regression testing. Maybe it was a parable that just wasn’t intended to be a literal description of the day-to-day workings of software writing. But whatever it started out as, it was soon since corrupted. Used as a hammer to beat down the nonbelievers, declare them unprofessional and unfit for writing software. A litmus test.

Enough. No more. My name is David, and I do not write software test-first. I refuse to apologize for that any more, much less hide it. I’m grateful for what TDD did to open my eyes to automated regression testing, but I’ve long since moved on from the design dogma.

Yes, test-first is dead to me. But rather than dance on its grave, I’d rather honor its contributions than linger on the travesties. It marked an important phase in our history, yet it’s time to move on.

The whole thing is worth reading, but I broke out some of the main points:

  1. TDD is a mind hack to encourage automated regression testing
  2. Angry test-first rhetoric leads to sadness and despair
  3. TDD leads to overly complex webs of objects and indirection
  4. You should prefer isolated system tests over unit tests
  5. But you shouldn’t turn that preference into another religion
  6. For those reasons, test-first isn’t a design practice you should use anymore.

In the essay, most of these points are tied together. They make sense on their own, but are not as intertwined as they’re made out to be.

If you don’t break them apart, arguing one point assumes other points that you may not actually agree with. When you combine that with twitter outrage and a 140-character limit, you end up with confusion, strawmen, and flamewars.

Instead, I’ll talk about a few of these points separately.

TDD leads to overly complex webs of objects and indirection

Recently, I’ve been seeing a preference in the Ruby testing community for “complex webs of objects and indirection.” I think this is a bad thing. (It’s one of the reasons I originally ran from Java!) But I don’t agree that TDD causes it.

I’ve TDD’d for close to a decade, and I don’t isolate my tests from the database. Controller-level functional tests and a robust integration test suite are also valuable parts of my total test suite, no matter how slowly they run.

Isolating your tests from the system is just optimization. And YAGNI says it’s probably not all that important yet. Besides, I’d rather get across-the-board speed boosts from SSD’s and app preloading instead of trying to optimize each individual test.

That said, TDD still has an effect on your system design.

TDD grows your code organically. This can be a great thing! But sometimes, a skilled software developer can write code that looks better than TDD would have designed it. It may be more tightly coupled with other code, it may not follow all of the OOD rules, but it’s clear, straightforward, and simple. (DHH Ping Pong has some good examples).

TDD can also lock you into an object API design before you really know what you’re building. It makes it harder to change the API later on. And that friction could cause you to settle on a worse design.

Test-first isn’t a design practice you should use anymore

TDD is a great tool for creating flexible, well-tested code. So I totally disagree with this point. I already talked about many benefits of TDD in an earlier article:

  • You’ll find yourself with a more flexible, tested object model.
  • Your system is by definition testable, making future tests less expensive to write.
  • You amortize your testing costs across the development process, making your estimates more accurate.
  • It keeps you in flow, because you never have to decide what to do next.

Besides those benefits, it helps you practice testing. Because you’re always writing your tests first, you’re going to test a lot more than if you tried to test after the fact. You’ll know right away on whether your tests are actually correct (because they’ll fail first). Plus, you don’t have an excuse to skip the tests “this one time.”

Angry test-first rhetoric leads to sadness and despair

Totally, 100% agree. Yes, I’m posting this again. But if someone isn’t TDDing because they don’t know about the benefits, or they haven’t quite learned how to do it yet, or they feel uncomfortable with it, the correct approach isn’t to shame them, it’s to help them. If TDD is useful to you, show them how it helps you. If they’re interested, help them get started.

If they’re not TDDing because they prefer the way the code turns out without TDD, let them.

As you build expertise, you’ll develop your own intuition, your own sense of taste, and your own preferred techniques for developing software. And no internet argument is going to teach an expert to ignore those things.

So, if you’re going to criticize someone for not following your idea of best practices, you’re either hurting their ability to learn or screaming into nothingness. Which is a waste of time in the best case, and hurting your cause in the worst.

So should you keep TDDing?

Yes! I absolutely suggest that you learn and practice it.

I credit TDD for teaching me how to test, and I still get huge benefits when I TDD. It’s not perfect, but no one technique is the key to magically producing better code.

Instead, you have to keep learning new patterns, techniques, and tools. Practice applying them until you can use the right pattern in the right place because it feels right.

As you learn, take some time to modify, refactor, and experiment with your code. Look at your old code and your new code. Get someone more experienced than you to review it. Did you make it better? That’s the way you grow.


How do you get the TDD flywheel moving?

(I’m in Chicago for RailsConf this week. If you see me around, say hi! I’d love to meet you. I’m the older one in the picture in the sidebar.)

A reader asked a great question about testing in a comment on one of my articles:

I know I also need to get the “TDD flywheel” moving, but TDD is very unfamiliar to me. Do you have any recommendations on how best to begin with RSpec or is it just a “get in there and hack away” kind of thing?

So I answered it:

If you’re not settled on one or the other, you might actually find minitest to be easier to get started with. It’s a little more ‘methods and classes’, so you can focus completely on learning and practicing TDD without also having to learn a new syntax.

Rails has a pretty good guide on test terminology, assertions, and how to get tests running in general in Rails.

The best way to learn TDD is to practice. These are the steps you should follow:

  1. Write a test that assumes that the code you need is already there.
  2. Run the tests, make sure your new test fails
  3. Write the simplest bit of code that will pass the test. Don’t overly abstract or refactor here.
  4. Run the tests, make sure your new test passes
  5. Refactor your code to remove duplication (or make the code more expressive).
  6. Run the tests again (to make sure they still pass).
  7. Return to step 1.

While you’re practicing TDD, you should keep asking yourself: Which step am I on? Did I skip a step? Which step is next? (You should always be in one of these steps.) This will help keep you on the right track, which is really important while you’re learning something new. “Practice makes permanent”, so you don’t want to practice the wrong thing!

But also, give yourself permission to mess up. You’re learning, it’s totally OK! It’ll get a lot easier as you do it more often.

Eric Steele, who’s also writing a book about testing, brought up a key point that I missed:

The part of TDD that gets left behind is the amount of planning that happens before step 1.

Tests are where the requirements of the app meet your planned implementation. Test driven development has a hard dependency on knowing what the app should do, and assumes that you have an idea of how to do it.

When we first start building with Rails, we don’t know a lot. We spend a lot of time writing experimental code until we get a result similar to what we’re looking for. You do this less and less as you learn more. I think this is why TDD starts with ‘write a test’ and not ‘figure out what you’re building’. By the time you’re testing, you have plenty of experience under your belt.

I’d add these steps before ‘Write a test’:

  1. Brainstorm, justify, and cull requirements. Avoid coding at all costs.
  2. Plan and design your code that will meet those requirements.
  3. Tinker with code to answer any unknown questions, but set that code aside.

My advice:

  • Focus on planning more than syntax
  • Minimize your tools (I found minitest/fixtures to be really helpful)
  • Practice, practice, practice.

Just like anything, getting started with TDD takes practice and the ability to self-correct (or having someone around to correct you). If you can follow a few semi-rigid steps, and stay mindful about where you are in the process, TDD will soon feel totally natural to you.


How to refactor a long, messy, badly tested controller

At some point in your Rails career, you’ll run into a controller that’ll make you want to give up programming forever. It may contain every line of code for an entire feature. It might have 15 before_filters that all communicate through instance variables and have to be called in a specific order or things blow up. And, inevitably, its tests will look like this:

1
2
3
4
test "index" do
  get :index
  assert_response :success
end

Awesome. 100% test coverage, right?

As great as it would be to just close your eyes and pretend it doesn’t exist, someday you’ll have to fix a bug in one of these controllers. And, being a good software developer, you want to leave the code better than you found it.

But how can you refactor, especially without good tests to rely on?

Get it tested (somehow)

You need good tests to feel safe while refactoring, but you can’t write good tests against this code until you refactor. So what can you do?

Well, no matter how badly written your controller is, you can still write integration tests that send some data to the controller and expect some response out of it. For now, you should write tests that make sure that the controller’s existing behavior doesn’t change as you refactor.

These tests won’t be as focused as unit tests. But these high-level tests can make you can feel comfortable that the refactoring you’re about to do won’t break everything. And the process of writing them will help you understand your code better, which will help you decide how to refactor it.

Break your dependencies

What makes bad controller code bad? Most of the time, it’s the implicit dependencies between before_filters, helper methods, or different parts of 200-line functions. It could also be a case of refactoring too early. To make the code better, you need to break these dependencies.

For Rails controllers, there’s an easy way to break dependencies. Just copy the code from your before_filters, helper methods, superclasses, and anywhere else controller-ish code could be hiding. Then, replace the calls to those methods with the code you copied.

You’re temporarily un-DRYing your code, so you can refactor it in a more understandable way later. It’s ugly, but now all of your code is out in the open. You should be able to see which pieces interact and how everything flows.

(During this process, you should be running your tests after every change to make sure your inlining isn’t breaking anything.)

Refactor toward testable code

Now, you’re ready to re-refactor your code. You’ll probably stick to the basic extract method, extract object, pull up method-type refactorings. Try refactoring your code a few different ways and see what feels best.

During the first few passes, you can rely on your high-level integration tests as a safety net. But soon, you’ll want something better.

As you refactor, look for opportunities to make your code more testable. This will usually mean creating places to inject test doubles, reducing dependencies between your objects, and making your controller rely on objects that can be easily created and unit tested. By moving code into testable objects, your controllers will get smaller, easier to understand, and easier to test themselves.

Can I get it in the form of a numbered list?

Once again, these are the steps to take to break down a large controller:

  1. Get high-level integration tests running against the controller.
  2. Run the tests, make sure they pass.
  3. Inline before_filters, superclass methods, and other abstractions that hide code.
  4. Run the tests, make sure they still pass.
  5. Perform a refactoring (extract method, extract service object, replace instance variable with local, etc.). See if the code feels better.
  6. Run the tests, make sure they still pass.
  7. If you just extracted code, write some unit tests against the object or method you extracted.
  8. Run the tests, make sure they still pass.
  9. If the controller still needs work, go back to step 5.

What’s next?

If you want to learn more, Working Effectively with Legacy Code is the bible of taking unmaintainable code with no tests and turning it into something you can work with. I highly recommend it, if you find yourself facing huge monolithic controllers often (and if refactoring is as much fun to you as it is to me).

So now, let’s hear your horror stories! What has the worst controller code you’ve ever worked on looked like?


A guide to choosing the best gems for your Ruby project

If you need something done in Ruby, a gem for it probably exists. Well, a dozen gems for it probably exist. Some of them are elegant, featureful, and well-maintained, and others were written to solve one use case the author ran into that one time. You have lots of gems to choose from, so how do you choose the right one? This choice is important – by the time you regret adding a gem to your project, it’s painful to change back.

The stats

When I need a library to solve a problem I’m having, I start with the Ruby Toolbox. The Ruby Toolbox is a great way to get a list of all the gems in a category (like “Subscription Management”), along with:

  • How many people use it?

    Popular gems are usually more popular for a reason. And it’ll be much easier to get help if everyone else on the internet is using the same gem.

  • How recently / often is it updated?

    Does the gem get consistent updates, or is that Rails 3.2 compatibility upgrade still sitting in the pull request queue? If the last release was years ago, you might want to skip it – it probably wouldn’t be compatible with your app, anyway.

With the gems you find on the Ruby Toolbox, filtered by those first two criteria, you should have a list of gems you’re actively considering. The Ruby Toolbox also gives you links to the source, the documentation, the website, and the bug tracker, which you’ll need next.

The code & documentation

  • Is the gem well documented?

    Check the website, if it has one. Is the documentation good? Does it have RDoc? Are you going to be able to find the answers you need when you need them?

  • Is it well-tested?

    Take a look at the tests. Does it even have a test/ or spec/ directory? Does it have any tests beyond test_truth? Is the author going to know if they cause a regression?

  • Look at the issues and pull requests

    Do bugs get fixed quickly? Is there discussion on the bugs and pull requests, and do they get fixed? Or do they just kind of sit there?

  • Search for the gem’s name on Google, StackOverflow, and reddit

    What do people say about it? Which gems get recommended, and which ones are discouraged?

  • Search GitHub for a few of the classes provided by the gem

    How are they used in real-world apps? Do they seem easy to use? Or do other developers have to hack around problems in the gem?

At this point you should have your choice narrowed down to a few remaining candidates.

The touchy-feely stuff

These criteria are a little subjective, which is why you have to find the right gem for your project. But if you miss on these, you’ll regret it later.

  • Based on the examples in the documentation, does the library feel Ruby-ish?

    Does it use Ruby idioms? Or do you feel like you’re actually writing Java or PHP? This happens incredibly often to API wrappers.

  • Is it built out of small parts that work well together?

    Can you use the objects the author gave you to build things the author never thought of? Because you’ll probably want to.

  • Is the gem already used by another gem that’s already in your project?

    Bonus, you just saved yourself some integration time, and you know you won’t have to fight with Bundler over versioning.

  • Does the gem make assumptions about your app? Does your app fit those assumptions, and will it always fit them?

    Some gems, like ActiveResource, make assumptions about the API of the server they talk to. Others, like ResourceController, make assumptions about what your controller will do. Gems that make assumptions can make you much faster and your code much simpler–while those assumptions hold. If those assumptions become incompatible with your app, though, you’ll find yourself spending days monkey patching, debugging, and eventually just tearing the gem out.

Whoa, that’s a lot.

Yep, there’s a ton of criteria you can use to pick the best gem for your project. And you usually won’t have to go through them all. Most of the time, you can just hop on the Ruby Toolbox and pick the most popular gem.

But there are some gems that become core parts of your app. Things like HTTP libraries, database libraries, or anything that promises to make your controllers cleaner. And it’s worth taking the extra time making sure that these gems are clean, well-maintained, well-documented, and fit your mental model of how they should work.

I put all these criteria in a Google Spreadsheet, so when you need to pick from an overwhelming number of seemingly similar gems, you can easily compare them. You can grab it here.


Debug a stuck cache with a simple code change

Partial Caching is a great way to get some major page speed improvements without a lot of work. But if you forget a touch: true on one of your associations, or your template dependencies aren’t working right, your cached partials won’t get updated.

Since the development environment usually runs with caching disabled, you’ll only discover this in staging, or even worse, production! To debug the problem, you’ll need to reproduce it in development mode by setting

config/environments/development.rb
1
config.action_controller.perform_caching             = false

in your config/environments/development.rb to true. You’ll have to do this every time you have to debug cache problems, and you’ll have to remember to change it back before you check in. I hate to do that kind of stuff, so when I start a new project, I set it to this instead:

config/environments/development.rb
1
config.action_controller.perform_caching             = ENV['CACHING'] == 'true'

That way, I can enable caching whenever I need to by starting my Rails server with

1
CACHING=true rails server

Running just rails server will run with caching disabled, as usual.

As you start tweaking configuration parameters more often, this pattern of turning hardcoded parameters into environment variables can save you a lot of time and fiddling. Give it a try in your own projects!


How to improve your software design with code that feels wrong

You know what you want to do, but your code just isn’t cooperating. Maybe it has a few too many levels of indentation, or chains a half dozen methods, or looks asymmetrical. Whatever it is, something just feels off. You could ignore it – I mean, you have a backlog full of features you still want to write, and it’s really not that bad. But that would be a mistake: Your code is trying to tell you something, and you don’t want to miss it.

If you can learn to tell when your code feels strange, you will quickly and drastically improve your software design skills. This intuition is hard to build, since it comes from experience, mentorship, and code reviews. But you can get some help from libraries that use syntactic vinegar to make bad code feel bad.

What does syntactic vinegar look like?

Here’s an example of syntactic vinegar using minitest/mock, a small mocking and stubbing library that ships with Ruby:

1
2
3
4
5
6
7
8
9
10
11
require 'minitest/mock'

class CartTest < MiniTest::Test
  def test_error_message_set_on_charge_failure
    cart = Cart.new(items)
    cart.stub(:charge!, false) do
      cart.checkout!
      assert_equal "The credit card could not be charged", cart.credit_card_error
    end
  end
end

When you run the test, the charge! method on Cart is stubbed, so the test won’t hit the payment processor. The block syntax is nice for making sure you only stub exactly when you want to. But what happens when you want to stub a bunch of methods?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
require 'minitest/mock'

class CartTest < MiniTest::Test
  def test_error_message_set_on_charge_failure
    payment_processor = PaymentProcessor.new
    cart = Cart.new(items, processor: payment_processor)

    payment_processor.stub(:charge!, false) do
      payment_processor.stub(:login!, true) do
        payment_processor.stub(:logout!, true) do
          cart.checkout!
          assert_equal "The credit card could not be charged", cart.credit_card_error
        end
      end
    end
  end
end

Ew. That’s a lot of indentation. And that’s just in a single test – you can imagine this code being repeated in a lot of other tests.

You could wrap all this nesting into a test helper method. But if you’re really listening to your code, it’s telling you that you should find a better way. It might be time to look into using a Test Double instead:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
class TestPaymentProcessor < PaymentProcessor
  def login!(account_id, key)
    true
  end

  def charge!(amount, credit_card)
    credit_card.can_be_charged?
  end

  def logout!
    true
  end
end

class CartTest < MiniTest::Test
  def test_error_message_set_on_charge_failure
    test_payment_processor = TestPaymentProcessor.new
    cart = Cart.new(items, processor: test_payment_processor)

    cart.credit_card = failing_credit_card
    cart.checkout!
    assert_equal "The credit card could not be charged", cart.credit_card_error
  end
end

Now your test is more readable. Plus, you have a TestPaymentProcessor that can be used in a lot of other places. You could even use it in development mode, if you don’t want to hit a real server!

Bad code should feel bad

By using opinionated libraries that make bad code obvious, you’ll start to notice bad code much faster and more reliably. And that’ll make your future code cleaner, easier to read, and less painful to work with.

What are your favorite opinionated libraries, and how do they help you find and fix bad code? Let me know in the comments below!


3 ways to start your ambitious Rails project

Last week, I talked about three conventions that can help you beat procrastination and get your new Rails project started. By now you should be less overwhelmed by the work in front of you. But you still have a tough choice to make. Which code do you write first? Authentication? The part that talks to Twilio? And how would you even begin to work on the forum post recommender engine?

Yep, at some point you actually have to write the code, and to do that, you need a place to start.

What makes a good starting point?

First, you should take a few minutes and think about the system you’re going to build. Focus on end-to-end paths and what kind of resources those paths call out for. Don’t go overboard! Once your system becomes a little clearer, you can look for paths that fit into these categories:

1. Is this core to the project?

By core to the project, I mean the path that you’d talk about if someone asked you to describe your app in 30 seconds. If you’re working on forum software, posting and replying to a topic would be core to the project. If you’re working on a push notification service, registering a device and sending a notification to it would be a core interaction.

Core interactions are good to write early, since you can start playing with your app with the least amount of effort up front. You’ll have a good foundation for building new features. And that will make the next thing you work on even easier to start.

2. Can this stand on its own?

You could look for features you can write without learning new libraries or new APIs. If you start here, you can stay in the code without constantly switching to a documentation window. You don’t have to switch between “writing mode” and “learning mode.” By avoiding interruption, you’ll also be avoiding opportunities to procrastinate.

If you’ve ever forced yourself to fix an easy bug first thing in the morning to keep from getting stuck in the email-twitter-reddit-email loop, you know how important it is to start in the right mindset. This is a great way to get in that mindset early, and stay there.

3. Does this seem really risky, hard, or complicated?

These can be dangerous to start on. Complicated features usually need more planning, which means more opportunity to get stuck. So how could this be a good place to start?

Developers crave solving interesting problems. Attacking a hard problem can be the most intrinsically motivating way to get started on a new project. Plus, the extra challenge will give you a better chance of hitting flow, which is where the good stuff really happens.

So, which do you choose?

It doesn’t even really matter which of these approaches you choose! All three are good starting points with side benefits, and will point you in the right direction. So pick one (at random, if you need to), and see how you like it.

Because no matter which approach you pick, the secret is getting the TDD flywheel spinning as early as possible, so the momentum can help carry you through to the day you ship.

Do you use any of these approaches when you start a new project? Or do you have your own favorite place to start a new app? Leave a comment and let me know!


How to beat procrastination on your new Rails project

You’ve just typed rails new on your next project. Now what? Where do you start? Do you write everything in a single app, or go with a Service Oriented Architecture? Is this a good time to learn RSpec? Should you start building out all of your data models, or get a few actions working end-to-end? There are tons of decisions you have to make up front, and it’s easy to feel lost, and procrastinate, and get frustrated, and never finish your app.

This problem is super common. It comes from having to make too many choices up front, when you have the least amount of information you need to make an educated decision.

Convention Over Decision-making

Convention over Configuration is my favorite Rails principle. Before Rails, I’d get stuck figuring out where to put files, what to name database tables, how to map them to objects, and tons of other minor meaningless decisions. I’d spend more time choosing a direction than it would have taken to change my mind if I discovered later on that I was wrong.

Lots of the decisions you face when starting a new project will be like these. But you can replace these decisions if you borrow conventions from books, blogs, or styleguides. And you’ll save yourself a lot of time, stress, and meaningless debate.

Here are some conventions I follow when I start a new app:

  • Start with a few full features.

    Your app doesn’t have to do everything you can think of right away. Start small, and start simple. Figure out the one or two features you can’t live without, and focus all your attention on those first. Build them really well. You can add other things later if you need it. But probably, YAGNI. (And you’ll usually find that what you think you need is totally different than what your customers think you need).

  • Start with a single app.

    I know, you want to avoid building a Monorail. And there are a half dozen RailsConf talks about how SOA is the best way to keep Rails apps maintainable in the long term. Shouldn’t you figure out which parts of your app can be independent, and build a service for each part?

    SOA is a great way to break apart a large, complex app. But it adds a ton of pain, and unless the pain of building, maintaining, and monitoring services outweighs the pain of working on your monolithic Rails app, it’s not worth doing. And if you haven’t started writing your app yet, you’re not feeling that pain. So, start with one app.

  • Start with a stack you’ve used before.

    If you don’t know a stack yet, just use the Omakase stack. But either way, you don’t want to spend half a day integrating a test framework you’ve never used before with Rails. You don’t want to fight version conflicts between VCR and WebMock. You don’t want to have to figure out if your tests crashing is your fault or resque_unit’s. If you want to try a new library out, you can always add it later (or try it on a toy project).

Shipping is hard. If you want to finish your app, you have to scrape for every advantage you can get. So start small, start simple, and start with what you know.

Conventions may not always be perfect, but they’ll help you get started. And starting is the hardest part.

What’s next?

What conventions do you follow on your projects? And how do you decide which features or tests you should start with? I’ll share my thoughts next time, but I’d love to hear some of yours.


I DRY-ed up my code and now it’s hard to work with. What happened?

“Don’t Repeat Yourself” is one of the most valuable ideas from one of the most valuable books I read during my software development career. If you can refactor away duplicate code, you will produce more general, more stable code. When you start DRY-ing up your code, though, you’ll start to run into some problems: code that doesn’t handle edge cases well, code that’s too generalized and hard to read, or code that’s hard to find. If refactoring toward DRYness doesn’t work all the time, how do you know when you should refactor?

Too-DRY code comes from a misunderstanding of what kind of duplication you should try to refactor away. You need to be able to identify the difference between essential duplication and accidental duplication.

Essential duplication is code that solves the class of problems you’re working on. This is the kind of duplication that you should kill right away.

Accidental duplication, though, is duplication that’s unrelated to the problem at hand, “duplication by coincidence.” If you clean up accidental duplication too aggressively, you’ll end up with more brittle code that has to be un-refactored as new cases are added.

But how can you tell the difference? With experience, you might get better at this, but even after decades of programming, I still can’t do this right even half the time. Luckily, there’s a general rule that helps a lot: Three Strikes and You Refactor.

The first time you do something, you just do it.

The second time you do something similar, you wince at the duplication, but you do the duplicate thing anyway.

The third time you do something similar, you refactor.

Can you see why this helps? By the third time, you should start to see where the patterns are. You should have an idea of which duplicate code is necessary for solving your problem, and which code just looks the same coincidentally. You can start to generalize (and DRY up) just the code that’s fundamentally the same across all instances of the problem you’re trying to solve.

Take a moment and think about the last time refactoring caused you pain. Were you trying to DRY up something that was duplicated between two copies of the problem, but the third copy was just a little different?

And the next time you feel like you have to duplicate some code, try waiting until the third copy before you refactor. (It’s really hard and feels terrible, but close your eyes and try it anyway). Then, think about it as you DRY up the code. Are you refactoring differently than you would have if you refactored right after writing the second copy?


5 reasons why you’re not writing tests

Good developers know they should test their code. Too often, though, the tests get skipped, rushed through, or never started. There are some really common traps I’ve seen people fall into, and they’ll kill your motivation to test every time.

1. Should I use RSpec? Cucumber? Capybara? Minitest?

When you start a new project, it’s way too easy to spend a lot of time trying to pick the best tools. This kind of procrastination hides the fact that you don’t know where to start, and until you pick your tools, you can feel productive without actually being productive.

Instead, just pick the stack you know best. If you’re not experienced with any particular test stack, take the default–start with what Rails gives you. You can always add more later. There’s nothing wrong with trying out new tools, but it’s a much better idea to introduce them over time, as you get to know your project and workflow better.

2. Do I start with unit tests? Functional tests? Integration tests?

When you begin a project, you should have an idea of which features or screens should be built first. This is a great place to start! Pick the first thing off your feature list and write a failing integration test for it. This should tell you what kind of controllers and routes you’re missing, which means you need to write some failing functional tests. The controllers need data and logic to do their job, so you’ll write unit tests and your models next. Then, once you get your unit tests passing, the functional tests should pass, and so should the integration test. Now you can move onto the next feature.

If you have a testing process that always has the next step defined, it’s a lot easier to stay motivated. If you don’t have to make decisions, there’s less of an opportunity for procrastination to creep in.

3. I don’t know how to test this network code, command line utility, or rake task!

The easiest thing to do here is move as much code out of the file or class that’s hard to test and into a new object that’s easy to test. That way, the hard-to-test thing just creates and passes parameters to your new object.

Of course, you still have the original thing that’s hard to test. But it’s probably just a few lines of code now, and it should be easier to stub or fake it out.

4. “The project’s almost done, now I just have to write the tests for it!”

Every developer I’ve met craves shipping code. If the tests are the last thing to do before you can ship, you’ll write the minimum number of tests you need to feel kind of confident that the code works, probably. If you get into this habit, you’ll start to see tests as annoying instead of helpful, and it’ll be that much harder to motivate yourself to write them.

One of my favorite things about TDD is that it mixes the testing with the design and the coding, which means you start to see testing as coding, which makes it much more fun (and you get the benefit of having tests a lot earlier).

5. What if I’m doing it wrong?

The Ruby community is known for really pushing code quality, unit testing, and object oriented design principles. This is a great thing! Unfortunately, it means that it’s really common to feel an enormous amount of pressure to ship perfect code with 100% test coverage on your first try.

This makes it really hard to start a project, especially open source, where you know other people will see the code. What will people say if they see that it doesn’t follow all of the SOLID principles? But there are a few things I’ve learned that have helped me deal with this pressure:

  • Every good developer writes code they’re embarassed by later.
  • Good code that ships is infinitely better than perfect code that doesn’t.
  • Some people are just jerks and will make fun of your code. This really sucks. It’s ruined entire weeks of mine. But good developers want to help you instead of criticize you. And I’d bet that if you showed that code to a programming hero of yours, they wouldn’t make fun of it–they’d help you make it better.

What else have you noticed?

Which of these traps have you fallen into? What’s helped you pull yourself out? And are there any that I didn’t mention that you’ve noticed?

If you do recognize yourself in any of these traps, what are you going to do to get yourself out?


Search and filter Rails models without bloating your controller

Searching, sorting, and filtering in Rails controllers can be a pain. ElasticSearch and Solr are great, high-powered solutions, but are really big dependencies for a small app.

Luckily, Rails includes scopes, which can provide you with a lot of what you need for simple searching, filtering, and sorting. If you take advantage of scope chaining, you can build the features you want without taking on big dependencies or writing a bunch of repetitive search code yourself.

Searching with scopes

Imagine an #index method on your RESTful controller that shows a table of products. The products can be active, pending, or inactive, are available in a single location, and have a name.

If you want to be able to filter these products, you can write some scopes:

app/models/product.rb
1
2
3
4
5
class Product < ActiveRecord::Base
  scope :status, -> (status) { where status: status }
  scope :location, -> (location_id) { where location_id: location_id }
  scope :starts_with, -> (name) { where("name like ?", "#{name}%")}
end

Each of these scopes defines a class method on Product that you can use to limit the results you get back.

1
@products = Product.status("active").starts_with("Ruby") # => All active products whose names start with 'Ruby'

Your controller can use these scopes to filter your results:

app/controllers/product_controller.rb
1
2
3
4
5
6
def index
  @products = Product.where(nil) # creates an anonymous scope
  @products = @products.status(params[:status]) if params[:status].present?
  @products = @products.location(params[:location]) if params[:location].present?
  @products = @products.starts_with(params[:starts_with]) if params[:starts_with].present?
end

And now you can show just the active products with names that start with ‘Ruby’.

http://example.com/products?status=active&starts_with=Ruby

Clearly, this needs some cleanup

You can see how this code starts to get unwieldy and repetitive! Of course, you’re using Ruby, so you can stuff this in a loop:

app/controllers/product_controller.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
def index
  @products = Product.where(nil)
  filtering_params(params).each do |key, value|
    @products = @products.public_send(key, value) if value.present?
  end
end

private

# A list of the param names that can be used for filtering the Product list
def filtering_params(params)
  params.slice(:status, :location, :starts_with)
end

A more reusable solution

You can move this code into a module and include it into any model that supports filtering:

app/models/concerns/filterable.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
module Filterable
  extend ActiveSupport::Concern

  module ClassMethods
    def filter(filtering_params)
      results = self.where(nil)
      filtering_params.each do |key, value|
        results = results.public_send(key, value) if value.present?
      end
      results
    end
  end
end
app/models/product.rb
1
2
3
4
class Product
  include Filterable
  ...
end
app/controllers/product_controller.rb
1
2
3
def index
  @products = Product.filter(params.slice(:status, :location, :starts_with))
end

You now have filtering and searching of your models with one line in the controller and one line in the model. How easy is that? You can also get built-in sorting by using the built-in order class method, but it’s probably a better idea to write your own scopes for sorting. That way you can sanity-check the input.

To save you some effort, I put Filterable into a gist. Give it a try in your own project, It’s saved me a lot of time and code.

Do you learn better with video?

You can watch every step, from starting a brand new app to adding searching and filtering, in the companion screencast. We’ll create an app, fill it with sample data, add a search form, and hook it up. And you’ll get the source along with the videos, so you can refer back when you add simple searching and filtering to your own Rails apps.

Learn more about the screencast here!


How much time does rendering a partial really take?

I’ve heard from some people that are worried about breaking apart large Rails views into smaller partials: How much time does rendering a partial really take? Will the performance impact of calling the partials outweigh the benefits in code readability?

I ran some numbers to give an example on how simple partial rendering compares to inline rendering, so we can discuss the tradeoff later. This is the code I used to benchmark, in a brand new Rails app (with config.cache_classes = true):

app/views/test/show.html.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
<% require 'benchmark'
   Benchmark.bmbm do |x|
     x.report "inline" do
       10000.times do -%>
         <p>Hello!</p>
    <% end
     end
     x.report "partial" do
       10000.times do -%>
         <%= render partial: "hello" %>
    <% end
     end-%>
<% end -%>
app/views/test/_hello.html.erb
1
<p>Hello!</p>

And the results (using Ruby 2.1 and Rails 4.0.2, on a 2013 15” Retina Macbook Pro):

1
2
3
4
5
6
7
8
Rehearsal -------------------------------------------
inline    0.010000   0.000000   0.010000 (  0.007045)
partial   0.970000   0.090000   1.060000 (  1.050433)
---------------------------------- total: 1.070000sec

              user     system      total        real
inline    0.010000   0.000000   0.010000 (  0.005529)
partial   0.920000   0.070000   0.990000 (  0.997491)

So, a partial render runs at an average of about 0.1ms on a speedy machine. It’s much slower than rendering inline, but fast enough that it would be hard to notice when you also have things like URL generation, Rails helpers, and browser rendering time to think about.

Speed isn’t everything

If the performance of rendering partials was a lot worse, I’d still probably make the decision to break apart huge views.

There’s a phrase, Make It Work, Make It Right, Make It Fast. When you’re trying to fix a view that’s too big, you’re trying to get out of “Make it Work” mode, and you can’t just skip to “Make it Fast.” You’ll make your code less maintainable, and you could miss some opportunities for refactoring, which could lead to opportunities for caching, which could lead to even bigger performance benefits.

You won’t know until you measure

You don’t have to believe my numbers. I’d recommend that you didn’t! Ruby’s Benchmark library gives you some simple tools to run your own benchmarks. There are also tools for profiling your full stack, like New Relic and MiniProfiler. I use all three of these pretty regularly in my day-to-day work.

When you profile, always profile in production mode! The Rails development environment auto-reloads classes and views, which is useful while developing, but will kill your performance numbers.


How should my Rails app talk to Redis?

While you’re writing your Rails app, you might run into a problem that you could solve more easily with a different data store. For example, you might have a leaderboard that ranks users by the number of points they got for answering questions on your site. With Redis Sorted Sets, a lot of the implementation is done for you. Awesome! But where do you put the code that interacts with Redis?

Your User model could talk to Redis:

1
2
3
4
5
class User < ActiveRecord::Base
  def award_points(points)
    Redis.current.zincrby("leaderboard", points, id)
  end
end

But now your User model holds responsibility for representing a user, talking to Redis, and managing the leaderboard! This makes User harder to understand and harder to test, and that’s the opposite of what you want.

Instead, you could wrap the Redis requests in a new object that represents what you’re using Redis for. For example, you could create a Leaderboard class that wraps the Redis communication, doesn’t inherit from ActiveRecord::Base, but still lives in the app/models directory. This would look like:

app/models/leaderboard.rb
1
2
3
4
5
class Leaderboard
  def award_points_to_user(user_id, points)
    Redis.current.zincrby("leaderboard", points, user_id)
  end
end
app/models/user.rb
1
2
3
4
5
class User < ActiveRecord::Base
  def award_points(leaderboard, points)
    leaderboard.award_points_to_user(id, points)
  end
end

Both of these classes can live in app/models, but now you’re not contaminating your ActiveRecord models with extra logic. The Leaderboard class manages the communication with Redis, so the User class no longer has to care about how leaderboards are implemented. This also makes it easier to test.

You gain a lot by creating new classes for new responsibilities, and app/models is a great place to keep them. You get the benefits of leaning on another service for your feature’s implementation, while also keeping your code easy to work with.

Try it out

Can you think of any network service communication that’s called directly from your ActiveRecord models, controllers, or views? Try moving that code into a non-ActiveRecord data model, and see if your code becomes easier to understand, work with, and test. Then, send me an email and let me know how it went!

Just remember:

All problems in computer science can be solved by another level of indirection.

David Wheeler

…except for the problem of too many layers of indirection.

Kevlin Henney

What happened to my test coverage?

Writing code feels so much easier than writing tests for it, and does that one-line method really need to be tested, anyway? It’s so trivial! Any tests you add would just double or triple development time, and the next time you change the code, you’ll have to change the test, too. It seems like such a waste, especially when you only have a little bit of time left on your estimate.

But soon, your code is only 20% covered by tests, and any change you make to your code feels like trying to replace the middle layer of a house of cards without knocking the whole thing over. Somewhere, something went wrong and even though the decisions you made seemed right at the time, you still ended up with a totally unmaintainable codebase.

How did you get here? You wanted your tests to provide a safety net and allow you to confidently refactor. They should have helped make your code better! Instead, you came back to low test coverage on code you don’t understand anymore, and the tests you have somehow make it harder to change the code.

This isn’t a skill failure. It happens to the best developers. It’s a process failure. With a few changes in how you write new features, you can protect your code without your tests slowing you down. Your tests can make your code more understandable and more flexible. You’ll be able to change your code with confidence, knowing every path through it is tested.

You shouldn’t have to make a decision

If you’re sitting at your keyboard trying to decide whether a bit of code needs to be tested, you’re already going down the wrong path. You should always default to “Test it!” Even if it seems too trivial to need tests, write the test.

If the code is trivial to write, it should be easy to test. And complicated code will never seem as trivial as it does right after you wrote it. How do you know it’ll still seem as trivial six months from now?

But you don’t want to overtest

A gigantic test suite can be its own problem. Tests that take 20 minutes to run are as good as no tests at all, since you won’t run them all the time. (You say you will, but I know you won’t). Even worse, if you have too many brittle tests, it makes refactoring even more of a pain than it was before, so you won’t do it. You’ll end up with methods longer than most novels.

Does this contradict my earlier point? Not necessarily. Your tests should always focus on your code’s interface, not its implementation. For example:

1
2
3
4
5
6
7
8
9
class Cart
  def initialize(item_params)
    @line_items = Array(item_params).map {|item| LineItem.new(item[:name], item[:price])}
  end

  def total
    @line_items.sum(&:price)
  end
end

It really feels like the code here needs to test both the Cart class and the LineItem class. But is the LineItem class used by anything else? If it’s just an implementation detail of Cart, and isn’t exposed to the outside world, how many tests does it really need? Can’t it just be tested through your Cart class?

Classes that are extracted by refactoring often don’t need their own test suite. They’re just an implementation detail. It’s only when they’re used on their own that they need those extra tests.

With a great test suite against a public interface, you have the flexibility to change your implementation without rewriting all your tests. You can do this with a lot less effort than writing even an average test suite against your object’s implementation.

Amortize your test costs with Test-Driven Development

In the first section, you learned that you should test everything. In the second section, you learned that you should only test public interfaces. It’s Test-Driven Development that brings these two opposing goals together.

With Test-Driven Development, your tests drive the design and implementation of your code by following this process:

  1. Write a failing test that assumes that the code you need is already there.
  2. Write the simplest implementation of the code that passes the test.
  3. Refactor to remove duplication (or make the code more expressive).
  4. Run the tests again (to make sure they still pass).
  5. Return to step 1.

By following these steps, you’ll test everything (since no code should be written without a failing test), while only testing public interfaces (since you don’t write new tests right after refactoring).

It’s never exactly that easy. But there are still ways to test-drive even the most complicated code.

TDD has some side benefits:

  • You’ll find yourself with a more flexible, tested object model (this is arguably the main benefit).
  • Your system is by definition testable, making future tests less expensive to write.
  • You amortize your testing costs across the development process, making your estimates more accurate.
  • It keeps you in flow, because you never have to decide what to do next.

So how do I get started?

Starting is the hard part! Once you get in the rhythm of writing test-driven code, it’s hard to stop.

The next time you work on a new feature, try following the TDD steps above. You’ll find yourself with close to 100% code coverage with the least amount of work possible, you’ll have a solid foundation to build on, and you’ll have total confidence that your test suite will protect you when you have to change the code next year. Or later today, when your requirements change again.

When you’re done, send me an email and let me know how it went.

By following a straightforward testing process, you can spend less time making simple decisions and chasing down bugs, and more time writing code that solves your customers’ and business’ needs.


Testing network services in Ruby is easier than you think

You’ve started a new project and it’s time for your code to depend on a third-party service. It could be something like ElasticSearch, Resque, a billing provider, or just an arbitrary HTTP API. You’re a good developer, so you want this code to be well tested. But how do you test code that fires off requests to a service that’s totally out of your control?

You could skip the tests, but you’ll soon be piling more code on a shaky foundation. Untested code tends to attract more complex code to it, and you’ll eventually feel like the code is too dangerous to refactor because you don’t have the test coverage you need to feel safe. You want to build a stable foundation for your future work, but instead you’ve ended up with an unmaintainable mess.

Avoiding this situation is a lot easier than it seems! With a few tools and a little up-front effort, you can decouple your tests from the services your code depends on, write simpler code, and have the confidence to improve the code you’ve written–without introducing bugs. Instead of procrastinating because you don’t know how to approach writing those next tests, you could look at an interaction between your code and the outside world, and know exactly how to jump right into the middle of it.

Mocha: the quick-and-dirty approach

Mocha is the easiest way to get in between your code and the outside world.

As an example, say you have a Cart object that triggers a credit card charge when it gets checked out. You want to make sure that the cart has an error message attached to it if the charge fails.

You probably won’t want your tests to actually hit the billing system every time the tests run. Even if you did, it might be hard to force that service to return a failure. Here’s what it would look like with Mocha:

1
2
3
4
5
6
def test_error_message_set_on_charge_failure
  cart = Cart.new(items)
  cart.stubs(:charge!).returns(false) # mocha in action
  cart.checkout!
  assert_equal "The credit card could not be charged", cart.credit_card_error
end

Mocha can also fail your tests if methods aren’t called the way you expect them to:

1
2
3
4
5
6
def test_only_bill_once_per_cart
  cart = Cart.new(items)
  cart.expects(:charge!).once # Don't double-bill, no matter how many times we check out
  cart.checkout!
  cart.checkout!
end

Mocha is simple to use, but can be incredibly handy. You have to be careful that you’re mocking out only the behavior you don’t want to happen – it’s easy to mock too much and hide real bugs. You also don’t want to go overboard with this approach: tests full of expects and stubs are hard to read and think about.

Test fakes: my preferred approach

If you mock or stub the same methods on the same objects all the time, you can promote your mocks to full-fledged objects (sometimes called test fakes), like this:

1
2
3
4
5
6
7
def test_billed_full_amount_minus_discount
  test_payment_provider = TestPaymentProvider.new # A fake payment provider
  cart = Cart.new(items, discount: 30, provider: test_payment_provider)
  cart.checkout!

  assert_equal items.sum(:&price) * 0.7, test_payment_provider.total_charges
end

Fakes are great:

  • Your fake can keep track of its internal state

    The fake can have custom assertion messages and helper functions that make writing your tests easier, like the total_charges method in the example above.

  • As a full-fledged object, you get extra editor and language support

    If you’re using an editor that supports it, you can get autocomplete, inline documentation, and other things you won’t get by stubbing out individual methods with Mocha. You’ll also get better validations, exception handling, and whatever else you want to build into your fake.

  • If you use a fake in development mode, you don’t have to have a connection to the real service

    You can write your app on the bus, you don’t have to have a forest of services running down your laptop’s battery, and you can set these fake services up to return the data you need to work through edge cases without needing a lot of setup.

  • These objects can be used outside of your tests

    This is probably my favorite part of fakes. You can have a logging client log to both a 3rd party service and your fake, backed by an in-memory array. You could then dump the contents of this array in an admin view on your site, making it much easier to verify that you’re logging what you think you’re logging.

You could do something like this:

1
2
3
4
  fake_backend = FakeBackend.new
  LoggingService.backends = [RealBackend.new, fake_backend]
  LoggingService.debug("TEST MESSAGE PLEASE IGNORE")
  fake_backend.messages.first # => [:debug, "TEST MESSAGE PLEASE IGNORE"]

Writing a fake takes more effort than stubbing out individual methods, but with practice it shouldn’t take more than an hour or two to get a helpful fake built. If you build one that would be useful to other people, share it! I built resque-unit a long time ago, and lots of people still use it today.

How do I get these objects injected, anyway?

You’ll have to get your objects under test to talk to these fakes somehow. Luckily, Ruby is so easy to abuse that injecting fakes usually isn’t hard.

If you control the API of the object under test, it’s best to add a default parameter, an attribute, or a constructor option where you can set your fake:

1
2
3
4
5
6
class Card
  attr_reader :provider
  def initialize(items, options={})
    @provider = options.fetch(:provider) { RealProvider.new }
  end
end

This is clean when you are talking to the real service and gives you a hook to add flexibility later.

If you don’t control the object or don’t want to add the extra parameter, you can always monkey patch:

1
2
3
4
5
6
# if in test mode
Card.class_eval do
  def provider
    @provider ||= TestProvider.new
  end
end

It’s uglier in test, but cleaner in environments that don’t use the fake.

Start building your own fake right now

Building fakes gets easier with practice, so you should give it a try now:

  • Find a test that talks to an external service. Tests that would fail if you disconnected from the internet are good candidates.
  • Figure out what object actually does the communication, and what calls your code makes to that object.
  • Create a mostly empty duplicate of that object’s class, and have it log the calls you make to an array.
  • Add a method to your fake to return the list of calls made.
  • Swap out the real object with your new fake object, and write some assertions against the calls your code makes.

If you give it a try, let me know how it goes!

With these techniques, it won’t be long until you’re able to tame the craziest interactions between your application and the outside world. A simple stub in the right place will let you ship your well-tested code with confidence.


Should I Upgrade to Ruby 2.1?

I’ll answer that with a screenshot:

We upgraded from Ruby 1.9 to 2.1 in production about a week ago, and this is what we saw. The gray section at the bottom is our garbage collection time going to almost nothing. Overall, the upgrade gave us a 15-20% improvement in server response time. That’s without doing any additional GC tuning!

We saw a big boost in Google’s and Bing’s crawl rates after shipping the upgrade, which is especially awesome for sites like ours that benefit heavily from search engine traffic.

Cool, so how hard was it?

On a large Rails app with close to 150 gem dependencies, the upgrade took a single small commit and a few days of testing. We run rvm on all of our servers and provision with chef, so upgrading all our servers to 2.1 was as simple as changing a version number in a chef role file.

What problems did we run into?

We still use iconv for transliteration, so we had to include the iconv gem, which replaces built-in functionality that was removed from 2.x. I was never able to get ruby-debug or the debugger gem working under 2.x, so we switched to pry and pry-byebug. I still prefer the debugger gem to pry-byebug, but pry is so much better than irb that it’s kind of a wash. Deivid Rodriguez, byebug’s author, pointed out that my problems with pry-byebug were probably not caused by byebug. He was totally right! (it was actually caused by the interaction between byebug and pry).

In a mixed 1.9 and 2.1 environment, you’ll also have to upgrade bundler to 1.5.1. With the new version of bundler, you can specify 2.1 as a platform requirement in your Gemfile:

1
gem 'iconv', '~> 1.0.2', :platform => [:mri_20, :mri_21]

It sounds like GitHub ran into some other problems with 2.1, mentioned by Aman Gupta in this gist: https://gist.github.com/tmm1/8393897, but we haven’t noticed any issues yet.

You’ll also want to tune the GC after you ship to get even more speed. Aman has a good post with lots of details on how to do that: http://tmm1.net/ruby21-rgengc/. The entire 2.1 series on his blog is worth reading if you want to get the most benefit from your cheap, simple Ruby upgrade.


Estimates Are Not a Goal, They’re a Communication Tool

“How long would it take to do this?”

“How hard would it be to make this change?”

If you’re starting to get angry and frustrated, you’re probably a software developer. Estimation is painful, especially when the person asking you to estimate a task (Let’s call them ‘Bob’) just has a rough idea of what they’re asking for. A lot of this pain is caused by a difference in what Bob wants and what you think Bob wants.

Estimates are not a goal

As a developer, I think of an estimate as “How long, on average, will it take me to accomplish a task?” I’m frustrated because I usually don’t have enough information to answer that question, and I know I’m going to be held to what I say. I assume that my ability to hit this estimate will be an indicator of my overall performance, so I get stressed and annoyed.

They’re a communication tool

Any product that ships has things that depend on it. Depending on how large the company is, it could be marketing, publicity, business analysis, future product planning, or lots of other things. This means that the answer Bob is looking for is “When can I start assuming this thing will be done, so I can start planning things that depend on it?”

I’m answering with “This should be done by…”, but Bob is looking for a “This will be done by…”.

David Bryant Copeland has a great book, The Senior Software Engineer, which talks about this in its first chapter:

A senior software engineer is trusted and expected to do the most important work and to do it reliably. The way to accomplish that is to focus everything you do on the delivery of results.

Keeping that in mind, it’s best to meet Bob on his terms. This means that:

  • Estimates should be made at 90% confidence, not 50%

    Bob wants to know when he can start planning things that depend on the work being done. If this means that the estimate needs to be padded (it probably does), that’s fine. If this means you have to get back him in an hour or two, that’s fine too.

  • Estimates should be based on time-to-ship, not time-to-test-handoff or time-to-deploy-queue or anything else

    This goes back to the focus on the delivery of results. If Bob is waiting for a project to ship, it does no good to him if it’s stuck in QA. This usually means the estimates need to be padded again, which is fine.

  • The best estimate is often “I can’t answer that now, but I’ll get back to you.”

    It’s much easier to get to a 90%-confidence time-to-ship estimate if the estimate is broken up into smaller parts, each of which is easier to understand. Many books discuss the tactics of making accurate estimates, but it’s out of the scope of this post.

What if I’m told that I’m not moving fast enough?

Everywhere I’ve worked, I’ve been told that management wishes we could move faster. No matter how fast products are delivered, even if quality is completely sacrificed, even if the estimates are absolutely insane, even if you work 20 hours a day, management will still wish you could move faster. It’s the job of a developer to see that it’s just a wish and do their job of keeping a fast but maintainable pace, being predictable, delivering results, and keeping a high bar of quality.

What about when I beat my estimates by a hilarious amount and it looks like my estimates are incredibly padded?

I’ve never seen someone who regulary overestimates get punished for it, and it’ll pay off the next time you have a project that goes much longer than you expected. Overestimation by a given amount is much, much better than underestimation by the same amount (I think it’s probably similar to Loss Aversion). Most of the time, there will be a backlog and you can start the next task earlier. Great!

If you targeted the 50% case, unexpected complications could demolish an entire product schedule. You’d do that about half the time! If you target the 90% case, even if things go badly, the rest of the schedule can operate as intended. The product ships on time, everyone’s happy, you get raises and promotions and the best reward of all: You get to make more estimates!