(I’m in Chicago for RailsConf this week. If you see me around, say hi! I’d love to meet you. I’m the older one in the picture in the sidebar.)
A reader asked a great question about testing in a comment on one of my articles:
I know I also need to get the “TDD flywheel” moving, but TDD is very unfamiliar to me. Do you have any recommendations on how best to begin with RSpec or is it just a “get in there and hack away” kind of thing?
So I answered it:
If you’re not settled on one or the other, you might actually find minitest to be easier to get started with. It’s a little more ‘methods and classes’, so you can focus completely on learning and practicing TDD without also having to learn a new syntax.
Rails has a pretty good guide on test terminology, assertions, and how to get tests running in general in Rails.
The best way to learn TDD is to practice. These are the steps you should follow:
- Write a test that assumes that the code you need is already there.
- Run the tests, make sure your new test fails
- Write the simplest bit of code that will pass the test. Don’t overly abstract or refactor here.
- Run the tests, make sure your new test passes
- Refactor your code to remove duplication (or make the code more expressive).
- Run the tests again (to make sure they still pass).
- Return to step 1.
While you’re practicing TDD, you should keep asking yourself: Which step am I on? Did I skip a step? Which step is next? (You should always be in one of these steps.) This will help keep you on the right track, which is really important while you’re learning something new. “Practice makes permanent”, so you don’t want to practice the wrong thing!
But also, give yourself permission to mess up. You’re learning, it’s totally OK! It’ll get a lot easier as you do it more often.
The part of TDD that gets left behind is the amount of planning that happens before step 1.
Tests are where the requirements of the app meet your planned implementation. Test driven development has a hard dependency on knowing what the app should do, and assumes that you have an idea of how to do it.
When we first start building with Rails, we don’t know a lot. We spend a lot of time writing experimental code until we get a result similar to what we’re looking for. You do this less and less as you learn more. I think this is why TDD starts with ‘write a test’ and not ‘figure out what you’re building’. By the time you’re testing, you have plenty of experience under your belt.
I’d add these steps before ‘Write a test’:
- Brainstorm, justify, and cull requirements. Avoid coding at all costs.
- Plan and design your code that will meet those requirements.
- Tinker with code to answer any unknown questions, but set that code aside.
- Focus on planning more than syntax
- Minimize your tools (I found minitest/fixtures to be really helpful)
- Practice, practice, practice.
Just like anything, getting started with TDD takes practice and the ability to self-correct (or having someone around to correct you). If you can follow a few semi-rigid steps, and stay mindful about where you are in the process, TDD will soon feel totally natural to you.
At some point in your Rails career, you’ll run into a controller that’ll make you want to give up programming forever. It may contain every line of code for an entire feature. It might have 15
before_filters that all communicate through instance variables and have to be called in a specific order or things blow up. And, inevitably, its tests will look like this:
1 2 3 4
Awesome. 100% test coverage, right?
As great as it would be to just close your eyes and pretend it doesn’t exist, someday you’ll have to fix a bug in one of these controllers. And, being a good software developer, you want to leave the code better than you found it.
But how can you refactor, especially without good tests to rely on?
Get it tested (somehow)
You need good tests to feel safe while refactoring, but you can’t write good tests against this code until you refactor. So what can you do?
Well, no matter how badly written your controller is, you can still write integration tests that send some data to the controller and expect some response out of it. For now, you should write tests that make sure that the controller’s existing behavior doesn’t change as you refactor.
These tests won’t be as focused as unit tests. But these high-level tests can make you can feel comfortable that the refactoring you’re about to do won’t break everything. And the process of writing them will help you understand your code better, which will help you decide how to refactor it.
Break your dependencies
What makes bad controller code bad? Most of the time, it’s the implicit dependencies between
before_filters, helper methods, or different parts of 200-line functions. It could also be a case of refactoring too early. To make the code better, you need to break these dependencies.
For Rails controllers, there’s an easy way to break dependencies. Just copy the code from your
before_filters, helper methods, superclasses, and anywhere else controller-ish code could be hiding. Then, replace the calls to those methods with the code you copied.
You’re temporarily un-DRYing your code, so you can refactor it in a more understandable way later. It’s ugly, but now all of your code is out in the open. You should be able to see which pieces interact and how everything flows.
(During this process, you should be running your tests after every change to make sure your inlining isn’t breaking anything.)
Refactor toward testable code
Now, you’re ready to re-refactor your code. You’ll probably stick to the basic extract method, extract object, pull up method-type refactorings. Try refactoring your code a few different ways and see what feels best.
During the first few passes, you can rely on your high-level integration tests as a safety net. But soon, you’ll want something better.
As you refactor, look for opportunities to make your code more testable. This will usually mean creating places to inject test doubles, reducing dependencies between your objects, and making your controller rely on objects that can be easily created and unit tested. By moving code into testable objects, your controllers will get smaller, easier to understand, and easier to test themselves.
Can I get it in the form of a numbered list?
Once again, these are the steps to take to break down a large controller:
- Get high-level integration tests running against the controller.
- Run the tests, make sure they pass.
before_filters, superclass methods, and other abstractions that hide code.
- Run the tests, make sure they still pass.
- Perform a refactoring (extract method, extract service object, replace instance variable with local, etc.). See if the code feels better.
- Run the tests, make sure they still pass.
- If you just extracted code, write some unit tests against the object or method you extracted.
- Run the tests, make sure they still pass.
- If the controller still needs work, go back to step 5.
If you want to learn more, Working Effectively with Legacy Code is the bible of taking unmaintainable code with no tests and turning it into something you can work with. I highly recommend it, if you find yourself facing huge monolithic controllers often (and if refactoring is as much fun to you as it is to me).
So now, let’s hear your horror stories! What has the worst controller code you’ve ever worked on looked like?
If you need something done in Ruby, a gem for it probably exists. Well, a dozen gems for it probably exist. Some of them are elegant, featureful, and well-maintained, and others were written to solve one use case the author ran into that one time. You have lots of gems to choose from, so how do you choose the right one? This choice is important — by the time you regret adding a gem to your project, it’s painful to change back.
When I need a library to solve a problem I’m having, I start with the Ruby Toolbox. The Ruby Toolbox is a great way to get a list of all the gems in a category (like “Subscription Management”), along with:
How many people use it?
Popular gems are usually more popular for a reason. And it’ll be much easier to get help if everyone else on the internet is using the same gem.
How recently / often is it updated?
Does the gem get consistent updates, or is that Rails 3.2 compatibility upgrade still sitting in the pull request queue? If the last release was years ago, you might want to skip it — it probably wouldn’t be compatible with your app, anyway.
With the gems you find on the Ruby Toolbox, filtered by those first two criteria, you should have a list of gems you’re actively considering. The Ruby Toolbox also gives you links to the source, the documentation, the website, and the bug tracker, which you’ll need next.
The code & documentation
Is the gem well documented?
Check the website, if it has one. Is the documentation good? Does it have RDoc? Are you going to be able to find the answers you need when you need them?
Is it well-tested?
Take a look at the tests. Does it even have a
spec/directory? Does it have any tests beyond
test_truth? Is the author going to know if they cause a regression?
Look at the issues and pull requests
Do bugs get fixed quickly? Is there discussion on the bugs and pull requests, and do they get fixed? Or do they just kind of sit there?
Search for the gem’s name on Google, StackOverflow, and reddit
What do people say about it? Which gems get recommended, and which ones are discouraged?
Search GitHub for a few of the classes provided by the gem
How are they used in real-world apps? Do they seem easy to use? Or do other developers have to hack around problems in the gem?
At this point you should have your choice narrowed down to a few remaining candidates.
The touchy-feely stuff
These criteria are a little subjective, which is why you have to find the right gem for your project. But if you miss on these, you’ll regret it later.
Based on the examples in the documentation, does the library feel Ruby-ish?
Does it use Ruby idioms? Or do you feel like you’re actually writing Java or PHP? This happens incredibly often to API wrappers.
Is it built out of small parts that work well together?
Can you use the objects the author gave you to build things the author never thought of? Because you’ll probably want to.
Is the gem already used by another gem that’s already in your project?
Bonus, you just saved yourself some integration time, and you know you won’t have to fight with Bundler over versioning.
Does the gem make assumptions about your app? Does your app fit those assumptions, and will it always fit them?
Some gems, like ActiveResource, make assumptions about the API of the server they talk to. Others, like ResourceController, make assumptions about what your controller will do. Gems that make assumptions can make you much faster and your code much simpler—while those assumptions hold. If those assumptions become incompatible with your app, though, you’ll find yourself spending days monkey patching, debugging, and eventually just tearing the gem out.
Whoa, that’s a lot.
Yep, there’s a ton of criteria you can use to pick the best gem for your project. And you usually won’t have to go through them all. Most of the time, you can just hop on the Ruby Toolbox and pick the most popular gem.
But there are some gems that become core parts of your app. Things like HTTP libraries, database libraries, or anything that promises to make your controllers cleaner. And it’s worth taking the extra time making sure that these gems are clean, well-maintained, well-documented, and fit your mental model of how they should work.
I put all these criteria in a Google Spreadsheet, so when you need to pick from an overwhelming number of seemingly similar gems, you can easily compare them. You can grab it here.
Partial Caching is a great way to get some major page speed improvements without a lot of work. But if you forget a
touch: true on one of your associations, or your template dependencies aren’t working right, your cached partials won’t get updated.
Since the development environment usually runs with caching disabled, you’ll only discover this in staging, or even worse, production! To debug the problem, you’ll need to reproduce it in development mode by setting
true. You’ll have to do this every time you have to debug cache problems, and you’ll have to remember to change it back before you check in. I hate to do that kind of stuff, so when I start a new project, I set it to this instead:
That way, I can enable caching whenever I need to by starting my Rails server with
rails server will run with caching disabled, as usual.
As you start tweaking configuration parameters more often, this pattern of turning hardcoded parameters into environment variables can save you a lot of time and fiddling. Give it a try in your own projects!
You know what you want to do, but your code just isn’t cooperating. Maybe it has a few too many levels of indentation, or chains a half dozen methods, or looks asymmetrical. Whatever it is, something just feels off. You could ignore it — I mean, you have a backlog full of features you still want to write, and it’s really not that bad. But that would be a mistake: Your code is trying to tell you something, and you don’t want to miss it.
If you can learn to tell when your code feels strange, you will quickly and drastically improve your software design skills. This intuition is hard to build, since it comes from experience, mentorship, and code reviews. But you can get some help from libraries that use syntactic vinegar to make bad code feel bad.
What does syntactic vinegar look like?
Here’s an example of syntactic vinegar using minitest/mock, a small mocking and stubbing library that ships with Ruby:
1 2 3 4 5 6 7 8 9 10 11
When you run the test, the
charge! method on
Cart is stubbed, so the test won’t hit the payment processor. The block syntax is nice for making sure you only stub exactly when you want to. But what happens when you want to stub a bunch of methods?
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Ew. That’s a lot of indentation. And that’s just in a single test — you can imagine this code being repeated in a lot of other tests.
You could wrap all this nesting into a test helper method. But if you’re really listening to your code, it’s telling you that you should find a better way. It might be time to look into using a Test Double instead:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Now your test is more readable. Plus, you have a
TestPaymentProcessor that can be used in a lot of other places. You could even use it in development mode, if you don’t want to hit a real server!
Bad code should feel bad
By using opinionated libraries that make bad code obvious, you’ll start to notice bad code much faster and more reliably. And that’ll make your future code cleaner, easier to read, and less painful to work with.
What are your favorite opinionated libraries, and how do they help you find and fix bad code? Let me know in the comments below!
Last week, I talked about three conventions that can help you beat procrastination and get your new Rails project started. By now you should be less overwhelmed by the work in front of you. But you still have a tough choice to make. Which code do you write first? Authentication? The part that talks to Twilio? And how would you even begin to work on the forum post recommender engine?
Yep, at some point you actually have to write the code, and to do that, you need a place to start.
What makes a good starting point?
First, you should take a few minutes and think about the system you’re going to build. Focus on end-to-end paths and what kind of resources those paths call out for. Don’t go overboard! Once your system becomes a little clearer, you can look for paths that fit into these categories:
1. Is this core to the project?
By core to the project, I mean the path that you’d talk about if someone asked you to describe your app in 30 seconds. If you’re working on forum software, posting and replying to a topic would be core to the project. If you’re working on a push notification service, registering a device and sending a notification to it would be a core interaction.
Core interactions are good to write early, since you can start playing with your app with the least amount of effort up front. You’ll have a good foundation for building new features. And that will make the next thing you work on even easier to start.
2. Can this stand on its own?
You could look for features you can write without learning new libraries or new APIs. If you start here, you can stay in the code without constantly switching to a documentation window. You don’t have to switch between “writing mode” and “learning mode.” By avoiding interruption, you’ll also be avoiding opportunities to procrastinate.
If you’ve ever forced yourself to fix an easy bug first thing in the morning to keep from getting stuck in the email-twitter-reddit-email loop, you know how important it is to start in the right mindset. This is a great way to get in that mindset early, and stay there.
3. Does this seem really risky, hard, or complicated?
These can be dangerous to start on. Complicated features usually need more planning, which means more opportunity to get stuck. So how could this be a good place to start?
Developers crave solving interesting problems. Attacking a hard problem can be the most intrinsically motivating way to get started on a new project. Plus, the extra challenge will give you a better chance of hitting flow, which is where the good stuff really happens.
So, which do you choose?
It doesn’t even really matter which of these approaches you choose! All three are good starting points with side benefits, and will point you in the right direction. So pick one (at random, if you need to), and see how you like it.
Because no matter which approach you pick, the secret is getting the TDD flywheel spinning as early as possible, so the momentum can help carry you through to the day you ship.
Do you use any of these approaches when you start a new project? Or do you have your own favorite place to start a new app? Leave a comment and let me know!
You’ve just typed
rails new on your next project. Now what? Where do you start? Do you write everything in a single app, or go with a Service Oriented Architecture? Is this a good time to learn RSpec? Should you start building out all of your data models, or get a few actions working end-to-end? There are tons of decisions you have to make up front, and it’s easy to feel lost, and procrastinate, and get frustrated, and never finish your app.
This problem is super common. It comes from having to make too many choices up front, when you have the least amount of information you need to make an educated decision.
Convention Over Decision-making
Convention over Configuration is my favorite Rails principle. Before Rails, I’d get stuck figuring out where to put files, what to name database tables, how to map them to objects, and tons of other minor meaningless decisions. I’d spend more time choosing a direction than it would have taken to change my mind if I discovered later on that I was wrong.
Lots of the decisions you face when starting a new project will be like these. But you can replace these decisions if you borrow conventions from books, blogs, or styleguides. And you’ll save yourself a lot of time, stress, and meaningless debate.
Here are some conventions I follow when I start a new app:
Start with a few full features.
Your app doesn’t have to do everything you can think of right away. Start small, and start simple. Figure out the one or two features you can’t live without, and focus all your attention on those first. Build them really well. You can add other things later if you need it. But probably, YAGNI. (And you’ll usually find that what you think you need is totally different than what your customers think you need).
Start with a single app.
I know, you want to avoid building a Monorail. And there are a half dozen RailsConf talks about how SOA is the best way to keep Rails apps maintainable in the long term. Shouldn’t you figure out which parts of your app can be independent, and build a service for each part?
SOA is a great way to break apart a large, complex app. But it adds a ton of pain, and unless the pain of building, maintaining, and monitoring services outweighs the pain of working on your monolithic Rails app, it’s not worth doing. And if you haven’t started writing your app yet, you’re not feeling that pain. So, start with one app.
Start with a stack you’ve used before.
If you don’t know a stack yet, just use the Omakase stack. But either way, you don’t want to spend half a day integrating a test framework you’ve never used before with Rails. You don’t want to fight version conflicts between VCR and WebMock. You don’t want to have to figure out if your tests crashing is your fault or resque_unit’s. If you want to try a new library out, you can always add it later (or try it on a toy project).
Shipping is hard. If you want to finish your app, you have to scrape for every advantage you can get. So start small, start simple, and start with what you know.
Conventions may not always be perfect, but they’ll help you get started. And starting is the hardest part.
What conventions do you follow on your projects? And how do you decide which features or tests you should start with? I’ll share my thoughts next time, but I’d love to hear some of yours.
“Don’t Repeat Yourself” is one of the most valuable ideas from one of the most valuable books I read during my software development career. If you can refactor away duplicate code, you will produce more general, more stable code. When you start DRY-ing up your code, though, you’ll start to run into some problems: code that doesn’t handle edge cases well, code that’s too generalized and hard to read, or code that’s hard to find. If refactoring toward DRYness doesn’t work all the time, how do you know when you should refactor?
Too-DRY code comes from a misunderstanding of what kind of duplication you should try to refactor away. You need to be able to identify the difference between essential duplication and accidental duplication.
Essential duplication is code that solves the class of problems you’re working on. This is the kind of duplication that you should kill right away.
Accidental duplication, though, is duplication that’s unrelated to the problem at hand, “duplication by coincidence.” If you clean up accidental duplication too aggressively, you’ll end up with more brittle code that has to be un-refactored as new cases are added.
But how can you tell the difference? With experience, you might get better at this, but even after decades of programming, I still can’t do this right even half the time. Luckily, there’s a general rule that helps a lot: Three Strikes and You Refactor.
The first time you do something, you just do it.
The second time you do something similar, you wince at the duplication, but you do the duplicate thing anyway.
The third time you do something similar, you refactor.
Can you see why this helps? By the third time, you should start to see where the patterns are. You should have an idea of which duplicate code is necessary for solving your problem, and which code just looks the same coincidentally. You can start to generalize (and DRY up) just the code that’s fundamentally the same across all instances of the problem you’re trying to solve.
Take a moment and think about the last time refactoring caused you pain. Were you trying to DRY up something that was duplicated between two copies of the problem, but the third copy was just a little different?
And the next time you feel like you have to duplicate some code, try waiting until the third copy before you refactor. (It’s really hard and feels terrible, but close your eyes and try it anyway). Then, think about it as you DRY up the code. Are you refactoring differently than you would have if you refactored right after writing the second copy?
Good developers know they should test their code. Too often, though, the tests get skipped, rushed through, or never started. There are some really common traps I’ve seen people fall into, and they’ll kill your motivation to test every time.
1. Should I use RSpec? Cucumber? Capybara? Minitest?
When you start a new project, it’s way too easy to spend a lot of time trying to pick the best tools. This kind of procrastination hides the fact that you don’t know where to start, and until you pick your tools, you can feel productive without actually being productive.
Instead, just pick the stack you know best. If you’re not experienced with any particular test stack, take the default—start with what Rails gives you. You can always add more later. There’s nothing wrong with trying out new tools, but it’s a much better idea to introduce them over time, as you get to know your project and workflow better.
2. Do I start with unit tests? Functional tests? Integration tests?
When you begin a project, you should have an idea of which features or screens should be built first. This is a great place to start! Pick the first thing off your feature list and write a failing integration test for it. This should tell you what kind of controllers and routes you’re missing, which means you need to write some failing functional tests. The controllers need data and logic to do their job, so you’ll write unit tests and your models next. Then, once you get your unit tests passing, the functional tests should pass, and so should the integration test. Now you can move onto the next feature.
If you have a testing process that always has the next step defined, it’s a lot easier to stay motivated. If you don’t have to make decisions, there’s less of an opportunity for procrastination to creep in.
3. I don’t know how to test this network code, command line utility, or rake task!
The easiest thing to do here is move as much code out of the file or class that’s hard to test and into a new object that’s easy to test. That way, the hard-to-test thing just creates and passes parameters to your new object.
Of course, you still have the original thing that’s hard to test. But it’s probably just a few lines of code now, and it should be easier to stub or fake it out.
4. “The project’s almost done, now I just have to write the tests for it!”
Every developer I’ve met craves shipping code. If the tests are the last thing to do before you can ship, you’ll write the minimum number of tests you need to feel kind of confident that the code works, probably. If you get into this habit, you’ll start to see tests as annoying instead of helpful, and it’ll be that much harder to motivate yourself to write them.
One of my favorite things about TDD is that it mixes the testing with the design and the coding, which means you start to see testing as coding, which makes it much more fun (and you get the benefit of having tests a lot earlier).
5. What if I’m doing it wrong?
The Ruby community is known for really pushing code quality, unit testing, and object oriented design principles. This is a great thing! Unfortunately, it means that it’s really common to feel an enormous amount of pressure to ship perfect code with 100% test coverage on your first try.
This makes it really hard to start a project, especially open sounce, where you know other people will see the code. What will people say if they see that it doesn’t follow all of the SOLID principles? But there are a few things I’ve learned that have helped me deal with this pressure:
- Every good developer writes code they’re embarassed by later.
- Good code that ships is infinitely better than perfect code that doesn’t.
- Some people are just jerks and will make fun of your code. This really sucks. It’s ruined entire weeks of mine. But good developers want to help you instead of criticize you. And I’d bet that if you showed that code to a programming hero of yours, they wouldn’t make fun of it—they’d help you make it better.
What else have you noticed?
Which of these traps have you fallen into? What’s helped you pull yourself out? And are there any that I didn’t mention that you’ve noticed?
If you do recognize yourself in any of these traps, what are you going to do to get yourself out?
Luckily, Rails includes scopes, which can provide you with a lot of what you need for simple searching, filtering, and sorting. If you take advantage of scope chaining, you can build the features you want without taking on big dependencies or writing a bunch of repetitive search code yourself.
Searching with scopes
#index method on your RESTful controller that shows a table of products. The products can be active, pending, or inactive, are available in a single location, and have a name.
If you want to be able to filter these products, you can write some scopes:
1 2 3 4 5
Each of these scopes defines a class method on Product that you can use to limit the results you get back.
Your controller can use these scopes to filter your results:
1 2 3 4 5 6
And now you can show just the active products with names that start with ‘Ruby’.
Clearly, this needs some cleanup
You can see how this code starts to get unwieldy and repetitive! Of course, you’re using Ruby, so you can stuff this in a loop:
1 2 3 4 5 6 7 8 9 10 11 12 13
A more reusable solution
You can move this code into a module and include it into any model that supports filtering:
1 2 3 4 5 6 7 8 9 10 11 12 13
1 2 3 4
1 2 3
You now have filtering and searching of your models with one line in the controller and one line in the model. How easy is that? You can also get built-in sorting by using the built-in
order class method, but it’s probably a better idea to write your own scopes for sorting. That way you can sanity-check the input.
To save you some effort, I put
Filterable into a gist. Give it a try in your own project, It’s saved me a lot of time and code.