Setting up Ruby development setup with rbenv

I wanted to setup ruby source code on my machine. The source on Github has detailed instructions on how to do the setup.

git clone github.com/ruby/ruby
cd ruby
autoconf
./configure
make
make install

worked correctly on Mac OS X and Ruby was installed in usr/local/bin.

I use rbenv for managing Ruby versions. After setting up the Ruby from source, i wanted to use it using rbenv. Why? It will make very easy to switch between dev version and normal version. rbenv shell ruby-dev, do something, make some changes, compile again, test again. rbenv shell 2.1.2, and back to normal.

But as the source Ruby was installed in usr/local/bin, rbenv was not able to find it.

I asked question on #ruby on IRC and Postmodern pointed me to this.

It mentions passing --prefix option to the ./configure command. We can pass the name of directory where we want to install Ruby to this option.

So we can run

./configure --prefix=PATH_TO_INSTALL_RUBY

rbenv by default, installs all rubies in ~/.rbnev/versions

And when we do rbenv versions, it looks for all rubies installed in this directory and lists them.

The content of my ~/.rbnev/versions directory looked like these:

prathamesh ~/.rbenv/versions 2.0.0
$ ls
2.0.0-p247 2.0.0-p353 2.0.0-p481 2.1.0      2.1.1      2.1.2      rbx-2.2.6

prathamesh ~/.rbenv/versions 2.0.0

Now if we give prefix path to configure command, the dev-ruby will be installed in ~/.rbenv/versions

./configure --prefix="$HOME/.rbenv/versions/ruby-dev"

After make and make install, dev ruby was installed in ~/.rbnenv/versions in ruby-dev directory.

Now rbenv versions output had ruby-dev also.

prathamesh ~/.rbenv/versions 2.0.0
$ rbenv versions
* system (set by /Users/prathamesh/.rbenv/version)
  2.0.0-p247
  2.0.0-p353
  2.0.0-p481
  2.1.0
  2.1.1
  2.1.2
  rbx-2.2.6
  ruby-dev

prathamesh ~/.rbenv/versions 2.0.0

As ruby-dev is listed by rbenv versions, i can switch to it easily:

rbenv shell ruby-dev

Update

On Mac OS X Sierra, I had to also pass the flag for openssl directory while configuring.

cd ruby
./configure --prefix="$HOME/.rbenv/versions/ruby-dev" --with-opt-dir="/usr/local/opt/openssl/"
make
make install
end

This sets up Ruby dev version on my machine and also allows me to switch back and forth using rbenv.

Weirdness of Bigdecimal.new

Recently i was bitten by this weirdness of BigDecimal in Ruby.

BigDecimal.new excepts initial value and number of significant digits. In one of my app, BigDecimal.new was called with string initial value without precision. Everything was working great and suddenly something changed. And this error started coming.

can't omit precision for a Float.

What happened was, the value passed to BigDecimal.new was no longer string. Due to some changes in other parts of code, float was passed to BigDecimal.new.

Now when a float or rational is passed to BigDecimal.new, it requires precision to be present. Otherwise raises error.

can't omit precision for a Float(/Rational).

Fix was very simple. Pass the precision digits as second argument to the BigDecimal.new call.

Setting up ispell and Emacs on Mac OS X

I was trying to setup flyspell mode on new Mac and whenever i do

flyspell-buffer

but Emacs was throwing error -

Problem - Flyspell does not work with Emacs. Can't find ispell.

Mac OS X does not come with ispell or its replacement aspell. So the first thing is to install aspell using brew.

brew install aspell

Now to tell emacs to use aspell instead of ispell.

(setq ispell-program-name "aspell")

Problem

Even though, aspell is installed properly, Emacs is not able to find it in the path.

Solution

Instead of just aspell, specify the full path to the aspell executable.

(setq ispell-program-name "/usr/local/bin/aspell")

There were other solutions mentioned on the internet like updating the PATH env variable and exec-path variable with usr/local/bin but it did not work for me.

Devise : Passing params to after_* paths

Devise allows us to customize after_signup_path by overriding a protected method in RegistrationsController.

class RegistrationsController < Devise::RegistrationsController
  protected

  def after_inactive_sign_up_path_for(resource)
    your_custom_path
  end

  def after_sign_up_path_for(resource)
    your_custom_path
  end
end

If we are using :confirmable with devise, sometimes we need to show user a message -

Confirmation email has been sent to you <user_email>.
Please confirm your email.

Lets say that route name for this is confirmation_email_sent_path.

class RegistrationsController < Devise::RegistrationsController
  protected

  def after_inactive_sign_up_path_for(resource)
    confirmation_email_sent_path
  end
end

Now how to pass user’s email to this path?

One way is to override create action of Devise’s RegistrationsController. But this is bad as we will get dependent on current Devise implementation.

Lets see how we can achieve this with minimum dependency.

resource has the User object. So we can use that to pass user’s email to the confirmation_email_sent action.

class RegistrationsController < Devise::RegistrationsController
  protected

  def after_inactive_sign_up_path_for(resource)
    confirmation_email_sent_path(email: resource.email)
  end
end

Now we can access this email in confirmation_email_sent action.

def confirmation_email_sent
  @email = params[:email]
end

VERP on Rails

We sent newsletter campaigns from our Rails app. One of the main requirement of such campaign is how many emails bounced?. We need to track all bounced emails and map them again to a specific campaign because there will be multiple campaigns going on all the time.

To track bounced emails efficiently, we need to set unique return path for every recipient. This technique is called Variable Envelope Return Path or VERP

Postfix supports VERP with -V switch. For eg.

sendmail.postfix -V -f bounced

will generate return path for anyone@example.com as bounced+anyone=example.com@yourdomain.com.

If you want more control and more information like from which campaign this email was sent, we need to send more information in the return-path.

So it is best to generate our own return path pattern and parse it once it is received. We generated a pattern where return-path will generated from all the required information for tracking. This pattern was given to mail method as follows:

    mail(
         from:        'hello@example.com',
         to:          'client@example.com',
         return_path: generate_verp_pattern
        )

So now the first problem was solved. We were sending unique return-path for every email that was going out from the system.

Next part is to track it once it bounces and update database.

Postfix allows piping incoming email to a particular address to a script. So if we pipe all incoming emails to bounce@yordomain.com to our script then we can parse the incoming address and update database.

For this, we have to edit /etc/aliases (or /etc/postfix/aliases)file as follows:

bounced: "|/path/to/your/script"

This means all incoming emails to bounced@yourdomain.com will be piped to our script. This actually means that the whole email with body, headers, attachments etc is forwarded to our script.

The /etc/aliases file is a text file that is used by postfix as a table to redirect mail for local recipients. To rebuild this table after a change, we need to run newaliases command

newaliases

This will rebuild the table for postfix. Now we have completed the second part of the process.

The shell script will get the bounced email content now. There can be multiple bounced emails generated at the same time. So we can’t directly pass them to rails runner scripts or rake tasks. Because that will kill our server by launching multiple rails instances. Instead we need to use some background tasks mechanism.

We were already using resque, so decided to use it for bounced emails also. So a resque worker will actually update the database. Our script just has to enqueue the job for resque.

We broke this enqueuing process into two parts.

First - A shell script which will use correct RVM Ruby version and call the ruby script.

Second - A Ruby script which will enqueue the job to resque.

So the shell script looked like -

   #!/bin/bash
   rvm use ruby_version@ruby_gemset
   ruby /path/to/ruby/script

And Ruby script looked like

    require 'rubygems'
    require 'resque'

    # Adds the incoming bounced_email to background job
    class BouncedEmail
      def initialize(content)
      Resque.enqueue_to(:bounced_email_receiver, 'BouncedEmailReceiver', content)
    end

    BouncedEmail.new($stdin.read)

We had to go in 2 steps here because we had multiple apps using multiple rubies on same server. If you have only one ruby then you can make a executable ruby script instead of shell script which decides which ruby to use.

Now its upto resque worker to parse the content and update database.

For that, we used bounced_email gem which detects lot of things such as bounced code, reason, type of failure etc. As it is integrated with mail gem, we got the recipient address(which was unique pattern generated by us only) and were able to parse it to update the database. With bounced_email we got some more relevant information for free :)

References :

  • http://keakaj.com/wisdom/2007/08/08/verp-on-rails/
  • http://blog.sosedoff.com/2011/08/10/processing-emails-with-postfix-and-rails/
  • https://github.com/mitio/bounce_email

jQuery append and clone

I was trying to append some part of the DOM to some other element of the page.

<div id='container'>
  <ul class='contacts_list'>
  </ul>
</div>
...
<div id='updated_contacts'>
</div>

On a click event, i wanted to append contacts_list to updated_contacts. Using jQuery and append method

  $("#updated_contacts").append($(".contacts_list"));

But every time, the original contacts_list from container was getting removed.

After searching documentation of append method, i found out that when an element is inserted to some other part of DOM, it is removed from the original location.

If an element selected this way is inserted into a single location elsewhere in the DOM, it will be moved into the target (not cloned):

Using clone method we can solve this problem.

clone creates a copy of the existing element and appends it to new location keeping original version as it is.

  $("#updated_contacts").append($(".contacts_list").clone());

Documentation of clone method is here

Using clone() has the side-effect of producing elements with duplicate id attributes, which are supposed to be unique. Where possible, it is recommended to avoid cloning elements with this attribute or using class attributes as identifiers instead.

is important point to consider while using clone.

Update :

Prashant pointed out that the clone and append methods are part of native javascript itself. And jQuery makes use of them to build simpler API.

Javascript has appendChild and cloneNode methods which can be used when we are not using jQuery in same situation.

More info about these methods can be found here.

Thanks Prashant :)

enable minitest again

If you have skipped minitest/test-unit while creating a rails app with -T or want to move to minitest from rspec or want to start with minitest in an existing rails project without tests, its very easy.

Just include

    require "rails/test_unit/railtie"

in config/application.rb.

After this rake test will start working. You may have to create test folder structure on your own or can copy from an existing project with tests.

preload associations with `find_by_sql`

I have a very complex query which is made up of more than 1 subqueries. Arel is awesome, but it can’t generate that query. So i generate those subqueries separately and combine them by union or intersection based on some condition. Finally the generated query is given to find_by_sql to get the data.

We found out that, in the jbuilder template that was rendering this data, that it was calling association objects resulting in n+1 query problem.

Generally this problem is solved by eager-loading in rails.

For eg. to load the company of the user, we do

@users = User.where(status: 'connected').includes(:company)

Rails will do the magic(load the company of the user also in the same query) and when we refer to @user.company from view, it would not trigger any new sql query and directly access it from ruby object.

But this doesn’t work with find_by_sql.

@users = User.find_by_sql(some_condition).includes(:company)

will throw error. Because find_by_sql returns Ruby array and not ActiveRecord::Relation object. So it doesn’t respond to includes.

If we try calling includes first,

@users = User.includes(:company).find_by_sql(some_conditions)

The includes part is silently dropped and only find_by_sql part gets executed.

ActiveRecord::Associations::Preloader class to our rescue.

Only applicable for Rails 3 and Rails 4.0.x

We can use the run method from Preloader class like follows:

@users = User.find_by_sql(some_condition)
ActiveRecord::Associations::Preloader.new(@users, :company).run

This will preload the company association for the @users object like it will do when we use includes.

Rails 4.1 and onwards

Rails 4.1 onwards, run method is not present in Preloader class. It is replaced by the preload method which was private in Rails 3. We can call preload method directly as follows.

@users = User.find_by_sql(some_condition)
ActiveRecord::Associations::Preloader.new.preload(@users, :company)

The first argument to this method is records array, second argument is associations and third is options which are optional.

We can pass a single association like :company.

We can also pass multiple associations in the form of array [:company, :account].

We can also pass a hash to eager load associations of associations like { company: :category }.

We can also mix last two options like [{ company: :category }, :account].

Rails 5?

Things can change. The Preloader class might be replaced by a module. I will update the blog post once that is done.

Happy Hacking!

facebook app with multiple subdomains

I had a requirement of adding facebook integration with our Rails app. I am using omniauth-facebook gem for the authentication part and it works great.

Our app has multiple subdomains for all clients, so i wanted to callback URLs separate for each client.

For example,

For foo.example.com, callback should be http://foo.example.com/auth/facebook For bar.example.com, callback should be http://bar.example.com/auth/facebook

We can do that in our Facebook app settings, using App Domains field.

The description of App Domains is as follows:

Enable auth on domain and subdomain(s) (e.g., “example.com” will enable *.example.com)

To test this locally, i added entries in /etc/hosts/ for testing

    127.0.0.1   foo.myapp.com
    127.0.0.1   bar.myapp.com
    127.0.0.1   baz.myapp.com

Then accessing foo.myapp.com:3000 and bar.myapp.com:3000 and clicking on facebook authenticaton, it redirected me correctly to foo.myapp.com:3000/auth/facebook and bar.myapp.com:3000/auth/facebook respectively.

In production, we have to replace the App Domain with actual URL of our website.

conditional where in rails

T.L.D.R

You can pass blank object to where clause and it will return current relation as it is.

Long version

Sometimes, we need to apply where clause conditionally. For eg. apply timeframe condition if timeframe is given in params. We can achieve this using simple if/unless.

  def most_recent(from = nil, to = Date.today)
    condition = Activity.where("created_at < ?", to)
    condition.where("created_at > ?", from) unless from.blank?
  end

This works but doesn’t look good. If we can chain this second conditional scope, it would be much better. Enter magic of Rails. If we read documentation of where clause,

# === blank condition
    #
    # If the condition is any blank-ish object, then #where is a no-op and returns
    # the current relation.

This is interesting. This means if we pass any blank value to where clause, it returns the current relation as it is.

Now we have to just write a method which will generate the query if condition is satisfied. It will return nil otherwise resulting in a no-op.

  def most_recent(from = nil, to = Date.today)
    Activity.where("created_at < ?", to).where(from_condition(from))
  end
  def from_condition(from)
    "created_at > #{from}" unless from.blank?
  end

This looks more cleaner than earlier solution. It will work the same way if we pass '', "", {}, [], false or any other object that respond to blank as true instead of nil.

Happy Hacking!