Configure Delayed Job with Elastic Beanstalk

Recently, I got a chance to work with Elastic Beanstalk and trust me, it is not as friendly as Heroku and it took me a little while to get used to with Beanstalk deployments.

I am using delayed_job in my project and the requirement was very simple, i.e. to automate background jobs. On heroku we simply add a worker dyno and jobs start working immediately. But, same is not the case with Beanstalk. I came across with various article where people suggested to add a shell script inside `.ebextensions` but I did not find this a full proof solution. Then my colleague suggested me to try `active_elastic_job` @ https://github.com/tawan/active-elastic-job

I find this gem extremely helpful and the installation instructions are super simple.

In this article, I will not dig into how to configure, install and start using this gem (because if you visit the link, the steps are defined in simplest form and should not block you). My idea, is to highlight/cover the areas which are not mentioned there or missing.

1. When you create a worker environment, skip selecting the RDS section (which means you’ll create your worker environment without a RDS DB).

2. Now you’ll have to use the existing database (which your web environment is using). For that, copy all RDS environment variables from web environment to your worker environment.

3. Once above is done, deploy your the code to your worker environment. In case your deployment fails with reason:


PG::ConnectionBad: could not connect to server: Connection timed out

Then, go to Services ~> Elastic Beanstalk ~> You Web Application ~> Configuration ~> Instances ~> Modify and note down the Security Group there (using new UI of Beanstalk)

Now, go to Services ~> Elastic Beanstalk ~> Your worker application ~> Configuration ~> Instances ~> Modify and check the security group which your Web Application is using (Note that you can check as many as you want) and Save your changes. This should allow your worker app to use the same DB.

4. Once you have successfully deployed and configured your worker app and you get:


ActiveJob::QueueAdapters::ActiveElasticJobAdapter::SerializedJobTooBig
The job contains bytes in its serialzed form, which exceeds the allowed maximum of 262144 bytes imposed by Amazon SQS

That means you are not allowed to send more than 256KB of data to delayed job. This could prove a bottleneck and you’ll have to think of a workaround to deal with this error. Although, folks on internet suggested that we could use Amazon Extended Client Library for JAVA @ https://github.com/awslabs/amazon-sqs-java-extended-client-lib but I wasn’t able to use and apply this in my Rails app.

Having said that, if anyone knows how to use this in Rails, please drop a comment or as an alternate try to send data < 256KB to SOS queue

That’s all I have for now. See you soon! Happy Coding. 🙂

Advertisements

What a leader should learn from M.S. Dhoni

M.S. Dhoni, yes the world knows him! What he has achieved so far in his cricketing career, nobody else does. He is Indian skipper, known as ‘Maahi’, ‘Capatin Cool’ and lots of other names.

Well, after India became T20 champions and then in 2011 became World champions, soon after that I started hating Dhoni. Trust me, but it’s true. Somewhere in my mind, I thought he is arrogant, proudy and a person who doesn’t respect his seniors and don’t give any value to his team. Trust me when I say this, I mean it.

But in June, 24th 2013, I’ve witnessed of another glory/achivement of Indian team and this time Indian team won the ICC champions trophy 2013 cup. And the Man in highlight is again the Indian skipper M.S. Dhoni.

He was given a team of those players who have either debuting their ODI career or played very limited number of ODI’s or have no experience of playing overseas matches. But in the end, everyone contributed and delivered their best. All credit goes to ‘Maahi’.

What he has done? He showed his trust in #IshantSharma and he proved to be the Game changer/Turning point for #TeamIndia.

* What mantra he (Dhoni) is using?
* Any divine power on him?
* Any Totka he do before every match starts?

Well, the answer to each and every question lies in his Captaincy skills. The way he manages his team, the level of trust he has and the decision making power, the ability to motivate team memeber even if he is not performing and the ability to stay calm, makes him what he is now.

In this article, I am going to talk about what we can learn from Dhoni, how we can adapt some extra ordinary skills and improve ourselves.

Personally, I won’t be giving a lecture kind of thing here.. We all want to be leaders and with the examples like this we can be the masters of our era.

Fetching Cotacts from Gmail, Yahoo and Hotmail

Hello Guys,

This is another post by me, hope this post will help you, if you are stuck in fetching or grab contacts from Yahoo, Gmail and Hotmail. This is possible using gem ‘contacts’.

So here I am posting my code, I know there are lot of things and code available for this, but this is the exact code which I am using.

If you are running an application or want to create a new go ahead, I am writing a new application here but you can add it in your existing code as well.

open your command or console prompt and go into your application directory, here I am creating new:

rails contact

this will create a particular set of files and directories in your application. Now go into your directory, before start working any further be sure that you have installed ‘contacts’ gem with the latest version i.e 1.0.13, to install ‘contacts’ type :

for linux users : sudo gem install contacts

for windows : gem install contacts

if already have but older version update it : sudo gem update contacts

Now open your environemnt.rb file and below this line write

Rails::Initializer.run do |config|

end

require ‘contacts’

be sure to restart your server.

Now open the controllers folder and create this method in any controller if you have or where you want to display this. In my case I am writing this in my account_controller.rb

def invite_friends
#@user = User.find(params[:id])
end

def import
@users = User.find(params[:id])
begin
@sites = {“gmail”  => Contacts::Gmail, “yahoo” => Contacts::Yahoo, “hotmail” => Contacts::Hotmail}
@contacts = @sites[params[:from]].new(params[:login], params[:password]).contacts
@users , @no_users = [], []
@contacts.each do |contact|
#if u = User.find(:first , :conditions => @users.email = ‘#{contact[1]}’ , :include =>[:user])
if u = User.find(:first , :conditions => “email = ‘#{contact[1]}'” )
@users << u
else
@no_users << {:name => contact[0] , :email => contact[1]}
end
end
respond_to do |format|
format.html {render :template => ‘shared/_contact_list’, :layout => false}
format.xml {render :xml => @contacts.to_xml}
end
end

before creating this two method, just be sure that you are giving the user id from invite_friends method to import method.

Open up your invite_friends.html.erb and paste this code :

<% form_tag :action => ‘import’, :id => @user do %>

<select name=”from” id=”from”>
<option value=””>Select Id</option>
<option value=”gmail”>Gmail</option>
<option value=”yahoo”>Yahoo</option>
<option value=”hotmail”>Hotmail</option>
</select>

<BR />
<p>Please Enter Your Email Address Below : <BR />
<input type=”text” name=”login”></p>
<p>Enter Your Password :<BR />
<input type=”password” name=”password”></p>

<p><h4>Note : we are not going to save your Password anywhere </h4></p>
<p><%= submit_tag ‘Find My Friends’ %>

<% end %>

Now start your server, open up firefox and type the address (in my case it is) : http://localhost:3000/account/invite_friends

Select any service like Yahoo, Hotmail or Gmail give the corresponding username and Password and hit submit.

You will get an error message that missing template, to remove that create a folder ‘shared’ in /app/views/ and create a new file name ‘_contact_list.html.erb’ and paste the below code :

<% for i in @contacts %>
<input type=”checkbox” name=”email[]” id=”email_<%= i %>” value=”<%= i %>” /><%= i %><br>
<% end %>

this will help you to take further actions on fetched email addresses, that’s it. You have done..

If you are facing any problems, leave a comment, I will get back in touch with you, and if your code works, don’t forget to leave a comment.

Thanks to ‘contacts’ and thanks to you all also.