Recently, I got a chance to work with Elastic Beanstalk and trust me, it is not as friendly as Heroku and it took me a little while to get used to with Beanstalk deployments.
I am using delayed_job in my project and the requirement was very simple, i.e. to automate background jobs. On heroku we simply add a worker dyno and jobs start working immediately. But, same is not the case with Beanstalk. I came across with various article where people suggested to add a shell script inside `.ebextensions` but I did not find this a full proof solution. Then my colleague suggested me to try `active_elastic_job` @ https://github.com/tawan/active-elastic-job
I find this gem extremely helpful and the installation instructions are super simple.
In this article, I will not dig into how to configure, install and start using this gem (because if you visit the link, the steps are defined in simplest form and should not block you). My idea, is to highlight/cover the areas which are not mentioned there or missing.
1. When you create a worker environment, skip selecting the RDS section (which means you’ll create your worker environment without a RDS DB).
2. Now you’ll have to use the existing database (which your web environment is using). For that, copy all RDS environment variables from web environment to your worker environment.
3. Once above is done, deploy your the code to your worker environment. In case your deployment fails with reason:
PG::ConnectionBad: could not connect to server: Connection timed out
Then, go to Services ~> Elastic Beanstalk ~> You Web Application ~> Configuration ~> Instances ~> Modify and note down the Security Group there (using new UI of Beanstalk)
Now, go to Services ~> Elastic Beanstalk ~> Your worker application ~> Configuration ~> Instances ~> Modify and check the security group which your Web Application is using (Note that you can check as many as you want) and Save your changes. This should allow your worker app to use the same DB.
4. Once you have successfully deployed and configured your worker app and you get:
ActiveJob::QueueAdapters::ActiveElasticJobAdapter::SerializedJobTooBig
The job contains bytes in its serialzed form, which exceeds the allowed maximum of 262144 bytes imposed by Amazon SQS
That means you are not allowed to send more than 256KB of data to delayed job. This could prove a bottleneck and you’ll have to think of a workaround to deal with this error. Although, folks on internet suggested that we could use Amazon Extended Client Library for JAVA @ https://github.com/awslabs/amazon-sqs-java-extended-client-lib but I wasn’t able to use and apply this in my Rails app.
Having said that, if anyone knows how to use this in Rails, please drop a comment or as an alternate try to send data < 256KB to SOS queue
That’s all I have for now. See you soon! Happy Coding. 🙂
PUNEET PANDEY's WEBLOG 