- Published on
Debugging Rails Connectivity Issues with the DB, Queue and Sidekiq on AWS
- Authors
- Name
- Yair Mark
- @yairmark
I recently had to debug connectivity issues to a Rails app in containers deployed to a cluster. Being new to Rails this was not immediately obvious. I had to debug what was causing DB, Sidekiq and queue connectivity issues on a newly deployed app.
I eventually managed to track down the issue by debugging as follows
Checking Environment Details
System Level
First, let us check if the app has the correct DB connection string. To do this go into the container running your rails app.
In the normal bash terminal run: env
In the output of env
, you should see an environment variable matching what you pass to the container. This variable should have the DB connection string as its value.
Database Connectivity
Next, we need to confirm that our container and host can actually physically connect to the DB. This will let us rule out AWS security group configuration issues if we can connect. If we cannot it likely means the DB security group's inbound rule has to be updated to allow connections from the host where the container is running.
Run the following on the host and then again in the container to confirm we can connect to the DB without issues:
telnet dbHostURI dbPort
- This article from AWS describes this process in more detail. Using this approach is very useful in picking up networking issues.
- If you are using an alpine container you will need to install telnet as follows:
apk add busybox-extras
- This is as per this thread
If you connect you will be given the option of disconnecting (this means that your host can see the DB and is authorized to hit the DB URI)
We now know we can successfully connect so there are no issues with security groups or other networking issues. Let us now confirm the DB connection settings are correct. In the host or the container run the following
psql -Atx -U yourDbUserName -d yourDbName -h dbHostURI -p dbPort
You will be prompted for a password. Type/paste it in. If all is good you should be dropped into a psql shell.
Rails Level
The last step in debugging is confirming that Rails has the right config.
To do this go onto the machine/container running your rails app, then:
- cd to the root of your Rails project
- Run
rails console
to put you into a console in your app's environment - Type:
ActiveRecord::Base.connection_config
- This is as per this answer
If the connection string returned matches the one you tested earlier then you should be all good.
Redis Queue Connectivity
To confirm Sidekiq will work properly we first need to ensure that we have no connectivity issues to the queue. To do this we will install the redis-cli on the host machine.
Then using the cli try to connect as follows:
redis-cli -h yourRedisURI
In the above:
- Use
-p
if the URL has a different port
To confirm that Redis responds in the cli type ping
you should get back PONG
If you need to install Redis on the host follow these steps
If connectivity is working at this point, Sidekiq should have no issues connecting to the queue as long as the connection string is correct.
ArgumentError: invalid uri scheme
Sidekiq: At one stage I ran into this error.
This happens if you do not prepend redis://
to the front of your Redis URI.