Overview

Rails applications in Engine Yard utilize utility instances for additional services or compute power. This article provides an introduction to utility instances, their roles and types, directory structure, and the location of main logs for investigation.

Contents

• The utility instance
  • Supported applications
  • Important directories
    • Utility application deployment: /data/<app_name>/
    • Other directories and files
  • Relevant logs
    • Deployment logs
    • Configuration & Chef-related logs
    • Utility application logs
    • System logs
• Summary

The utility instance

Utility instances in Engine Yard provide either additional infrastructure or compute power to the application. Like application instances, the codebase is also deployed in utility instances; this is because their work is related to application code: either use the code directly or provide a related service.

Supported applications

While, theoretically, any required application can be deployed in a utility instances, there are some prebuilt options, which differ depending on the stack:

Deployed through environment variables in v6 and v7:

• Redis
• Memcached
• Sidekiq

Deployed through recipes in V7:

• Resque
• Resque Scheduler
• Elasticsearch
• Delayed Job

v6 offers everything in v7 plus:

• Thinking Sphinx
• Tinyproxy

v5 offers all of the above recipes, plus:

• Solr

Another important way to split the supported apps is between services and compute:

• Services: Redis, Memcached, Elasticsearch, Thinking Sphinx, Tinyproxy, and Solr.
• Compute: Sidekiq, Resque, Resque Scheduler, and Delayed Job.

When deployed by Engine Yard, the applications listed above are managed similarly to an application server in the application instance:

• They are managed by Monit or systemd (Sidekiq, Redis, and Memcached in v7 only).
• All have a control script that manages the application.
• Chef will install and overwrite configuration files under /data/<app_name>/shared/config

<supportagent>

Note: Customers may run anything else that they wish, such as Mongo, any Python apps, Docker, etc. These are considered not to be in the scope of support. While we can provide basic help, it is suggested that PS services are offered and escalation considered.

In addition, the two following utility instance types are considered PS customizations and issues should be escalated accordingly:

• Jumpbox
• IDS (Wazuh)

</supportagent>

Important directories

In Engine Yard, the following directories will normally need to be accessed when investigating and addressing issues with utility instances:

Utility application deployment: /data/<app_name>/

The application deployment directory is used in the deployment process, and contains the following relevant elements:

Subdirectories & files	Description
releases	Contains the directories for successful deployments made on the instance.
releases_failed	Contains the directories for failed deployments made on the instance.
current	It's a symlink to the current 'release' folder.
current/deploy	Contains the deploy hooks, which allow for customizing the deployment process and are managed by the customer.
shared	Contains files used by all versions of the code, to prevent duplication. These are managed by Chef.
shared/config	Contains shared config files. Env files within the folder contain the environmental variables set in the Engine Yard Web UI.
shared/log	Symlink identical to that in /data/<app_name>/current/log/, pointing at /var/log/engineyard/apps/<app_name>.
shared/tmp	Used by the Rails app to store temporary files.
shared/bin	Contains the control files used by the deployment process.
shared/pids	Has the PIDs of the applications being managed by Monit.
shared/cached-copy	Contains a copy of the codebase, so pulling the code from GitHub does not require downloading all files during deployment.
shared/bundled-gems	Contains the installed Ruby libraries ("Gems"), so that only new and modified files are installed on deployment.
shared/assets	Similar to bundled-gems, but contains the JavaScript assets installed by Yarn during deployment.

Other directories and files

Directories & files	Description
/lib/systemd/system/	Contains the systemd configuration files. systemd is used in place of Monit in newer versions of the stack, and is a native OS utility.
/etc/monit.d/	Contains the relevant Monit configuration files relevant to Engine Yard.
/etc/init.d/	Contains service initialization scripts for Nginx, MySQL, etc.
/etc/chef/recipes/	Contains the installed cookbooks. In the v4 stack, etc/chef-custom/recipes exists as well for custom recipes.
/etc/chef/dna.json	This file contains all information about the instance type and use: When the instance is created or 'Apply' is clicked, triggering a Chef run, AWSM creates a new version of the file based on the existing configuration and pushes it to the instance. The file contains data such as instance role, environment type, Ruby version, etc. Variables contained here are used by Enzyme as an input when running Chef.
/etc/engineyard/	Contains configuration files: dracul.yml: Contains the URLs from which recipes are downloaded, among other data. instance_api.yml: Contains an instance ID and token for talking back to AWSM. Used by collectd to send alerts to the UI.  collectd.conf: Contains the collectd configuration file.
/engineyard/bin/	Contains the control scripts for application servers.

Relevant logs

The existing flows and applications in the EY stack log their operations and errors to a variety of logs. This is a list of the logs that should be accessed during troubleshooting:

Deployment logs

Logs related to the deployment of an application's codebase to the instance.

Location	Description
/home/deploy/	Contains the last deployment log; for older logs, the Engine Yard web UI must be used instead. The application master contains all deploy-related logs, while other applications only contain the deploy hooks.

Configuration & Chef-related logs

Logs related to instance configuration and Chef runs.

Version	Location
v4	/var/log/chef.main.log /var/log/chef.custom.log /var/log/enzyme.log
v5	/var/log/chef.log /var/log/ey-enzyme.log /var/log/ey-primer.log
v6 & v7	/var/log/chef.log /var/log/ey-enzyme.log /var/log/ey-mnml.*.log

Utility application logs

Logs related to the execution of the utility application and the relevant web stack.

Location	Description
/data/<app_name>/current/log/	Contains logs for the rails application under <environment_framework>.*.log.
/var/log/engineyard/	Contains, potentially, logs used by the application: Passenger, PHP-FPM, etc.

System logs

Other logs concerning OS operations.

Location	Description
/var/log/collectd.log	Lists all activities from the monitoring and alerting stack, corresponding to the alerts shown in the web UI. collectd can be configured as required.
/var/log/monit.log	Internal Monit logs, showing its actions.
/var/log/messages.log	OS log with info from applications run in the user's space. While Engine Yard rarely uses it directly, applications in utility instances and other apps may write to it.
/var/log/syslog	OS-related logs, logging information for cronjobs, collectd errors, Monit activities, kernel messages, etc.
/var/log/auth.log	Shows user access to the system, including all SSH attempts.

Summary

This article provides an overview of utility instances in Engine Yard, their roles, types, and directory structure. It also provides a guide on how to work with these instances and the location of the main logs for investigation.