Ethel Docs

Deploying ethel

These instructions are for customer deployments and assume a Windows server:

Building ethel

ethel is currently distributed as zipped archives. These are built and attached to the recent releases in github. You should use these unless testing a proposed fix.

Use npm run build-current in the repository root to create core and client archives of the current project state. This will transfer the state of any npm link based updates for the shared or runners projects.

Plugin runners

Job specific runners are called plugins. These can be copied to the default plugins folder location, which is a ./plugins folder at the root of the project. Or the plugins can be located anywhere on the server and the path to the plugins added to the ETHEL_PLUGINS_PATHS environment variable. This environment variable accepts an array of paths. Using the environment variable is simpler when you are using NPM modules not available in the the ethel core project.

A starter project for custom runners is available in github. If you are using this project the simplest approach is to add the path to the dist folder of that project to the ETHEL_PLUGINS_PATHS array. This also is the best way if there are any NPM modules used by your plugins that are not in the core project.

Database options

Ethel supports SQLite, SQL Server and Postgres. SQLite is the default option and by far the most performant option. If the job involves the migration of millions of items, runs will need to be deleted, or exported and deleted, regularly to control the size of the database.

If a different database is used, it is expected that the customer or a PaaS solution would be used. Once accesss is provided update the ETHEL_DATA environment variable to use the database.

RabbitMQ

While the default queue solution, BetterQueue, works well in most cases, RabbitMQ provides extra flexibility that can be useful in many scenarios.There are two install options, manual or using Chocolately. The Chocolately install can cause problems in some scenarios.

  • Use Chocolately if you have an internet connection and are not using a service account to run RabbitMQ.
  • Use the manual install if you are using a service account or don't have an internet connection.
Chocolatey install
  • Create a directory to hold the application data, e.g. D:\rabbitmq. Then create the RABBITMQ_BASE system environment variable and point it to this directory.
  • Run choco install rabbitmq.
Manual install

Grab the RabbitMQ installer first. Then check the Erlang Version Requirements to determine the version of Erlang to download. Download the installer from the Erlang downloads page. Once you've transferred the installers to the server, do the following:

  • If you're not using a service account create a directory to hold the RabbitMQ application data, e.g. D:\RabbitMQ. Then create the RABBITMQ_BASE system environment variable and point it to this directory.
  • Run the Erlang OTP installer, accepting the defaults.
  • After the installer completes, create the ERLANG_HOME system environment variable. This should point to the bin folder in the Erlang directory, e.g. C:\Program Files\Erlang OTP\bin.
  • Confirm that the required environment variables are available by opening a PowerShell console and typing $env:ERLANG_HOME and if not using a service account $env:RABBITMQ_BASE
  • Run the RabbitMQ installer, accepting the defaults. If you are using a service acccount do not start the service.
  • RabbitMQ defaults to using %APPDATA%/RabbitMQ to store application data. The installer creates a number of files in this path and these should be moved to the equivalent path for the service account folder path. Once moved, update the service log on details and start the service.
Finalise configuration

Complete the following configuration steps:

  • Make sure the RabbitMQ service is running, then check cli access by opening a PowerShell session and navigating to the sbin folder in the RabbitMQ folder. Run .\rabbitmqctl.bat status to get the current configuration.
  • If there is a connection error the installer probably copied an out of date Erlang cookie to your user folder. Get the current version of the cookie by copying C:\Windows\System32\config\systemprofile\.erlang.cookie to the root of your C:\Users\{username} folder. Run .\rabbitmqctl.bat status again to check if this resolved the connection issue. If not it's google time.
  • Add a user for ethel .\rabbitmqctl.bat add_user ethel <password> and then give them permissions .\rabbitmqctl.bat set_permissions -p / ethel ".*" ".*" ".*". Or you could do this via the management GUI.
Other configuration

Optional configuration changes:

  • If you want the management web GUI install the plugin via .\rabbitmq-plugins.bat enable rabbitmq_management. The default URL for the GUI is http://localhost:15672 and the default user is guest with a password of guest. The guest user can only log from the server. Remote login is disabled for this account.
  • You can use environment variables to change most common configuration options, like port numbers. A full list is here. To change the port the nodes are listening on use RABBITMQ_NODE_PORT. When changing an environment variable recreate the service using the rabbitmq-service.bat tool, running remove, install and start.

Windows server

PM2 is used to run ethel on Windows servers. Install instructions and configuration information is available in the ethel repository at ./resources/pm2.