Component B Component A Component C Engine-ering Rails Applications Vladimir Dementyev, Saint-P Ruby 2019

We* are Doomed * Rails developers

Can you escape your destiny? V.Yurlov, for “The war of the worlds” by H.G.Wells

Components 👍

Monolith 😞

Hanami

Elixir

Shopify: Modular Monolith

SHOPIFY all others

Rails

The Book?

What we’ve done Or how we decided to go with engines

Co-living rentals service

Our mission • They had a property and lease managements system (admin panel) • They needed a community app for users • They wanted to keep everything in the same Rails app

Community • • • • Events Perks Chat Billing

Phase #1: Namespaces app/ controllers/ chat/… models/ chat/… jobs/ chat/…

Namespaces • Quick start • Fake isolation

Our mission evolves • There is a property and lease management application (admin panel) • We need a community app for users • We (they) want to keep everything in the same Rails app • And we (again, they) want to re-use the app’s code in the future for a side-project

Phase #2: Engines & Gems app/… engines/ chat/ app/ controllers/… … lib/… gems/…

The Modular Monolith: Rails Architecture

rails plugin new \ my_engine !—mountable Engems Building Rails apps from engines and gems

Crash Course in Engines $ rails plugin new my_engine \ !—mountable # or —full create create create create … create README.md Rakefile Gemfile my_engine.gemspec Engine is a gem

Crash Course in Engines my_engine/ app/ controllers/… … config/routes.rb lib/ my_engine/engine.rb my_engine.rb Added to paths

Crash Course in Engines # my_engine/lib/my_engine/engine.rb module MyEngine class Engine < !::Rails!::Engine # only in mountable engines isolate_namespace MyEngine end end

Crash Course in Engines # my_engine/config/routes.rb MyEngine!::Engine.routes.draw do get “/best_ruby_conference”, to: redirect(“https:!//spbrubyconf.ru”) end

Crash Course in Engines # <root>/Gemfile gem “my_engine”, path: “my_engine”

Crash Course in Engines # <root>/config/routes.rb Rails.application.routes.draw do mount MyEngine!::Engine !=> “/my_engine” end

Crash Course in Engines $ rake routes Prefix Verb … my_engine URI Pattern Controller#Action /my_engine MyEngine!::Engine Routes for MyEngine!::Engine: best_ruby_conference GET /best_ruby_conference(:format)

The end

Common Engines common-events perks_by connect_by active-storage-proxy chat_by meet_by manage_by main app

Dependencies • How to use non-Rubygems deps? • How to share common deps? • How to sync versions?

Path/Git problem # engines/my_engine/Gemfile gem “local-lib”, path: “!../local-lib” gem “git-lib”, github: “palkan/git-lib” # <root>/Gemfile gem “my_engine”, path: “engines/my_engine” gem “local-lib”, path: “!../local-lib” gem “git-lib”, github: “palkan/git-lib”

eval_gemfile # engines/my_engine/Gemfile eval_gemfile “./Gemfile.runtime” # engines/my_engine/Gemfile.runtime gem “local-lib”, path: “!../local-lib” gem “git-lib”, github: “palkan/git-lib” # <root>/Gemfile gem “my_engine”, path: “engines/my_engine” eval_gemfile “engines/my_engine/Gemfile.runtime”

eval_gemfile

Shared Gemfiles gemfiles/ profilers.gemfile rails.gemfile gem “stackprof”, “0.2.12” gem “ruby-prof”, “0.17.0” gem “memory_profiler”, “0.9.12” gem “rails”, “6.0.0.rc1”

Shared Gemfiles # engines/my_engine/Gemfile eval_gemfile “!../gemfiles/rails.runtime” eval_gemfile “!../gemfiles/profilers.runtime” # <root>/Gemfile eval_gemfile “gemfiles/rails.runtime” eval_gemfile “gemfiles/profilers.runtime”

Keep Versions N’Sync gem ‘transdeps’ A gem to find inconsistent dependency versions in component-based Ruby apps. NOT VER IFIED

DB vs. Engines • How to manage migrations? • How to write seeds? • How to namespace tables?

Migrations. Option #1 Install migrations: rails my_engine:install:migrations 🤔

Migrations. Option #2 “Mount“ migrations: # engines/my_engine/lib/my_engine/engine.rb initializer “my_engine.migrations” do |app| app.config.paths[“db/migrate”].concat( config.paths[“db/migrate”].expanded ) # For checking pending migrations ActiveRecord!::Migrator.migrations_paths += config.paths[“db/migrate”].expanded.flatten end 😺

Seeds # <root>/db/seed.rb ActiveRecord!::Base.transaction do ConnectBy!::Engine.load_seed PerksBy!::Engine.load_seed ChatBy!::Engine.load_seed MeetBy!::Engine.load_seed end

table_name_prefix class CreateConnectByInterestTags < ActiveRecord!::Migration include ConnectBy!::MigrationTablePrefix def change create_table :interest_tags do |t| t.string :name, null: false end end end

table_name_prefix module ConnectBy module MigrationTablePrefix def table_prefix ConnectBy.table_name_prefix end def table_name_options(config = ActiveRecord!::Base) { table_name_prefix: “!#{table_prefix}!#{config.table_name_prefix}”, table_name_suffix: config.table_name_suffix } end end end

table_name_prefix

config/application.rb config.connect_by.table_name_prefix = “connect_”

Factories # engines/my_engine/lib/my_engine/engine.rb initializer “my_engine.factories” do |app| factories_path = root.join(“spec”, “factories”) Custom load hook ActiveSupport.on_load(:factory_bot) do require “connect_by/ext/factory_bot_dsl” FactoryBot.definition_file_paths.unshift factories_path end end

Factories: Aliasing using ConnectBy!::FactoryBotDSL Takes engine namespace into account FactoryBot.define do # Uses ConnectBy.factory_name_prefix + “city” as the name factory :city do sequence(:name) { |n| Faker!::Address.city + ” (!#{n})”} trait :private do visibility { :privately_visible } end end end

Factories: Aliasing # spec/support/factory_aliases.rb unless ConnectBy.factory_name_prefix.empty? RSpec.configure do |config| config.before(:suite) do FactoryBot.factories.map do |factory| next unless factory.name.to_s.starts_with?(ConnectBy.factory_name_prefix) FactoryBot.factories.register( factory.name.to_s.sub(/^!#{ConnectBy.factory_name_prefix}/, “”).to_sym, factory ) end end end end

Factories: Aliasing • Use short (local) name when testing the engine • Use long (namespaced) when using in other engines

Shared Contexts my_engine/ lib/ my_engine/ testing/ shared_contexts/… shared_examples/… testing.rb

Shared Contexts # other_engine/spec/rails_helper/rb require “connect_by/testing/shared_contexts” require “connect_by/testing/shared_examples”

How to test engines?

Testing. Option #1 Using a full-featured Dummy app: spec/ dummy/ app/ controllers/ … config/ db/ test/ … 😕

Testing. Option #2 gem ‘combustion’ A library to help you test your Rails Engines in a simple and effective manner, instead of creating a full Rails application in your spec or test folder. 🙃

Combustion begin Combustion.initialize! :active_record, :active_job do config.logger = Logger.new(nil) config.log_level = :fatal config.autoloader = :zeitwerk config.active_storage.service = :test config.active_job.queue_adapter = :test end rescue !=> e # Fail fast if application couldn’t be loaded $stdout.puts “Failed to load the app: !#{e.message}\n” \ “!#{e.backtrace.take(5).join(“\n”)}” exit(1) end

Combustion spec/ internal/ config/ storage.yml db/ schema.rb

Combustion • Load only required Rails frameworks • No boilerplate, only the files you need • Automatically re-runs migrations for every test run

CI commands: engem: description: Run engine/gem build parameters: target: type: string steps: - run: Skip if no relevant changes name: “[!<< parameters.target !>>] bundle install” command: | .circleci/is-dirty !<< parameters.target !>> !|| \ bundle exec bin/engem !<< parameters.target !>> build

.circleci/is-dirty pattern = File.join(!dir!, “!../{engines,gems}//.gemspec”) gemspecs = Dir.glob(pattern).map do |gemspec_file| Gem!::Specification.load(gemspec_file) end names = gemspecs.each_with_object({}) do |gemspec, hash| hash[gemspec.name] = [] end # see next slide

.circleci/is-dirty tree = gemspecs.each_with_object(names) do |gemspec, hash| deps = Set.new(gemspec.dependencies.map(&:name)) + Set.new(gemspec.development_dependencies.map(&:name)) local_deps = deps & Set.new(names.keys) local_deps.each do |local_dep| hash[local_dep] !<< gemspec.name end end # invert tree to show which gem depends on what tree.each_with_object(Hash.new { |h, k| h[k] = Set.new }) do |(name, deps), index| deps.each { |dep| index[dep] !<< name } index end

.circleci/is-dirty def dirty_libraries changed_files = git diff $(git merge-base origin/master HEAD) !--name-only .split(“\n”) raise “failed to get changed files” unless $!?.success? changed_files.each_with_object(Set.new) do |file, changeset| if file =~ %r{^(engines|gems)/([^/]+)} changeset !<< Regexp.last_match[2] end changeset end end

engines: executor: rails steps: - attach_workspace: at: . - engem: target: connect_by - engem: target: perks_by - engem: target: chat_by - engem: target: manage_by - engem: target: meet_by CI

Dev Tools • rails plugins new is too simple • Use generators: rails g engine

Generators # lib/generators/engine/engine_generator.rb class EngineGenerator < Rails!::Generators!::NamedBase source_root File.expand_path(“templates”, !dir!) def create_engine directory(“.”, “engines/!#{name}”) end end

Dev Tools • rails plugins new is too simple • Use generators: • Manage engines: rails g engine bin/engem

bin/engem # run a specific test $ ./bin/engem connect_by rspec spec/models/connect_by/city.rb:5

runs bundle install, rubocop and rspec by default $ ./bin/engem connect_by build

generate a migration $ ./bin/engem connect_by rails g migration <name>

you can run command for all engines/gems at once by using “all” name $ ./bin/engem all build

Engine Engine Main app

How to modify other engine’s entities?

Base & Behaviour • Base Rails classes within an engine MUST be configurable • They also MAY require to have a certain “interface”

Base & Behaviour module ConnectBy class ApplicationController < Engine.config.application_controller.constantize raise “Must include ConnectBy!::ControllerBehaviour” unless self < ConnectBy!::ControllerBehaviour raise “Must implement #current_user method” unless instance_methods.include?(:current_user) end end

Base & Behaviour module ConnectBy class ApplicationController < Engine.config.application_controller.constantize raise “Must include ConnectBy!::ControllerBehaviour” unless self < ConnectBy!::ControllerBehaviour raise “Must implement #current_user method” unless instance_methods.include?(:current_user) end end Configurable

Base & Behaviour # config/application.rb config.connect_by.application_controller = “MyApplicationController”

Base & Behaviour module ConnectBy class ApplicationController < Engine.config.application_controller.constantize raise “Must include ConnectBy!::ControllerBehaviour” unless self < ConnectBy!::ControllerBehaviour raise “Must implement #current_user method” unless instance_methods.include?(:current_user) end end “Interface”

Base & Behaviour class ApplicationController < ActionController!::API include ConnectBy!::ControllerBehaviour include JWTAuth include RavenContext if defined?(Raven) end

Modify Models • Prefer extensions over patches • Use load hooks (to support autoload/ reload)

Load Hooks # engines/connect_by/app/models/connect_by/city.rb module ConnectBy class City < ActiveRecord!::Base # !!… ActiveSupport.run_load_hooks( “connect_by/city”, self ) end end

Load Hooks # engines/meet_by/lib/meet_by/engine.rb initializer “meet_by.extensions” do ActiveSupport.on_load(“connect_by/city”) do include !::MeetBy!::Ext!::ConnectBy!::City end end

Load Hooks # engines/meet_by/app/models/ext/connect_by/city.rb module MeetBy module Ext module ConnectBy module City extend ActiveSupport!::Concern included do has_many :events, class_name: “MeetBy!::Events!::Member”, inverse_of: :user, foreign_key: :user_id, dependent: :destroy end end end end end

How to communicate between engines?

Problem • Some “events” trigger actions in multiple engines • E.g. , user registration triggers chat membership initialization, manager notifications, etc. • But user registration “lives” in connect_by and have no idea about chats, managers, whatever 🤔

Solution • Events • Events • Events

Tools • Hanami Events • Rails Events Store • Wisper (?) • dry-events (?)

Railsy RES

Railsy RES vs RES • Class-independent event types • Uses RES as interchangeable adapter • Better testing tools • Less verbose API and convention over configuration

Railsy RES class ProfileCompleted < Railsy!::Events!::Event # (optional) event identifier is used for transmitting events # to subscribers. # # By default, identifier is equal to name.underscore.gsub('/', '.'). self.identifier = “profile_completed” # Add attributes accessors attributes :user_id # Sync attributes only available for sync subscribers sync_attributes :user end

Railsy RES event = ProfileCompleted.new(user_id: user.id) # or with metadata event = ProfileCompleted.new( user_id: user.id, metadata: { ip: request.remote_ip } ) # then publish the event Railsy!::Events.publish(event)

Railsy RES initializer “my_engine.subscribe_to_events” do ActiveSupport.on_load “railsy-events” do |store| # async subscriber is invoked from background job, # enqueued after the current transaction commits store.subscribe MyEventHandler, to: ProfileCreated # anonymous handler (could only be synchronous) store.subscribe(to: ProfileCreated, sync: true) do |event| # do something end # subscribes to ProfileCreated automatically store.subscribe OnProfileCreated!::DoThat end end Convention over configuration

Railsy RES engines/ connect_by/ events/ connect_by/ users/ registered.rb chat_by/ subscribers/ connect_by/ users/ on_registered/ create_chat_account.rb

What we implement in the main app?

Main App • Authentication

Authentication module ConnectBy class ApplicationController < Engine.config.application_controller.constantize raise “Must include ConnectBy!::ControllerBehaviour” unless self < ConnectBy!::ControllerBehaviour raise “Must implement #current_user method” unless instance_methods.include?(:current_user) end end Engines don’t care about how do you obtain the current user

Main App • Authentication • Feature/system tests • Locales and mailers templates • Instrumentation and exception handling • Configuration

Gems Or stop putting everything into lib/ folder

Gems • Shared non-application specific code between engines • Isolated tests • Ability to share between applications in the future (e.g. , GitHub Package Registry)

Gems gems/ common-rubocop/ common-testing/ common-graphql/ …

common-rubocop • Standard RuboCop configuration (based on standard gem) • RuboCop plugins (e.g. , rubocoprspec) • Custom cops

common-rubocop # .rubocop.yml inherit_gem: common-rubocop: config/base.yml

common-testing • RSpec tools bundle (factory_bot, faker, test-prof, formatters) • Common spec_helper.rb and rails_helper.rb

common-testing # rails_helper.rb require “common/testing/rails_configuration”

common-graphql • Base classes (object, mutation) • Additional scalar types • Batch loaders • Testing helpers

Why engines? And why not

Why engines? • Modular monolith instead of monolith monsters or micro-services hell • Code (and tests) isolation • Loose coupling (no more spaghetti logic)

Why engines? • Easy to upgrade frameworks (e.g. , Rails) component by component • Easy to re-use logic in another app • …and we can migrate to micro-services in the future (if we’re brave enough)

Why not engines? • Not so good third-party gems support • Not so good support within Rails itself

Thanks! Спасибо!