This document describes how to bootstrap into the webapplication. The documentation is updated for Ubuntu 18.04 LTS and Debian 10 (Buster).
The following other libraries and programs are needed
-
Ruby > 2.3
-
Apache 2.4
-
MySQL > 8.0.4 or MariaDB 10.x
-
Git
-
Passenger for Apache 2
-
rails
-
ruby-bundler
-
libmysqlclient-dev
-
imagemagick
-
libmagickwand-dev
NOTE From Muscat 6.1 MySQL *8.0.4 is required* for autocomplete and comments to properly work, as the REGEX library was changed. MariaDB 10.x, as provided by Debian 10 (Buster) also seems to work properly.
Depending on your system, a package for Mysql 8 may not exist. In this case use the official package provided by Oracle dev.mysql.com/downloads/repo/apt/
NOTE: Download the latest version or the GPG keys may have been expired! Copy the file from the link provided in that page, there is no need for an account, follow the “No thanks, just start my download” link. In this case it points to dev.mysql.com/get/mysql-apt-config_0.8.15-1_all.deb
wget https://dev.mysql.com/get/mysql-apt-config_0.8.15-1_all.deb sudo dpkg -i mysql-apt-config_0.8.15-1_all.deb
(obviously subsitute with the exact file name in the link). dpk will prompt a screen to configure tha package, select option to configure the server version and select 8, then exit.
Then update and install mysql:
sudo apt-get update sudo apt-get install mysql-server
When prompted, select *use legacy passwords* as not all connectors for the moment work with the new system. This will be updated in the future. Also install the newest versions of the connector libraries:
sudo apt-get install libmysqlclient21 libmysqlclient-dev
If you are using Debian 10 (Buster), the default MariaDB server (mariadb-server, that pulls 10.3) and clients (ruby-mysql2) are just fine.
Muscat 6 requires Ruby 2.3 or higher and Bundler 2.1.4 or higher. Rails (5.2) is installed with the correct version with bundle.
Recent Ubuntu releases (from 18.4 LTS) and Debian 10 (Buster) provide come Ruby 2.5 by default so no actions are necessary. On Debian Buster, Bundler 2.1.4 is provided as a backport. Versions for Passenger, Java (as required for Solr) are also fine.
Get the sources if necessary (github.com/rism-ch/muscat and github.com/rism-ch/muscat-guidelines):
git clone https://github.com/rism-ch/muscat.git --recursive sudo bundle install #--deployment # deployment is only for the production system
Alternatively, you can just download and use the stable release tarballs found in github.com/rism-ch/muscat/releases.
Install base configuration:
cd muscat/config cp sample_database.yml database.yml cp sample_application.rb application.rb
Install the base css:
cd muscat/config cp muscat-custom-sample.scss ../vendor/assets/stylesheets/muscat-custom.scss
Make sure the directory is owned by the right user (www-data on Ubuntu or Debian)
sudo chown -R www-data:www-data muscat/
Set up databases access:
sudo mysql # Or log in with a user that has user creation privileges
Create the user, substitute with an appropriate user and password!
CREATE DATABASE muscat_development CHARACTER SET utf8 COLLATE utf8_general_ci; CREATE USER 'rism'@'localhost'; ALTER USER 'rism'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password'; GRANT ALL ON muscat_development.* TO 'rism'@'localhost';
Remember username and pass should be the same as in database.yml. Then, migrate the database (that us, update the structure):
bundle exec rake db:migrate # development
NOTE: in a production environment all the tasks must declare RAILS_ENV=production to run. For local development this is not necessary
sudo RAILS_ENV=production bundle exec rake db:migrate ## on a production system
Add basic dataset:
bundle exec rake db:seed
All the secret keys are now stored in the rails 5.2 credentials file, which is not included in the repo. A blank on must be created with:
rails credentials:edit
This will automatically create credentials.yml.enc and the master key, along with a cookie secret key. If you are not familiarised with this Rails 5.2 feature, this guide is very didactic: youtu.be/BHgvPPr2nLE.
Default configuration for Muscat requires the user to check the recaptcha challenge in order to download bibliographic records. So you will have to create a (free) Google Recaptcha account for your site (www.google.com/recaptcha/about/) in order to get the site_key and the secret_key.
The user and password are needed for background search and export, and should be a muscat user with minimum guest access. So, to use the other services on Muscat, such as history, add to this file:
recaptcha: site_key: 'xx' secret_key: 'xx' export: user: "xx" password: "xx"
Remember that they key must be readable from the user running apache, so check the permissions and change the user is needed:
chown www-data:www-data config/master.key
Default (development) startup:
bundle exec rake sunspot:solr:start rails s -e development
Try to open $IP_ADDRESS:3000/
For startup in production mode see “Starting Daemons in Production Mode”
For refreshing an installation in production mode (with sudo, as root or sudo -u www-data depending on the permissions on the installation dir):
bundle exec rake RAILS_ENV=production assets:clean bundle exec rake RAILS_ENV=production assets:precompile
A default administrative user has been created as part of the installation process. To log in, go to ‘ip:3000/admin` and log in with the following credentials:
username: [email protected] password: password
It is advised that you delete this account after creating a new administrative user in the admin interface.
rake sunspot:reindex
In production mode run:
RAILS_ENV=production rake sunspot:reindex
Specify only a model:
rake sunspot:reindex[,Person]
Do reindex in 1 record batches, useful if reindex crashes to see in which one (very slow to start)
rake sunspot:reindex[1]
With MySQL 64 bit binaries, add:
export DYLD_LIBRARY_PATH="/usr/local/mysql/lib:$DYLD_LIBRARY_PATH"
Example Apache configuration:
# create /usr/local/etc/apache/Includes/default.conf
# and add default site:
<VirtualHost *:80>
# ServerName www..org
DocumentRoot /var/rails/rism-ch/public
<Directory "/var/rails/rism-ch/public">
Options All -Indexes +ExecCGI -Includes +MultiViewsny
Require all granted
</Directory>
RailsEnv production
</VirtualHost>
Double check permissions in the muscat installation. Also make sure DocumentRoot and Directory point to public/ in the muscat installation location.
Start Apache, Solr and the related services in production mode (see below).
Muscat has three running daemons that should be started:
sudo RAILS_ENV=production bundle exec crono start # Make sure solr is not running as root! sudo -u www-data bundle exec rake RAILS_ENV=production sunspot:solr:start
‘DelayedJob` now is a bit more tricky, as different queues get different worker pools:
sudo -u www-data RAILS_ENV=production bin/delayed_job start --pool=reindex,triggers,folders:10 --pool=sub_reindex:10 --pool=resave --pool=export
Other ways to start delayed_job:
sudo RAILS_ENV=production bin/delayed_job start rake jobs:work # Run in foreground rake jobs:workoff # Run all jobs in foreground and exit rake jobs:clear # Clear failed jobs ## in production it is then... RAILS_ENV=production bundle exec rake jobs:<xxx>
Depending on the env, remove sudo if necessary. RAILS_ENV is needed only on production, and has to come after sudo.
It is also good to block access to Solr:
sudo iptables -A INPUT -p tcp -s localhost --dport 8983 -j ACCEPT sudo iptables -A INPUT -p tcp --dport 8983 -j DROP
Also make sure that if there is a proxy, it is configured in ‘config/initializers/recaptcha.rb`:
config.proxy = 'http://myproxy.com.au:8080'
0 2 * * * cd $PATH_TO && $PATH_TO/bin/rake blacklight:delete_old_searches[7] RAILS_ENV=production
Add to /etc/mysql.cnf
[mysqld] innodb_buffer_pool_size = 8G innodb_log_buffer_size = 512M innodb_log_file_size = 1G innodb_write_io_threads = 16 innodb_flush_log_at_trx_commit = 0
To speed up imports. (See here for more)
On a full muscat DB there can be many many _old versions_ in the “‘versions“` table, since version snapshots are kept for each saved item. When doing development this can be quite annoying since it can take up to 30 minutes to restore a db.
sed '/INSERT INTO `versions`/d' muscat_dump.sql > muscat_no_versions.sql
Removes all the old versions. From 30 minutes to 9.