Skip to content

CLI

Jump to bottom
rHorsey edited this page Oct 27, 2016 · 11 revisions

Openstudio-server CLI

Please begin by referring to the getting-started section of this document, which ensures that the ruby dependencies of the server are installed via openstudio_meta and that all other required programs are installed and updated. The remainder of the wiki is broken down by target type, local and remote, with a two finial sections, one devoted to analysis jobs and the one to troubleshooting.

Local targets are started and stopped using programs installed on your operating system and are highly-system dependent. As such, we only support OSX Yosemite and above, as well as Windows 7 and up. Currently, we provided limited support for Ubuntu 14.04 and above, as well as most CentOs 6 and 7 systems. For assistance with these systems, please refer to the troubleshooting section or the HPC knowledge store.

Remote targets are generally grouped into two sets: one's managed externally to the openstudio_meta, and those managed by openstudio_meta (currently only AWS OpenStudio-Server AMI instances.) The former are typically defined by a DNS which is used for running analysis jobs, however the later are significantly more complicated and make use of the AWS-SKD. Please refer to the remote section for more information.

##Getting-Started

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla ut porta enim, vitae hendrerit ligula. Sed lectus elit, cursus id nisi non, porta gravida magna. Nam vel accumsan nisl, eu porta dui. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In non nunc malesuada elit accumsan mollis. Vestibulum vel massa nec metus condimentum mollis. Fusce non diam quam. In non viverra felis, at euismod est. Maecenas quis porta dui. In quis lobortis tortor. Cras ac turpis in ipsum tempor suscipit a ut neque. Vivamus ultricies quam ut nisl porta, a imperdiet metus fringilla. Morbi vel pellentesque augue.

Suspendisse iaculis mollis interdum. Suspendisse ut augue at risus semper tristique. Cras laoreet aliquet nibh, id sagittis ex dignissim et. Sed non placerat odio, eget condimentum metus. Maecenas nec tincidunt nisi. Quisque imperdiet lacus sem, eu porta massa rhoncus sit amet. Nam vel malesuada urna, eget elementum eros. Vestibulum sollicitudin bibendum tortor vel molestie.

Pellentesque quis enim maximus, finibus sem quis, auctor odio. Etiam in tellus vitae elit lacinia euismod. Phasellus porta, enim ut facilisis feugiat, arcu turpis maximus tellus, vitae eleifend est lectus sed ex. Sed enim quam, feugiat nec eros a, sagittis faucibus orci. Vestibulum neque tortor, lacinia eu ante nec, varius ornare elit. Ut gravida urna ligula, non vehicula ex interdum eget. Suspendisse mattis sed mauris ut consequat.

Nam eget velit in massa auctor finibus et ac turpis. Sed tristique venenatis velit ut porta. Cras eu mollis neque. Praesent vitae libero justo. Suspendisse potenti. Praesent fringilla felis non ligula semper iaculis. Cras efficitur tempor nisi aliquam facilisis.

Aliquam erat volutpat. Donec eget laoreet ipsum. Curabitur id purus posuere, viverra est non, dapibus lacus. Nunc blandit ligula leo, in vestibulum erat gravida nec. Sed urna magna, varius at porta eu, tincidunt et augue. Cras nibh dui, pellentesque et blandit et, dictum id mi. Nullam eleifend, turpis a interdum efficitur, sem odio euismod urna, ac accumsan turpis elit nec mauris. Nam sed ipsum facilisis, ultrices sapien non, aliquam sapien. Nunc ultricies tortor aliquet metus accumsan, fringilla elementum eros mollis. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse vel leo ut velit egestas malesuada vitae at erat.

Suspendisse elementum purus laoreet ante ultricies, efficitur dignissim velit bibendum. Donec vel enim faucibus, sagittis felis nec, placerat ante. Duis bibendum scelerisque laoreet. Fusce rutrum volutpat imperdiet. Ut tempus interdum dignissim. Phasellus sodales commodo auctor. Duis convallis, ex ac ullamcorper egestas, nisl elit sodales lacus, ut bibendum enim leo ut mi. Duis vitae quam eget augue convallis consectetur a eu ipsum. Integer eu ex blandit, blandit erat eu, feugiat augue. Cras ut metus magna. Pellentesque mi felis, dictum et mauris accumsan, pulvinar ultrices libero. Integer ornare ut erat ac efficitur. Vestibulum a varius sapien. Morbi pulvinar sapien nec est pretium pretium. Phasellus sit amet ipsum ac lectus lacinia feugiat. Pellentesque magna elit, lacinia quis mattis aliquet, rhoncus in felis.

Proin consequat erat at tortor vehicula, ut sollicitudin arcu molestie. Proin tincidunt leo vehicula quam condimentum vestibulum. Sed laoreet dapibus lectus et malesuada. Donec mollis justo sed aliquet pretium. Nulla sed malesuada est. Etiam lorem mi, tincidunt at bibendum a, fringilla in metus. Mauris diam erat, tempus et luctus a, rhoncus vitae magna. Praesent feugiat, urna vitae sollicitudin faucibus, velit magna tempor erat, a commodo tellus urna nec sem. Aliquam interdum leo vulputate ultricies maximus. Curabitur mattis consectetur felis ut congue. Duis tortor justo, hendrerit vitae felis id, condimentum vehicula dui. Donec et sapien ultricies, dignissim tellus in, posuere massa. Fusce a dapibus lectus. Morbi vitae velit eu ipsum rutrum varius.

Nam venenatis tempor augue ut pulvinar. Nam suscipit urna nisl, eu volutpat diam ultrices in. Donec vestibulum porttitor nisl, quis dignissim ex. Quisque arcu enim, euismod ac dolor vitae, elementum placerat nibh. Fusce aliquam turpis vel diam tincidunt placerat. Phasellus eget eros efficitur, volutpat velit sit amet, vestibulum leo. Morbi blandit lorem vel magna vulputate luctus. Curabitur at odio ante. Maecenas et porta tellus. Aliquam arcu leo, mattis vel maximus in, fringilla ac ante. Fusce tristique sollicitudin lacinia. Ut egestas dui sit amet erat laoreet dapibus. Pellentesque commodo est id semper gravida. Aenean odio nibh, interdum non posuere sit amet, facilisis sit amet risus.

Duis ornare ipsum ac odio sodales scelerisque. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Mauris vestibulum at metus nec mattis. In at urna purus. Nulla facilisi. Pellentesque vestibulum condimentum laoreet. Donec semper pellentesque tellus, vitae feugiat est ullamcorper vel. Nullam est diam, gravida in mattis sit amet, suscipit non elit. Donec eleifend pulvinar pretium. Proin consequat faucibus ipsum, consectetur venenatis nulla gravida sed. Aliquam erat volutpat. Integer porta pretium lorem, at tempus orci molestie vel.

Morbi eu mi cursus, vulputate lacus sed, pellentesque ante. Suspendisse potenti. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nulla at tempor enim, id blandit metus. Donec eu blandit magna. Vestibulum aliquam sollicitudin odio vel malesuada. Sed tempor felis sit amet mauris interdum viverra. Sed et lectus posuere, rutrum ex in, efficitur sapien. Vivamus metus sem, viverra et erat ut, consectetur pharetra sem. Pellentesque ac sapien ac nibh sollicitudin tincidunt a ac dui. Donec fringilla nisi vel libero venenatis, luctus tincidunt nulla convallis. In feugiat a eros vel dapibus. Pellentesque accumsan laoreet efficitur. Aenean placerat rhoncus dictum.

##Local

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla ut porta enim, vitae hendrerit ligula. Sed lectus elit, cursus id nisi non, porta gravida magna. Nam vel accumsan nisl, eu porta dui. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In non nunc malesuada elit accumsan mollis. Vestibulum vel massa nec metus condimentum mollis. Fusce non diam quam. In non viverra felis, at euismod est. Maecenas quis porta dui. In quis lobortis tortor. Cras ac turpis in ipsum tempor suscipit a ut neque. Vivamus ultricies quam ut nisl porta, a imperdiet metus fringilla. Morbi vel pellentesque augue.

Suspendisse iaculis mollis interdum. Suspendisse ut augue at risus semper tristique. Cras laoreet aliquet nibh, id sagittis ex dignissim et. Sed non placerat odio, eget condimentum metus. Maecenas nec tincidunt nisi. Quisque imperdiet lacus sem, eu porta massa rhoncus sit amet. Nam vel malesuada urna, eget elementum eros. Vestibulum sollicitudin bibendum tortor vel molestie.

Pellentesque quis enim maximus, finibus sem quis, auctor odio. Etiam in tellus vitae elit lacinia euismod. Phasellus porta, enim ut facilisis feugiat, arcu turpis maximus tellus, vitae eleifend est lectus sed ex. Sed enim quam, feugiat nec eros a, sagittis faucibus orci. Vestibulum neque tortor, lacinia eu ante nec, varius ornare elit. Ut gravida urna ligula, non vehicula ex interdum eget. Suspendisse mattis sed mauris ut consequat.

Nam eget velit in massa auctor finibus et ac turpis. Sed tristique venenatis velit ut porta. Cras eu mollis neque. Praesent vitae libero justo. Suspendisse potenti. Praesent fringilla felis non ligula semper iaculis. Cras efficitur tempor nisi aliquam facilisis.

Aliquam erat volutpat. Donec eget laoreet ipsum. Curabitur id purus posuere, viverra est non, dapibus lacus. Nunc blandit ligula leo, in vestibulum erat gravida nec. Sed urna magna, varius at porta eu, tincidunt et augue. Cras nibh dui, pellentesque et blandit et, dictum id mi. Nullam eleifend, turpis a interdum efficitur, sem odio euismod urna, ac accumsan turpis elit nec mauris. Nam sed ipsum facilisis, ultrices sapien non, aliquam sapien. Nunc ultricies tortor aliquet metus accumsan, fringilla elementum eros mollis. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse vel leo ut velit egestas malesuada vitae at erat.

Suspendisse elementum purus laoreet ante ultricies, efficitur dignissim velit bibendum. Donec vel enim faucibus, sagittis felis nec, placerat ante. Duis bibendum scelerisque laoreet. Fusce rutrum volutpat imperdiet. Ut tempus interdum dignissim. Phasellus sodales commodo auctor. Duis convallis, ex ac ullamcorper egestas, nisl elit sodales lacus, ut bibendum enim leo ut mi. Duis vitae quam eget augue convallis consectetur a eu ipsum. Integer eu ex blandit, blandit erat eu, feugiat augue. Cras ut metus magna. Pellentesque mi felis, dictum et mauris accumsan, pulvinar ultrices libero. Integer ornare ut erat ac efficitur. Vestibulum a varius sapien. Morbi pulvinar sapien nec est pretium pretium. Phasellus sit amet ipsum ac lectus lacinia feugiat. Pellentesque magna elit, lacinia quis mattis aliquet, rhoncus in felis.

Proin consequat erat at tortor vehicula, ut sollicitudin arcu molestie. Proin tincidunt leo vehicula quam condimentum vestibulum. Sed laoreet dapibus lectus et malesuada. Donec mollis justo sed aliquet pretium. Nulla sed malesuada est. Etiam lorem mi, tincidunt at bibendum a, fringilla in metus. Mauris diam erat, tempus et luctus a, rhoncus vitae magna. Praesent feugiat, urna vitae sollicitudin faucibus, velit magna tempor erat, a commodo tellus urna nec sem. Aliquam interdum leo vulputate ultricies maximus. Curabitur mattis consectetur felis ut congue. Duis tortor justo, hendrerit vitae felis id, condimentum vehicula dui. Donec et sapien ultricies, dignissim tellus in, posuere massa. Fusce a dapibus lectus. Morbi vitae velit eu ipsum rutrum varius.

Nam venenatis tempor augue ut pulvinar. Nam suscipit urna nisl, eu volutpat diam ultrices in. Donec vestibulum porttitor nisl, quis dignissim ex. Quisque arcu enim, euismod ac dolor vitae, elementum placerat nibh. Fusce aliquam turpis vel diam tincidunt placerat. Phasellus eget eros efficitur, volutpat velit sit amet, vestibulum leo. Morbi blandit lorem vel magna vulputate luctus. Curabitur at odio ante. Maecenas et porta tellus. Aliquam arcu leo, mattis vel maximus in, fringilla ac ante. Fusce tristique sollicitudin lacinia. Ut egestas dui sit amet erat laoreet dapibus. Pellentesque commodo est id semper gravida. Aenean odio nibh, interdum non posuere sit amet, facilisis sit amet risus.

Duis ornare ipsum ac odio sodales scelerisque. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Mauris vestibulum at metus nec mattis. In at urna purus. Nulla facilisi. Pellentesque vestibulum condimentum laoreet. Donec semper pellentesque tellus, vitae feugiat est ullamcorper vel. Nullam est diam, gravida in mattis sit amet, suscipit non elit. Donec eleifend pulvinar pretium. Proin consequat faucibus ipsum, consectetur venenatis nulla gravida sed. Aliquam erat volutpat. Integer porta pretium lorem, at tempus orci molestie vel.

Morbi eu mi cursus, vulputate lacus sed, pellentesque ante. Suspendisse potenti. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nulla at tempor enim, id blandit metus. Donec eu blandit magna. Vestibulum aliquam sollicitudin odio vel malesuada. Sed tempor felis sit amet mauris interdum viverra. Sed et lectus posuere, rutrum ex in, efficitur sapien. Vivamus metus sem, viverra et erat ut, consectetur pharetra sem. Pellentesque ac sapien ac nibh sollicitudin tincidunt a ac dui. Donec fringilla nisi vel libero venenatis, luctus tincidunt nulla convallis. In feugiat a eros vel dapibus. Pellentesque accumsan laoreet efficitur. Aenean placerat rhoncus dictum.

##Remote

There are two primary CLI commands which control remote OpenStudio-server instances. These are start_remote and stop_remote. Both of these commands use the Amazon Web Services (AWS) Ruby SDK to instantiate instances of the OpenStudio-Server on AWS using pre-built Amazon Machine Images (AMIs) deployed on the Elastic Cloud Computing (EC2) infrastructure. To use AWS first an account needs to be created and configured, next several environment variables must be set, followed by defining the server parameters via a JSON document, and finally openstudio_meta start_remote must be called with the correct parameters.

Setting Up an Account

  1. Begin by going to [AWS] (http://aws.amazon.com). Click the Sign Up button in the top right corner.
  2. Enter your email or mobile number and click I am a new user. before clicking on the Sign in button.
  3. Be aware while progressing through the next screens that it is important to use strong passwords, as malicious activity can easily go undetected. Additionally it is important to provide Amazon with a phone number you are accessible at so you can be reached in case of potentially fraudulent activity on your account.
  4. When you reach 'aws.amazon.com/registration-confirmation' click on the Launch Management Console button, which should redirect you to 'console.aws.amazon.com/console/home'. Under Administration & Security click on Identity & Access Management. See the links for setting up an MFA, non-root user, non-root group permission, and using the management console. It is critical to complete all five items listed under Security Status.
  5. Download, Screenshot, and save your Access Key ID and Secret Access Key when creating a IAM User. These will never again be available to you.
    *When creating an individual IAM user it is recommended that Internet Explorer not be used, as there is a known bug on the page only experienced by IE users.

Once all five steps have been completed, consider using an [alias] (http://docs.aws.amazon.com/IAM/latest/UserGuide/AccountAlias.html) for your user account and log into your non-root account.

Setting the Environment

Three environment variables are required for the start_remote command. They are AWS_ACCESS_KEY, AWS_SECRET_KEY, and AWS_DEFAULT_REGION.

AWS_ACCESS_KEY

This key specifies the account to create the AMIs on. It is not secret, however should not be shared publicly regardless.

AWS_SECRET_KEY

This key serves as authentication of any requests to start, stop, or otherwise alter the state of AMIs. It is critically important that it is not shared, and if it is you must immediately suspend the key on your account. See this help page for step-by-step instructions.

AWS_DEFAULT_REGION

This key defines which region the EC2 servers should be started in. Currently, OpenStudio-Server AMIs are only made available by default in the us-east-1 region. For help using other regions please contact @rhorsey or @nllong at [email protected] or [email protected].

Setting Environment Variables

We recommend only setting these values in a single shell / cmd.exe at a time. To do so type the following commands for each environment variable specified above, substituting the correct environment variable key and value.

For UNIX/LINUX users:

export TYPE_ENV_VAR_KEY_HERE=the-environment-variable-value

For Windows users:

set TYPE_ENV_VAR_KEY_HERE=the-environment-variable-value

Defining the Remote Server Configuration

A final piece of input is required to utilize EC2 servers, namely, a specification of which servers to use with which AMIs. A server configuration file is required by the start_remote command of the following form:

{
   "cluster_name":"example123",
   "user_id":"jdoe",
   "server_instance_type":"c3.xlarge",
   "worker_instance_type":"c3.xlarge",
   "worker_node_number":1,
   "aws_tags":[

   ],
   "openstudio_server_version":"1.20.0-rc1"
}

The cluster_name field should not contain spaces or unusual characters, however can largely be set at the users descrection. The user_id field is used to determine the instance creater / owner in the AWS EC2 meta-data. The viable values of server_instance_type and worker_instance_type are defined by Amazon here in the server model column. The server should have at least 6 cores and substantial storage. The worker_node_number can be set to 0, although this tends to result in long analysis runs. the aws_tags field allows for institutional identifiers to be passed through to the AWS EC2 metadata. Finally, the openstudio_server_version defines which AMI version of the OpenStudio-Server to use. These are documented here. Once this JSON file is written to disk, the actual openstudio_meta start_remote command can be executed.

Executing start_remote

You should now be able to use the openstudio_meta start_remote AWS functionality. The -s or --server_config FILE argument is used to specify the server configuration JSON file, and the -p or --project DIRECTORY argument is used to specify where SSH keys and system configuration details should be saved to. The target of the start_remote command is aws. As such, the command would look like

ruby openstudio_meta start_remote -s /path/to/my/server/config.json -p /path/to/a/directory aws
# or, with substantial logging
ruby openstudio_meta start_remote -s /path/to/my/server/config.json -p /path/to/a/directory aws --verbose --debug

Jobs

Troubleshooting

Clone this wiki locally