Skip to content

Commit

Permalink
Merge pull request #67 from nchammas/packaging
Browse files Browse the repository at this point in the history
Prepare Flintrock for its first release
  • Loading branch information
nchammas committed Dec 12, 2015
2 parents ee27fc2 + 5091dc2 commit 37b7908
Show file tree
Hide file tree
Showing 31 changed files with 246 additions and 99 deletions.
3 changes: 1 addition & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ python:
- "3.4"
- "3.5"
install:
- "pip install -r requirements.txt"
- "pip install -r requirements-dev.txt"
- "pip install -r requirements/developer.pip"
script:
- "py.test ./tests/test_style.py"
11 changes: 11 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Flintrock Change Log


## Unreleased

* Nothing yet.


## 0.1.0 - 2015-12-11

* Initial release.
15 changes: 6 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,16 @@
# Contributing Guide

*Note: This guide was inspired in part by the excellent [Phabricator contributor docs](https://secure.phabricator.com/book/phabcontrib/).*

**Flintrock is still under heavy development and has no releases at this time.**

**Until we make an initial release, most PRs will be rejected.** We are likely already fixing or making that thing you want to open a PR for, and major components are rapidly evolving so your PR is likely to be invalidated by upcoming changes you may not know about.

There are many ways to contribute to Flintrock.

## Contributing Thanks

When we put our time and enthusiasm into an open source project like this, we hope that somewhere out there we are helping people by saving them time and frustration. But most of the time, we'll never know what kind of impact we made, if any, on people's lives.
When we put our time and enthusiasm into an open source project like this, we hope that somewhere out there we are putting a smile on someone's face.

Most of the time we'll never know, though. When people reach out within an open source community, it's typically to report a problem, ask for help, or share an idea.

Hearing first-hand of the positive impact we had on someone else's day, even if it is minor, can be a huge boost of joy and motivation.
That's a bummer, because hearing first-hand that we made a positive impact on someone else's day, even if it's minor, can be a huge boost of joy and motivation.

Don't underestimate the power of a thank you. If Flintrock helped you in some way, share your story, no matter how long or short, and know that at times it can be the most valuable way to contribute to the project.
Don't underestimate the power of a thank you. If Flintrock helped you in some way, share your story, even if it's "trivial", and know that at times this can be the most valuable way to contribute to the project.


## Contributing Money
Expand Down
10 changes: 10 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# See: https://docs.python.org/3/distutils/commandref.html
graft flintrock

include README.md
include CHANGELOG.md
include COPYRIGHT
include LICENSE

global-exclude *.py[cod] __pycache__ .DS_Store
global-exclude config.yaml
61 changes: 40 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
![Flintrock logo](flintrock-logo.png)
![Flintrock logo](https://raw.githubusercontent.com/nchammas/flintrock/master/flintrock-logo.png)

[![Build Status](https://travis-ci.org/nchammas/flintrock.svg)](https://travis-ci.org/nchammas/flintrock)
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/nchammas/flintrock)

Flintrock is a command-line tool and library for launching [Apache Spark](http://spark.apache.org/) clusters.

**Flintrock is currently undergoing heavy development. Until we make a 1.0 release, you probably should not use Flintrock unless you are ready to keep up with frequent changes to how it works.** Python hackers or heavy spark-ec2 users who are looking to experiment with something new are welcome to try Flintrock out and potentially even [contribute](CONTRIBUTING.md).
**Flintrock is currently undergoing heavy development. Until we make a 1.0 release, you probably should not use Flintrock unless you are ready to keep up with frequent changes to how it works.** Python hackers or heavy spark-ec2 users who are looking to experiment with something new are welcome to try Flintrock out and potentially even [contribute](https://github.com/nchammas/flintrock/blob/master/CONTRIBUTING.md).


## Usage
Expand All @@ -16,7 +16,7 @@ Here's a quick way to launch a cluster on EC2, assuming you already have an [AWS
flintrock launch test-cluster \
--num-slaves 1 \
--no-install-hdfs \
--spark-version 1.5.1 \
--spark-version 1.5.2 \
--ec2-key-name key_name \
--ec2-identity-file /path/to/key.pem \
--ec2-ami ami-60b6c60a \
Expand All @@ -40,7 +40,7 @@ Other things you can do with Flintrock include:
```sh
flintrock login test-cluster
flintrock describe test-cluster
flintrock run-command test-cluster 'yum install -y package'
flintrock run-command test-cluster 'sudo yum install -y package'
flintrock copy-file test-cluster /local/path /remote/path
```

Expand All @@ -56,31 +56,48 @@ That's not all. Flintrock has a few more [features](#features) that you may find

## Installation

Before using Flintrock, take a quick look at the [copyright](COPYRIGHT) notice and [license](LICENSE) and make sure you're OK with their terms.
Before using Flintrock, take a quick look at the [copyright](https://github.com/nchammas/flintrock/blob/master/COPYRIGHT) notice and [license](https://github.com/nchammas/flintrock/blob/master/LICENSE) and make sure you're OK with their terms.

Flintrock requires Python 3.4 or newer. Since we don't have any releases yet, the only way to install Flintrock at the moment is as follows:
Flintrock requires Python 3.4 or newer. It's currently been tested only on OS X, but it should run on all POSIX systems. We have plans to [add Windows support](https://github.com/nchammas/flintrock/issues/46) in the future, too.

Eventually, we also plan to release stand-alone executables so that you can install Flintrock without having to worry about having Python installed.

### Release version

To get the latest release of Flintrock, simply run [pip](https://pip.pypa.io/en/stable/):

```
python3 -m pip install flintrock
```

This will install Flintrock and place it on your path. You should be good to go now!

You'll probably want to get started with the following two commands:

```sh
# Download Flintrock.
git clone https://github.com/nchammas/flintrock
flintrock --help
flintrock configure
```

### Development version

If you like living on the edge, or if you want to [contribute](https://github.com/nchammas/flintrock/blob/master/CONTRIBUTING.md), you can install the development version of Flintrock like this:

# Set your defaults.
```sh
git clone https://github.com/nchammas/flintrock
cd flintrock
cp config.yaml.template config.yaml
# vi config.yaml

# Install Flintrock's dependencies.
# Setup a virtual environment.
# Optional, but *strongly recommended*.
python3 -m venv venv
source venv/bin/activate
python3 -m pip install -r requirements.txt
deactivate

# You're good to go now.
./flintrock --help
# Install Flintrock.
# If you want to contribute, install the developer requirements.
python3 -m pip install -r requirements/user.pip
python3 -m pip install -r requirements/developer.pip
```

Eventually, we plan to release binaries so that you can install Flintrock without having to worry about having Python installed.


## Use Cases

Expand Down Expand Up @@ -137,7 +154,9 @@ flintrock launch test-cluster --num-slaves 10

### Configurable CLI Defaults

Flintrock lets you persist your desired configuration to a file (called `config.yaml` by default) so that you don't have to keep typing out the same options over and over at the command line.
Flintrock lets you persist your desired configuration to a YAML file so that you don't have to keep typing out the same options over and over at the command line.

To setup and edit the default config file, call `flintrock configure`. You can also point Flintrock to a non-default config file by using the `--config` option.

#### Sample `config.yaml`

Expand Down Expand Up @@ -204,7 +223,7 @@ Flintrock automatically configures any available [ephemeral storage](http://docs

### Tests

Flintrock comes with a set of automated, end-to-end [tests](./tests). These tests help us develop Flintrock with confidence and guarantee a certain level of quality.
Flintrock comes with a set of automated, end-to-end [tests](https://github.com/nchammas/flintrock/tree/master/tests). These tests help us develop Flintrock with confidence and guarantee a certain level of quality.

### Low-level Provider Options

Expand Down Expand Up @@ -257,4 +276,4 @@ There are a few additional peeves I had with spark-ec2 -- some of which are diff

## About the Flintrock Logo

The [Flintrock logo](flintrock-logo.png) was created using [Highbrow Cafetorium JNL](http://www.myfonts.com/fonts/jnlevine/highbrow-cafetorium/) and [this icon](https://thenounproject.com/term/stars/40856/). Licenses to use both the font and icon were purchased from their respective owners.
The [Flintrock logo](https://github.com/nchammas/flintrock/blob/master/flintrock-logo.png) was created using [Highbrow Cafetorium JNL](http://www.myfonts.com/fonts/jnlevine/highbrow-cafetorium/) and [this icon](https://thenounproject.com/term/stars/40856/). Licenses to use both the font and icon were purchased from their respective owners.
3 changes: 0 additions & 3 deletions flintrock

This file was deleted.

1 change: 1 addition & 0 deletions flintrock/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__version__ = '0.1.0'
5 changes: 5 additions & 0 deletions flintrock/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# See: https://docs.python.org/3/library/__main__.html
from .flintrock import main

if __name__ == '__main__':
main()
File renamed without changes.
Loading

0 comments on commit 37b7908

Please sign in to comment.