Skip to content

Commit

Permalink
Merge branch 'feature/DOPS-1585/remove-jenkins' of github.com:hyperle…
Browse files Browse the repository at this point in the history
…dger/iroha into feature/DOPS-1585/remove-jenkins
  • Loading branch information
safinsaf committed Feb 18, 2022
2 parents 5cda6f7 + 2079685 commit 510d697
Show file tree
Hide file tree
Showing 48 changed files with 1,169 additions and 569 deletions.
1 change: 1 addition & 0 deletions .github/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/matrix*
68 changes: 68 additions & 0 deletions .github/_README.make-workflows.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
make-workflows.sh for GitHub Actions
====================================

GitHub Workflow description in YAML does not support anchors.
There are several workarounds => anyway [they](#links) come to building-editing workflow yaml from source.
So I suggest yet another one `make-workflows.sh` based on YAML tool [`yq`](https://github.com/mikefarah/yq) version 4.

<sub>\<spoiler\>All these code and repo is written around `yq eval 'explode(.)'`\</spoiler\></sub>

### USAGE
0. [Install](#ways-to-install) `make-workflows.sh` to reachable place inside or outside of your repo, i.e. '.github/'
1. Put your workflows to `.github/*.src.yml`
2. (recommended) `pre-commit install` and edit [`.pre-commit-config.yaml`](/.pre-commit-config.yaml) according to where [`make-workflows.sh`](./make-workflows.sh) is placed.
<sub>(altenative optional) Copy or link `pre-commit-hook.sh` to `.git/hooks/pre-commit`
Like `ln -s ../../.github/pre-commit-hook.sh .git/hooks/pre-commit`</sub>

```
$ ./make-workflows.sh --help
make-workflows:
This script expands '*.src.yml' from $1..$[N-1] (default: REPO_ROOT/.github/)
to $N (default:REPO_ROOT/.github/workflows/) with corresponding name '*.yml'
Main goal is to dereference YAML anchors.
Deals only with Git cached/indexed files until --worktree passed.
DEBUG: use option -x
NOTE: spaces in filenames are not allowed to keep code simplicity.
Usage:
make-workflows.sh [--worktree] [dirs_from... [dir_to]]
make-workflows.sh [--help]
Options:
--worktree List files and get contents from working tree
instead of git index
-h, --help show this help
-x, --trace, +x, --no-trace enable/disable bash trace
-i, --install
--update
-V, --version
```

### Automate using pre-commit (recommended)
There is a nice tool [pre-commit](https://pre-commit.com) to do checks and some actions just before commit. The tool is called by Git pre-commit hook.

Making workflows is better being automated – just
```sh
$ pre-commit install
```
and add next sample to [`.pre-commit-config.yaml`](/.pre-commit-config.yaml)
```yaml
repos:
- repo: local
hooks:
- id: make-workflows
name: Make GitHub workflows from *.src.yml
entry: bash -c '.github/make-workflows.sh && git add .github/workflows'
language: system
files: '.github/.*\.src\.ya?ml'
pass_filenames: false
```
> NOTE: pay attention to path to `make-workflows.sh`

> NOTE2: pay attention to path(s) where source files are stored `files: 'PATH_REGEXP'`


### Links
1. https://stackoverflow.com/questions/67368724/share-same-steps-for-different-github-actions-jobs
2. https://github.community/t/support-for-yaml-anchors/16128/60
3. https://github.com/mithro/actions-includes
4. https://github.com/allejo/gha-workflows
5. dedicated repo https://github.com/kuvaldini/make-workflows.sh
90 changes: 63 additions & 27 deletions .github/_README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,67 @@
GitHub Actions
==============
GitHub Actions CI for Iroha
===========================

GitHub Workflow description in YAML does not support anchors.
There are several workarounds => anyway they come to building-editing workflow yaml from source.
So I suggest yet another one `make-workflows.sh` based on YAML tool `yq`.

### USAGE
0. Move your workflows to `.github/*.src.yml`
1. Put `make-workflows.sh` to directory `.github/`
2. (optional) Copy or link `pre-commit-hook.sh` to `.git/hooks/pre-commit`
Like `ln -s ../../.github/pre-commit-hook.sh .git/hooks/pre-commit`
> ### For the **smooth experience** please `pre-commit install` after `git clone`.
### Using pre-commit
```yaml
repos:
- repo: local
hooks:
- id: make-workflows
name: Make GitHub workflows from *.src.yml
entry: bash -c '.github/make-workflows.sh && git add .github/workflows'
language: system
types: [yaml]
pass_filenames: false
```
---------------------

### Links
1. https://stackoverflow.com/questions/67368724/share-same-steps-for-different-github-actions-jobs
2. https://github.community/t/support-for-yaml-anchors/16128/60
3. https://github.com/mithro/actions-includes
4. https://github.com/allejo/gha-workflows
USAGE
-----
GitHub Actions Workflow [`Iroha1`](build-iroha1.src.yml) solves task of automated build and deployment Iroha1.
There are events when it is running:
- on **pull request** to Iroha1 main and development branches
- on **push** to main or development branches including event when PR is **merged**
- on **workflow dispatch** to run WF manually on special branch with defined buildspec through web interface or via CLI tool
- **scheduled** every night
- _(under construction PR #XX) on **comment to PR** which contains buildspec._

Default `buildspec` is _`/build all`_

### Buildspec
Build matrix is a way to select among number of configurations to be built.
Build matrix is generated from buildspec string and handled by script [`chatops-gen-matrix.sh`](./chatops-gen-matrix.sh)

## List of files
- `build-iroha1.src.yml`
Main file here. GitHub workflow YAML description with ANCHORS, code is not duplicated.
IMPORTANT: regeneration required after after edit, which is automated with pre-commit.
- `workflows/build-iroha1.yml`
Result worflow taken by GitHub and generated with make-workflows script. Long file of repeated code. DO NOT EDIT MANUALLY.
- `make-workflows.sh`
A tool to generate workflows/*.yml from *.src.yml - evaluates anchors. [Read the docs](_README.make-workflows.md).
- `chatops-gen-matrix.sh`
Generates build matrixes form convenient user input. See `--help`
```
USAGE:
chatops-gen-matrix.sh --help
chatops-gen-matrix.sh /build ubuntu clang
chatops-gen-matrix.sh '/build ubuntu clang; /build macos release ursa'
echo /build [build_spec...] | chatops-gen-matrix.sh
EXAMPLE build_spec:
/build ubuntu release gcc10
/build macos llvm release; /build macos clang ursa release
/build all
/build ubuntu all ## build all possible configurations on Ubuntu
/build ubuntu burrow all ## build release and debug on Ubuntu with Burrow
AVAILABLE build_spec keywords:
ubuntu|linux
macos
windows
normal
burrow
ursa
release|Release
debug|Debug
gcc|gcc-9|gcc9
gcc-10|gcc10
clang|clang-10|clang10
llvm
msvc
skip-testing|skip_testing
all|everything|beforemerge|before_merge|before-merge|readytomerge|ready-to-merge|ready_to_merge
```
- `pre-commit-hook.sh`
See docs of make-workflows. Use instead of pre-commit as `ln -s ../../.github/pre-commit-hook.sh .git/hooks/pre-commit`, reserv alternative.
- `TESTS_ALLOWED_TO_FAIL`
One day tests of Iroha became failing. To fix CI and postpone fixing tests, this file was invented. It allows CI to pass even when listed tests are failing. DO NOT USE UNLESS YOU DEFINITELY KNOW WHAT'S GOING. KEEP IT EMPTY.
Loading

0 comments on commit 510d697

Please sign in to comment.