Skip to content
This repository has been archived by the owner on Apr 23, 2022. It is now read-only.

Latest commit

 

History

History
370 lines (245 loc) · 7.12 KB

slides.md

File metadata and controls

370 lines (245 loc) · 7.12 KB
transition logo
fade

CI4Py

Continuous Integration for Python

JDEV T4.A07

Marseille (France) - 2017/07/05

$ whoami

  • :fa-twitter: @julienmaupetit
  • PhD in structural bioinformatics (Paris Diderot)
  • Research engineer (RPBS platform, Paris Diderot)
  • Co-founder of TailorDev (Clermont-Ferrand)

What about you?

Disclaimer

Do you git, bash, python?

Outline

  1. A brief theoretical introduction to CI
  2. Configure GitLab-CI
  3. Test an application with py.test

Install party!

For the practical part, you will need:

  • a UNIX shell (bash, zsh, …)
  • git
  • python >= 3.4
  • ping www.google.fr

Ready?

Goal

Test & add CI to climate, a python application that analyzes average temperature records by country since the 18th century.

Climate example output

Continuous Integration 101

Captain Obvious ™️

There is no continuous integration for applications with no tests.

Open Source good practices

Shipping an application (or a feature) is an inherent bundle including code, tests and documentation.

:fa-github: github.com

:fa-gitlab: gitlab.com

CI Platforms

(Third party) services can be integrated to GitHub or GitLab to provide continuous/automated testing (aka Continuous Integration).

CI pipeline

  1. Build the software from sources,
  2. Run the test suite,
  3. Report results to the development team,
  4. Integrate changes to the main repository/branch.

CI triggers

  • New branch
  • New Pull/Merge Request (PR/MR)
  • Merged feature branch

git workflow (1)

We recommend to use the Simple Git branching model

git workflow (2)

  • The master branch is the current stable version
  • The master branch can be safely deployed in production
  • All other branches are feature branches
  • Releases are performed by tagging the master branch

For more information about Continuous Delivery (CD), see: https://tailordev.fr/blog/2017/04/05/why-we-always-start-with-ci-cd/

Ready?

Bootstraping

GitLab

Virtualenv

Clone the project

$ cd my/projects/directory
# substitute your-account with… your GitLab login
$ git clone [email protected]:your-account/jdev-2017-ci4py.git

Create a virtualenv for your CI project:

$ cd jdev-2017-ci4py
$ python3 -m venv venv

Activate the newly created virtualenv:

$ source venv/bin/activate
# Now your prompt should be modified:
(venv) $

Install dependencies

# Install development dependencies
(venv) $ pip install -r requirements-dev.txt
# Install the app
(venv) $ python setup.py develop

Now, you should be able to work on the project 🎉

Project tree

├── AUTHORS
├── LICENSE
├── MANIFEST.in
├── README.md
├── bin
│   └── factrump
├── climate
│   ├── __init__.py
│   ├── data
│   │   ├── temperature_by_country.csv
│   │   └── temperature_by_country_light.csv
│   ├── exceptions.py
│   ├── temperature.py
│   └── utils.py
├── examples
│   └── madagascar.png
├── requirements-dev.txt
├── requirements.txt
├── setup.cfg
├── setup.py
└── tests
    ├── __init__.py
    └── test_temperature.py

Check py.test & flake8 configuration

# setup.cfg
[tool:pytest]
addopts = -vs --cov-report term-missing --cov=climate
testpaths = tests

[flake8]
exclude =
  examples,
  venv

Run the test suite

(venv) $ pytest
============================== test session starts ==============================
platform darwin -- Python 3.6.0, pytest-3.1.2, py-1.4.34, pluggy-0.4.0 -- /Users/maupetit/projects/TailorDev/Workshops/2017/JDEV/ci4py/venv/bin/python3
cachedir: .cache
rootdir: /Users/maupetit/projects/TailorDev/Workshops/2017/JDEV/ci4py, inifile: setup.cfg
plugins: cov-2.5.1
collected 1 items

tests/test_temperature.py::test_temperature_data_loading PASSED

---------- coverage: platform darwin, python 3.6.0-final-0 -----------
Name                     Stmts   Miss  Cover   Missing
------------------------------------------------------
climate/__init__.py          1      0   100%
climate/exceptions.py        2      0   100%
climate/temperature.py      43     23    47%   45, 53-58, 65-67, 75-104
climate/utils.py            10      7    30%   10-27
------------------------------------------------------
TOTAL                       56     30    46%


=========================== 1 passed in 1.77 seconds ============================

Integrate GitLab CI

Read the docs!

.gitlab-ci.yml

# First things first: create a new branch to work on
(venv) $ git checkout -b add-gitlab-ci
(venv) $ touch .gitlab-ci.yml

Steps:

  • install dependencies
  • lint your code with flake8
  • run the test suite

You should read the documentation first: https://docs.gitlab.com/ce/ci/quick_start/README.html

Test project CI

# Commit your changes
(venv) $ git add .gitlab-ci.yml
(venv) $ git commit -m 'Add draft gitlab-ci integration'
# Push your branch to YOUR repo
(venv) $ git push origin add-gitlab-ci

Now it's time to open your first Merge Request! 🎉

Go to gitlab.com/your-account/jdev-2017-ci4py

Configure your project

Go to Settings > General > Merge Request

:fa-check: Only allow merge requests to be merged if the pipeline succeeds

.gitlab-ci.yml (solution)

image: tailordev/pandas

# This folder is cached between builds
# http://docs.gitlab.com/ce/ci/yaml/README.html#cache
cache:
  paths:
  - ~/.cache/pip/

# Install the project and dependencies
before_script:
  # Print out python version for debugging
  - python -V
  - pip install -r requirements-dev.txt
  - python setup.py install

stages:
  - lint
  - test

flake8:
  stage: lint
  script:
    flake8

test climate:
  stage: test
  script:
    pytest

Test the app with py.test

Test, test, test

Increase test coverage to ~100% 💪

https://docs.pytest.org/en/latest/

That's all folks!