Skip to content

Commit

Permalink
initial version
Browse files Browse the repository at this point in the history
  • Loading branch information
kamil-olszewski-devskiller committed Dec 13, 2024
0 parents commit 99fd4ca
Show file tree
Hide file tree
Showing 5 changed files with 334 additions and 0 deletions.
4 changes: 4 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FROM python:3.13-slim
COPY entrypoint.py /entrypoint.py
RUN pip install "requests>=2.32.3"
ENTRYPOINT [ "python3", "/entrypoint.py" ]
33 changes: 33 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# DevSkiller TalentScore - Import tasks from YAML

This GitHub Action uploads custom MCQ/Essay/CodeGaps tasks to the DevSkiller TalentScore platform. It allows you to easily integrate the import from YAML process into your CI/CD pipeline.

## Inputs

### `api_key`
**Required**: The TalentScore API key. This key is needed to authenticate the upload request.

### `path`
**Required**: Path to a YAML file with tasks. [Description of the file structure](yaml-file-structure.md)

## Example

```yaml
name: Sample YAML import

on:
push:
branches:
- master

jobs:
upload-task:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- uses: Devskiller/[email protected]
with:
api_key: ${{ secrets.TALENTSCORE_API_KEY }}
path: ./java-tasks.yml
```
18 changes: 18 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Import tasks from YAML to TalentScore
description: Upload your custom MCQ/Essay/CodeGaps tasks to the DevSkiller TalentScore platform
author: DevSkiller
branding:
icon: upload-cloud
color: blue

inputs:
api_key:
description: "TalentScore API key"
required: true
path:
description: "The path of the yaml file with tasks to import"
required: true

runs:
using: docker
image: Dockerfile
58 changes: 58 additions & 0 deletions entrypoint.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import os
import requests

TASKS_API = os.environ.get("PLATFORM_URL", "https://api.devskiller.com") + "/tasks"

input_api_key = os.environ["INPUT_API_KEY"]
input_path = os.environ["INPUT_PATH"]

def validate_input_path():
if not os.path.isfile(input_path) or not (input_path.endswith('.yaml') or input_path.endswith('.yml')):
print("::error::PATH parameter is not a yaml file")
exit(1)

def import_file():
print(f"::info::Importing the YAML file: {input_path}")
with open(input_path, "rb") as f:
response = requests.put(
f"{TASKS_API}/yaml",
data=f.read(),
headers={"Content-Type": "application/yaml", "Devskiller-Api-Key": input_api_key},
)

if response.status_code not in [200, 422]:
print(f"::error::Upload failed with status code: {response.status_code}, response: {response.text}")
exit(1)

return response

def write_summary(import_response):
with open(os.environ['GITHUB_STEP_SUMMARY'], 'a') as f:
body = import_response.json()

markdown = ""
if import_response.status_code == 422:
markdown = "### Validation errors:\n"
for violation in body.get('violations', []):
markdown += f"- {violation}\n"
f.write(markdown)
exit(1)
elif import_response.status_code == 200:
if body.get('created'):
markdown += "### Created tasks:\n"
for task in body.get('created', []):
markdown += f"- {task}\n"
if body.get('updated'):
markdown += "\n### Updated tasks:\n"
for task in body.get('updated', []):
markdown += f"- {task}\n"
f.write(markdown)


def main():
validate_input_path()
import_response = import_file()
write_summary(import_response)

if __name__ == "__main__":
main()
221 changes: 221 additions & 0 deletions yaml-file-structure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
# YAML imports

If your subscription plan includes YAML imports, you can use the structure described below to import Choice, Code Gaps or Essay tasks.

## Formatting tips
Below are a few hints on how to create a perfect YAML import file.

1. Divide tasks with one empty paragraph in between.
2. Markdown can be used for formatting, including the syntax for code blocks:

a) For the inline code snippets use backticks around the snippet, like in markdown.

```
`echo "Hello World!";`
```
b) For the block of code use 3 backticks at the beginning and 3 backticks at the end of code
block.
```
```
echo "Hello";
echo "World!";
```
```
3. Pay attention to whitespaces and quoting of special characters - they can break your task and it will not pass the validation. You can use [YAML linters](https://yamlchecker.com/) to streamline your YAML files creation.
There are multiple ways to achieve a line continuation in YAML:
- **Folded style** - removes the newlines within the string (but adds one at the end).
```yaml
title: >-
JavaScript DEMO question
```
- **Literal style** - turns newlines within the string into literal newlines
```yaml
title: |-
JavaScript DEMO question
```
## Choice questions
### Correct and wrong answers
You can create two types of questions: multiple and single choice questions.
It is pretty obvious that in single choice questions there could be a lot of possible answers but one correct, but in multiple choice questions there can be any combination of correct vs wrong answers.
For example: 2 correct and 5 wrong; 4 correct and 1 wrong and etc.
### YAML format
A YAML file should consist of a list of questions.
Every question is a hash, which must contain the following keys.
- `uuid` - id of the task. With this id you will be able to update tasks. You can use existing ones or generate unique values using [generators](https://www.uuidgenerator.net/).
- `difficulty` - the difficulty level, one of `EASY`, `MEDIUM`, `HARD`.
- `duration` - time duration in minutes or in the ISO 8601 duration format.
- `points` - default number of points for this task.
- `tags` - task tags, ie. what kind of knowledge this questions tests.
- `question` - the question which will be presented to the candidate.
- `type` - type of a task: `MULTI_CHOICE`.
- `action` - put this key to decide whether you want your question be automatically published on addition. Possible values are: `PUBLISH` or `CREATE_DRAFT` _(default)_.
- `mode` - add this one with SINGLE key word if you need a single choice question.
- `choices` - list of hashes with choices. The key in these hashes can be either `correct`, for correct answers, or `wrong`, for wrong ones.
**Single choice question example:**
```yaml
- uuid: 8d002491-1e45-4f57-9f2b-149393cbd47d
title: JavaScript | Demo question
difficulty: EASY
duration: 2
points: 5
tags: [JavaScript, SecondTag]
question: |-
Select the correct statement about JavaScript.
type: MULTI_CHOICE
action: PUBLISH
mode: SINGLE
choices:
- correct: |-
JavaScript is a dynamic programming language, which can be used to write client-side scripts for web browsers.
- wrong: |-
JavaScript applications are compiled to bytecode that can run on a Java Virtual Machine.
- wrong: |-
JavaScript was originally developed by James Gosling at Sun Microsystems
```

**Multiple choice question example:**
```yaml
- uuid: 8d002491-1e45-4f57-9f2b-149393cbd47d
title: JavaScript | Demo question
difficulty: EASY
duration: 'PT2M'
points: 5
tags: [JavaScript, SecondTag]
question: |-
Select the correct statement about JavaScript.
type: MULTI_CHOICE
action: CREATE_DRAFT
choices:
- correct: |-
JavaScript is a dynamic programming language, which can be used to write client-side scripts for web browsers.
- wrong: |-
JavaScript applications are compiled to bytecode that can run on a Java Virtual Machine.
- wrong: |-
JavaScript was originally developed by James Gosling at Sun Microsystems
- correct: |-
I am also correct.
```
## Code Gaps questions
Code gaps is a multipurpose task. You can provide a snippet of code with gaps in it where the candidate must fill in the blanks to make it complete.
### YAML format
A YAML file should consist of a list of questions.
Every question is a hash, which must contain the following keys.
- `uuid` - id of the task. With this id you will be able to update tasks. You can use existing ones or generate unique values using [generators](https://www.uuidgenerator.net/).
- `title` - title of a task.
- `difficulty` - the difficulty level, one of `EASY`, `MEDIUM`, `HARD`.
- `duration` - time duration in minutes or in the ISO 8601 duration format.
- `points` - default number of points for this task.
- `tags` - task tags, ie. what kind of knowledge this questions tests.
- `question` - the question which will be presented to the candidate.
- `type` - type of a task: `CODE_GAPS`.
- `action` - put this key to decide whether you want your question be automatically published on addition. Possible values are: `PUBLISH` or `CREATE_DRAFT` _(default)_.
- `mode` - syntax highlight mode, e.g.: `PLAIN` _(regular text)_, `Dockerfile`, `PHP`, `Java`
- `content` - the task code.

### Content field format:
Code gaps should be covered with brackets `{{{gap}}}`.

If more than 2 choices can be correct, put `||` in front of each correct answer. Each task can have more than one gap.

You can also combine rules for code gaps. For example:

`{{{|CW|"hello"|CW|'hello'}}}`

This means that the correct answer can have `'` or `"` quotes and ignore case + ignore whitespace options enabled.

- `|C|` - stands for **ignore case** in the answer.
- `|R|` - stands for **regexp** in the answer.
- `|W|` - stands for **ignore whitespace** in the answer. If the answer consists only from 1 word no `|W|` is needed.

If your task uses `{` and `}` as part of the code, you should use the following format:

```
{{{ {this is all code gap} }}}
```

**Single answer example:**
```yaml
- uuid: 4da801c5-b132-43d1-a211-8e5efb43cffa
title: DevOps | Running docker containers - cleanup
difficulty: EASY
duration: 3
points: 2
tags: [DevOps, Docker]
question: |-
Ensure that the container will be removed when it exits
type: CODE_GAPS
mode: SHELL
action: PUBLISH
content: |-
$ docker run {{{--rm}}} hello-world
```

**Multiple answers example:**
```yaml
- uuid: 54454be6-38f8-4707-871e-31ccedef79f1
title: JavaScript | Some unique task name
difficulty: EASY
duration: 'PT10M'
points: 10
tags: [JavaScript]
skills: [Software Development, JavaScript]
question: |-
Fill in the JavaScript code gap to make the code do this and that.
type: CODE_GAPS
mode: JAVASCRIPT
action: PUBLISH
content: |-
here goes the code and here goes the {{{|C|gap|C|gaps}}}
```

## Essay questions
Essay is a open-ended and manually evaluated task where the candidate can write a short answer.

### YAML format

A YAML file should consist of a list of questions.
Every question is a hash, which must contain the following keys.

- `uuid` - id of the task. With this id you will be able to update tasks. You can use existing ones or generate unique values using [generators](https://www.uuidgenerator.net/).
- `title` - title of a task.
- `difficulty` - the difficulty level, one of `EASY`, `MEDIUM`, `HARD`.
- `duration` - time duration in minutes or in the ISO 8601 duration format.
- `points` - default number of points for this task.
- `tags` - task tags, ie. what kind of knowledge this questions tests.
- `question` - the question which will be presented to the candidate.
- `type` - type of a task: `ESSAY`.
- `action` - put this key to decide whether you want your question be automatically published on addition. Possible values are: `PUBLISH` or `CREATE_DRAFT` _(default)_.

**Example:**
```yaml
- uuid: 4da801c5-b132-43d1-a211-8e5efb43cffa
title: Soft Skills | Some unique task name
difficulty: MEDIUM
duration: 30
points: 15
tags: ['Soft Skills']
question: |-
Describe a situation when you had to work in a team.
type: ESSAY
action: PUBLISH
```

0 comments on commit 99fd4ca

Please sign in to comment.