Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub Actions CI Port #16082

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
12 changes: 10 additions & 2 deletions .ci/build-linux-aarch64.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,16 @@ git submodule -q update --init $(awk '/path/ && !/llvm/ && !/opencv/ { print $3

mkdir build && cd build || exit 1

export CC="${CLANG_BINARY}"
export CXX="${CLANGXX_BINARY}"
if [ "$COMPILER" = "gcc" ]; then
# These are set in the dockerfile
export CC="${GCC_BINARY}"
export CXX="${GXX_BINARY}"
export LINKER=gold
export CFLAGS="-fuse-linker-plugin -fuse-ld=${LINKER}"
else
export CC="${CLANG_BINARY}"
export CXX="${CLANGXX_BINARY}"
fi

cmake .. \
-DCMAKE_INSTALL_PREFIX=/usr \
Expand Down
18 changes: 13 additions & 5 deletions .ci/setup-windows.sh
Original file line number Diff line number Diff line change
Expand Up @@ -103,16 +103,24 @@ COMM_HASH=$(git rev-parse --short=8 HEAD)
# Format the above into filenames
if [ -n "$PR_NUMBER" ]; then
AVVER="${COMM_TAG}-${COMM_HASH}"
BUILD="rpcs3-v${AVVER}_win64.7z"
BUILD_RAW="rpcs3-v${AVVER}_win64"
BUILD="${BUILD_RAW}.7z"
else
AVVER="${COMM_TAG}-${COMM_COUNT}"
BUILD="rpcs3-v${AVVER}-${COMM_HASH}_win64.7z"
BUILD_RAW="rpcs3-v${AVVER}-${COMM_HASH}_win64"
BUILD="${BUILD_RAW}.7z"
fi

# BRANCH is used for experimental build warnings for pr builds, used in main_window.cpp.
# BUILD is the name of the release artifact
# BUILD_RAW is just filename
# AVVER is used for GitHub releases, it is the version number.
BRANCH="${REPO_NAME}/${REPO_BRANCH}"
echo "BRANCH=$BRANCH" > .ci/ci-vars.env
echo "BUILD=$BUILD" >> .ci/ci-vars.env
echo "AVVER=$AVVER" >> .ci/ci-vars.env

# SC2129
{
echo "BRANCH=$BRANCH"
echo "BUILD=$BUILD"
echo "BUILD_RAW=$BUILD_RAW"
echo "AVVER=$AVVER"
} >> .ci/ci-vars.env
171 changes: 171 additions & 0 deletions .github/workflows/rpcs3.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
name: Build RPCS3

on:
push:
paths-ignore:
- '.cirrus.yml'
- '.azure-pipelines.yml'
- 'README.md'
pull_request:
paths-ignore:
- '.cirrus.yml'
- '.azure-pipelines.yml'
- 'README.md'
workflow_dispatch:

concurrency:
group: ${{ github.ref }}-${{ github.event_name }}
cancel-in-progress: true

env:
BUILD_REPOSITORY_NAME: ${{ github.repository }}
BUILD_SOURCEBRANCHNAME: ${{ github.ref_name }}
BUILD_ARTIFACTSTAGINGDIRECTORY: ${{ github.workspace }}/build

jobs:
Linux_Build:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
docker_img: "rpcs3/rpcs3-ci-focal:1.9"
build_sh: "/rpcs3/.ci/build-linux.sh"
compiler: clang
- os: ubuntu-24.04
docker_img: "rpcs3/rpcs3-ci-focal:1.9"
build_sh: "/rpcs3/.ci/build-linux.sh"
compiler: gcc
- os: ubuntu-24.04-arm
docker_img: "rpcs3/rpcs3-ci-focal-aarch64:1.0"
build_sh: "/rpcs3/.ci/build-linux-aarch64.sh"
compiler: clang
runs-on: ${{ matrix.os }}
env:
CCACHE_DIR: ${{ github.workspace }}/ccache
CI_HAS_ARTIFACTS: true
DEPLOY_APPIMAGE: true
APPDIR: "/rpcs3/build/appdir"
ARTDIR: "/root/artifacts"
RELEASE_MESSAGE: "/rpcs3/GitHubReleaseMessage.txt"
COMPILER: ${{ matrix.compiler }}

steps:
- name: Checkout repository
uses: actions/checkout@main
with:
submodules: recursive
fetch-depth: 0

- name: Setup Cache
uses: actions/cache@main
with:
path: ${{ env.CCACHE_DIR }}
key: ${{ runner.os }}-ccache-${{ matrix.compiler }}

- name: Docker setup and build
run: |
docker pull --quiet ${{ matrix.docker_img }}
docker run \
-v $PWD:/rpcs3 \
--env-file .ci/docker.env \
-v ${{ env.CCACHE_DIR }}:/root/.ccache \
-v ${{ github.workspace }}/artifacts:/root/artifacts \
${{ matrix.docker_img }} \
${{ matrix.build_sh }}

- name: Upload artifacts
#TODO: Upload artifact to release repository
#condition for release
#if: |
# github.event_name != 'pull_request' &&
# github.repository == 'RPCS3/rpcs3' &&
# github.ref == 'refs/heads/master' &&
# matrix.compiler == 'clang'
uses: actions/upload-artifact@main
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with actions/upload-artifact@main, is this going to upload binaries to RPCS3/rpcs3-binaries-win/linux/linux-arm64/etc ? I don't see where we set any variables related to the upload repository, so I don't think this will work how we want it to.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also only run this step with the same restrictions we use in Azure:
condition: and(ne(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.Repository.Name'], 'RPCS3/rpcs3'), eq(variables['Build.SourceBranch'], 'refs/heads/master'), eq(variables['COMPILER'], 'clang'))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

        if: |
          github.event_name != 'pull_request' &&
          github.repository == 'RPCS3/rpcs3' &&
          github.ref == 'refs/heads/master' &&
          matrix.compiler == 'clang'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with actions/upload-artifact@main, is this going to upload binaries to RPCS3/rpcs3-binaries-win/linux/linux-arm64/etc ? I don't see where we set any variables related to the upload repository, so I don't think this will work how we want it to.

Nope, there will have to be a separate step for it. Using appropriate tokens like azure to upload it

with:
name: RPCS3 for Linux (${{ runner.arch }}, ${{ matrix.compiler }})
path: ${{ env.BUILD_ARTIFACTSTAGINGDIRECTORY }}/*.AppImage
compression-level: 0

Windows_Build:
runs-on: windows-latest
env:
COMPILER: msvc
QT_VER_MAIN: '6'
QT_VER: '6.8.1'
QT_VER_MSVC: 'msvc2022'
QT_DATE: '202411221531'
VULKAN_VER: '1.3.268.0'
VULKAN_SDK_SHA: '8459ef49bd06b697115ddd3d97c9aec729e849cd775f5be70897718a9b3b9db5'
CACHE_DIR: ./cache
steps:

- name: Checkout repository
uses: actions/checkout@main
with:
submodules: recursive
fetch-depth: 0

- name: Setup env
run: |
echo "QTDIR=C:\Qt\${{ env.QT_VER }}\${{ env.QT_VER_MSVC }}_64" >> ${{ github.env }}
echo "VULKAN_SDK=C:\VulkanSDK\${{ env.VULKAN_VER }}" >> ${{ github.env }}

- name: Get Cache Keys
shell: bash
run: .ci/get_keys-windows.sh

- name: Setup Cache
uses: actions/cache@main
with:
path: ${{ env.CACHE_DIR }}
key: ${{ runner.os }}-${{ env.COMPILER }}-${{ env.QT_VER }}-${{ env.VULKAN_SDK_SHA }}-llvm.lock-glslang.lock
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if the llvm.lock and glslang.lock keys are working? In Azure providing the filename means it will hash the file for the key. However, in https://github.com/actions/cache#creating-a-cache-key I see that they use this function: hashFiles('**/lockfiles') to hash files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will check

restore-keys: ${{ runner.os }}-${{ env.COMPILER }}

- name: Download and unpack dependencies
shell: bash
run: .ci/setup-windows.sh

- name: Export Variables
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should should be in a script file. Also, is it possible to do this step in bash?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be done in bash, as for making it a file, I'm afraid that github.env won't be resolved there, hence why it's a step.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh that's unfortunate, if you could rewrite in bash that'd be great for consistency.

run: |
Get-Content ".ci/ci-vars.env" | ForEach-Object {
# Skip over lines containing comments.
if ($_ -notmatch '^\s*#')
{
$line = $_ -split '='
$key = $line[0]
$val = $line[1]
echo "$key=$val" >> $env:GITHUB_ENV
}
}

- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@main

- name: Compile RPCS3
run: msbuild rpcs3.sln /p:Configuration=Release /p:Platform=x64

- name: Pack up build artifacts
shell: bash
run: |
mkdir -p ${{ env.BUILD_ARTIFACTSTAGINGDIRECTORY }}
.ci/deploy-windows.sh

- name: Upload artifacts (7z)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same problems with the linux version of this step

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be sorted

#TODO: Upload artifact to release repository
#condition for release
#if: |
# github.event_name != 'pull_request' &&
# github.repository == 'RPCS3/rpcs3' &&
# github.ref == 'refs/heads/master'
uses: actions/upload-artifact@main
with:
name: RPCS3 for Windows (MSVC)
# 7z
# 7z.sha256
path: |
${{ env.BUILD_ARTIFACTSTAGINGDIRECTORY }}/${{ env.BUILD }}
${{ env.BUILD_ARTIFACTSTAGINGDIRECTORY }}/${{ env.BUILD }}.sha256
compression-level: 0
illusion0001 marked this conversation as resolved.
Show resolved Hide resolved
if-no-files-found: error