Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync with main #4049

Open
wants to merge 109 commits into
base: dev/shared_heap
Choose a base branch
from
Open

Sync with main #4049

wants to merge 109 commits into from

Conversation

TianlongLiang
Copy link
Collaborator

Sync with main, as preparation for shared heap enhancement.

wenyongh and others added 30 commits September 25, 2024 16:43
No need to compile win_file.c when uvwasi is enabled.
Add symbol __atomic_compare_exchange_4 and __atomic_store_4.
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.2.4 to 3.26.9.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](github/codeql-action@v2.2.4...v3.26.9)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3.1.0 to 4.4.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v3.1.0...v4.4.0)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action) from 2.3.1 to 2.4.0.
- [Release notes](https://github.com/ossf/scorecard-action/releases)
- [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md)
- [Commits](ossf/scorecard-action@0864cf1...62b2cac)

---
updated-dependencies:
- dependency-name: ossf/scorecard-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
…ons/github/codeql-action-3.26.9

build(deps): bump github/codeql-action from 2.2.4 to 3.26.9
…ons/actions/upload-artifact-4.4.0

build(deps): bump actions/upload-artifact from 3.1.0 to 4.4.0
…ons/ossf/scorecard-action-2.4.0

build(deps): bump ossf/scorecard-action from 2.3.1 to 2.4.0
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.9 to 3.26.11.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](github/codeql-action@v3.26.9...v3.26.11)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Currently, the open-source builds of wamrc set WASM_ENABLE_DUMP_CALL_STACK,
which causes these two fields to be emitted. They are required by aot_emit_exception.c.

Internally at Google, we don't enable call stack dumps, so we've been using the
attached patch to make sure the fields are emitted anyway.
The execute_post_instantiate_functions may be triggered by wasm_cluster_spawn_exec_env,
in which the exec_env_tls can be NULL and cause the assertion invalid.

p.s. #3839.
)

Add table64 extension(in Memory64 proposal) support in classic-interp
and AOT running modes, currently still use uint32 to represent table's
initial and maximum size to keep AOT ABI unchanged.
The default iwasm building in Windows MSVC enables libc-uvwasi because
libc-wasi isn't supported at the beginning. Since libc-wasi had been refactored
and is supported in Windows msys2 building, and libc-wasi supports more
functionalities(e.g. sockets) than libc-uvwasi, this PR fixes some issues to
enable libc-wasi in windows MSVC buidlings.
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.4.0 to 4.4.3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v4.4.0...v4.4.3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.11 to 3.26.12.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](github/codeql-action@v3.26.11...v3.26.12)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Implement the shared heap feature for interpreter, aot and llvm jit.
Add below runtime APIs:
```C
wasm_shared_heap_t
wasm_runtime_create_shared_heap(SharedHeapInitArgs *init_args);

bool
wasm_runtime_attach_shared_heap(wasm_module_inst_t module_inst,
                                wasm_shared_heap_t shared_heap);

void
wasm_runtime_detach_shared_heap(wasm_module_inst_t module_inst);

uint64_t
wasm_runtime_shared_heap_malloc(wasm_module_inst_t module_inst, uint64_t size,
                                void **p_native_addr);

void
wasm_runtime_shared_heap_free(wasm_module_inst_t module_inst, uint64_t ptr);
```

And allow wasm app to call API shared_heap_malloc and shared_heap_free:
```C
void *shared_heap_malloc(uint32_t size);
void shared_heap_free(void *ptr);
```
- Clear some compile warnings
- Fix some typos
- Fix llvm LICENSE link error
- Remove unused aot file and binarydump bin
- Add checks when loading AOT exports
- For Windows, llvm libs need to cache more directories, so use a multi-line
  environment variable for paths
- Remove conditionally build directories `win32build`, just use `build` for all platform
- Add Windows wamrc and iwasm(disable lib pthread semaphore and fast jit for now)
  build in release CI
Add runtime API wasm_runtime_set_native_stack_boundary.

p.s. #3816
Sort the module instance's export functions with the function name,
and use binary search to lookup the wasm/aot function.
… and Exception Handling (#3866)

As suggested in #3829, in release CI, we add zip/tar.gz artifacts named
iwasm-gc-eh-{version}-{platform} for `iwasm` which supports features
garbage collection and exception handling(classic interpreter only).

Also, add a command line option to control GC heap size for `iwasm` on
the Windows platform.
woodsmc and others added 27 commits December 22, 2024 18:48
This fixes link errors seen on my environment.
(macOS 15.2, x86-64, Xcode 16.2)

Tested as:
```
mkdir b
cd b
cmake -D WAMR_BUILD_JIT=1 -D LLVM_DIR=/usr/local/opt/llvm@19/lib/cmake/llvm ..
make
```
For boundary checking, WAMR calls `pthread_attr_np`, which is
unfortunately quite slow on Linux when not called on the main thread
(see #3966
for discussion).

This change moves the cost of stack bounds checking earlier in the
wasm_exec_env creation process. The idea is that it's perhaps better to
pay the price when creating the execution environment rather than in the
first function call.

The original code is left in place inside
`call_wasm_with_hw_bound_check` in case the `wasm_exec_env` is created
via `wasm_runtime_spawn_exec_env`.
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.27.9 to 3.28.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](github/codeql-action@v3.27.9...v3.28.0)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.4.3 to 4.5.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v4.4.3...v4.5.0)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Save memory if the file buffer is always exist before exit.

Signed-off-by: Huang Qi <[email protected]>
Break aot_create_comp_data into small functions
Improve error message in the scenario where the runtime was built
with ref types disabled but the module uses reference types feature.
* Update CMakeLists.txt of fuzzing

- enable software bound-check
- enable wasi
- disable libc builtin and multiple modules

* Fix off-by-one error in result offset calculation for function calls
…3999)

* check whether table has funcref elem in call_indirect
* check whether table has funcref elem in call_indirect when gc is enabled
…ets (#4011)

One more corner case: if the `frame_offset` increases and becomes equal to
the `frame_offset_boundary` after the last assignment within the for loop.
…oposals (#3972)

Add documentation regarding security issues and the status of Wasm proposals.
- Enable shrunk memory by default and add related configurations
- Improve error messages for memory access alignment checks
- Add documentation for WAMR shrunk memory build option
- Update NuttX workflow to disable shrunk memory build option
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.5.0 to 4.6.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v4.5.0...v4.6.0)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.0 to 3.28.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](github/codeql-action@v3.28.0...v3.28.1)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
So far, no workflows would attempt to build the shared version of the
iwasm library (namely, vmlib).

Note that, as opposed to GC_EH_BUILD_OPTIONS and DEFAULT_BUILD_OPTIONS,
the actual default options defined by the build system are assumed, for
the sake of simplicity and avoiding repeated code.
- Refine const offsets in loader for fast-interp
- handle const cell num overflow
- Use const array, remove list
…4022)

- Synchronize the GC spec tests to the commit from December 9. 2024.
- Revise the error messages to be consistent with the spec test cases.
- bypass gc spec test on the nuttx platform as a workaround
correctly report error when datacount section has non-zero data segment count while the data section is not present
@github-advanced-security
Copy link

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

@@ -4606,6 +4713,21 @@
return false;
}

#if WASM_ENABLE_BULK_MEMORY != 0
static bool
check_data_count_consistency(bool has_datacount_section, int datacount_len,

Check notice

Code scanning / CodeQL

Unused static function Note

Static function check_data_count_consistency is unreachable (
load
must be removed at the same time)
Static function check_data_count_consistency is unreachable (
load_from_sections
must be removed at the same time)
Static function check_data_count_consistency is unreachable (
load_data_segment_section
must be removed at the same time)
@@ -9367,108 +9524,116 @@
return true;
}

static int
cmp_i64_const(const void *p_i64_const1, const void *p_i64_const2)

Check notice

Code scanning / CodeQL

Unused static function Note

Static function cmp_i64_const is unreachable (
wasm_loader_prepare_bytecode
must be removed at the same time)
Static function cmp_i64_const is unreachable (
wasm_loader_get_const_offset
must be removed at the same time)
Static function cmp_i64_const is unreachable (
load
must be removed at the same time)
Static function cmp_i64_const is unreachable (
load_from_sections
must be removed at the same time)
}

static int
cmp_i32_const(const void *p_i32_const1, const void *p_i32_const2)

Check notice

Code scanning / CodeQL

Unused static function Note

Static function cmp_i32_const is unreachable (
wasm_loader_prepare_bytecode
must be removed at the same time)
Static function cmp_i32_const is unreachable (
wasm_loader_get_const_offset
must be removed at the same time)
Static function cmp_i32_const is unreachable (
load
must be removed at the same time)
Static function cmp_i32_const is unreachable (
load_from_sections
must be removed at the same time)

/* Read leb without malformed format check */
static uint64
read_leb_quick(uint8 **p_buf, uint32 maxbits, bool sign)

Check notice

Code scanning / CodeQL

Unused static function Note

Static function read_leb_quick is unreachable (
wasm_loader_prepare_bytecode
must be removed at the same time)
Static function read_leb_quick is unreachable (
load
must be removed at the same time)
Static function read_leb_quick is unreachable (
load_from_sections
must be removed at the same time)
@@ -107,7 +119,20 @@
- run: brew install ccache ninja
if: steps.retrieve_llvm_libs.outputs.cache-hit != 'true' && startsWith(inputs.os, 'macos')

- uses: actions/cache@v4

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: GitHub-owned GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help

# Install tools on Windows
- run: choco install -y ccache ninja
if: steps.retrieve_llvm_libs.outputs.cache-hit != 'true' && inputs.os == 'windows-latest'

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: chocoCommand not pinned by hash
Click Remediation section below to solve this issue
@@ -49,7 +53,7 @@

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
uses: github/codeql-action/init@v3.28.1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: GitHub-owned GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help
@@ -66,7 +70,7 @@
- run: |
./.github/scripts/codeql_buildscript.sh
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
uses: github/codeql-action/analyze@v3.28.1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: GitHub-owned GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help
@@ -95,14 +99,14 @@
output: ${{ steps.step1.outputs.sarif-output }}/cpp.sarif

- name: Upload CodeQL results to code scanning
uses: github/codeql-action/upload-sarif@v3
uses: github/codeql-action/upload-sarif@v3.28.1

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: GitHub-owned GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help
with:
sarif_file: ${{ steps.step1.outputs.sarif-output }}
category: "/language:${{matrix.language}}"

- name: Upload CodeQL results as an artifact
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v4.6.0

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: GitHub-owned GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help
@@ -341,7 +351,7 @@

- name: upload the log
if: always()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v4.6.0

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 0: GitHub-owned GitHubAction not pinned by hash
Remediation tip: update your workflow using https://app.stepsecurity.io
Click Remediation section below for further remediation help
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.