Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Macros] Add support for wasm macros #73031

Open
wants to merge 132 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
132 commits
Select commit Hold shift + click to select a range
e5c191d
wip
kabiroberai Apr 15, 2024
e5342e7
wip
kabiroberai Apr 15, 2024
d5d1cb7
wip
kabiroberai Apr 15, 2024
f776c28
revert swift-plugin-server
kabiroberai Apr 15, 2024
43fc233
use jsc in-proc
kabiroberai Apr 15, 2024
961fd54
Allow wasm in sandbox, don’t disable it
kabiroberai Apr 15, 2024
a771631
fix warning, prune version check
kabiroberai Apr 15, 2024
3f8ef81
Factor out JSCWasmPlugin
kabiroberai Apr 15, 2024
23426e0
the continuation just... works now?
kabiroberai Apr 15, 2024
9fe505f
more tweaks to JSCWasmPlugin
kabiroberai Apr 15, 2024
89158e2
Create SwiftPluginServerSupport
kabiroberai Apr 15, 2024
f084e25
JSCWasmPlugin: memoize, improve errors
kabiroberai Apr 15, 2024
5ef86da
steps towards x-plat
kabiroberai Apr 15, 2024
02f4324
more modularity
kabiroberai Apr 15, 2024
bddde66
Add wasm-plugin-server-path frontend opt
kabiroberai Apr 16, 2024
ca22e2d
Start adding WasmKit support
kabiroberai Apr 16, 2024
6a9bbe7
Update WasmKitPlugin
kabiroberai Apr 16, 2024
e598064
safety, tweaks
kabiroberai Apr 16, 2024
7d1510e
tweaks to JSC
kabiroberai Apr 16, 2024
37b8a61
Merge remote-tracking branch 'upstream/main' into kabir/wasm-plugins
kabiroberai Apr 17, 2024
da74f38
Proper WasmKit integration
kabiroberai Apr 22, 2024
7a18da2
tweaks
kabiroberai Apr 22, 2024
1076b16
Check ABI version
kabiroberai Apr 22, 2024
805c659
Merge remote-tracking branch 'upstream/main' into kabir/wasm-plugins
kabiroberai Apr 22, 2024
4527842
move ABI check into WasmEnginePlugin
kabiroberai Apr 22, 2024
f9dbbb1
build fixes
kabiroberai Apr 22, 2024
6511672
Elide copies of wasm blob
kabiroberai Apr 22, 2024
dd481fe
fix duplicate libraries warning
kabiroberai Apr 23, 2024
4ce4749
Merge remote-tracking branch 'upstream/main' into kabir/wasm-plugins
kabiroberai Apr 23, 2024
482871c
add SWIFT_WASM_USE_JSC
kabiroberai Apr 23, 2024
771acb0
Merge remote-tracking branch 'upstream/main' into kabir/wasm-plugins
kabiroberai Apr 23, 2024
2460e98
Skip wasmkit CLI for now
kabiroberai Apr 24, 2024
150f3d2
Or actually build wasmkit-cli
kabiroberai Apr 24, 2024
34ed5b0
use stdio for ipc
kabiroberai Apr 24, 2024
94d525e
No need for memory witness
kabiroberai Apr 24, 2024
a99f6b3
Byte-swap length if needed
kabiroberai Apr 24, 2024
c5dc6bd
Improve error handling
kabiroberai Apr 24, 2024
6375faf
Merge branch 'main' into kabir/wasm-plugins
kabiroberai Apr 30, 2024
826a80f
Fix linkage
kabiroberai Apr 30, 2024
f053859
No C dep needed for wasm
kabiroberai Apr 30, 2024
6c0b40c
Drop foundation (mostly); lower min os
kabiroberai Apr 30, 2024
d72e165
More foundation pruning
kabiroberai Apr 30, 2024
564f844
interop mode cxx -> c
kabiroberai Apr 30, 2024
a670f32
Target
kabiroberai Apr 30, 2024
aaaaae3
tweaks
kabiroberai Apr 30, 2024
7b16cbd
We don’t need WasmKitPluginError
kabiroberai Apr 30, 2024
7d9d015
Change wasm versioning approach
kabiroberai May 5, 2024
ea81405
Merge branch 'main' into kabir/wasm-plugins
kabiroberai May 5, 2024
b472f4f
Merge branch 'main' into kabir/wasm-plugins
kabiroberai May 11, 2024
3baf627
Merge branch 'main' into kabir/wasm-plugins
kabiroberai May 25, 2024
eee48e0
swift-wasm-plugin-server => swift-plugin-server
kabiroberai May 25, 2024
4fca102
swift-plugin-server doesn’t need a product
kabiroberai May 25, 2024
d465d36
We don’t need Dispatch here
kabiroberai May 25, 2024
c4efcee
Improve diagnostics
kabiroberai May 25, 2024
b3f53c2
Remove JSC for now
kabiroberai May 25, 2024
b629484
Explicitly depend on System
kabiroberai May 26, 2024
af2b87e
Nevermind SwiftPM didn’t like that
kabiroberai May 26, 2024
bbb916c
Lower swift-tools-version
kabiroberai May 26, 2024
77c273f
Fix WASI dep
kabiroberai May 26, 2024
b56e28a
Add missing header
kabiroberai May 26, 2024
94f0ad7
Merge branch 'main' into kabir/wasm-plugins
kabiroberai May 26, 2024
14b5b79
Include position in diagnostics if possible
kabiroberai May 26, 2024
af68825
Merge branch 'main' into kabir/wasm-plugins
kabiroberai May 26, 2024
92aef4e
this comment is now stale
kabiroberai May 26, 2024
9b1e4ee
No wasm_plugin_server_path
kabiroberai May 26, 2024
f93df4d
Remove debug code
kabiroberai May 26, 2024
95f254c
error_load_plugin diagnostic
kabiroberai May 26, 2024
ca27b45
Merge branch 'main' into kabir/wasm-plugins
kabiroberai May 26, 2024
caa1a8c
Elide copy
kabiroberai May 26, 2024
2ee417f
Merge branch 'main' into kabir/wasm-plugins
kabiroberai Jun 12, 2024
fa52ecd
Feedback
kabiroberai Jun 12, 2024
f2f2ea2
nit
kabiroberai Jun 12, 2024
c0a75e1
record ServerPath
kabiroberai Jun 12, 2024
011a0a5
Basic test
kabiroberai Jun 12, 2024
d2d3b94
Parameterize
kabiroberai Jun 12, 2024
e2a455a
Merge branch 'main' into kabir/wasm-plugins
kabiroberai Jun 29, 2024
b307eb4
Fix deprecation warning
kabiroberai Jun 29, 2024
782956a
working test
kabiroberai Jun 30, 2024
8a88dc5
simplify
kabiroberai Jun 30, 2024
fcb838d
remove wasm_sdk_root
kabiroberai Jun 30, 2024
469d84a
error tests
kabiroberai Jun 30, 2024
cc41cf1
Update WasmKit to 0.0.5
kateinoigakukun Jul 1, 2024
7499295
Merge branch 'main' into kabir/wasm-plugins
kabiroberai Jul 14, 2024
e89fd6a
post-merge fixup
kabiroberai Jul 14, 2024
2b47724
no opaque types
kabiroberai Jul 14, 2024
4285cf1
tentative fix for lldb build
kabiroberai Jul 14, 2024
56b6cae
don’t depend on ArgumentParser
kabiroberai Jul 14, 2024
96363e1
temporarily update wasmkit checkout
kabiroberai Jul 14, 2024
913041b
Revert "temporarily update wasmkit checkout"
kateinoigakukun Jul 15, 2024
6cd389e
Update WasmKit to 0.0.6
kateinoigakukun Jul 15, 2024
c3f4276
[test] Annotate macro tests that require the WebAssembly code generator
kateinoigakukun Jul 15, 2024
4341058
Update WasmKit to 0.0.7
kateinoigakukun Jul 21, 2024
393933a
Add temporary workaround for Windows pipe support of swift-system
kateinoigakukun Jul 22, 2024
9c5e7fc
[test] Remove usage of unused lit variables for building wasm plugins
kateinoigakukun Jul 22, 2024
5890c3e
Change the first `-load-plugin` path separator from `:` to `#`
kateinoigakukun Jul 23, 2024
0f39ec8
[test] Fix path segment separator of plugin path for Windows
kateinoigakukun Jul 23, 2024
48dd97f
Merge branch 'main' into kabir/wasm-plugins
MaxDesiatov Aug 20, 2024
c3f41c7
Merge branch 'main' into kabir/wasm-plugins
MaxDesiatov Aug 21, 2024
ee82b85
Merge branch 'main' of github.com:apple/swift into kabir/wasm-plugins
MaxDesiatov Aug 23, 2024
1dcdd7b
Merge branch 'main' of github.com:apple/swift into kabir/wasm-plugins
MaxDesiatov Aug 27, 2024
6e56934
Attempt to use `main` branch of WasmKit
MaxDesiatov Aug 27, 2024
dd13597
Merge branch 'main' into kabir/wasm-plugins
kabiroberai Sep 21, 2024
5770553
Revert wasmkit bump
kabiroberai Sep 21, 2024
94f870c
Merge branch 'main' into kabir/wasm-plugins
MaxDesiatov Oct 10, 2024
ed54400
Bump WasmKit to 0.1.0 in `update-checkout-config.json`
MaxDesiatov Oct 10, 2024
d568d67
Update for WasmKit 0.1.0 API
MaxDesiatov Oct 10, 2024
75bd7ca
Fix missing imports in `WasmKitEngine.swift`
MaxDesiatov Oct 11, 2024
c1e928d
Bump wasmkit to 0.1.1 to enable WMO
kateinoigakukun Oct 13, 2024
0d2c140
Merge remote-tracking branch 'origin/main' into kabir/wasm-plugins
kateinoigakukun Oct 13, 2024
7683e83
Bump WasmKit to 0.1.2 in `update-checkout-config.json`
MaxDesiatov Oct 21, 2024
e0427b2
Bump stack size in `WasmKitEngine.swift` to avoid OOB
MaxDesiatov Oct 21, 2024
be04533
Merge branch 'main' of github.com:apple/swift into kabir/wasm-plugins
MaxDesiatov Dec 4, 2024
1852df9
Remove outdated use of `PluginSearchOption::LoadPlugin`
MaxDesiatov Dec 4, 2024
e49c980
Bump WasmKit to 0.1.3 in `update-checkout-config.json`
MaxDesiatov Dec 4, 2024
052058a
Remove outdated `case OPT_load_plugin` in `CompilerInvocation.cpp`
MaxDesiatov Dec 4, 2024
f860528
Remove outdated use of `LoadPlugin` from `swift-ide-test.cpp`
MaxDesiatov Dec 4, 2024
28a9efa
Update lit tests to use `-load-resolved-plugin`
MaxDesiatov Dec 6, 2024
19eef9a
Merge branch 'main' of github.com:apple/swift into kabir/wasm-plugins
MaxDesiatov Dec 6, 2024
db325e8
Merge branch 'main' of github.com:apple/swift into kabir/wasm-plugins
MaxDesiatov Dec 12, 2024
54eb483
Adopt new `shutDown` API introduced in `swift-syntax`
MaxDesiatov Dec 12, 2024
224ac09
Add a trivial echo macro test to the package
MaxDesiatov Dec 13, 2024
dc8cd3c
Test `WasmEnginePlugin` APIs in `WasmEngineTests`
MaxDesiatov Dec 16, 2024
6817517
Merge branch 'main' of github.com:apple/swift into kabir/wasm-plugins
MaxDesiatov Dec 16, 2024
f56ec6a
Make formatting consistent in `swift-plugin-server`
MaxDesiatov Dec 16, 2024
1626ad8
Create stdin/stdout pipes for plugins with `mkfifo`
MaxDesiatov Dec 16, 2024
4204eef
Clean up temp Wasm fixture file in `WasmEngineTests`
MaxDesiatov Dec 16, 2024
1cf1e20
Make `init` parameter names more explicit in `WasmEngine`
MaxDesiatov Dec 16, 2024
6277098
Merge branch 'main' into kabir/wasm-plugins
MaxDesiatov Dec 17, 2024
0da4f22
Remove use of named pipes as engine-specific
MaxDesiatov Dec 17, 2024
3991b60
Fix renaming in `tools/swift-plugin-server/CMakeLists.txt`
MaxDesiatov Dec 17, 2024
636fa0e
Fix Windows compatibility
MaxDesiatov Dec 17, 2024
f915c05
Merge branch 'main' of github.com:apple/swift into kabir/wasm-plugins
MaxDesiatov Jan 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
/*
This source file is part of the Swift System open source project

Copyright (c) 2024 Apple Inc. and the Swift System project authors
Licensed under Apache License v2.0 with Runtime Library Exception

See https://swift.org/LICENSE.txt for license information
*/

#if canImport(Darwin)
import Darwin
#elseif canImport(Glibc)
import Glibc
#elseif canImport(Musl)
import Musl
#elseif canImport(WASILibc)
import WASILibc
#else
#error("Unsupported platform")
#endif

import SystemPackage

// MARK: - API

/// Create a temporary path for the duration of the closure.
///
/// - Parameters:
/// - basename: The base name for the temporary path.
/// - body: The closure to execute.
///
/// Creates a temporary directory with a name based on the given `basename`,
/// executes `body`, passing in the path of the created directory, then
/// deletes the directory and all of its contents before returning.
func withTemporaryFilePath<R>(
basename: FilePath.Component,
_ body: (FilePath) throws -> R
) throws -> R {
let temporaryDir = try createUniqueTemporaryDirectory(basename: basename)
defer {
try? _recursiveRemove(at: temporaryDir)
}

return try body(temporaryDir)
}

// MARK: - Internals

fileprivate let base64 = Array<UInt8>(
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_".utf8
)

/// Create a directory that is only accessible to the current user.
///
/// - Parameters:
/// - path: The path of the directory to create.
/// - Returns: `true` if a new directory was created.
///
/// This function will throw if there is an error, except if the error
/// is that the directory exists, in which case it returns `false`.
fileprivate func makeLockedDownDirectory(at path: FilePath) throws -> Bool {
return try path.withPlatformString {
if mkdir($0, 0o700) == 0 {
return true
}
let err = errno
if err == Errno.fileExists.rawValue {
return false
} else {
throw Errno(rawValue: err)
}
}
}

/// Generate a random string of base64 filename safe characters.
///
/// - Parameters:
/// - length: The number of characters in the returned string.
/// - Returns: A random string of length `length`.
fileprivate func createRandomString(length: Int) -> String {
return String(
decoding: (0..<length).map{
_ in base64[Int.random(in: 0..<64)]
},
as: UTF8.self
)
}

/// Given a base name, create a uniquely named temporary directory.
///
/// - Parameters:
/// - basename: The base name for the new directory.
/// - Returns: The path to the new directory.
///
/// Creates a directory in the system temporary directory whose name
/// starts with `basename`, followed by a `.` and then a random
/// string of characters.
func createUniqueTemporaryDirectory(
basename: FilePath.Component
) throws -> FilePath {
var tempDir = try _getTemporaryDirectory()
tempDir.append(basename)

while true {
tempDir.extension = createRandomString(length: 16)

if try makeLockedDownDirectory(at: tempDir) {
return tempDir
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
/*
This source file is part of the Swift System open source project

Copyright (c) 2024 Apple Inc. and the Swift System project authors
Licensed under Apache License v2.0 with Runtime Library Exception

See https://swift.org/LICENSE.txt for license information
*/

#if !os(Windows)

#if canImport(Darwin)
import Darwin
#elseif canImport(Glibc)
import Glibc
#elseif canImport(Musl)
import Musl
#elseif canImport(WASILibc)
import WASILibc
#else
#error("Unsupported platform")
#endif

import SystemPackage

/// Get the path to the system temporary directory.
internal func _getTemporaryDirectory() throws -> FilePath {
guard let tmp = getenv("TMPDIR") else {
return "/tmp"
}

return FilePath(stringLiteral: .init(cString: tmp))
}

/// Delete the entire contents of a directory, including its subdirectories.
///
/// - Parameters:
/// - path: The directory to be deleted.
///
/// Removes a directory completely, including all of its contents.
internal func _recursiveRemove(
at path: FilePath
) throws {
let dirfd = try FileDescriptor.open(path, .readOnly, options: .directory)
defer {
try? dirfd.close()
}

let dot: (CInterop.PlatformChar, CInterop.PlatformChar) = (46, 0)
try withUnsafeBytes(of: dot) {
try recursiveRemove(
in: dirfd.rawValue,
name: $0.assumingMemoryBound(to: CInterop.PlatformChar.self).baseAddress!
)
}

try path.withPlatformString {
let error = rmdir($0)
if error != 0 {
throw Errno(rawValue: error)
}
}
}

internal let SYSTEM_AT_REMOVE_DIR = AT_REMOVEDIR
internal let SYSTEM_DT_DIR = DT_DIR
internal typealias system_dirent = dirent
#if os(Linux) || os(Android)
internal typealias system_DIRPtr = OpaquePointer
#else
internal typealias system_DIRPtr = UnsafeMutablePointer<DIR>
#endif


/// Open a directory by reference to its parent and name.
///
/// - Parameters:
/// - dirfd: An open file descriptor for the parent directory.
/// - name: The name of the directory to open.
/// - Returns: A pointer to a `DIR` structure.
///
/// This is like `opendir()`, but instead of taking a path, it uses a
/// file descriptor pointing at the parent, thus avoiding path length
/// limits.
fileprivate func impl_opendirat(
_ dirfd: CInt,
_ name: UnsafePointer<CInterop.PlatformChar>
) -> system_DIRPtr? {
let fd = openat(dirfd, name,
FileDescriptor.AccessMode.readOnly.rawValue
| FileDescriptor.OpenOptions.directory.rawValue)
if fd < 0 {
return nil
}
return fdopendir(fd)
}

/// Invoke a closure for each file within a particular directory.
///
/// - Parameters:
/// - dirfd: The parent of the directory to be enumerated.
/// - subdir: The subdirectory to be enumerated.
/// - body: The closure that will be invoked.
///
/// We skip the `.` and `..` pseudo-entries.
fileprivate func forEachFile(
in dirfd: CInt,
subdir: UnsafePointer<CInterop.PlatformChar>,
_ body: (system_dirent) throws -> ()
) throws {
guard let dir = impl_opendirat(dirfd, subdir) else {
throw Errno(rawValue: errno)
}
defer {
_ = closedir(dir)
}

while let dirent = readdir(dir) {
// Skip . and ..
if dirent.pointee.d_name.0 == 46
&& (dirent.pointee.d_name.1 == 0
|| (dirent.pointee.d_name.1 == 46
&& dirent.pointee.d_name.2 == 0)) {
continue
}

try body(dirent.pointee)
}
}

/// Delete the entire contents of a directory, including its subdirectories.
///
/// - Parameters:
/// - dirfd: The parent of the directory to be removed.
/// - name: The name of the directory to be removed.
///
/// Removes a directory completely, including all of its contents.
fileprivate func recursiveRemove(
in dirfd: CInt,
name: UnsafePointer<CInterop.PlatformChar>
) throws {
// First, deal with subdirectories
try forEachFile(in: dirfd, subdir: name) { dirent in
if dirent.d_type == SYSTEM_DT_DIR {
try withUnsafeBytes(of: dirent.d_name) {
try recursiveRemove(
in: dirfd,
name: $0.assumingMemoryBound(to: CInterop.PlatformChar.self)
.baseAddress!
)
}
}
}

// Now delete the contents of this directory
try forEachFile(in: dirfd, subdir: name) { dirent in
let flag: CInt

if dirent.d_type == SYSTEM_DT_DIR {
flag = SYSTEM_AT_REMOVE_DIR
} else {
flag = 0
}

let result = withUnsafeBytes(of: dirent.d_name) {
unlinkat(dirfd, $0.assumingMemoryBound(to: CInterop.PlatformChar.self).baseAddress!, flag)
}

if result != 0 {
throw Errno(rawValue: errno)
}
}
}

#endif // !os(Windows)

Loading