Two major changes here:
1. We store the CWD as a simple `[]const u8` and lookup Preopens for
every absolute or CWD-referenced file operation, based on the
Preopen with the longest match (i.e. most specific path)
2. Preorders are normalized to POSIX absolute paths at init time.
Behavior depends on the "cwd_root" parameter of `initPreopensWasi`:
`cwd_root` is used for any Preopens that start with "."
For example:
"./foo/bar" - inits to -> "{cwd_root}/foo/bar"
"foo/bar" - inits to -> "/foo/bar"
"/foo/bar" - inits to -> "/foo/bar"
`cwd_root` must be an absolute path.
Using "/" as `cwd_root` gives behavior similar to wasi-libc.
Fixes#11353
The renderer treats comments and doc comments differently since doc
comments are parsed into the Ast. This commit adds a check after getting
the text for the doc comment and trims whitespace at the end before
rendering.
The `a = 0,` in the test is here to avoid a ParseError while parsing the
test.
Currently, the new API will only be available on macOS with
the intention of adding more POSIX systems to it incrementally
(such as Linux, etc.).
Changes:
* add `posix_spawn` wrappers in a separate container in
`os/posix_spawn.zig`
* rewrite `ChildProcess.spawnPosix` using `posix_spawn` targeting macOS
as `ChildProcess.spawnMacos`
* introduce a `posix_spawn` specific `std.c.waitpid` wrapper which
does return an error in case the child process failed to exec - this
is required for any process that was spawned using `posix_spawn`
mechanism as, by definition, the errors returned by `posix_spawn`
routine cover only the `fork`-equivalent; `pre-exec()` and `exec()`
steps are covered by a catch-all error `ECHILD` returned by `waitpid`
on unsuccessful execution, e.g., no such file error, etc.
This adds a special CWD file descriptor, AT.FDCWD (-2), to refer to the
current working directory. The `*at(...)` functions look for this and
resolve relative paths against the stored CWD. Absolute paths are
dynamically matched against the stored Preopens.
"os.initPreopensWasi()" must be called before std.os functions will
resolve relative or absolute paths correctly. This is asserted at
runtime.
Support has been added for: `open`, `rename`, `mkdir`, `rmdir`, `chdir`,
`fchdir`, `link`, `symlink`, `unlink`, `readlink`, `fstatat`, `access`,
and `faccessat`.
This also includes limited support for `getcwd()` and `realpath()`.
These return an error if the CWD does not correspond to a Preopen with
an absolute path. They also do not currently expand symlinks.
* os/linux/io_uring: add recvmsg and sendmsg
* Use std.os.iovec and std.os.iovec_const
* Remove msg_ prefix in msghdr and msghdr_const in arm64 etc
* Strip msg_ prefix in msghdr and msghdr_const for linux arm-eabi
* Copy msghdr and msghdr_const from i386 to mips
* Add sockaddr to lib/std/os/linux/mips.zig
* Copy msghdr and msghdr_const from x86_64 to riscv64
As of 6249a24, align() is not allowed on packed struct fields
and as such the align(4) was removed from the first field of
EdidOverrideProtocolAttributes. The endgame here is packed struct
backed by explicit integers, in this case a u32, but until that
is implemented put the align(4) on the pointer to avoid breaking
someone's UEFI code in a hard to debug way.
This implements #10113 for the self-hosted compiler only. It removes the
ability to override alignment of packed struct fields, and removes the
ability to put pointers and arrays inside packed structs.
After this commit, nearly all the behavior tests pass for the stage2 llvm
backend that involve packed structs.
I didn't implement the compile errors or compile error tests yet. I'm
waiting until we have stage2 building itself and then I want to rework
the compile error test harness with inspiration from Vexu's arocc test
harness. At that point it should be a much nicer dev experience to work
on compile errors.
Windows does Unicode-aware case-insensitivity comparisons for environment variable names. Before, os.getenvW was only doing ASCII case-insensitivity. We can take advantage of RtlEqualUnicodeString in NtDll to get the proper Unicode case insensitivity.
I found that after switching from my custom Guid parser to the one in std that it increased zigwin32 build times substantially (from 40 seconds to over 10 minutes). More information can be found in the benchmark PR I created here: https://github.com/ziglang/gotta-go-fast/pull/21 . This PR ports my GUID parser to std so all projects can leverage the faster comptime performance.
The size of a GUID is not platform-dependent, it's always a fixed number of bits. So I've updated guid to use fixed bit integer types rather than platform-dependent C integer types.
Beyond adding default zero-initialization, this commit changes undefined
initialization to zero, as some cases reserved the padding and on other
cases, I've found some systems act strange when giving uninit instead of
zero even when it shouldn't be an issue, one example being
FileProtocol.Open's attributes, which *should* be ignored when not
creating a file, but ended up giving an unrelated error.
I've seen having this be wrong break some cross-compilers, and it's
also how it is in other files so it's best to be consistent.
It's also just the actual casing of the file.
This allows users to add file paths to device paths, which is often used
in methods like `boot_services.loadImage` and `boot_services.startImage`,
which take a device path with an additional file path appended to locate
the image.
copy_cqes() is not guaranteed to return as many CQEs as provided in the
`wait_nr` argument, meaning the assert in `copy_cqe` can trigger.
Instead, loop until we do get at least one CQE returned.
This mimics the behaviour of liburing's _io_uring_get_cqe.
The semantics of this function are that it moves both files and
directories. Previously we had this `is_dir` boolean field of
`std.os.windows.OpenFile` which required the API user to choose: are we
opening a file or directory? And the other kind would either cause
error.IsDir or error.NotDir. But that is not a limitation of the Windows
file system API; it was self-imposed.
On Windows, rename is implemented internally with `NtCreateFile` so we
need to allow it to open either files or directories. This is now done
by `std.os.windows.OpenFile` accepting enum{file_only,dir_only,any}
instead of a boolean.
For renameat, unlinkat, mkdirat, symlinkat and linkat the error code
differs between kernel 5.4 which returns EBADF and kernel 5.10 which returns EINVAL.
Fixes#10466
Adds the `tcflag_t` type to the termios constants.
This is made to allow bitwise operations on the termios
constants without an integer cast, e.g.:
```zig
var raw = try std.os.tcgetattr(std.os.STDIN_FILENO);
raw.lflag &= std.os.linux.ECHO | std.os.linux.ICANON;
```
instead of
```zig
var raw = try std.os.tcgetattr(std.os.STDIN_FILENO);
raw.lflag &= ~@intCast(u32, std.os.linux.ECHO | std.os.linux.ICANON);
```
Contributes to #10181
The old test "timeout_link_chain1" was ported from liburing test_timeout_link_chain1
509873c445/test/link-timeout.c (L539-L628)
However it turns out that both fails with EBADF (-9) on Linux kernel 5.4.
The this new test skips properly on Linux kernel 5.4
and passes on Linux kernel 5.11.
This test can fail due to a fix in musl for 32 bit MIPS, where musl changes limits greater than -1UL/2 to RLIM_INFINITY.
See http://git.musl-libc.org/cgit/musl/commit/src/misc/getrlimit.c?id=8258014fd1e34e942a549c88c7e022a00445c352
Depending on the system where the test is run getrlimit can return
RLIM_INFINITY for example if RLIMIT_MEMLOCK is bigger than ~2GiB.
If that happens, the setrlimit call will fail with PermissionDenied.
This is a breaking change. Before, usage looked like this:
```zig
const held = mutex.acquire();
defer held.release();
```
Now it looks like this:
```zig
mutex.lock();
defer mutex.unlock();
```
The `Held` type was an idea to make mutexes slightly safer by making it
more difficult to forget to release an aquired lock. However, this
ultimately caused more problems than it solved, when any data structures
needed to store a held mutex. Simplify everything by reducing the API
down to the primitives: lock() and unlock().
Closes#8051Closes#8246Closes#10105
The TLS area may be located in the upper part of the address space and,
if the platform expects a constant offset to be applied, may make the tp
register calculation overflow.
Use +% instead of +, the overflow is harmless.
Some systems (Solaris, OpenBSD, AIX) change their definitions of
sockaddr_storage to be larger than 128 bytes. This comment adds a new
constant in the `sockaddr` that defines the size for every system.
Fixes#9759
Extract existing constants to do with TCP socket options into a 'TCP'
namespace.
Export 'MSG' and 'TCP' from std.os.{linux, windows} into std.c.
Fix compile errors to do with std.x.os.Socket methods related to setting
TCP socket options.
Handle errors in the case that an interface could not be resolved in an
IPv6 address on Windows. Tested using Wine with the loopback interface
disabled.
Have all instantiations of std.x.os.Socket on Windows instantiate an
overlapped socket descriptor. Fixes the '1ms read timeout' test in
std.x.net.tcp.Client. The test would previously deadlock, as read
timeouts only apply to overlapped sockets.
Windows documentation by default recommends that most instantiations of
sockets on Windows be overlapped sockets (s.t. they may operate in both
blocking or nonblocking mode when operated with WSA* syscalls). Refer to
the documentation for WSASocketA for more info.
I incorrectly assumed that __kernel_timespec was used when not linking
libc, however that is not the case. `std.os.timespec` is used both for
libc and non-libc cases. `__kernel_timespec` is a special struct that is
used only for io_uring.
Tests with no names are executed when using `zig test` regardless of the
`--test-filter` used. Non-named tests should be used when simply
importing unit tests from another file. This allows `zig test` to find
all the appropriate tests, even when using `--test-filter`.
* std lib tests are passing on x86_64-linux with and without -lc
* stage2 is building from source on x86_64-linux
* down to 38 remaining uses of `usingnamespace`
The main purpose of this branch is to explore avoiding the
`usingnamespace` feature of the zig language, specifically with regards
to `std.os` and related functionality.
If this experiment is successful, it will provide a data point on
whether or not it would be practical to entirely remove `usingnamespace`
from the language.
In this commit, `usingnamespace` has been completely eliminated from
the Linux x86_64 compilation path, aside from io_uring.
The behavior tests pass, however that's as far as this branch goes. It is
very breaking, and a lot more work is needed before it could be
considered mergeable. I wanted to put a pull requset up early so that
zig programmers have time to provide feedback.
This is progress towards closing #6600 since it clarifies where the
actual "owner" of each declaration is, and reduces the number of
different ways to import the same declarations.
One of the main organizational strategies used here is to do namespacing
with real namespaces (e.g. structs) rather than by having declarations
share a common prefix (the C strategy). It's no coincidence that
`usingnamespace` has similar semantics to `#include` and becomes much
less necessary when using proper namespaces.
We already have a LICENSE file that covers the Zig Standard Library. We
no longer need to remind everyone that the license is MIT in every single
file.
Previously this was introduced to clarify the situation for a fork of
Zig that made Zig's LICENSE file harder to find, and replaced it with
their own license that required annual payments to their company.
However that fork now appears to be dead. So there is no need to
reinforce the copyright notice in every single file.
The primary purpose of this change is to eliminate one usage of
`usingnamespace` in the standard library - specifically the usage for
errno values in `std.os.linux`.
This is accomplished by truncating the `E` prefix from error values, and
making errno a proper enum.
A similar strategy can be used to eliminate some other `usingnamespace`
sites in the std lib.
Implement io_uring submission queue entry preparation methods for
epoll_ctl, poll_add and poll_remove.
Poll masks are designated as 32-bit little-endian integers as
specified in liburing's definitions.
Updated io_uring_prep_rw to take in an unsigned 64-bit address instead
of an anytype. io_uring_sqe by default assumes that the address
specified in a submission queue entry is an unsigned 64-bit integer.
Previously the fd parameter was ignored and so the result would not get
populated. Now it passes the fd pointer to the inline assembly so that
the results can be observed.
* Don't skip the TLS initialization (Fixes#9083)
* Add a test case where a PIE program is built and run
* Refactor the common initialization code in the Linux startup
sequence.
This finishes LemonBoy's Draft PR ziglang#6750. It updates ChildProcess to collect the output from stdout/stderr asynchronously using Overlapped IO and named pipes.
Reading stdin&stderr at different times may lead to nasty deadlocks (eg.
when stdout is read before stderr and the child process doesn't write
anything onto stdout).
Implement a polling mechanism to make sure this won't happen: we read
data from stderr/stdout as it becomes ready and then it's copied into an
ArrayList provided by the user, avoiding any kind of blocking read.
This patch adjusts the exit code for a child process to be a u8. Since
the WEXITSTATUS macro returns the lower eight bits, it's safe to assume
that we can truncate the returned u32.
The documentation (e.g. `man 7 rtnetlink`) states that ifi_change "is reserved for future use and should be always set to 0xFFFFFFFF". This is no longer true, even though the text hasn't been updated.
`msghdr` and `msghdr_const` definitions have been added back the way
they were in std.os. std.os.sendmsg has also been modified to accept a
msghdr_const again to ensure backwards-compatibility with this PR.
Underneath the hood, std.os.sendmsg will @ptrCast the provided
msghdr_const into a std.x.os.Socket.Message.
`sockaddr_storage` definitions have been added back the way they were in
std.os, except that it now simply aliases
std.x.os.Socket.Address.Native.Storage as all of
std.x.os.Socket.Address.Native.Storage's fields are equivalent to the
fields that were previously defined for std.x.os.bits.sockaddr_storage.
std.x.os.Socket.sendMessage now no longer is a stub that aliases
std.os.sendmsg, but instead calls and handles
errors from std.os.system.sendmsg directly.
Addresses feedback to urge backwards compatibility from @andrewrk.
Cross-platform versions of msghdr, sendmsg, recvmsg, linger, and iovec
were provided based on findings from glibc, musl, and Microsoft's
documentation.
Implemented initial Reactor interface for epoll (linux) which wraps
around I/O reactor subsystems such as epoll, kqueue, select, etc. across
different platforms. The Reactor interface allows for driving async I/O
in Zig applications.
A test was added for the Reactor interface to drive a TCP
client/listener socket pair.
A greatest-common-subset of possible socket initialization flags (close
socket on exec syscalls, initialize socket to be non-blocking) were
implemented.
A test was added for using sendmsg/recvmsg syscalls across different
platforms for a TCP client/listener socket pair.
All other uses of `ws2_32.socklen_t` in windows.zig casts the value to an i32. `recvfrom` should do so as well; it currently errors out with `expected type '?*i32', found '?*u32'`.
- more support for linux, android, freebsd, netbsd, openbsd, dragonfly
- centralize musl utils; musl logic is no longer intertwined with csu
- fix musl compilation to build crti/crtn for full archs list
- fix openbsd to support `zig build-lib -dynamic`
- initial dragonfly linking success (with a warning)
ancillary:
- fix emutls (openbsd) tests to use `try`
When WSASocketW gets WSANOTINITIALISED, now it will lock a mutex to
safely call WSAStartup and then try again one time.
This implementation:
* Does not use recursion
* Contains a detailed doc comment explaining why things are how they are
* Is careful about which errors are surfaced in the respective error
sets. `std.os.socket` intentionally does not have "not initialised"
as one of the possible errors.
Conflicts:
* build.zig
* src/Compilation.zig
* src/codegen/spirv/spec.zig
* src/link/SpirV.zig
* test/stage2/darwin.zig
- this one might be problematic; start.zig looks for `main` in the
root source file, not `_main`. Not sure why there is an underscore
there in master branch.
Conflicts:
* lib/std/os/linux.zig
* lib/std/os/windows/bits.zig
* src/Module.zig
* src/Sema.zig
* test/stage2/test.zig
Mainly I wanted Jakub's new macOS code for respecting stack size, since
we now depend on it for debug builds able to pass one of the test cases
for recursive comptime function calls with `@setEvalBranchQuota`.
The conflicts were all trivial.
Implement loading Winsock extensions.
Add missing Winsock extension GUID's.
Implement readVectorized() for POSIX sockets and
readVectorized() / writeVectorized() for Windows
sockets.
Inverse how mixins are used to implement platform-independent syscalls
for the std.x.os.Socket abstraction. This cleans up the API as suggested
by @komuw.
Add missing constants to ws2_32 such as SIO_BASE_HANDLE.
Made all declared constants in ws2_32 comptime_int, and fixed a few
broken constants (i.e. GUID's).
Fixed a few syscalls not having callconv(WINAPI) in ws2_32.
Fixed a typo in std.x.net.tcp.Client.
Removed unnecessary @alignCast's for socket addresses in
std.x.net.Socket.
Added a warning on using timeout methods for std.x.net.Socket.
Fixed compilation error spotted by CI in std.os that references
std.os.windows.ws2_32.SD_RECEIVE.
Renamed std.x.os.Socket.setOption()'s parameter `name: u32` to `code:
u32`.
Socket I/O methods such as read, readv, write, writev, send, recv,
sendmsg, recvmsg have been generalized to read(buf, flags), write(buf,
flags), readVectorized(vectors, flags), and writeVectorized(vectors,
flags). There is still some work left to be done abstracting both
readVectorized and writeVectorized properly across platforms, which is
work to be done in a future PR.
Support for setting the linger timeout of a socket, querying the remote
address of a socket, setting whether or not keep-alive messages are to
be sent through a connection-oriented socket periodically depending on
host operating system settings has been added.
`std.io.Reader` and `std.io.Writer` wrappers around `Socket` has been
implemented, which wrap around Socket.read(buf, flags) and
Socket.write(buf, flags). Both wrappers may be provided flags which are
passed to Socket.read / Socket.write accordingly.
Cross-platform support for `getpeername()` has been implemented.
Windows support for the new `std.x.os.Socket` has been implemented. To
accomplish this, a full refactor of `std.os.windows.ws2_32` has been
done to supply any missing definitions and constants based on
auto-generated Windows syscall bindings by @marler8997.
`std.x.net.TCP.Listener.setQuickACK` has been moved to
`std.x.net.TCP.Client.setQuickACK`.
Windows support for resolving the scope ID of an interface name
specified in an IPv6 address has been provided.
`sockaddr_storage` definitions have been provided for Windows, Linux,
and Darwin. `sockaddr_storage` is used to allocate space before any
socket addresses are queried via. calls such as accept(), getsockname(),
and getpeername().
Zig-friendly wrappers for GetQueuedCompletionStatusEx(), getpeername(),
SetConsoleCtrlHandler(), SetFileCompletionNotificationModes() syscalls
on Windows have been provided.
Socket.setOption() was provided to set the value of a socket option in
place of os.setsockopt. Socket.getOption() will be provided in a future
PR.
There is still further work to be done regarding querying socket option
values on Windows, which is to be done in a subsequent PR.
The system `stat` structure includes padding, and, on some
operating systems such as all BSDs, "spare" bytes at the end.
We can't reliably compare two `Stat` values if these are
uninitialized, while being later compared.
This is what was causing the `fstatat` test to fail on FreeBSD
since the update to LLVM 12. It was previously only passing by
accident.
Conflicts:
* doc/langref.html.in
* lib/std/enums.zig
* lib/std/fmt.zig
* lib/std/hash/auto_hash.zig
* lib/std/math.zig
* lib/std/mem.zig
* lib/std/meta.zig
* test/behavior/alignof.zig
* test/behavior/bitcast.zig
* test/behavior/bugs/1421.zig
* test/behavior/cast.zig
* test/behavior/ptrcast.zig
* test/behavior/type_info.zig
* test/behavior/vector.zig
Master branch added `try` to a bunch of testing function calls, and some
lines also had changed how to refer to the native architecture and other
`@import("builtin")` stuff.
There are some small problems here and there, mostly due to the pointers
having the lsb set and disrupting the fn alignment tests and the
`@FrameSize` implementation.
