The TLS area may be located in the upper part of the address space and,
if the platform expects a constant offset to be applied, may make the tp
register calculation overflow.
Use +% instead of +, the overflow is harmless.
Some systems (Solaris, OpenBSD, AIX) change their definitions of
sockaddr_storage to be larger than 128 bytes. This comment adds a new
constant in the `sockaddr` that defines the size for every system.
Fixes#9759
Extract existing constants to do with TCP socket options into a 'TCP'
namespace.
Export 'MSG' and 'TCP' from std.os.{linux, windows} into std.c.
Fix compile errors to do with std.x.os.Socket methods related to setting
TCP socket options.
Handle errors in the case that an interface could not be resolved in an
IPv6 address on Windows. Tested using Wine with the loopback interface
disabled.
Have all instantiations of std.x.os.Socket on Windows instantiate an
overlapped socket descriptor. Fixes the '1ms read timeout' test in
std.x.net.tcp.Client. The test would previously deadlock, as read
timeouts only apply to overlapped sockets.
Windows documentation by default recommends that most instantiations of
sockets on Windows be overlapped sockets (s.t. they may operate in both
blocking or nonblocking mode when operated with WSA* syscalls). Refer to
the documentation for WSASocketA for more info.
I incorrectly assumed that __kernel_timespec was used when not linking
libc, however that is not the case. `std.os.timespec` is used both for
libc and non-libc cases. `__kernel_timespec` is a special struct that is
used only for io_uring.
Tests with no names are executed when using `zig test` regardless of the
`--test-filter` used. Non-named tests should be used when simply
importing unit tests from another file. This allows `zig test` to find
all the appropriate tests, even when using `--test-filter`.
* std lib tests are passing on x86_64-linux with and without -lc
* stage2 is building from source on x86_64-linux
* down to 38 remaining uses of `usingnamespace`
The main purpose of this branch is to explore avoiding the
`usingnamespace` feature of the zig language, specifically with regards
to `std.os` and related functionality.
If this experiment is successful, it will provide a data point on
whether or not it would be practical to entirely remove `usingnamespace`
from the language.
In this commit, `usingnamespace` has been completely eliminated from
the Linux x86_64 compilation path, aside from io_uring.
The behavior tests pass, however that's as far as this branch goes. It is
very breaking, and a lot more work is needed before it could be
considered mergeable. I wanted to put a pull requset up early so that
zig programmers have time to provide feedback.
This is progress towards closing #6600 since it clarifies where the
actual "owner" of each declaration is, and reduces the number of
different ways to import the same declarations.
One of the main organizational strategies used here is to do namespacing
with real namespaces (e.g. structs) rather than by having declarations
share a common prefix (the C strategy). It's no coincidence that
`usingnamespace` has similar semantics to `#include` and becomes much
less necessary when using proper namespaces.
We already have a LICENSE file that covers the Zig Standard Library. We
no longer need to remind everyone that the license is MIT in every single
file.
Previously this was introduced to clarify the situation for a fork of
Zig that made Zig's LICENSE file harder to find, and replaced it with
their own license that required annual payments to their company.
However that fork now appears to be dead. So there is no need to
reinforce the copyright notice in every single file.
The primary purpose of this change is to eliminate one usage of
`usingnamespace` in the standard library - specifically the usage for
errno values in `std.os.linux`.
This is accomplished by truncating the `E` prefix from error values, and
making errno a proper enum.
A similar strategy can be used to eliminate some other `usingnamespace`
sites in the std lib.
Implement io_uring submission queue entry preparation methods for
epoll_ctl, poll_add and poll_remove.
Poll masks are designated as 32-bit little-endian integers as
specified in liburing's definitions.
Updated io_uring_prep_rw to take in an unsigned 64-bit address instead
of an anytype. io_uring_sqe by default assumes that the address
specified in a submission queue entry is an unsigned 64-bit integer.
Previously the fd parameter was ignored and so the result would not get
populated. Now it passes the fd pointer to the inline assembly so that
the results can be observed.
* Don't skip the TLS initialization (Fixes#9083)
* Add a test case where a PIE program is built and run
* Refactor the common initialization code in the Linux startup
sequence.
This finishes LemonBoy's Draft PR ziglang#6750. It updates ChildProcess to collect the output from stdout/stderr asynchronously using Overlapped IO and named pipes.
Reading stdin&stderr at different times may lead to nasty deadlocks (eg.
when stdout is read before stderr and the child process doesn't write
anything onto stdout).
Implement a polling mechanism to make sure this won't happen: we read
data from stderr/stdout as it becomes ready and then it's copied into an
ArrayList provided by the user, avoiding any kind of blocking read.
This patch adjusts the exit code for a child process to be a u8. Since
the WEXITSTATUS macro returns the lower eight bits, it's safe to assume
that we can truncate the returned u32.
The documentation (e.g. `man 7 rtnetlink`) states that ifi_change "is reserved for future use and should be always set to 0xFFFFFFFF". This is no longer true, even though the text hasn't been updated.
`msghdr` and `msghdr_const` definitions have been added back the way
they were in std.os. std.os.sendmsg has also been modified to accept a
msghdr_const again to ensure backwards-compatibility with this PR.
Underneath the hood, std.os.sendmsg will @ptrCast the provided
msghdr_const into a std.x.os.Socket.Message.
`sockaddr_storage` definitions have been added back the way they were in
std.os, except that it now simply aliases
std.x.os.Socket.Address.Native.Storage as all of
std.x.os.Socket.Address.Native.Storage's fields are equivalent to the
fields that were previously defined for std.x.os.bits.sockaddr_storage.
std.x.os.Socket.sendMessage now no longer is a stub that aliases
std.os.sendmsg, but instead calls and handles
errors from std.os.system.sendmsg directly.
Addresses feedback to urge backwards compatibility from @andrewrk.
Cross-platform versions of msghdr, sendmsg, recvmsg, linger, and iovec
were provided based on findings from glibc, musl, and Microsoft's
documentation.
Implemented initial Reactor interface for epoll (linux) which wraps
around I/O reactor subsystems such as epoll, kqueue, select, etc. across
different platforms. The Reactor interface allows for driving async I/O
in Zig applications.
A test was added for the Reactor interface to drive a TCP
client/listener socket pair.
A greatest-common-subset of possible socket initialization flags (close
socket on exec syscalls, initialize socket to be non-blocking) were
implemented.
A test was added for using sendmsg/recvmsg syscalls across different
platforms for a TCP client/listener socket pair.
All other uses of `ws2_32.socklen_t` in windows.zig casts the value to an i32. `recvfrom` should do so as well; it currently errors out with `expected type '?*i32', found '?*u32'`.
- more support for linux, android, freebsd, netbsd, openbsd, dragonfly
- centralize musl utils; musl logic is no longer intertwined with csu
- fix musl compilation to build crti/crtn for full archs list
- fix openbsd to support `zig build-lib -dynamic`
- initial dragonfly linking success (with a warning)
ancillary:
- fix emutls (openbsd) tests to use `try`
When WSASocketW gets WSANOTINITIALISED, now it will lock a mutex to
safely call WSAStartup and then try again one time.
This implementation:
* Does not use recursion
* Contains a detailed doc comment explaining why things are how they are
* Is careful about which errors are surfaced in the respective error
sets. `std.os.socket` intentionally does not have "not initialised"
as one of the possible errors.
Conflicts:
* build.zig
* src/Compilation.zig
* src/codegen/spirv/spec.zig
* src/link/SpirV.zig
* test/stage2/darwin.zig
- this one might be problematic; start.zig looks for `main` in the
root source file, not `_main`. Not sure why there is an underscore
there in master branch.
Conflicts:
* lib/std/os/linux.zig
* lib/std/os/windows/bits.zig
* src/Module.zig
* src/Sema.zig
* test/stage2/test.zig
Mainly I wanted Jakub's new macOS code for respecting stack size, since
we now depend on it for debug builds able to pass one of the test cases
for recursive comptime function calls with `@setEvalBranchQuota`.
The conflicts were all trivial.
Implement loading Winsock extensions.
Add missing Winsock extension GUID's.
Implement readVectorized() for POSIX sockets and
readVectorized() / writeVectorized() for Windows
sockets.
Inverse how mixins are used to implement platform-independent syscalls
for the std.x.os.Socket abstraction. This cleans up the API as suggested
by @komuw.
Add missing constants to ws2_32 such as SIO_BASE_HANDLE.
Made all declared constants in ws2_32 comptime_int, and fixed a few
broken constants (i.e. GUID's).
Fixed a few syscalls not having callconv(WINAPI) in ws2_32.
Fixed a typo in std.x.net.tcp.Client.
Removed unnecessary @alignCast's for socket addresses in
std.x.net.Socket.
Added a warning on using timeout methods for std.x.net.Socket.
Fixed compilation error spotted by CI in std.os that references
std.os.windows.ws2_32.SD_RECEIVE.
Renamed std.x.os.Socket.setOption()'s parameter `name: u32` to `code:
u32`.
Socket I/O methods such as read, readv, write, writev, send, recv,
sendmsg, recvmsg have been generalized to read(buf, flags), write(buf,
flags), readVectorized(vectors, flags), and writeVectorized(vectors,
flags). There is still some work left to be done abstracting both
readVectorized and writeVectorized properly across platforms, which is
work to be done in a future PR.
Support for setting the linger timeout of a socket, querying the remote
address of a socket, setting whether or not keep-alive messages are to
be sent through a connection-oriented socket periodically depending on
host operating system settings has been added.
`std.io.Reader` and `std.io.Writer` wrappers around `Socket` has been
implemented, which wrap around Socket.read(buf, flags) and
Socket.write(buf, flags). Both wrappers may be provided flags which are
passed to Socket.read / Socket.write accordingly.
Cross-platform support for `getpeername()` has been implemented.
Windows support for the new `std.x.os.Socket` has been implemented. To
accomplish this, a full refactor of `std.os.windows.ws2_32` has been
done to supply any missing definitions and constants based on
auto-generated Windows syscall bindings by @marler8997.
`std.x.net.TCP.Listener.setQuickACK` has been moved to
`std.x.net.TCP.Client.setQuickACK`.
Windows support for resolving the scope ID of an interface name
specified in an IPv6 address has been provided.
`sockaddr_storage` definitions have been provided for Windows, Linux,
and Darwin. `sockaddr_storage` is used to allocate space before any
socket addresses are queried via. calls such as accept(), getsockname(),
and getpeername().
Zig-friendly wrappers for GetQueuedCompletionStatusEx(), getpeername(),
SetConsoleCtrlHandler(), SetFileCompletionNotificationModes() syscalls
on Windows have been provided.
Socket.setOption() was provided to set the value of a socket option in
place of os.setsockopt. Socket.getOption() will be provided in a future
PR.
There is still further work to be done regarding querying socket option
values on Windows, which is to be done in a subsequent PR.
The system `stat` structure includes padding, and, on some
operating systems such as all BSDs, "spare" bytes at the end.
We can't reliably compare two `Stat` values if these are
uninitialized, while being later compared.
This is what was causing the `fstatat` test to fail on FreeBSD
since the update to LLVM 12. It was previously only passing by
accident.
Conflicts:
* doc/langref.html.in
* lib/std/enums.zig
* lib/std/fmt.zig
* lib/std/hash/auto_hash.zig
* lib/std/math.zig
* lib/std/mem.zig
* lib/std/meta.zig
* test/behavior/alignof.zig
* test/behavior/bitcast.zig
* test/behavior/bugs/1421.zig
* test/behavior/cast.zig
* test/behavior/ptrcast.zig
* test/behavior/type_info.zig
* test/behavior/vector.zig
Master branch added `try` to a bunch of testing function calls, and some
lines also had changed how to refer to the native architecture and other
`@import("builtin")` stuff.
There are some small problems here and there, mostly due to the pointers
having the lsb set and disrupting the fn alignment tests and the
`@FrameSize` implementation.
Address comments from @ifreund and @MasterQ32 to address unsafeness and
ergonomics of the `Address` API.
Rename the `TCP` namespace to `tcp` as it does not contain any
top-level fields.
Fix missing reference to `sockaddr` which was identified by @kprotty in
os/bits/linux/arm64.zig.
The `Socket` abstraction was refactored to only comprise of methods that
can be generically used/applied to all socket domains and protocols.
A more comprehensive IPv4/IPv6 module derived from @LemonBoy's earlier
work was implemented under `std.x.os.IPv4` and `std.x.os.IPv6`. Using
this module, one can then combine them together into a union for example
in order to optimize memory usage when dealing with socket addresses.
A `TCP.Client` and `TCP.Listener` abstraction is introduced that is one
layer over the `Socket` abstraction, which isolates methods that can
only be applied to a "client socket" and a "listening socket". All prior
tests from the `Socket` abstraction, which all previously operated
assuming the socket is operating via. TCP/IP, were moved. All TCP socket
options were also moved into the `TCP.Client` and `TCP.Listener`
abstractions respectively away from the `Socket` abstraction.
Some additional socket options from @LemonBoy's prior PR for Darwin were
also moved in (i.e. SIGNOPIPE).
* `comptime const` is redundant
* don't use `extern enum`; specify a tag type.
`extern enum` is only when you need tags to alias. But aliasing tags
is a smell. I will be making a proposal shortly to remove `extern enum`
from the language.
* there is no such thing as `packed enum`.
* instead of `catch |_|`, omit the capture entirely.
* unused function definition with missing parameter name
* using `try` outside of a function or test
liburing commit: 1bafb3ce5f
As stated in the liburing commit message, this fixes a regression,
reverting code that was added specutively to avoid a syscall in some
cases.
Beside the new order being consistent with the ThreadPool API and making
more sense, this shuffling allows to write the context argument type in
terms of the startFn arguments, reducing the use of anytype (eg. less
explicit casts when using comptime_int parameters, yay).
Sorry for the breakage.
Closes#8082
* move concurrency primitives that always operate on kernel threads to
the std.Thread namespace
* remove std.SpinLock. Nobody should use this in a non-freestanding
environment; the other primitives are always preferable. In
freestanding, it will be necessary to put custom spin logic in there,
so there are no use cases for a std lib version.
* move some std lib files to the top level fields convention
* add std.Thread.spinLoopHint
* add std.Thread.Condition
* add std.Thread.Semaphore
* new implementation of std.Thread.Mutex for Windows and non-pthreads Linux
* add std.Thread.RwLock
Implementations provided by @kprotty
... and mem.copy operations. Requires slightly larger input buffers than result length. Add helper functions std.mem.alignInBytes and std.mem.alignInSlice.
The NtQueryInformationFile with .FileNormalizedNameInformation is only available in Windows 10 1803 (rs4) and later, however there is probably still another route we can go via ntdll.
* read directly into the ArrayList buffers.
* respect max_output_bytes
* std.ArrayList:
- make `allocatedSlice` public.
- add `unusedCapacitySlice`.
I removed the Windows implementation of this stuff; I am doing a partial
merge of LemonBoy's patch with the understanding that a later patch can
add the Windows implementation after it is vetted.
Reading stdin&stderr at different times may lead to nasty deadlocks (eg.
when stdout is read before stderr and the child process doesn't write
anything onto stdout).
Implement a polling mechanism to make sure this won't happen: we read
data from stderr/stdout as it becomes ready and then it's copied into an
ArrayList provided by the user, avoiding any kind of blocking read.
* Truncate user and group ids
Calls to `getuid`, `getgid` and their `eid` variants fail to compile on
64bit Linux systems because the return value of the syscall is of
`usize` and needs to be truncated to fit the size of `uid_t` that is 32
bit.
Thanks to @FireFox317 for figuring this out in Zig's Discord channel!
* Add a regression test for user and group ids
* Replace @truncate with @intCast
This should be safe because `uid_t` will be 32-bit.
* Add missing import for getauxval
* Add missing package names
* Revert "Add missing import for getauxval"
This reverts commit 38f93dc89e.
* Skip user and group test if builtin.link_libc
Everybody gets what they want!
* AT_RANDOM is completely ignored.
* On Linux, MADV_WIPEONFORK is used to provide fork safety.
* On pthread systems, `pthread_atfork` is used to provide fork safety.
* For systems that do not have the capability to provide fork safety,
the implementation falls back to calling getrandom() every time.
* If madvise is unavailable or returns an error, or pthread_atfork
fails for whatever reason, it falls back to calling getrandom() every
time.
* Applications may choose to opt-out of fork safety.
* Applications may choose to opt-in to unconditionally calling
getrandom() for every call to std.crypto.random.fillFn.
* Added `std.meta.globalOption`.
* Added `std.os.madvise` and related bits.
* Bumped up the size of the main thread TLS buffer. See the comment
there for justification.
* Simpler hot path in TLS initialization.
ad05509 introduced a fix for the wrong problem, the logic to align the
start of main_thread_tls_buffer was already there but was flawed.
Fix it for good and avoid wasting too many bytes for alignment purposes.
