Commit Graph

9525 Commits

Author SHA1 Message Date
Andrew Kelley
4e32edbff5 fuzzing: comptime assertions to protect the ABI
compile errors are nice
2024-08-08 21:46:36 -07:00
YANG Xudong
a9b65b6fd4
std: add loongarch64 support (#20915)
Co-authored-by: Alex Rønne Petersen <alex@alexrp.com>
2024-08-09 00:30:57 +00:00
John Benediktsson
a854ce3021
std.hash_map: adding a rehash() method (#19923)
see #17851
2024-08-08 11:59:22 -07:00
Andrew Kelley
8031251c33
Merge pull request #20997 from xxxbxxx/debuglink
std.debug.Dwarf: improve loading debug symbol from separate files
2024-08-08 11:06:14 -07:00
Andrew Kelley
cf87a1a7cf language: add module name field to @src
closes #20963
2024-08-08 07:47:14 -07:00
Jan Hendrik Farr
ca012e5b69 std.posix: read on timerfd can return error.Canceled 2024-08-08 01:12:26 -07:00
Andrew Kelley
468b976f63
Merge pull request #20975 from squeek502/cpu-features-update
update_cpu_features: Fixes and updates
2024-08-07 22:48:57 -07:00
Xavier Bouchoux
cbdf9bf5ee std.debug.Dwarf: try to load the debuginfo from the debuginfod cache.
The previous mecanism for linux distributions to delivers debug info into `/usr/lib/debug` no longer seems in use.
the current mecanism often is using `debuginfod` (https://sourceware.org/elfutils/Debuginfod.html)

This commit only tries to load already available debuginfo but does not try to make any download requests.
the user can manually run `debuginfod-find debuginfo PATH` to populate the cache.
2024-08-08 07:28:59 +02:00
Xavier Bouchoux
7e966de45e std.debug.Dwarf: fix loading external debuginfo in the ".debuglink" case.
- look up the debuglink file in the directory of the executable file (instead of the cwd)
 - fix parsing of debuglink section (the 4-byte alignement is within the file, unrelated to the in-memory address)
2024-08-08 07:15:00 +02:00
Andrew Kelley
7a7421c749
Merge pull request #20973 from ziglang/fix-json-utf8
std.json: fix partial strings getting dropped when multi-byte codepoints span input buffers
2024-08-07 18:37:49 -07:00
Andrew Kelley
4381bac792 build runner: output hint when no --port provided 2024-08-07 16:55:46 -07:00
YANG Xudong
b8705ed652
loongarch: various architecture specific fixes (#20912) 2024-08-07 13:06:29 -07:00
Pat Tullmann
4d6429fc4f POSIX link() syscall only takes two arguments (no flags)
The signature is documented as:

   int link(const char *, const char *);

(see https://man7.org/linux/man-pages/man2/link.2.html or https://man.netbsd.org/link.2)

And its not some Linux extension, the [syscall
implementation](21b136cc63/fs/namei.c (L4794-L4797))
only expects two arguments too.

It probably *should* have a flags parameter, but its too late now.

I am a bit surprised that linking glibc or musl against code that invokes
a 'link' with three parameters doesn't fail (at least, I couldn't get any
local test cases to trigger a compile or link error).

The test case in std/posix/test.zig is currently disabled, but if I
manually enable it, it works with this change.
2024-08-07 13:05:42 -07:00
Jeffrey C. Ollie
979fd12be9 Add getppid to std.c and std.os.linux.
The std lib is missing getppid, this patch adds it.
2024-08-07 13:03:21 -07:00
Andrew Kelley
0e99f517f2
Merge pull request #20958 from ziglang/fuzz
introduce a fuzz testing web interface
2024-08-07 11:55:30 -07:00
Ryan Liptak
e912411fbe Run update_cpu_features against LLVM 18.1.8
- RISCV unaligned-scalar-mem was added in LLVM 18.1.6
- Fixes arm v9_5a confusion
2024-08-07 09:03:05 -07:00
Josh Wolfe
a805454dea fix partial strings getting dropped when multi-byte codepoints span input buffers 2024-08-07 06:29:34 -04:00
Josh Wolfe
8bec737ca2 fix tests actually checking string contents 2024-08-07 06:29:34 -04:00
Alex Rønne Petersen
f9f8942008 std.os.linux: Move clone() here and stop exporting it. 2024-08-07 01:19:51 -07:00
Andrew Kelley
75f78bfb77
Merge pull request #20922 from alexrp/vdso
`std.os.linux`: Fix VDSO for mips, add VDSO for riscv
2024-08-07 01:18:35 -07:00
Andrew Kelley
cd5f673cae
Merge pull request #20909 from alexrp/glibc-riscv
Support building glibc for riscv32/riscv64
2024-08-07 01:14:54 -07:00
Andrew Kelley
8184912a98
Merge pull request #20925 from alexrp/windows-tls
`std`: Some Windows TLS cleanup and fixes
2024-08-07 01:12:11 -07:00
Alex Rønne Petersen
c0681d6b6e start: Add sparc32 support. 2024-08-07 01:09:18 -07:00
Alex Rønne Petersen
8cbf091a9a std.os.linux.start_pie: Add sparc/sparc64 support. 2024-08-07 01:09:06 -07:00
Andrew Kelley
b071b10ce8
Merge pull request #20894 from alexrp/target-cleanup-4
`std.Target`: Minor rework to some `isArch()` functions, fix some related issues throughout `std`
2024-08-07 01:08:44 -07:00
Jakub Dóka
a6486492be
std.crypto.ecdsa: use separate function for null seed (#20953)
Due to the `std.crypto.ecdsa.KeyPair.create` taking and optional of seed, even if the seed is generated, cross-compiling to the environments without standard random source (eg. wasm) (`std.crypto.random.bytes`) will fail to compile.

This commit changes the API of the problematic function and moves the random seed generation to a new utility function.
2024-08-07 01:06:15 -07:00
Alex Rønne Petersen
ebd0c6ffd0 std.zig.target: Add arc-linux-gnu.
https://lists.gnu.org/archive/html/info-gnu/2020-08/msg00002.html
2024-08-07 01:03:22 -07:00
Alex Rønne Petersen
00097c3bb8 std.Target: Some corrections and additions to charSignedness().
Sourced from GCC backends and some manual testing with clang.
2024-08-07 00:56:02 -07:00
Andrew Kelley
40edd11516 std.debug: fix compile errors on windows and macos 2024-08-07 00:48:32 -07:00
Andrew Kelley
8dae629c4f update branch for latest std.sort changes 2024-08-07 00:48:32 -07:00
Andrew Kelley
bfc2ee03e5 fuzzer web ui: resolve cwd in sources.tar
because the wasm code needs to string match against debug information
2024-08-07 00:48:32 -07:00
Andrew Kelley
e64a00950e fuzzer web ui: introduce entry points
so you can have somewhere to start browsing
2024-08-07 00:48:32 -07:00
Andrew Kelley
dec7e45f7c fuzzer web UI: receive coverage information
* libfuzzer: track unique runs instead of deduplicated runs
  - easier for consumers to notice when to recheck the covered bits.
* move common definitions to `std.Build.Fuzz.abi`.

build runner sends all the information needed to fuzzer web interface
client needed in order to display inline coverage information along with
source code.
2024-08-07 00:48:32 -07:00
Andrew Kelley
22925636f7 std.debug.Coverage: use extern structs
helps the serialization use case
2024-08-07 00:48:32 -07:00
Andrew Kelley
b9fd0eeca6 add std.http.WebSocket 2024-08-07 00:48:32 -07:00
Andrew Kelley
d36c182748 std.posix: add some more void bits
prevents unnecessary compilation errors on wasm32-freestanding
2024-08-07 00:48:32 -07:00
Andrew Kelley
517cfb0dd1 fuzzing: progress towards web UI
* libfuzzer: close file after mmap
* fuzzer/main.js: connect with EventSource and debug dump the messages.
  currently this prints how many fuzzer runs have been attempted to
  console.log.
* extract some `std.debug.Info` logic into `std.debug.Coverage`.
  Prepares for consolidation across multiple different executables which
  share source files, and makes it possible to send all the
  PC/SourceLocation mapping data with 4 memcpy'd arrays.
* std.Build.Fuzz:
  - spawn a thread to watch the message queue and signal event
    subscribers.
  - track coverage map data
  - respond to /events URL with EventSource messages on a timer
2024-08-07 00:48:32 -07:00
Andrew Kelley
53aa9d75a9 std.debug.Info.resolveSourceLocations: O(N) implementation 2024-08-07 00:48:32 -07:00
Andrew Kelley
c2ab4614b6 std.Debug.Info: remove std.Progress integration
it's too fast to need it now
2024-08-07 00:48:32 -07:00
Andrew Kelley
1792258dc8 std.debug.Dwarf: precompute .debug_line table
yields a 60x speedup for resolveSourceLocations in debug builds
2024-08-07 00:48:32 -07:00
Andrew Kelley
66954e8330 std.debug.FixedBufferReader is fine
it does not need to be deprecated
2024-08-07 00:48:32 -07:00
Andrew Kelley
de47acd732 code coverage dumping tool basic implementation
* std.debug.Dwarf: add `sortCompileUnits` along with a field to track
  the state for the purpose of assertions and correct API usage.
  This makes batch lookups faster.
  - in the future, findCompileUnit should be enhanced to rely on sorted
    compile units as well.
* implement `std.debug.Dwarf.resolveSourceLocations` as well as
  `std.debug.Info.resolveSourceLocations`. It's still pretty slow, since
  it calls getLineNumberInfo for each array element, repeating a lot of
  work unnecessarily.
* integrate these APIs with `std.Progress` to understand what is taking
  so long.

The output I'm seeing from this tool shows a lot of missing source
locations. In particular, the main area of interest is missing for my
tokenizer fuzzing example.
2024-08-07 00:48:32 -07:00
Andrew Kelley
2e12b45d8b introduce tool for dumping coverage file
with debug info resolved.

begin efforts of providing `std.debug.Info`, a cross-platform
abstraction for loading debug information into an in-memory format that
supports queries such as "what is the source location of this virtual
memory address?"

Unlike `std.debug.SelfInfo`, this API does not assume the debug
information in question happens to match the host CPU architecture, OS,
or other target properties.
2024-08-07 00:48:32 -07:00
Andrew Kelley
107b272766 fuzzer: share zig to html rendering with autodocs 2024-08-07 00:48:32 -07:00
Andrew Kelley
e0ffac4e3c introduce a web interface for fuzzing
* new .zig-cache subdirectory: 'v'
  - stores coverage information with filename of hash of PCs that want
    coverage. This hash is a hex encoding of the 64-bit coverage ID.
* build runner
  * fixed bug in file system inputs when a compile step has an
    overridden zig_lib_dir field set.
  * set some std lib options optimized for the build runner
    - no side channel mitigations
    - no Transport Layer Security
    - no crypto fork safety
  * add a --port CLI arg for choosing the port the fuzzing web interface
    listens on. it defaults to choosing a random open port.
  * introduce a web server, and serve a basic single page application
    - shares wasm code with autodocs
    - assets are created live on request, for convenient development
      experience. main.wasm is properly cached if nothing changes.
    - sources.tar comes from file system inputs (introduced with the
      `--watch` feature)
  * receives coverage ID from test runner and sends it on a thread-safe
    queue to the WebServer.
* test runner
  - takes a zig cache directory argument now, for where to put coverage
    information.
  - sends coverage ID to parent process
* fuzzer
  - puts its logs (in debug mode) in .zig-cache/tmp/libfuzzer.log
  - computes coverage_id and makes it available with
    `fuzzer_coverage_id` exported function.
  - the memory-mapped coverage file is now namespaced by the coverage id
    in hex encoding, in `.zig-cache/v`
* tokenizer
  - add a fuzz test to check that several properties are upheld
2024-08-07 00:48:32 -07:00
Alex Rønne Petersen
ef502daafe
std.Target: Handle mesa3d in c_type_bit_size(). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
23b5a6c71e
std.Target: Treat spirv as identical to spirv64 for ABI size/alignment purposes.
This is arbitrary since spirv (as opposed to spirv32/spirv64) refers to the
version with logical memory layout, i.e. no 'real' pointers. This change at
least matches what clang does.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
231f322a65
std.Target: Fix C long double size for ve (16, not 8). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
7b47ebe576
std.Target: Fix C long double size for sparc32 (8, not 16). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
eef499812c
std.Target: Fix C long double size for amdhsa, amdpal, and mesa3d (8, not 16). 2024-08-06 20:17:03 +02:00