Commit Graph

30466 Commits

Author SHA1 Message Date
Andrew Kelley
b071b10ce8
Merge pull request #20894 from alexrp/target-cleanup-4
`std.Target`: Minor rework to some `isArch()` functions, fix some related issues throughout `std`
2024-08-07 01:08:44 -07:00
Alex Rønne Petersen
8268d7be52 process_headers: Add arc-linux-gnu. 2024-08-07 01:07:09 -07:00
Jakub Dóka
a6486492be
std.crypto.ecdsa: use separate function for null seed (#20953)
Due to the `std.crypto.ecdsa.KeyPair.create` taking and optional of seed, even if the seed is generated, cross-compiling to the environments without standard random source (eg. wasm) (`std.crypto.random.bytes`) will fail to compile.

This commit changes the API of the problematic function and moves the random seed generation to a new utility function.
2024-08-07 01:06:15 -07:00
Alex Rønne Petersen
ebd0c6ffd0 std.zig.target: Add arc-linux-gnu.
https://lists.gnu.org/archive/html/info-gnu/2020-08/msg00002.html
2024-08-07 01:03:22 -07:00
Ryan Liptak
304519da27 Make Type.intAbiAlignment match LLVM alignment for x86-windows target
During the LLVM 18 upgrade, two changes were made that changed `@alignOf(u64)` to 4 for the x86-windows target:

- `Type.maxIntAlignment` was made to return 16 for x86 (200e06b). Before that commit, `maxIntAlignment` was 8 for windows/uefi and 4 for everything else
- `Type.intAbiAlignment` was made to return 4 for 33...64 (7e1cba7 + e89d6fc). Before those commits, `intAbiAlignment` would return 8, since the maxIntAlignment for x86-windows was 8 (and for other targets, the `maxIntAlignment` of 4 would clamp the `intAbiAlignment` to 4)

`src/codegen/llvm.zig` has its own alignment calculations that no longer match the values returned from the `Type` functions. For the x86-windows target, this loop:

ddcb7b1c11/src/codegen/llvm.zig (L558-L567)

when the `size` is 64 will set `abi` and `pref` to 64 (meaning an align of 8 bytes), which doesn't match the `Type` alignment of 4.

This commit makes `Type.intAbiAlignment` match the alignment calculated in `codegen/llvm.zig`.

Fixes #20047
Fixes #20466
Fixes #20469
2024-08-07 00:59:46 -07:00
Alex Rønne Petersen
00097c3bb8 std.Target: Some corrections and additions to charSignedness().
Sourced from GCC backends and some manual testing with clang.
2024-08-07 00:56:02 -07:00
Andrew Kelley
d721d9af69 update coff_dwarf standalone test to new API
and make it still test compilation on non-Windows
2024-08-07 00:48:32 -07:00
Andrew Kelley
2a651eab45 build runner: --fuzz not yet supported on Windows 2024-08-07 00:48:32 -07:00
Andrew Kelley
904fcda736 Compilation: fix -femit-docs 2024-08-07 00:48:32 -07:00
Andrew Kelley
ff503edc04 Compilation: fix not showing sub-errors for autodocs 2024-08-07 00:48:32 -07:00
Andrew Kelley
40edd11516 std.debug: fix compile errors on windows and macos 2024-08-07 00:48:32 -07:00
Andrew Kelley
8dae629c4f update branch for latest std.sort changes 2024-08-07 00:48:32 -07:00
Andrew Kelley
529df8c007 libfuzzer: fix looking at wrong memory for pc counters
this fix bypasses the slice bounds, reading garbage data for up to the
last 7 bits (which are technically supposed to be ignored). that's going
to need to be fixed, let's fix that along with switching from byte elems
to usize elems.
2024-08-07 00:48:32 -07:00
Andrew Kelley
5f5a7b53a4 wasm zig source rendering: fix annotation location off-by-one 2024-08-07 00:48:32 -07:00
Andrew Kelley
1484f174ea fuzzer web ui: fail scrolling into view gracefully 2024-08-07 00:48:32 -07:00
Andrew Kelley
895fa87d77 dump-cov: show seen PCs 2024-08-07 00:48:32 -07:00
Andrew Kelley
bfc2ee03e5 fuzzer web ui: resolve cwd in sources.tar
because the wasm code needs to string match against debug information
2024-08-07 00:48:32 -07:00
Andrew Kelley
38227e9289 fuzzer web UI: render PCs with red or green depending on coverage 2024-08-07 00:48:32 -07:00
Andrew Kelley
3d48602c99 fuzzer web UI: annotated PCs in source view 2024-08-07 00:48:32 -07:00
Andrew Kelley
ef4c2193fc fuzzer web UI: navigate by source location index
This will help scroll the point of interest into view
2024-08-07 00:48:32 -07:00
Andrew Kelley
db69641061 fuzzing web ui: make entry point links clickable 2024-08-07 00:48:32 -07:00
Andrew Kelley
e64a00950e fuzzer web ui: introduce entry points
so you can have somewhere to start browsing
2024-08-07 00:48:32 -07:00
Andrew Kelley
6e6164f8a6 fuzzer web ui: add coverage stat 2024-08-07 00:48:32 -07:00
Andrew Kelley
f56d113503 fuzzer web ui: render stats 2024-08-07 00:48:32 -07:00
Andrew Kelley
dec7e45f7c fuzzer web UI: receive coverage information
* libfuzzer: track unique runs instead of deduplicated runs
  - easier for consumers to notice when to recheck the covered bits.
* move common definitions to `std.Build.Fuzz.abi`.

build runner sends all the information needed to fuzzer web interface
client needed in order to display inline coverage information along with
source code.
2024-08-07 00:48:32 -07:00
Andrew Kelley
22925636f7 std.debug.Coverage: use extern structs
helps the serialization use case
2024-08-07 00:48:32 -07:00
Andrew Kelley
b9fd0eeca6 add std.http.WebSocket 2024-08-07 00:48:32 -07:00
Andrew Kelley
d36c182748 std.posix: add some more void bits
prevents unnecessary compilation errors on wasm32-freestanding
2024-08-07 00:48:32 -07:00
Andrew Kelley
517cfb0dd1 fuzzing: progress towards web UI
* libfuzzer: close file after mmap
* fuzzer/main.js: connect with EventSource and debug dump the messages.
  currently this prints how many fuzzer runs have been attempted to
  console.log.
* extract some `std.debug.Info` logic into `std.debug.Coverage`.
  Prepares for consolidation across multiple different executables which
  share source files, and makes it possible to send all the
  PC/SourceLocation mapping data with 4 memcpy'd arrays.
* std.Build.Fuzz:
  - spawn a thread to watch the message queue and signal event
    subscribers.
  - track coverage map data
  - respond to /events URL with EventSource messages on a timer
2024-08-07 00:48:32 -07:00
Andrew Kelley
5f92a036f9 README: update how std lib docs are found in a release build 2024-08-07 00:48:32 -07:00
Andrew Kelley
53aa9d75a9 std.debug.Info.resolveSourceLocations: O(N) implementation 2024-08-07 00:48:32 -07:00
Andrew Kelley
c2ab4614b6 std.Debug.Info: remove std.Progress integration
it's too fast to need it now
2024-08-07 00:48:32 -07:00
Andrew Kelley
1792258dc8 std.debug.Dwarf: precompute .debug_line table
yields a 60x speedup for resolveSourceLocations in debug builds
2024-08-07 00:48:32 -07:00
Andrew Kelley
66954e8330 std.debug.FixedBufferReader is fine
it does not need to be deprecated
2024-08-07 00:48:32 -07:00
Andrew Kelley
de47acd732 code coverage dumping tool basic implementation
* std.debug.Dwarf: add `sortCompileUnits` along with a field to track
  the state for the purpose of assertions and correct API usage.
  This makes batch lookups faster.
  - in the future, findCompileUnit should be enhanced to rely on sorted
    compile units as well.
* implement `std.debug.Dwarf.resolveSourceLocations` as well as
  `std.debug.Info.resolveSourceLocations`. It's still pretty slow, since
  it calls getLineNumberInfo for each array element, repeating a lot of
  work unnecessarily.
* integrate these APIs with `std.Progress` to understand what is taking
  so long.

The output I'm seeing from this tool shows a lot of missing source
locations. In particular, the main area of interest is missing for my
tokenizer fuzzing example.
2024-08-07 00:48:32 -07:00
Andrew Kelley
2e12b45d8b introduce tool for dumping coverage file
with debug info resolved.

begin efforts of providing `std.debug.Info`, a cross-platform
abstraction for loading debug information into an in-memory format that
supports queries such as "what is the source location of this virtual
memory address?"

Unlike `std.debug.SelfInfo`, this API does not assume the debug
information in question happens to match the host CPU architecture, OS,
or other target properties.
2024-08-07 00:48:32 -07:00
Andrew Kelley
107b272766 fuzzer: share zig to html rendering with autodocs 2024-08-07 00:48:32 -07:00
Andrew Kelley
e0ffac4e3c introduce a web interface for fuzzing
* new .zig-cache subdirectory: 'v'
  - stores coverage information with filename of hash of PCs that want
    coverage. This hash is a hex encoding of the 64-bit coverage ID.
* build runner
  * fixed bug in file system inputs when a compile step has an
    overridden zig_lib_dir field set.
  * set some std lib options optimized for the build runner
    - no side channel mitigations
    - no Transport Layer Security
    - no crypto fork safety
  * add a --port CLI arg for choosing the port the fuzzing web interface
    listens on. it defaults to choosing a random open port.
  * introduce a web server, and serve a basic single page application
    - shares wasm code with autodocs
    - assets are created live on request, for convenient development
      experience. main.wasm is properly cached if nothing changes.
    - sources.tar comes from file system inputs (introduced with the
      `--watch` feature)
  * receives coverage ID from test runner and sends it on a thread-safe
    queue to the WebServer.
* test runner
  - takes a zig cache directory argument now, for where to put coverage
    information.
  - sends coverage ID to parent process
* fuzzer
  - puts its logs (in debug mode) in .zig-cache/tmp/libfuzzer.log
  - computes coverage_id and makes it available with
    `fuzzer_coverage_id` exported function.
  - the memory-mapped coverage file is now namespaced by the coverage id
    in hex encoding, in `.zig-cache/v`
* tokenizer
  - add a fuzz test to check that several properties are upheld
2024-08-07 00:48:32 -07:00
Andrew Kelley
ffc050e055 fuzzer: log errors and move deduplicated runs to shared mem 2024-08-07 00:48:32 -07:00
Andrew Kelley
97643c1ecc fuzzer: track code coverage from all runs
When a unique run is encountered, track it in a bit set memory-mapped
into the fuzz directory so it can be observed by other processes, even
while the fuzzer is running.
2024-08-07 00:48:32 -07:00
Alex Rønne Petersen
746f20d21f
llvm: Use unreachable in targetTriple() for targets without LLVM support. 2024-08-07 09:39:42 +02:00
Alex Rønne Petersen
a9f68410d0
llvm: Clarify in initializeLLVMTarget() that there's no kalimba backend. 2024-08-07 09:39:42 +02:00
Alex Rønne Petersen
650785c718
target: Fix hasLlvmSupport() for dxil, spirv[32,64], and kalimba. 2024-08-07 09:39:39 +02:00
Kyle Schwarz
a60810b5a3 cmake: add ZIG_EXTRA_BUILD_ARGS option 2024-08-06 19:28:48 -07:00
Alex Rønne Petersen
e9b5377b8e musl: Disable warnings in all compilations.
This is what upstream's configure does.

Previously, we only disabled warnings in some musl compilations, with `rcrt1.o`
notably being one for which we didn't. This resulted in a warning in `dlstart.c`
which is included in `rcrt1.c`. So let's just be consistent and disable warnings
for all musl code.

Closes #13385.
2024-08-06 15:00:47 -07:00
Andrew Kelley
c8a226a429
Merge pull request #20960 from alexrp/size-align-fixes
`std.Target`: A bunch of C type size/alignment fixes
2024-08-06 14:55:15 -07:00
Andrew Kelley
de1fb19909
Merge pull request #20951 from ehaas/c-size-and-align-check
tools: Add tool for checking size and alignment of C types
2024-08-06 13:06:28 -07:00
Alex Rønne Petersen
ef502daafe
std.Target: Handle mesa3d in c_type_bit_size(). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
23b5a6c71e
std.Target: Treat spirv as identical to spirv64 for ABI size/alignment purposes.
This is arbitrary since spirv (as opposed to spirv32/spirv64) refers to the
version with logical memory layout, i.e. no 'real' pointers. This change at
least matches what clang does.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
231f322a65
std.Target: Fix C long double size for ve (16, not 8). 2024-08-06 20:17:03 +02:00