Commit Graph

164 Commits

Author SHA1 Message Date
mlugg
5d7fa5513f std.Build: allow packages to expose arbitrary LazyPaths by name 2024-09-15 17:24:44 -07:00
Linus Groh
8588964972 Replace deprecated default initializations with decl literals 2024-09-12 16:01:23 +01:00
mlugg
0fe3fd01dd
std: update std.builtin.Type fields to follow naming conventions
The compiler actually doesn't need any functional changes for this: Sema
does reification based on the tag indices of `std.builtin.Type` already!
So, no zig1.wasm update is necessary.

This change is necessary to disallow name clashes between fields and
decls on a type, which is a prerequisite of #9938.
2024-08-28 08:39:59 +01:00
Andrew Kelley
d9e8671d96 fix merge conflict from previous commit 2024-08-23 21:46:36 -07:00
Eric Joldasov
03c297eea6 std.Build: call handleVerbose2 in runAllowFail
Ensures that all runned command are visible when using `--verbose` flag,
for example `pkg-config` from Step.Compile or `git describe` from build.zig.

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2024-08-23 21:45:32 -07:00
Robin Voetter
43f73af359
fix various issues related to Path handling in the compiler and std
A compilation build step for which the binary is not required could not
be compiled previously. There were 2 issues that caused this:

- The compiler communicated only the results of the emitted binary and
  did not properly communicate the result if the binary was not emitted.

  This is fixed by communicating the final hash of the artifact path (the
  hash of the corresponding /o/<hash> directory) and communicating this
  instead of the entire path. This changes the zig build --listen protocol
  to communicate hashes instead of paths, and emit_bin_path is accordingly
  renamed to emit_digest.

- There was an error related to the default llvm object path when
  CacheUse.Whole was selected. I'm not really sure why this didn't manifest
  when the binary is also emitted.

  This was fixed by improving the path handling related to flush() and
  emitLlvmObject().

In general, this commit also improves some of the path handling throughout
the compiler and standard library.
2024-08-19 19:09:11 +02:00
Andrew Kelley
e0ffac4e3c introduce a web interface for fuzzing
* new .zig-cache subdirectory: 'v'
  - stores coverage information with filename of hash of PCs that want
    coverage. This hash is a hex encoding of the 64-bit coverage ID.
* build runner
  * fixed bug in file system inputs when a compile step has an
    overridden zig_lib_dir field set.
  * set some std lib options optimized for the build runner
    - no side channel mitigations
    - no Transport Layer Security
    - no crypto fork safety
  * add a --port CLI arg for choosing the port the fuzzing web interface
    listens on. it defaults to choosing a random open port.
  * introduce a web server, and serve a basic single page application
    - shares wasm code with autodocs
    - assets are created live on request, for convenient development
      experience. main.wasm is properly cached if nothing changes.
    - sources.tar comes from file system inputs (introduced with the
      `--watch` feature)
  * receives coverage ID from test runner and sends it on a thread-safe
    queue to the WebServer.
* test runner
  - takes a zig cache directory argument now, for where to put coverage
    information.
  - sends coverage ID to parent process
* fuzzer
  - puts its logs (in debug mode) in .zig-cache/tmp/libfuzzer.log
  - computes coverage_id and makes it available with
    `fuzzer_coverage_id` exported function.
  - the memory-mapped coverage file is now namespaced by the coverage id
    in hex encoding, in `.zig-cache/v`
* tokenizer
  - add a fuzz test to check that several properties are upheld
2024-08-07 00:48:32 -07:00
Sammy James
00d6ea4764 Add libcpp option to test options 2024-07-28 20:40:53 -07:00
Andrew Kelley
a3c74aca99 add --debug-rt CLI arg to the compiler + bonus edits
The flag makes compiler_rt and libfuzzer be in debug mode.

Also:
* fuzzer: override debug logs and disable debug logs for frequently
  called functions
* std.Build.Fuzz: fix bug of rerunning the old unit test binary
* report errors from rebuilding the unit tests better
* link.Elf: additionally add tsan lib and fuzzer lib to the hash
2024-07-25 18:52:21 -07:00
Andrew Kelley
711ed56ce3 build runner: extract logic to std.Build.Fuzz 2024-07-25 18:52:21 -07:00
Andrew Kelley
047640383e add --fuzz CLI argument to zig build
This flag makes the build runner rebuild unit tests after the pipeline
finishes, if it finds any unit tests.

I did not make this integrate with file system watching yet.

The test runner is updated to detect which tests are fuzz tests.

Run step is updated to track which test indexes are fuzz tests.
2024-07-25 18:52:20 -07:00
Andrew Kelley
33c7984183 add std.testing.random_seed
closes #17609
2024-07-23 11:43:12 -07:00
L zard
7aea384e44 std.Build: add support for passing comptime_int, float and comptime_float options 2024-07-21 01:56:51 -07:00
Andrew Kelley
eb4028bf30 add std.fmt.hex
converts an unsigned integer into an array
2024-07-20 01:06:29 -07:00
Andrew Kelley
445bd7a06f build runner: update watch caption to include subprocesses 2024-07-14 22:27:51 -07:00
Andrew Kelley
716b128a24 frontend: add -fincremental, -fno-incremental flag
Remove --debug-incremental

This flag is also added to the build system. Importantly, this tells
Compile step whether or not to keep the compiler running between
rebuilds. It defaults off because it is currently crashing
zirUpdateRefs.
2024-07-14 21:18:09 -07:00
Andrew Kelley
abf8955951 make zig compiler processes live across rebuilds
Changes the `make` function signature to take an options struct, which
additionally includes `watch: bool`. I intentionally am not exposing
this information to configure phase logic.

Also adds global zig cache to the compiler cache prefixes.

Closes #20600
2024-07-14 19:51:16 -07:00
Andrew Kelley
a3c20dffae integrate Compile steps with file watching
Updates the build runner to unconditionally require a zig lib directory
parameter. This parameter is needed in order to correctly understand
file system inputs from zig compiler subprocesses, since they will refer
to "the zig lib directory", and the build runner needs to place file
system watches on directories in there.

The build runner's fanotify file watching implementation now accounts
for when two or more Cache.Path instances compare unequal but ultimately
refer to the same directory in the file system.

Breaking change: std.Build no longer has a zig_lib_dir field. Instead,
there is the Graph zig_lib_directory field, and individual Compile steps
can still have their zig lib directories overridden. I think this is
unlikely to break anyone's build in practice.

The compiler now sends a "file_system_inputs" message to the build
runner which shares the full set of files that were added to the cache
system with the build system, so that the build runner can watch
properly and redo the Compile step. This is implemented for whole cache
mode but not yet for incremental cache mode.
2024-07-12 00:14:08 -07:00
Andrew Kelley
26bdc836d2 std.Build.LazyPath: add getPath3; deprecate getPath2 and getPath
The goal is to move towards using `std.Build.Cache.Path` instead of
absolute path names.

This was helpful for implementing file watching integration to
the InstallDir Step
2024-07-12 00:14:08 -07:00
Andrew Kelley
d1c14f2f52 std.Build.Step.WriteFile: extract UpdateSourceFiles
This has been planned for quite some time; this commit finally does it.

Also implements file system watching integration in the make()
implementation for UpdateSourceFiles and fixes the reporting of step
caching for both.

WriteFile does not yet have file system watching integration.
2024-07-12 00:14:08 -07:00
Andrew Kelley
0cc492a272 make more build steps integrate with the watch system 2024-07-12 00:14:08 -07:00
Andrew Kelley
5ee3971b18 proof-of-concept --watch implementation based on fanotify
So far, only implemented for InstallFile steps.

Default debounce interval bumped to 50ms. I think it should be
configurable.

Next I have an idea to simplify the fanotify implementation, but other
OS implementations might want to refer back to this commit before I make
those changes.
2024-07-12 00:14:07 -07:00
Andrew Kelley
bbd90a562e build runner: implement --watch (work-in-progress)
I'm still learning how the fanotify API works but I think after playing
with it in this commit, I finally know how to implement it, at least on
Linux. This commit does not accomplish the goal but I want to take the
code in a different direction and still be able to reference this point
in time by viewing a source control diff.

I think the move is going to be saving the file_handle for the parent
directory, which combined with the dirent names is how we can correlate
the events back to the Step instances that have registered file system
inputs. I predict this to be similar to implementations on other
operating systems.
2024-07-12 00:14:07 -07:00
Andrew Kelley
26d506c0f8 std.Build: remove the "push installed file" mechanism
Tracked by #14943
2024-07-12 00:14:07 -07:00
Andrew Kelley
6e025fc2e2 build system: add --watch flag and report source file in InstallFile
This direction is not quite right because it mutates shared state in a
threaded context, so the next commit will need to fix this.
2024-07-12 00:14:07 -07:00
Jora Troosh
13070448f5
std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
Alex Rønne Petersen
9be9b8ca90 std.Build.findProgram(): Try with and without the Windows executable extensions.
I renamed std.process.Child.CreateProcessSupportedExtension to WindowsExtension
and made it public to avoid duplicating the list of extensions.

While here, I also improved it to not misreport OOM from std.fs.realpathAlloc()
as a generic failure to find the program, but instead panic like the rest of the
build system does for OOM.

Closes #20314.
2024-06-21 15:41:03 -04:00
Alex Kladov
5afd774db3 std: mark Build.args as const
I don't think these are intended to be mutated!
2024-06-20 15:17:31 -04:00
Alex Rønne Petersen
6220c754cf std.Build: Add Step.Fail and addFail() function.
Closes #15373.
2024-06-16 23:13:12 -04:00
Robin Voetter
44443b833b
build: inherit setExecCmd from test compile steps when creating run steps
This should fix #17756
2024-06-09 01:55:50 +02:00
Krzysztof Wolicki
6327a3994a
std.Build: remove deprecated b.host in favor of b.graph.host (#20237) 2024-06-08 12:22:38 -07:00
Maciej 'vesim' Kuliński
a4e01074b5 Add support for []enum in Build.option 2024-06-06 12:27:29 -04:00
Andrew Kelley
f97c2f28fd update the codebase for the new std.Progress API 2024-05-27 20:56:48 -07:00
Andrew Kelley
f47824f24d std: restructure child process namespace 2024-05-26 09:31:55 -07:00
Jacob Young
dee9f82f69 Run: add output directory arguments
This allows running commands that take an output directory argument. The
main thing that was needed for this feature was generated file subpaths,
to allow access to the files in a generated directory. Additionally, a
minor change was required to so that the correct directory is created
for output directory args.
2024-05-05 15:58:08 -04:00
Jacob Young
e3424332d3 Build: cleanup
* `doc/langref` formatting
 * upgrade `.{ .path = "..." }` to `b.path("...")`
 * avoid using arguments named `self`
 * make `Build.Step.Id` usage more consistent
 * add `Build.pathResolve`
 * use `pathJoin` and `pathResolve` everywhere
 * make sure `Build.LazyPath.getPath2` returns an absolute path
2024-05-05 09:42:51 -04:00
Andrew Kelley
3c221463fd
Merge pull request #19689 from nektro/nektro-patch-29794
std: fix and add test for Build.dependencyFromBuildZig
2024-04-18 16:38:11 -07:00
Meghan Denny
0820aa4a46 std: fix and add test for Build.dependencyFromBuildZig 2024-04-18 03:07:44 -07:00
Andrew Kelley
22a97cd235 std.Build: revert --host-target, --host-cpu, --host-dynamic-linker
This is a partial revert of 105db13536.

As we learned from Void Linux packaging, these options are not actually
helpful since the distribution package manager may very well want to
cross-compile the packages that it is building.

So, let's not overcomplicate things. There are already the standard
options: -Dtarget, -Dcpu, and -Ddynamic-linker.

These options are generally provided when the project generates machine
code artifacts, however, there may be a project that does no such thing,
in which case it makes sense for these options to be missing. The Zig
Build System is a general-purpose build system, after all.
2024-04-18 03:02:13 -07:00
Andrew Kelley
499a202a13 std.Build: revert the breaking changes only
The previous commit deleted the deprecated API and then made all the
follow-up changes; this commit reverts only the breaking API changes.
This commit can be reverted once 0.12.0 is tagged.
2024-04-11 14:09:58 -07:00
Andrew Kelley
b30ad74908 remove deprecated LazyPath.path union tag 2024-04-11 14:02:47 -07:00
Andrew Kelley
7fb5a0b18b introduce std.Build.path; deprecate LazyPath.relative
This adds the *std.Build owner to LazyPath so that lazy paths returned
from a dependency can be used in the application without friction or
footguns.

closes #19313
2024-04-10 15:02:20 -07:00
Andrew Kelley
1b27146792
Merge pull request #18590 from castholm/move-standalone-test-cases
std.Build: Remove `anonymousDependency`
2024-04-07 16:08:23 -07:00
Carl Åstholm
e5bf1a5d50 Make std.Build.dependencyInner private 2024-04-07 16:06:20 -07:00
Carl Åstholm
ef90d3af54 Remove std.Build.anonymousDependency 2024-04-07 16:06:19 -07:00
Carl Åstholm
09f1bbe11b std.Build: Remove unused c_std field
This field has not been referenced by compile steps since
e76ce2c1d0, all the way back in 2019.
To specify the language standard, pass `-std=[value]` as a regular
C flag instead.
2024-04-07 16:06:15 -07:00
Carl Åstholm
05126fc4c5 std.Build: add dependencyFromBuildZig 2024-04-07 15:54:30 -07:00
Andrew Kelley
e204a6edb8
Merge pull request #18988 from castholm/lazy-build-zig
std.Build: add `lazyImport` (`@import` for lazy dependencies)
2024-04-07 15:43:54 -07:00
Carl Åstholm
d99e44a157 Document added/updated functions
Also renames `addHeaders` to `addHeadersDirectory` for clarity.
2024-04-07 15:34:47 +02:00
Carl Åstholm
ce71eb31da Update addInstallHeaderFile to take a LazyPath 2024-04-07 15:34:46 +02:00