Commit Graph

296 Commits

Author SHA1 Message Date
Andrew Kelley
5f5a7b53a4 wasm zig source rendering: fix annotation location off-by-one 2024-08-07 00:48:32 -07:00
Andrew Kelley
3d48602c99 fuzzer web UI: annotated PCs in source view 2024-08-07 00:48:32 -07:00
Andrew Kelley
107b272766 fuzzer: share zig to html rendering with autodocs 2024-08-07 00:48:32 -07:00
Andrew Kelley
e0ffac4e3c introduce a web interface for fuzzing
* new .zig-cache subdirectory: 'v'
  - stores coverage information with filename of hash of PCs that want
    coverage. This hash is a hex encoding of the 64-bit coverage ID.
* build runner
  * fixed bug in file system inputs when a compile step has an
    overridden zig_lib_dir field set.
  * set some std lib options optimized for the build runner
    - no side channel mitigations
    - no Transport Layer Security
    - no crypto fork safety
  * add a --port CLI arg for choosing the port the fuzzing web interface
    listens on. it defaults to choosing a random open port.
  * introduce a web server, and serve a basic single page application
    - shares wasm code with autodocs
    - assets are created live on request, for convenient development
      experience. main.wasm is properly cached if nothing changes.
    - sources.tar comes from file system inputs (introduced with the
      `--watch` feature)
  * receives coverage ID from test runner and sends it on a thread-safe
    queue to the WebServer.
* test runner
  - takes a zig cache directory argument now, for where to put coverage
    information.
  - sends coverage ID to parent process
* fuzzer
  - puts its logs (in debug mode) in .zig-cache/tmp/libfuzzer.log
  - computes coverage_id and makes it available with
    `fuzzer_coverage_id` exported function.
  - the memory-mapped coverage file is now namespaced by the coverage id
    in hex encoding, in `.zig-cache/v`
* tokenizer
  - add a fuzz test to check that several properties are upheld
2024-08-07 00:48:32 -07:00
Ryan Liptak
ac2459327f autodoc: Preserve whitespace in inline code spans
Fixes #20754
2024-07-23 10:02:54 -07:00
Ian Johnson
2511830442 Autodoc: only group structs under "namespaces"
The old heuristic of checking only for the number of fields has the
downside of classifying all opaque types, such as `std.c.FILE`, as
"namespaces" rather than "types".
2024-07-09 15:58:03 -04:00
Ryan Liptak
76fb2b685b std: Convert deprecated aliases to compile errors and fix usages
Deprecated aliases that are now compile errors:

- `std.fs.MAX_PATH_BYTES` (renamed to `std.fs.max_path_bytes`)
- `std.mem.tokenize` (split into `tokenizeAny`, `tokenizeSequence`, `tokenizeScalar`)
- `std.mem.split` (split into `splitSequence`, `splitAny`, `splitScalar`)
- `std.mem.splitBackwards` (split into `splitBackwardsSequence`, `splitBackwardsAny`, `splitBackwardsScalar`)
- `std.unicode`
  + `utf16leToUtf8Alloc`, `utf16leToUtf8AllocZ`, `utf16leToUtf8`, `fmtUtf16le` (all renamed to have capitalized `Le`)
  + `utf8ToUtf16LeWithNull` (renamed to `utf8ToUtf16LeAllocZ`)
- `std.zig.CrossTarget` (moved to `std.Target.Query`)

Deprecated `lib/std/std.zig` decls were deleted instead of made a `@compileError` because the `refAllDecls` in the test block would trigger the `@compileError`. The deleted top-level `std` namespaces are:

- `std.rand` (renamed to `std.Random`)
- `std.TailQueue` (renamed to `std.DoublyLinkedList`)
- `std.ChildProcess` (renamed/moved to `std.process.Child`)

This is not exhaustive. Deprecated aliases that I didn't touch:
  + `std.io.*`
  + `std.Build.*`
  + `std.builtin.Mode`
  + `std.zig.c_translation.CIntLiteralRadix`
  + anything in `src/`
2024-06-13 10:18:59 -04:00
expikr
7cf6650663
autodoc: fix misaligned table header when alignment is default (#20220) 2024-06-08 12:37:07 -07:00
mlugg
d0e74ffe52
compiler: rework comptime pointer representation and access
We've got a big one here! This commit reworks how we represent pointers
in the InternPool, and rewrites the logic for loading and storing from
them at comptime.

Firstly, the pointer representation. Previously, pointers were
represented in a highly structured manner: pointers to fields, array
elements, etc, were explicitly represented. This works well for simple
cases, but is quite difficult to handle in the cases of unusual
reinterpretations, pointer casts, offsets, etc. Therefore, pointers are
now represented in a more "flat" manner. For types without well-defined
layouts -- such as comptime-only types, automatic-layout aggregates, and
so on -- we still use this "hierarchical" structure. However, for types
with well-defined layouts, we use a byte offset associated with the
pointer. This allows the comptime pointer access logic to deal with
reinterpreted pointers far more gracefully, because the "base address"
of a pointer -- for instance a `field` -- is a single value which
pointer accesses cannot exceed since the parent has undefined layout.
This strategy is also more useful to most backends -- see the updated
logic in `codegen.zig` and `codegen/llvm.zig`. For backends which do
prefer a chain of field and elements accesses for lowering pointer
values, such as SPIR-V, there is a helpful function in `Value` which
creates a strategy to derive a pointer value using ideally only field
and element accesses. This is actually more correct than the previous
logic, since it correctly handles pointer casts which, after the dust
has settled, end up referring exactly to an aggregate field or array
element.

In terms of the pointer access code, it has been rewritten from the
ground up. The old logic had become rather a mess of special cases being
added whenever bugs were hit, and was still riddled with bugs. The new
logic was written to handle the "difficult" cases correctly, the most
notable of which is restructuring of a comptime-only array (for
instance, converting a `[3][2]comptime_int` to a `[2][3]comptime_int`.
Currently, the logic for loading and storing work somewhat differently,
but a future change will likely improve the loading logic to bring it
more in line with the store strategy. As far as I can tell, the rewrite
has fixed all bugs exposed by #19414.

As a part of this, the comptime bitcast logic has also been rewritten.
Previously, bitcasts simply worked by serializing the entire value into
an in-memory buffer, then deserializing it. This strategy has two key
weaknesses: pointers, and undefined values. Representations of these
values at comptime cannot be easily serialized/deserialized whilst
preserving data, which means many bitcasts would become runtime-known if
pointers were involved, or would turn `undefined` values into `0xAA`.
The new logic works by "flattening" the datastructure to be cast into a
sequence of bit-packed atomic values, and then "unflattening" it; using
serialization when necessary, but with special handling for `undefined`
values and for pointers which align in virtual memory. The resulting
code is definitely slower -- more on this later -- but it is correct.

The pointer access and bitcast logic required some helper functions and
types which are not generally useful elsewhere, so I opted to split them
into separate files `Sema/comptime_ptr_access.zig` and
`Sema/bitcast.zig`, with simple re-exports in `Sema.zig` for their small
public APIs.

Whilst working on this branch, I caught various unrelated bugs with
transitive Sema errors, and with the handling of `undefined` values.
These bugs have been fixed, and corresponding behavior test added.

In terms of performance, I do anticipate that this commit will regress
performance somewhat, because the new pointer access and bitcast logic
is necessarily more complex. I have not yet taken performance
measurements, but will do shortly, and post the results in this PR. If
the performance regression is severe, I will do work to to optimize the
new logic before merge.

Resolves: #19452
Resolves: #19460
2024-04-17 13:41:25 +01:00
andrewkraevskii
f7a76bdfe3 autodoc: fix tokenizer 2024-04-10 20:44:10 -07:00
Ian Johnson
6dcbad780c
Autodoc: fix Markdown indented lists (#19577)
Previously, indentation was not being handled correctly in some cases,
causing examples such as `std.json.WriteStream` to be rendered with
improper list nesting.

Additionally, some more test cases have been added to ensure
indentation (or lack of indentation) is handled correctly in some other
constructs.
2024-04-08 09:49:22 +00:00
Jacob Young
eb723a4070 Update uses of @fieldParentPtr to use RLS 2024-03-30 20:50:48 -04:00
Andrew Kelley
abadad4640
Merge pull request #19402 from ianprime0509/markdown-autolinks
Autodoc: hyperlink URLs in text
2024-03-25 15:36:37 -07:00
HydroH
ef31d8f48f
autodoc: better indentation handling when rendering source code (#19422)
Closes #19293
2024-03-25 18:20:15 -04:00
Ian Johnson
ad34ed5a63 Autodoc: recognize Markdown links in plain text
This extension to the typical `<>` Markdown autolink syntax allows
HTTP(S) links to be recognized in normal text without being delimited by
`<>`. This is the most natural way to write links in text, so it makes
sense to support it and allow documentation comments to be written in a
more natural way.
2024-03-22 20:53:25 -04:00
Ian Johnson
d3ca9d55d9 Autodoc: implement Markdown autolinks
Closes #19265

This commit implements support for Markdown autolinks delimited by angle
brackets. The precise syntax accepted is documented in the doc comment
of `markdown.zig`.
2024-03-22 20:03:32 -04:00
Jacob Young
c11b6adf13 Ast: fix comptime destructure
A preceding `comptime` keyword was being ignored if the first
destructure variable was an expression.
2024-03-17 15:23:16 -07:00
Carl Åstholm
2008b14bc7 autodoc: Use code for keyboard events 2024-03-13 18:37:00 -07:00
Igor Anić
c974e19816 docs: make docs work with recent tar changes
It is no longer need to call skip if file content is not consumed. It is
handled internally now. File types are now same as in os.File.
2024-03-11 13:30:26 +01:00
Andrew Kelley
b13a55db97 update autodocs web application to latest
upstream commit 1f921d540e1a8bb40839be30239019c820eb663d

after this branch is merged, ziglang/zig becomes the new repository for
this code.
2024-03-10 18:13:25 -07:00
Andrew Kelley
0b1b3f0225 upstream new autodocs implementation 2024-03-10 17:51:06 -07:00
Andrew Kelley
94daf87335 no 2024-03-10 17:50:54 -07:00
Michael Scott
e5dc9b1d09 Update styles in std docs to correct display glitch 2024-01-12 16:25:55 -08:00
Krzysztof Wolicki
1880c799ae
autodoc: Some support for field_call (#16853)
* autodoc: Some support for field_call

* autodoc: Change handling of field_call to respect tryResolveRefPath, add fieldVal to Expr

* autodoc: Fixed errors

* autodoc: sync with latest master changes

---------

Co-authored-by: Loris Cro <kappaloris@gmail.com>
2023-10-30 21:04:49 +00:00
Krzysztof Wolicki
606f0e496e
autodoc: Display type kinds for containers (#17667) 2023-10-22 16:15:00 +02:00
Krzysztof Wolicki
4d1e8ef1eb
autodoc: Add hidden class to [src] link when starting renderApi to only show it for functions (#17322) 2023-10-04 18:42:49 +02:00
Krzysztof Wolicki
9c3165b81b
autodoc: Add handling for array_cat, array_mul, struct_init_empty_result. (#17367)
Fix issue with getting docs for src-less types in main.js
2023-10-04 18:25:00 +02:00
Jay Petacat
731fd217db Add embedded SVG favicon to reference doc templates
The SVG looks way better than the pixelated PNG and will adapt best to
whatever screen it is being displayed on. The PNG continues to be used
because Apple Safari does not support SVG favicons yet. All other major
browsers do. See https://caniuse.com/link-icon-svg.

This is a companion PR to ziglang/www.ziglang.org#310.
2023-09-25 12:24:06 +03:00
Loris Cro
78ebf8f577 autodoc: give explicit width to logo
fix #17251
2023-09-24 01:17:06 +02:00
Loris Cro
c481510c99 autodoc: show more doc comments for namespaces and types
previously, in the container view (the type of view that you see when
you look at `std` for example), when listing types and namespaces, we
would only show doc comments places on the direct child decl, which in
the case of the `std` namespace, for example, it's just a bunch of
re-exports.

now, if we don't find a direct doc comment, we chase indirection and
display doc comments placed directly on the definition, if any.

this is the precise priority order:

```
/// 1
pub const Foo = _Foo;

/// 2
const _Foo = struct {
   //! 3
};
```
The numbers show the priority order for autodoc.
2023-09-20 19:17:00 +02:00
Loris Cro
c1e94b28a3 autodoc: split json payload per field
this will make s3 re-enable compression for the stdlib's autodoc
and improve loading times (and data usage) for users

alongside this commit the deploy script for the official website is also
being updated
2023-09-17 18:23:21 +02:00
Loris Cro
33f3a4d840 autodoc: rename the doctest section in the frontend to 'Usage Examples' 2023-09-17 18:23:21 +02:00
Krzysztof Wolicki
2651363c9d
autodoc: Implement various missing expressions in ex (#17129)
Co-authored-by: Loris Cro <kappaloris@gmail.com>
2023-09-16 17:38:06 +02:00
Krzysztof Wolicki
f2026e7dd6 autodoc: Implement builtin function rendering.
Implement unary ops handling.
Fix getType in main.js
Minor cleanup of builtin function handling.
2023-09-16 17:35:11 +02:00
Krzysztof Wolicki
9a326b22d5 autodoc: Fix styling of the navbar 2023-09-16 17:31:43 +02:00
Krzysztof Wolicki
da28379d6c autodoc: Remove unnecessary Expr tag 2023-09-16 17:30:49 +02:00
Krzysztof Wolicki
8a9aa9e112 autodoc: Handle ref ZIR instruction 2023-09-16 17:30:49 +02:00
Ian Johnson
d2014fe971 Autodoc: simplify search text on mobile
This prevents the placeholder text from spilling out of the search bar
on smaller screens.
2023-09-09 19:48:18 +02:00
Ian Johnson
51d7700c8c Autodoc: tweak page layout
Closes #17011
Closes #17012

This commit allows the logo to scale more freely to fit its container,
and removes some extra margins so that the content scroll bar is flush
with the right side of the viewport.
2023-09-09 19:48:18 +02:00
Ian Johnson
2f26b15995 Autodoc: fix search results navigation
Closes #17013
2023-09-09 19:47:57 +02:00
Loris Cro
fda087ed81 autodoc: style links in source code 2023-09-04 18:56:45 +02:00
Krzysztof Wolicki
5cc1831ca4
autodoc: Fix rendering of enum types (#17058) 2023-09-03 18:34:29 +02:00
Loris Cro
e5c72a32a7 autodoc: fix rendering of std.json.ObjectMap
closes #17014
supersedes #17022
follow up issue #17061
2023-09-03 18:32:26 +02:00
Krzysztof Wolicki
555028086a
autodoc: Implement @call, @unionInit, @mulAdd support (#16918)
* autodoc: Implement `@call`, `@unionInit`, `@mulAdd` support

* autodoc: Implement builtinIndex in ex
2023-09-03 17:20:23 +02:00
Krzysztof Wolicki
86a9e1deaf
autodoc: Implement @bitSizeOf rendering in main.js (#16895) 2023-09-03 17:18:51 +02:00
Krzysztof Wolicki
80c1d48ccb
autodoc: Implement a[b], a.?, and a.* (#16863)
* autodoc: Implement elem_val_node and basic optional_payload_*

* autodoc: Add `.*` loads
2023-09-03 17:18:16 +02:00
Loris Cro
0516a4d629 autodoc: fix typo 2023-09-03 17:15:15 +02:00
Loris Cro
c0da41582b autodoc: better light mode colors
fix #15799
2023-09-03 17:12:52 +02:00
James Chen-Smith
c3a8f1fe92
autodoc: Extract decl ref style and fix light mode color (#15990)
Co-authored-by: James Chen-Smith <james@chen-smith.net>
Co-authored-by: Loris Cro <kappaloris@gmail.com>
2023-09-03 16:51:07 +02:00
Loris Cro
373e48c983
autodoc: new layout (#16715)
* autodoc: init guide TOC work

* autodoc: working guides toc navigation

* autodoc: more improvements

* autodoc: ui refinements

* autodoc: new layout and init descriptions for namespaces in std.zig
2023-08-06 18:12:05 +02:00