Commit Graph

82 Commits

Author SHA1 Message Date
PauloCampana
3e62cb5c90
Remove old deprecated symbols in std (#21584)
Also, actually run tests inside std/tar/writer.zig
2024-10-04 13:50:25 -07:00
Linus Groh
8588964972 Replace deprecated default initializations with decl literals 2024-09-12 16:01:23 +01:00
Igor Anić
72bcc3b7a5
std.tar: add writer (#19603)
Simplifies code in docs creation where we used `std.tar.output.Header`.
Writer uses that Header internally and provides higher level interface.
Updates checksum on write, handles long file names, allows setting mtime and file permission mode. Provides handy interface for passing `Dir.WalkerEntry`.
2024-08-15 22:05:53 -07:00
Jora Troosh
13070448f5
std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
Igor Anić
035c1b6522 std.tar: add strip components error to diagnostics
This was the only kind of error which was raised in pipeToFileSystem and
not added to Diagnostics.
Shell tar silently ignores paths which are stripped out when used with
`--strip-components` switch. This enables that same behavior, errors
will be collected in diagnostics but caller is free to ignore that type
of diagnostics errors.
Enables use case where caller knows structure of the tar file and want
to extract only some deeply nested folders ignoring upper files/folders.

Fixes: #17620 by giving caller options:
- not provide diagnostic and get errors
- provide diagnostics and analyze errors
- provide diagnostics and ignore errors
2024-07-03 12:12:50 -07:00
Ryan Liptak
76fb2b685b std: Convert deprecated aliases to compile errors and fix usages
Deprecated aliases that are now compile errors:

- `std.fs.MAX_PATH_BYTES` (renamed to `std.fs.max_path_bytes`)
- `std.mem.tokenize` (split into `tokenizeAny`, `tokenizeSequence`, `tokenizeScalar`)
- `std.mem.split` (split into `splitSequence`, `splitAny`, `splitScalar`)
- `std.mem.splitBackwards` (split into `splitBackwardsSequence`, `splitBackwardsAny`, `splitBackwardsScalar`)
- `std.unicode`
  + `utf16leToUtf8Alloc`, `utf16leToUtf8AllocZ`, `utf16leToUtf8`, `fmtUtf16le` (all renamed to have capitalized `Le`)
  + `utf8ToUtf16LeWithNull` (renamed to `utf8ToUtf16LeAllocZ`)
- `std.zig.CrossTarget` (moved to `std.Target.Query`)

Deprecated `lib/std/std.zig` decls were deleted instead of made a `@compileError` because the `refAllDecls` in the test block would trigger the `@compileError`. The deleted top-level `std` namespaces are:

- `std.rand` (renamed to `std.Random`)
- `std.TailQueue` (renamed to `std.DoublyLinkedList`)
- `std.ChildProcess` (renamed/moved to `std.process.Child`)

This is not exhaustive. Deprecated aliases that I didn't touch:
  + `std.io.*`
  + `std.Build.*`
  + `std.builtin.Mode`
  + `std.zig.c_translation.CIntLiteralRadix`
  + anything in `src/`
2024-06-13 10:18:59 -04:00
Igor Anić
d841f84b8b
std.tar: fix finding tar root_dir (#19829)
For issue #19820.
2024-05-01 12:03:05 -07:00
Igor Anić
3d5a9237f7 fetch: use empty string instead of null for root_dir
Make it consistent with Cache.Path sub_path.
Remove null check in many locations.
2024-04-09 15:00:22 +02:00
Igor Anić
b1e70edd90 tar: find package root dir in pipeToFileSystem
While iterating over all files in tarball set root_dir in diagnostic if
there is single root in tarball. Will be used in package manager with
strip_components = 0 to find the root of the fetched package.
2024-04-03 19:44:51 +02:00
Igor Anić
b88ae8dbd8 std.tar: implement executable bit only 2024-04-02 14:02:43 -07:00
Andrew Kelley
12191c8a22 std: promote tests to doctests
Now these show up as "example usage" in generated documentation.
2024-03-21 14:11:46 -07:00
Igor Anić
0cca7e732e std.tar: fix broken public interface 2024-03-11 12:25:51 +01:00
Igor Anić
c4868b2bbc std.tar: use doctest
Make std.tar look better in docs. Remove from public interface what is
not necessary. Add comment to the public methods. Add doctest as usage
examples for iterator and pipeToFileSystem.
2024-03-11 12:25:03 +01:00
Igor Anić
a9e7abda20 std.tar: fix test hanging on windows
Problem was manifested only on windows with target `-target
aarch64-windows-gnu`.

I was creating new files but not closing any of them. Tmp dir cleanup
hangs looping in deleteTree forever.
2024-03-11 12:24:11 +01:00
Igor Anić
8ec990d6d7 disable test which is hanging on windows in ci
When this test is enabled something like:
`zig build test docs --zig-lib-dir .\lib\ -Dstatic-llvm -Dskip-non-native -Denable-symlinks-windows`
never finishes.

Those are failed runs from ci:
https://github.com/ziglang/zig/actions/runs/8137710393
https://github.com/ziglang/zig/actions/runs/8129619923
https://github.com/ziglang/zig/actions/runs/8125845128

Isolating that test and running it is not a problem. Running something
like `zig test .\lib\std\std.zig  --zig-lib-dir .\lib\` is fine.
2024-03-11 12:24:11 +01:00
Igor Anić
67336ca8c6 std.tar: fix build on 32 bit platform
Fixing error from ci:

std/tar.zig:423:54: error: expected type 'usize', found 'u64'
std/tar.zig:423:54: note: unsigned 32-bit int cannot represent all possible unsigned 64-bit values
2024-03-11 12:24:11 +01:00
Igor Anić
f5fd4691e5 std.tar: document iterator interface with example 2024-03-11 12:24:06 +01:00
Igor Anić
614161a7cf std.tar make iterator interface more ergonomic
for the then end users:

1. Don't require user to call file.skip() on file returned from
iterator.next if file is not read. Iterator will now handle this.
Previously that returned header parsing error, without knowing some tar
internals it is hard to understand what is required from user.

2. Use iterator.File.kind enum which is similar to fs.File.Kind,
something familiar. Internal Header.Kind has many types which are not
exposed but the user needs to have else in kind switch to cover those
cases.

3. Add reader interface to the iterator.File.
2024-03-11 12:22:12 +01:00
Igor Anić
5ccbb196ad std.tar: don't return in iterator init
Don't assert min buffer size on iterator init. That was changing public
interface. This way we don't break that interface.
2024-03-11 12:22:12 +01:00
Igor Anić
10add7c677 std.tar: remove redundant test name prefixes 2024-03-11 12:22:12 +01:00
Igor Anić
04e8bbd932 std.tar: test buffers provided to the iterator
Tar header stores name in max 256 bytes and link name in max 100 bytes.
Those are minimums for provided buffers. Error is raised during iterator
init if buffers are not long enough.

Pax and gnu extensions can store longer names. If such extension is
reached during unpack and don't fit into provided buffer error is
returned.
2024-03-11 12:22:12 +01:00
Igor Anić
af0502f6c4 std.tar: add tests for file and symlink create
Should do that before I broke package manager!
2024-03-11 12:22:00 +01:00
Andrew Kelley
ffd53a459e -femit-docs: creating sources.tar
It's always a good day when you get to use File.writeFileAll 😎
2024-03-10 17:51:07 -07:00
Andrew Kelley
af06584241
Merge pull request #19126 from ianic/tar_case_sensitive
std.tar: fix unconditional error return
2024-02-29 10:12:39 -08:00
Andrew Kelley
beca85e644 std.tar.iterator: make the buffers configurable
Provides more API flexibility and correctness, while still preserving
the handy high level `pipeToFileSystem` API.
2024-02-28 20:33:29 -07:00
Andrew Kelley
b3ad45f267 std.tar: avoid dependency on file system
In the iterator function which is the low-level API, don't depend on
`std.fs.MAX_PATH_BYTES` because this is not defined on all operating
systems, such as freestanding.

However in such environments it still makes sense to be able to extract
from a tar file.

An even more flexible solution would be to accept the buffers as
arguments to iterator() which I think is a good idea, but for now let's
just set the same upper limmit across all operating systems.
2024-02-28 20:14:21 -07:00
Igor Anić
7d536e8970 std.tar: fix unconditional error return 2024-02-29 01:22:49 +01:00
Andrew Kelley
81aa74e7e1
Merge pull request #19081 from ianic/tar_case_sensitive
std.tar don't overwrite files on unpack
2024-02-26 12:23:22 -08:00
Igor Anić
65e5c46d61 std.tar fix refactored function
It was not returning error in all cases. Bug in refactoring.
2024-02-26 00:24:23 +01:00
Igor Anić
f086ea856c std.tar skip test on windows
Or other platform which don't support symlinks.
2024-02-25 15:57:20 +01:00
Igor Anić
30a319be6d std.tar improve error reporting
Report file name which failed to create in all cases.
2024-02-25 12:03:23 +01:00
Igor Anić
b84301c8e5 std.tar don't overwrite existing file
Fail with error if file already exists. File is not silently overwritten
but an error is raised.

Fixes: #18089
2024-02-24 23:37:55 +01:00
Igor Anić
8d651f512b std.tar fix assert exploited by fuzzing 2024-02-24 13:17:04 -08:00
Igor Anić
0a86b117bf std.tar fix integer overflow in header size parse
Found by fuzzing. Fixing code and adding test.
2024-02-23 21:57:40 +01:00
Igor Anić
f67aa8b9b3 std.tar fix parsing mode field in tar header
Found by fuzzing. Previous numeric function assumed that is is getting
buffer of size 12, mode is size 8. Fuzzing found overflow.
Fixing and adding test cases.
2024-02-23 21:57:15 +01:00
Andrew Kelley
256c5934bf std.tar: remove abuse of inline fn
In general, any `inline fn` should document why it is using `inline`
because the rule of thumb is: don't use inline.
2024-02-23 01:16:44 -08:00
Igor Anić
30f15e3afe fix crash in tar found by fuzzing
Running fuzzing tar test with [zig std lib
fuzzing](https://github.com/squeek502/zig-std-lib-fuzzing) reached and
assert in tar implementation. Assert (in std lib) should not be
reachable by external input, so I'm fixing this to return error.
2024-02-22 18:20:05 -08:00
cipharius
f25c499d12 Permits tar directory path without trailing slash 2024-01-15 18:31:32 +02:00
Igor Anić
7d3a31872e tar: improve diagnostic reporting
Using Python testtar file (mentioned in #14310) to test diagnostic
reporting.
Added computing checksum by using both unsigned and signed header bytes
values.
Added skipping gnu exteneded sparse headers while reporting unsupported
header in diagnostic.

Note on testing:

wget https://github.com/python/cpython/raw/3.11/Lib/test/testtar.tar -O
/tmp/testtar.tar

```
test "Python testtar.tar file" {
    const file_name = "testtar.tar";

    var file = try std.fs.cwd().openFile("/tmp/" ++ file_name, .{});
    defer file.close();

    var diag = Options.Diagnostics{ .allocator = std.testing.allocator };
    defer diag.deinit();

    var iter = iterator(file.reader(), &diag);
    while (try iter.next()) |f| {
        std.debug.print("supported: {} {s} {d}\n", .{ f.kind, f.name, f.size });
        try f.skip();
    }
    for (diag.errors.items) |e| {
        switch (e) {
            .unsupported_file_type => |u| {
                std.debug.print("unsupported: {} {s}\n", .{ u.file_type, u.file_name });
            },
            else => unreachable,
        }
    }
}
```
2024-01-13 19:37:33 -07:00
Igor Anić
7923a53996 tar: rename reader to iterator
Itarator has `next` function, iterates over tar files. When using from
outside of module with `tar.` prefix makes more sense.

var iter = tar.iterator(reader, null);
while (try iter.next()) |file| {
...
}
2024-01-13 19:37:33 -07:00
Igor Anić
76fe1f53d5 tar: fix tests on 32-bit platforms 2024-01-13 19:37:33 -07:00
Igor Anić
a75fd4ff15 tar: move test cases to std/tar/testdata
Create std/tar/test.zig for test which uses cases from testdata.
2024-01-13 19:37:33 -07:00
Igor Anić
f8e42d6b30 tar: add Go test case files to the project 2024-01-13 19:37:33 -07:00
Igor Anić
c07527abac tar: reorganize file, functions before tests 2024-01-13 19:37:33 -07:00
Igor Anić
c76abe0e18 tar: use file word in less places 2024-01-13 19:37:33 -07:00
Igor Anić
4a6d67ab1a tar: remove stratch from tar reader
Use explicit buffers for name, link_name instead.
It is cleaner that way.
2024-01-13 19:37:33 -07:00
Igor Anić
9f7dd32308 tar: refactor pax attribute
Make it little readable.
2024-01-13 19:37:33 -07:00
Igor Anić
dbab45cfc6 tar: replace custom buffered reader with std.io 2024-01-13 19:37:33 -07:00
Igor Anić
58e0e509c6 tar: add module comment and references 2024-01-13 19:37:33 -07:00
Igor Anić
a3cf8ec71e tar: add pax file reader tests 2024-01-13 19:37:33 -07:00