Nixyri/zig
1
0
Fork 0
mirror of https://github.com/ziglang/zig.git synced 2024-11-16 09:03:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
6c0719fd5f
zig/lib/std/heap.zig

1244 lines
44 KiB
Zig
Raw Normal View History

rename std lib files to new convention
2019-03-02 21:46:04 +00:00
const std = @import("std.zig");
migrate from `std.Target.current` to `@import("builtin").target` closes #9388 closes #9321
2021-10-05 06:47:27 +00:00
const builtin = @import("builtin");
bring your own OS layer in the std lib closes #3784
2019-12-02 20:02:17 +00:00
const root = @import("root");
move std/debug.zig to a subdirectory self hosted compiler parser tests do some fuzz testing
2017-12-24 03:08:53 +00:00
const debug = std.debug;
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
const assert = debug.assert;
std.debug.assert: remove special case for test builds Previously, std.debug.assert would `@panic` in test builds, if the assertion failed. Now, it's always `unreachable`. This makes release mode test builds more accurately test the actual code that will be run. However this requires tests to call `std.testing.expect` rather than `std.debug.assert` to make sure output is correct. Here is the explanation of when to use either one, copied from the assert doc comments: Inside a test block, it is best to use the `std.testing` module rather than assert, because assert may not detect a test failure in ReleaseFast and ReleaseSafe mode. Outside of a test block, assert is the correct function to use. closes #1304
2019-02-08 23:18:47 +00:00
const testing = std.testing;
move std/debug.zig to a subdirectory self hosted compiler parser tests do some fuzz testing
2017-12-24 03:08:53 +00:00
const mem = std.mem;
const os = std.os;
const c = std.c;
remove @minValue,@maxValue; add std.math.minInt,maxInt closes #1466 closes #1476
2018-10-26 18:59:58 +00:00
const maxInt = std.math.maxInt;
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
fixups * move LoggingAllocator to its own file * style conventions * add documentation * use `anyerror` instead of `error{}` for the stream
2019-06-27 16:04:14 +00:00
pub const LoggingAllocator = @import("heap/logging_allocator.zig").LoggingAllocator;
update standard library to new I/O streams API
2020-03-10 22:44:30 +00:00
pub const loggingAllocator = @import("heap/logging_allocator.zig").loggingAllocator;
Add a logging allocator that uses std.log (#8511)
2021-06-09 18:42:07 +00:00
pub const ScopedLoggingAllocator = @import("heap/logging_allocator.zig").ScopedLoggingAllocator;
pub const LogToWriterAllocator = @import("heap/log_to_writer_allocator.zig").LogToWriterAllocator;
pub const logToWriterAllocator = @import("heap/log_to_writer_allocator.zig").logToWriterAllocator;
rework self-hosted compiler for incremental builds * introduce std.ArrayListUnmanaged for when you have the allocator stored elsewhere * move std.heap.ArenaAllocator implementation to its own file. extract the main state into std.heap.ArenaAllocator.State, which can be stored as an alternative to storing the entire ArenaAllocator, saving 24 bytes per ArenaAllocator on 64 bit targets. * std.LinkedList.Node pointer field now defaults to being null initialized. * Rework self-hosted compiler Package API * Delete almost all the bitrotted self-hosted compiler code. The only bit rotted code left is in main.zig and compilation.zig * Add call instruction to ZIR * self-hosted compiler ir API and link API are reworked to support a long-running compiler that incrementally updates declarations * Introduce the concept of scopes to ZIR semantic analysis * ZIR text format supports referencing named decls that are declared later in the file * Figure out how memory management works for the long-running compiler and incremental compilation. The main roots are top level declarations. There is a table of decls. The key is a cryptographic hash of the fully qualified decl name. Each decl has an arena allocator where all of the memory related to that decl is stored. Each code block has its own arena allocator for the lifetime of the block. Values that want to survive when going out of scope in a block must get copied into the outer block. Finally, values must get copied into the Decl arena to be long-lived. * Delete the unused MemoryCell struct. Instead, comptime pointers are based on references to Decl structs. * Figure out how caching works. Each Decl will store a set of other Decls which must be recompiled when it changes. This branch is still work-in-progress; this commit breaks the build.
2020-05-10 06:05:54 +00:00
pub const ArenaAllocator = @import("heap/arena_allocator.zig").ArenaAllocator;
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
pub const GeneralPurposeAllocator = @import("heap/general_purpose_allocator.zig").GeneralPurposeAllocator;
fixups * move LoggingAllocator to its own file * style conventions * add documentation * use `anyerror` instead of `error{}` for the stream
2019-06-27 16:04:14 +00:00
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
const Allocator = mem.Allocator;
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
const CAllocator = struct {
comptime {
if (!builtin.link_libc) {
@compileError("C allocator is only available when linking against libc");
}
run zig fmt on std lib and self hosted
2020-07-11 11:09:04 +00:00
}
new allocator interface
2020-04-17 20:15:36 +00:00
std: remove redundant comptime keyword @g-w1's fancy new compile error in action
2021-04-29 05:58:12 +00:00
usingnamespace if (@hasDecl(c, "malloc_size"))
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
struct {
pub const supports_malloc_size = true;
pub const malloc_size = c.malloc_size;
}
std: remove redundant comptime keyword @g-w1's fancy new compile error in action
2021-04-29 05:58:12 +00:00
else if (@hasDecl(c, "malloc_usable_size"))
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
struct {
pub const supports_malloc_size = true;
pub const malloc_size = c.malloc_usable_size;
}
std: remove redundant comptime keyword @g-w1's fancy new compile error in action
2021-04-29 05:58:12 +00:00
else if (@hasDecl(c, "_msize"))
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
struct {
pub const supports_malloc_size = true;
pub const malloc_size = c._msize;
}
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
else
struct {
pub const supports_malloc_size = false;
};
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
pub const supports_posix_memalign = @hasDecl(c, "posix_memalign");
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
fn getHeader(ptr: [*]u8) *[*]u8 {
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
return @intToPtr(*[*]u8, @ptrToInt(ptr) - @sizeOf(usize));
run zig fmt on std lib and self hosted
2020-07-11 11:09:04 +00:00
}
new allocator interface
2020-04-17 20:15:36 +00:00
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
fn alignedAlloc(len: usize, alignment: usize) ?[*]u8 {
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
if (supports_posix_memalign) {
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
// The posix_memalign only accepts alignment values that are a
// multiple of the pointer size
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
const eff_alignment = std.math.max(alignment, @sizeOf(usize));
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
var aligned_ptr: ?*anyopaque = undefined;
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
if (c.posix_memalign(&aligned_ptr, eff_alignment, len) != 0)
return null;
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
return @ptrCast([*]u8, aligned_ptr);
new allocator interface
2020-04-17 20:15:36 +00:00
}
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
// Thin wrapper around regular malloc, overallocate to account for
// alignment padding and store the orignal malloc()'ed pointer before
// the aligned address.
var unaligned_ptr = @ptrCast([*]u8, c.malloc(len + alignment - 1 + @sizeOf(usize)) orelse return null);
const unaligned_addr = @ptrToInt(unaligned_ptr);
const aligned_addr = mem.alignForward(unaligned_addr + @sizeOf(usize), alignment);
var aligned_ptr = unaligned_ptr + (aligned_addr - unaligned_addr);
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
getHeader(aligned_ptr).* = unaligned_ptr;
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
return aligned_ptr;
new allocator interface
2020-04-17 20:15:36 +00:00
}
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
fn alignedFree(ptr: [*]u8) void {
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
if (supports_posix_memalign) {
return c.free(ptr);
new allocator interface
2020-04-17 20:15:36 +00:00
}
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
const unaligned_ptr = getHeader(ptr).*;
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
c.free(unaligned_ptr);
}
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
fn alignedAllocSize(ptr: [*]u8) usize {
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
if (supports_posix_memalign) {
std: reorganization that allows new usingnamespace semantics The proposal #9629 is now accepted, usingnamespace stays but no longer puts identifiers in scope.
2021-08-28 22:34:17 +00:00
return CAllocator.malloc_size(ptr);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
}
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
const unaligned_ptr = getHeader(ptr).*;
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
const delta = @ptrToInt(ptr) - @ptrToInt(unaligned_ptr);
std: reorganization that allows new usingnamespace semantics The proposal #9629 is now accepted, usingnamespace stays but no longer puts identifiers in scope.
2021-08-28 22:34:17 +00:00
return CAllocator.malloc_size(unaligned_ptr) - delta;
new allocator interface
2020-04-17 20:15:36 +00:00
}
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
fn alloc(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
_: *anyopaque,
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
len: usize,
alignment: u29,
len_align: u29,
return_address: usize,
) error{OutOfMemory}![]u8 {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = return_address;
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
assert(len > 0);
assert(std.math.isPowerOfTwo(alignment));
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
var ptr = alignedAlloc(len, alignment) orelse return error.OutOfMemory;
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
if (len_align == 0) {
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
return ptr[0..len];
Implement a fallback mechanism for posix_memalign Do the alignment dance by ourselves whenever posix_memalign is not available. Don't try to use malloc as it has too many edge cases, figuring out whether a block of memory is manually aligned by the mechanism above or is directly coming from malloc becomes too hard to be valuable.
2020-09-22 11:18:40 +00:00
}
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
const full_len = init: {
std: reorganization that allows new usingnamespace semantics The proposal #9629 is now accepted, usingnamespace stays but no longer puts identifiers in scope.
2021-08-28 22:34:17 +00:00
if (CAllocator.supports_malloc_size) {
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
const s = alignedAllocSize(ptr);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
assert(s >= len);
break :init s;
}
break :init len;
};
return ptr[0..mem.alignBackwardAnyAlign(full_len, len_align)];
new allocator interface
2020-04-17 20:15:36 +00:00
}
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
fn resize(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
_: *anyopaque,
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
buf: []u8,
buf_align: u29,
new_len: usize,
len_align: u29,
return_address: usize,
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
) ?usize {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = buf_align;
_ = return_address;
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
if (new_len <= buf.len) {
return mem.alignAllocLen(buf.len, new_len, len_align);
new allocator interface
2020-04-17 20:15:36 +00:00
}
std: reorganization that allows new usingnamespace semantics The proposal #9629 is now accepted, usingnamespace stays but no longer puts identifiers in scope.
2021-08-28 22:34:17 +00:00
if (CAllocator.supports_malloc_size) {
Fix typo, remove debug leftover, rename few fns
2020-09-22 12:42:01 +00:00
const full_len = alignedAllocSize(buf.ptr);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
if (new_len <= full_len) {
return mem.alignAllocLen(full_len, new_len, len_align);
}
new allocator interface
2020-04-17 20:15:36 +00:00
}
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
return null;
new allocator interface
2020-04-17 20:15:36 +00:00
}
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
fn free(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
_: *anyopaque,
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
buf: []u8,
buf_align: u29,
return_address: usize,
) void {
_ = buf_align;
_ = return_address;
alignedFree(buf.ptr);
}
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
};
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
/// Supports the full Allocator interface, including alignment, and exploiting
/// `malloc_usable_size` if available. For an allocator that directly calls
/// `malloc`/`free`, see `raw_c_allocator`.
allocgate: dont use a dummy temporary for stateless allocators
2021-10-28 23:41:58 +00:00
pub const c_allocator = Allocator{
.ptr = undefined,
allocgate: utilize a *const vtable field
2021-10-29 03:17:21 +00:00
.vtable = &c_allocator_vtable,
};
const c_allocator_vtable = Allocator.VTable{
.alloc = CAllocator.alloc,
.resize = CAllocator.resize,
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
.free = CAllocator.free,
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
};
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
/// Asserts allocations are within `@alignOf(std.c.max_align_t)` and directly calls
/// `malloc`/`free`. Does not attempt to utilize `malloc_usable_size`.
/// This allocator is safe to use as the backing allocator with
std/heap: fix documentation of raw_c_allocator This is not in fact safe to use with GeneralPurposeAllocator as GPA requires align(page_size) but raw_c_allocator provides only @alignOf(std.c.max_align_t).
2020-12-28 14:06:19 +00:00
/// `ArenaAllocator` for example and is more optimal in such a case
/// than `c_allocator`.
allocgate: dont use a dummy temporary for stateless allocators
2021-10-28 23:41:58 +00:00
pub const raw_c_allocator = Allocator{
.ptr = undefined,
allocgate: utilize a *const vtable field
2021-10-29 03:17:21 +00:00
.vtable = &raw_c_allocator_vtable,
};
const raw_c_allocator_vtable = Allocator.VTable{
.alloc = rawCAlloc,
.resize = rawCResize,
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
.free = rawCFree,
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
};
fn rawCAlloc(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
_: *anyopaque,
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
len: usize,
ptr_align: u29,
len_align: u29,
ret_addr: usize,
) Allocator.Error![]u8 {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = len_align;
_ = ret_addr;
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
assert(ptr_align <= @alignOf(std.c.max_align_t));
const ptr = @ptrCast([*]u8, c.malloc(len) orelse return error.OutOfMemory);
return ptr[0..len];
}
fn rawCResize(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
_: *anyopaque,
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
buf: []u8,
old_align: u29,
new_len: usize,
len_align: u29,
ret_addr: usize,
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
) ?usize {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = old_align;
_ = ret_addr;
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
if (new_len <= buf.len) {
return mem.alignAllocLen(buf.len, new_len, len_align);
}
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
return null;
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
}
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
fn rawCFree(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
_: *anyopaque,
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
buf: []u8,
old_align: u29,
ret_addr: usize,
) void {
_ = old_align;
_ = ret_addr;
c.free(buf.ptr);
}
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
/// This allocator makes a syscall directly for every allocation and free.
M:N threading * add std.atomic.QueueMpsc.isEmpty * make std.debug.global_allocator thread-safe * std.event.Loop: now you have to choose between - initSingleThreaded - initMultiThreaded * std.event.Loop multiplexes coroutines onto kernel threads * Remove std.event.Loop.stop. Instead the event loop run() function returns once there are no pending coroutines. * fix crash in ir.cpp for calling methods under some conditions * small progress self-hosted compiler, analyzing top level declarations * Introduce std.event.Lock for synchronizing coroutines * introduce std.event.Locked(T) for data that only 1 coroutine should modify at once. * make the self hosted compiler use multi threaded event loop * make std.heap.DirectAllocator thread-safe See #174 TODO: * call sched_getaffinity instead of hard coding thread pool size 4 * support for Windows and MacOS * #1194 * #1197
2018-07-05 19:09:02 +00:00
/// Thread-safe and lock-free.
migrate from `std.Target.current` to `@import("builtin").target` closes #9388 closes #9321
2021-10-05 06:47:27 +00:00
pub const page_allocator = if (builtin.target.isWasm())
allocgate: dont use a dummy temporary for stateless allocators
2021-10-28 23:41:58 +00:00
Allocator{
.ptr = undefined,
allocgate: utilize a *const vtable field
2021-10-29 03:17:21 +00:00
.vtable = &WasmPageAllocator.vtable,
allocgate: dont use a dummy temporary for stateless allocators
2021-10-28 23:41:58 +00:00
}
else if (builtin.target.os.tag == .freestanding)
bring your own OS layer in the std lib closes #3784
2019-12-02 20:02:17 +00:00
root.os.heap.page_allocator
allocgate: dont use a dummy temporary for stateless allocators
2021-10-28 23:41:58 +00:00
else
Allocator{
.ptr = undefined,
allocgate: utilize a *const vtable field
2021-10-29 03:17:21 +00:00
.vtable = &PageAllocator.vtable,
allocgate: dont use a dummy temporary for stateless allocators
2021-10-28 23:41:58 +00:00
};
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
new allocator interface
2020-04-17 20:15:36 +00:00
/// Verifies that the adjusted length will still map to the full length
pub fn alignPageAllocLen(full_len: usize, len: usize, len_align: u29) usize {
const aligned_len = mem.alignAllocLen(full_len, len, len_align);
assert(mem.alignForward(aligned_len, mem.page_size) == full_len);
return aligned_len;
}
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
/// TODO Utilize this on Windows.
pub var next_mmap_addr_hint: ?[*]align(mem.page_size) u8 = null;
rename std.heap.direct_allocator to std.heap.page_allocator std.heap.direct_allocator is still available for now but it is marked deprecated.
2019-11-25 22:25:06 +00:00
const PageAllocator = struct {
allocgate: utilize a *const vtable field
2021-10-29 03:17:21 +00:00
const vtable = Allocator.VTable{
.alloc = alloc,
.resize = resize,
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
.free = free,
allocgate: utilize a *const vtable field
2021-10-29 03:17:21 +00:00
};
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
fn alloc(_: *anyopaque, n: usize, alignment: u29, len_align: u29, ra: usize) error{OutOfMemory}![]u8 {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = ra;
new allocator interface
2020-04-17 20:15:36 +00:00
assert(n > 0);
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
const aligned_len = mem.alignForward(n, mem.page_size);
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
introduce operating system version ranges as part of the target * re-introduce `std.build.Target` which is distinct from `std.Target`. `std.build.Target` wraps `std.Target` so that it can be annotated as "the native target" or an explicitly specified target. * `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a struct which has the tag as well as version range information. * `std.elf` gains some more ELF header constants. * `std.Target.parse` gains the ability to parse operating system version ranges as well as glibc version. * Added `std.Target.isGnuLibC()`. * self-hosted dynamic linker detection and glibc version detection. This also adds the improved logic using `/usr/bin/env` rather than invoking the system C compiler to find the dynamic linker when zig is statically linked. Related: #2084 Note: this `/usr/bin/env` code is work-in-progress. * `-target-glibc` CLI option is removed in favor of the new `-target` syntax. Example: `-target x86_64-linux-gnu.2.27` closes #1907
2020-02-25 06:52:27 +00:00
if (builtin.os.tag == .windows) {
clean up references to posix
2019-05-25 17:07:44 +00:00
const w = os.windows;
// Although officially it's at least aligned to page boundary,
// Windows is known to reserve pages on a 64K boundary. It's
// even more likely that the requested alignment is <= 64K than
// 4K, so we're just allocating blindly and hoping for the best.
// see https://devblogs.microsoft.com/oldnewthing/?p=42223
const addr = w.VirtualAlloc(
null,
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
aligned_len,
clean up references to posix
2019-05-25 17:07:44 +00:00
w.MEM_COMMIT | w.MEM_RESERVE,
w.PAGE_READWRITE,
) catch return error.OutOfMemory;
// If the allocation is sufficiently aligned, use it.
std: Add helpers to safely align pointers Add two helpers to ensure people won't ignore some edge cases such as pointers overflowing the address space. Also fix #8924 to some degree, the amount of unchecked alignForward is still scary.
2021-06-06 09:12:53 +00:00
if (mem.isAligned(@ptrToInt(addr), alignment)) {
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
return @ptrCast([*]u8, addr)[0..alignPageAllocLen(aligned_len, n, len_align)];
clean up references to posix
2019-05-25 17:07:44 +00:00
}
// If it wasn't, actually do an explicitely aligned allocation.
w.VirtualFree(addr, 0, w.MEM_RELEASE);
new allocator interface
2020-04-17 20:15:36 +00:00
const alloc_size = n + alignment - mem.page_size;
clean up references to posix
2019-05-25 17:07:44 +00:00
new allocator interface
2020-04-17 20:15:36 +00:00
while (true) {
clean up references to posix
2019-05-25 17:07:44 +00:00
// Reserve a range of memory large enough to find a sufficiently
// aligned address.
const reserved_addr = w.VirtualAlloc(
Implement Windows' DirectAllocator on top of VirtualAlloc and VirtualFree.
2019-05-08 16:02:37 +00:00
null,
clean up references to posix
2019-05-25 17:07:44 +00:00
alloc_size,
w.MEM_RESERVE,
w.PAGE_NOACCESS,
) catch return error.OutOfMemory;
const aligned_addr = mem.alignForward(@ptrToInt(reserved_addr), alignment);
// Release the reserved pages (not actually used).
w.VirtualFree(reserved_addr, 0, w.MEM_RELEASE);
// At this point, it is possible that another thread has
// obtained some memory space that will cause the next
// VirtualAlloc call to fail. To handle this, we will retry
// until it succeeds.
fixes for Windows and WASI
2019-05-27 04:48:56 +00:00
const ptr = w.VirtualAlloc(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
@intToPtr(*anyopaque, aligned_addr),
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
aligned_len,
Implement Windows' DirectAllocator on top of VirtualAlloc and VirtualFree.
2019-05-08 16:02:37 +00:00
w.MEM_COMMIT | w.MEM_RESERVE,
w.PAGE_READWRITE,
clean up references to posix
2019-05-25 17:07:44 +00:00
) catch continue;
fixes for Windows and WASI
2019-05-27 04:48:56 +00:00
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
return @ptrCast([*]u8, ptr)[0..alignPageAllocLen(aligned_len, n, len_align)];
new allocator interface
2020-04-17 20:15:36 +00:00
}
clean up references to posix
2019-05-25 17:07:44 +00:00
}
Implement Windows' DirectAllocator on top of VirtualAlloc and VirtualFree.
2019-05-08 16:02:37 +00:00
stage2: fix inline assembly with expression output Thanks @g-w1 for the print_air.zig implementation for inline assembly. I copied it and slightly modified it from your open pull request.
2021-10-20 21:50:07 +00:00
const max_drop_len = alignment - @minimum(alignment, mem.page_size);
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
const alloc_len = if (max_drop_len <= aligned_len - n)
aligned_len
else
mem.alignForward(aligned_len + max_drop_len, mem.page_size);
const hint = @atomicLoad(@TypeOf(next_mmap_addr_hint), &next_mmap_addr_hint, .Unordered);
behavior tests passing on Linux
2019-05-26 23:56:37 +00:00
const slice = os.mmap(
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
hint,
alloc_len,
std.os reorganization, avoiding `usingnamespace` The main purpose of this branch is to explore avoiding the `usingnamespace` feature of the zig language, specifically with regards to `std.os` and related functionality. If this experiment is successful, it will provide a data point on whether or not it would be practical to entirely remove `usingnamespace` from the language. In this commit, `usingnamespace` has been completely eliminated from the Linux x86_64 compilation path, aside from io_uring. The behavior tests pass, however that's as far as this branch goes. It is very breaking, and a lot more work is needed before it could be considered mergeable. I wanted to put a pull requset up early so that zig programmers have time to provide feedback. This is progress towards closing #6600 since it clarifies where the actual "owner" of each declaration is, and reduces the number of different ways to import the same declarations. One of the main organizational strategies used here is to do namespacing with real namespaces (e.g. structs) rather than by having declarations share a common prefix (the C strategy). It's no coincidence that `usingnamespace` has similar semantics to `#include` and becomes much less necessary when using proper namespaces.
2021-08-24 20:43:41 +00:00
os.PROT.READ | os.PROT.WRITE,
os.MAP.PRIVATE | os.MAP.ANONYMOUS,
clean up references to posix
2019-05-25 17:07:44 +00:00
-1,
0,
) catch return error.OutOfMemory;
new allocator interface
2020-04-17 20:15:36 +00:00
assert(mem.isAligned(@ptrToInt(slice.ptr), mem.page_size));
clean up references to posix
2019-05-25 17:07:44 +00:00
std: Add helpers to safely align pointers Add two helpers to ensure people won't ignore some edge cases such as pointers overflowing the address space. Also fix #8924 to some degree, the amount of unchecked alignForward is still scary.
2021-06-06 09:12:53 +00:00
const result_ptr = mem.alignPointer(slice.ptr, alignment) orelse
return error.OutOfMemory;
clean up references to posix
2019-05-25 17:07:44 +00:00
// Unmap the extra bytes that were only requested in order to guarantee
// that the range of memory we were provided had a proper alignment in
// it somewhere. The extra bytes could be at the beginning, or end, or both.
std: Add helpers to safely align pointers Add two helpers to ensure people won't ignore some edge cases such as pointers overflowing the address space. Also fix #8924 to some degree, the amount of unchecked alignForward is still scary.
2021-06-06 09:12:53 +00:00
const drop_len = @ptrToInt(result_ptr) - @ptrToInt(slice.ptr);
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
if (drop_len != 0) {
os.munmap(slice[0..drop_len]);
clean up references to posix
2019-05-25 17:07:44 +00:00
}
new allocator interface
2020-04-17 20:15:36 +00:00
// Unmap extra pages
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
const aligned_buffer_len = alloc_len - drop_len;
if (aligned_buffer_len > aligned_len) {
os.munmap(result_ptr[aligned_len..aligned_buffer_len]);
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
}
clean up references to posix
2019-05-25 17:07:44 +00:00
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
const new_hint = @alignCast(mem.page_size, result_ptr + aligned_len);
_ = @cmpxchgStrong(@TypeOf(next_mmap_addr_hint), &next_mmap_addr_hint, hint, new_hint, .Monotonic, .Monotonic);
return result_ptr[0..alignPageAllocLen(aligned_len, n, len_align)];
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
}
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
fn resize(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
_: *anyopaque,
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
buf_unaligned: []u8,
buf_align: u29,
new_size: usize,
len_align: u29,
return_address: usize,
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
) ?usize {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = buf_align;
_ = return_address;
new allocator interface
2020-04-17 20:15:36 +00:00
const new_size_aligned = mem.alignForward(new_size, mem.page_size);
introduce operating system version ranges as part of the target * re-introduce `std.build.Target` which is distinct from `std.Target`. `std.build.Target` wraps `std.Target` so that it can be annotated as "the native target" or an explicitly specified target. * `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a struct which has the tag as well as version range information. * `std.elf` gains some more ELF header constants. * `std.Target.parse` gains the ability to parse operating system version ranges as well as glibc version. * Added `std.Target.isGnuLibC()`. * self-hosted dynamic linker detection and glibc version detection. This also adds the improved logic using `/usr/bin/env` rather than invoking the system C compiler to find the dynamic linker when zig is statically linked. Related: #2084 Note: this `/usr/bin/env` code is work-in-progress. * `-target-glibc` CLI option is removed in favor of the new `-target` syntax. Example: `-target x86_64-linux-gnu.2.27` closes #1907
2020-02-25 06:52:27 +00:00
if (builtin.os.tag == .windows) {
clean up references to posix
2019-05-25 17:07:44 +00:00
const w = os.windows;
std.heap.page_allocator: Windows support for growing without remapping
2020-08-08 20:04:30 +00:00
if (new_size <= buf_unaligned.len) {
new allocator interface
2020-04-17 20:15:36 +00:00
const base_addr = @ptrToInt(buf_unaligned.ptr);
const old_addr_end = base_addr + buf_unaligned.len;
const new_addr_end = mem.alignForward(base_addr + new_size, mem.page_size);
if (old_addr_end > new_addr_end) {
clean up references to posix
2019-05-25 17:07:44 +00:00
// For shrinking that is not releasing, we will only
// decommit the pages not needed anymore.
w.VirtualFree(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
@intToPtr(*anyopaque, new_addr_end),
new allocator interface
2020-04-17 20:15:36 +00:00
old_addr_end - new_addr_end,
clean up references to posix
2019-05-25 17:07:44 +00:00
w.MEM_DECOMMIT,
);
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
}
new allocator interface
2020-04-17 20:15:36 +00:00
return alignPageAllocLen(new_size_aligned, new_size, len_align);
clean up references to posix
2019-05-25 17:07:44 +00:00
}
std.heap.page_allocator: Windows support for growing without remapping
2020-08-08 20:04:30 +00:00
const old_size_aligned = mem.alignForward(buf_unaligned.len, mem.page_size);
if (new_size_aligned <= old_size_aligned) {
new allocator interface
2020-04-17 20:15:36 +00:00
return alignPageAllocLen(new_size_aligned, new_size, len_align);
clean up references to posix
2019-05-25 17:07:44 +00:00
}
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
return null;
new allocator interface
2020-04-17 20:15:36 +00:00
}
clean up references to posix
2019-05-25 17:07:44 +00:00
new allocator interface
2020-04-17 20:15:36 +00:00
const buf_aligned_len = mem.alignForward(buf_unaligned.len, mem.page_size);
if (new_size_aligned == buf_aligned_len)
return alignPageAllocLen(new_size_aligned, new_size, len_align);
clean up references to posix
2019-05-25 17:07:44 +00:00
new allocator interface
2020-04-17 20:15:36 +00:00
if (new_size_aligned < buf_aligned_len) {
std: Add helpers to safely align pointers Add two helpers to ensure people won't ignore some edge cases such as pointers overflowing the address space. Also fix #8924 to some degree, the amount of unchecked alignForward is still scary.
2021-06-06 09:12:53 +00:00
const ptr = @alignCast(mem.page_size, buf_unaligned.ptr + new_size_aligned);
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
// TODO: if the next_mmap_addr_hint is within the unmapped range, update it
new allocator interface
2020-04-17 20:15:36 +00:00
os.munmap(ptr[0 .. buf_aligned_len - new_size_aligned]);
return alignPageAllocLen(new_size_aligned, new_size, len_align);
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
}
new allocator interface
2020-04-17 20:15:36 +00:00
// TODO: call mremap
std: introduce GeneralPurposeAllocator `std.GeneralPurposeAllocator` is now available. It is a function that takes a configuration struct (with default field values) and returns an allocator. There is a detailed description of this allocator in the doc comments at the top of the new file. The main feature of this allocator is that it is *safe*. It prevents double-free, use-after-free, and detects leaks. Some deprecation compile errors are removed. The Allocator interface gains `old_align` as a new parameter to `resizeFn`. This is useful to quickly look up allocations. `std.heap.page_allocator` is improved to use mmap address hints to avoid obtaining the same virtual address pages when unmapping and mapping pages. The new general purpose allocator uses the page allocator as its backing allocator by default. `std.testing.allocator` is replaced with usage of this new allocator, which does leak checking, and so the LeakCheckAllocator is retired. stage1 is improved so that the `@typeInfo` of a pointer has a lazy value for the alignment of the child type, to avoid false dependency loops when dealing with pointers to async function frames. The `std.mem.Allocator` interface is refactored to be in its own file. `std.Mutex` now exposes the dummy mutex with `std.Mutex.Dummy`. This allocator is great for debug mode, however it needs some work to have better performance in release modes. The next step will be setting up a series of tests in ziglang/gotta-go-fast and then making improvements to the implementation.
2020-08-08 05:35:15 +00:00
// TODO: if the next_mmap_addr_hint is within the remapped range, update it
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
return null;
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
}
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
fn free(_: *anyopaque, buf_unaligned: []u8, buf_align: u29, return_address: usize) void {
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
_ = buf_align;
_ = return_address;
if (builtin.os.tag == .windows) {
os.windows.VirtualFree(buf_unaligned.ptr, 0, os.windows.MEM_RELEASE);
} else {
const buf_aligned_len = mem.alignForward(buf_unaligned.len, mem.page_size);
const ptr = @alignCast(mem.page_size, buf_unaligned.ptr);
os.munmap(ptr[0..buf_aligned_len]);
}
}
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
};
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
const WasmPageAllocator = struct {
comptime {
migrate from `std.Target.current` to `@import("builtin").target` closes #9388 closes #9321
2021-10-05 06:47:27 +00:00
if (!builtin.target.isWasm()) {
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
@compileError("WasmPageAllocator is only available for wasm32 arch");
}
}
allocgate: utilize a *const vtable field
2021-10-29 03:17:21 +00:00
const vtable = Allocator.VTable{
.alloc = alloc,
.resize = resize,
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
.free = free,
allocgate: utilize a *const vtable field
2021-10-29 03:17:21 +00:00
};
Switch bitmask to enums
2019-12-05 23:59:43 +00:00
const PageStatus = enum(u1) {
used = 0,
free = 1,
Add WasmPageAllocator tests
2019-12-06 03:54:57 +00:00
pub const none_free: u8 = 0;
Switch bitmask to enums
2019-12-05 23:59:43 +00:00
};
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
Brain dump new wasm allocator
2019-11-28 00:46:42 +00:00
const FreeBlock = struct {
Standardize around bigger slices
2019-12-06 00:28:32 +00:00
data: []u128,
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
Strip out an unnecessary memset
2019-12-05 03:41:01 +00:00
const Io = std.packed_int_array.PackedIntIo(u1, .Little);
Initialize memory segments
2019-11-28 03:19:08 +00:00
fn totalPages(self: FreeBlock) usize {
Add WasmPageAllocator tests
2019-12-06 03:54:57 +00:00
return self.data.len * 128;
}
fn isInitialized(self: FreeBlock) bool {
return self.data.len > 0;
Use raw PackedIo to shave ~150b
2019-12-05 03:21:54 +00:00
}
Standardize around bigger slices
2019-12-06 00:28:32 +00:00
fn getBit(self: FreeBlock, idx: usize) PageStatus {
Strip out an unnecessary memset
2019-12-05 03:41:01 +00:00
const bit_offset = 0;
remove @bytesToSlice, @sliceToBytes from std lib
2020-02-21 18:46:53 +00:00
return @intToEnum(PageStatus, Io.get(mem.sliceAsBytes(self.data), idx, bit_offset));
Initialize memory segments
2019-11-28 03:19:08 +00:00
}
Standardize around bigger slices
2019-12-06 00:28:32 +00:00
fn setBits(self: FreeBlock, start_idx: usize, len: usize, val: PageStatus) void {
Strip out an unnecessary memset
2019-12-05 03:41:01 +00:00
const bit_offset = 0;
Extract setBits
2019-12-03 23:41:05 +00:00
var i: usize = 0;
while (i < len) : (i += 1) {
remove @bytesToSlice, @sliceToBytes from std lib
2020-02-21 18:46:53 +00:00
Io.set(mem.sliceAsBytes(self.data), start_idx + i, bit_offset, @enumToInt(val));
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
Extract setBits
2019-12-03 23:41:05 +00:00
}
Rejuggle how offsets are calculated
2019-12-05 00:12:25 +00:00
// Use '0xFFFFFFFF' as a _missing_ sentinel
// This saves ~50 bytes compared to returning a nullable
// We can guarantee that conventional memory never gets this big,
Use raw PackedIo to shave ~150b
2019-12-05 03:21:54 +00:00
// and wasm32 would not be able to address this memory (32 GB > usize).
Rejuggle how offsets are calculated
2019-12-05 00:12:25 +00:00
// Revisit if this is settled: https://github.com/ziglang/zig/issues/3806
const not_found = std.math.maxInt(usize);
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
WasmPageAllocator: fix bug not aligning allocations
2020-06-28 04:19:15 +00:00
fn useRecycled(self: FreeBlock, num_pages: usize, alignment: u29) usize {
Tighten recycled search
2019-12-04 05:49:56 +00:00
@setCold(true);
Standardize around bigger slices
2019-12-06 00:28:32 +00:00
for (self.data) |segment, i| {
Implement block-based skipping
2019-12-03 23:24:50 +00:00
const spills_into_next = @bitCast(i128, segment) < 0;
const has_enough_bits = @popCount(u128, segment) >= num_pages;
if (!spills_into_next and !has_enough_bits) continue;
Get stuff vaguely working
2019-11-28 04:02:54 +00:00
Implement block-based skipping
2019-12-03 23:24:50 +00:00
var j: usize = i * 128;
Tighten recycled search
2019-12-04 05:49:56 +00:00
while (j < (i + 1) * 128) : (j += 1) {
var count: usize = 0;
Switch bitmask to enums
2019-12-05 23:59:43 +00:00
while (j + count < self.totalPages() and self.getBit(j + count) == .free) {
Tighten recycled search
2019-12-04 05:49:56 +00:00
count += 1;
WasmPageAllocator: fix bug not aligning allocations
2020-06-28 04:19:15 +00:00
const addr = j * mem.page_size;
if (count >= num_pages and mem.isAligned(addr, alignment)) {
Switch bitmask to enums
2019-12-05 23:59:43 +00:00
self.setBits(j, num_pages, .used);
Rejuggle how offsets are calculated
2019-12-05 00:12:25 +00:00
return j;
Fix bugs
2019-12-02 18:26:14 +00:00
}
Get stuff vaguely working
2019-11-28 04:02:54 +00:00
}
Tighten recycled search
2019-12-04 05:49:56 +00:00
j += count;
Get stuff vaguely working
2019-11-28 04:02:54 +00:00
}
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
Rejuggle how offsets are calculated
2019-12-05 00:12:25 +00:00
return not_found;
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
Standardize around bigger slices
2019-12-06 00:28:32 +00:00
fn recycle(self: FreeBlock, start_idx: usize, len: usize) void {
Switch bitmask to enums
2019-12-05 23:59:43 +00:00
self.setBits(start_idx, len, .free);
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
Brain dump new wasm allocator
2019-11-28 00:46:42 +00:00
};
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
Add WasmPageAllocator tests
2019-12-06 03:54:57 +00:00
var _conventional_data = [_]u128{0} ** 16;
// Marking `conventional` as const saves ~40 bytes
Actually use `const conventional` as the comment indicates
2019-12-06 21:16:07 +00:00
const conventional = FreeBlock{ .data = &_conventional_data };
Standardize around bigger slices
2019-12-06 00:28:32 +00:00
var extended = FreeBlock{ .data = &[_]u128{} };
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
Rejuggle how offsets are calculated
2019-12-05 00:12:25 +00:00
fn extendedOffset() usize {
return conventional.totalPages();
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
Brain dump new wasm allocator
2019-11-28 00:46:42 +00:00
fn nPages(memsize: usize) usize {
new allocator interface
2020-04-17 20:15:36 +00:00
return mem.alignForward(memsize, mem.page_size) / mem.page_size;
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
fn alloc(_: *anyopaque, len: usize, alignment: u29, len_align: u29, ra: usize) error{OutOfMemory}![]u8 {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = ra;
new allocator interface
2020-04-17 20:15:36 +00:00
const page_count = nPages(len);
WasmPageAllocator: fix bug not aligning allocations
2020-06-28 04:19:15 +00:00
const page_idx = try allocPages(page_count, alignment);
run zig fmt on std lib and self hosted
2020-07-11 11:09:04 +00:00
return @intToPtr([*]u8, page_idx * mem.page_size)[0..alignPageAllocLen(page_count * mem.page_size, len, len_align)];
new allocator interface
2020-04-17 20:15:36 +00:00
}
WasmPageAllocator: fix bug not aligning allocations
2020-06-28 04:19:15 +00:00
fn allocPages(page_count: usize, alignment: u29) !usize {
new allocator interface
2020-04-17 20:15:36 +00:00
{
WasmPageAllocator: fix bug not aligning allocations
2020-06-28 04:19:15 +00:00
const idx = conventional.useRecycled(page_count, alignment);
new allocator interface
2020-04-17 20:15:36 +00:00
if (idx != FreeBlock.not_found) {
return idx;
}
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
WasmPageAllocator: fix bug not aligning allocations
2020-06-28 04:19:15 +00:00
const idx = extended.useRecycled(page_count, alignment);
Rejuggle how offsets are calculated
2019-12-05 00:12:25 +00:00
if (idx != FreeBlock.not_found) {
return idx + extendedOffset();
}
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
WasmPageAllocator: fix bug not aligning allocations
2020-06-28 04:19:15 +00:00
const next_page_idx = @wasmMemorySize(0);
const next_page_addr = next_page_idx * mem.page_size;
const aligned_addr = mem.alignForward(next_page_addr, alignment);
const drop_page_count = @divExact(aligned_addr - next_page_addr, mem.page_size);
const result = @wasmMemoryGrow(0, @intCast(u32, drop_page_count + page_count));
if (result <= 0)
Rejuggle how offsets are calculated
2019-12-05 00:12:25 +00:00
return error.OutOfMemory;
WasmPageAllocator: fix bug not aligning allocations
2020-06-28 04:19:15 +00:00
assert(result == next_page_idx);
const aligned_page_idx = next_page_idx + drop_page_count;
if (drop_page_count > 0) {
freePages(next_page_idx, aligned_page_idx);
Rejuggle how offsets are calculated
2019-12-05 00:12:25 +00:00
}
WasmPageAllocator: fix bug not aligning allocations
2020-06-28 04:19:15 +00:00
return @intCast(usize, aligned_page_idx);
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
new allocator interface
2020-04-17 20:15:36 +00:00
fn freePages(start: usize, end: usize) void {
if (start < extendedOffset()) {
stage2: fix inline assembly with expression output Thanks @g-w1 for the print_air.zig implementation for inline assembly. I copied it and slightly modified it from your open pull request.
2021-10-20 21:50:07 +00:00
conventional.recycle(start, @minimum(extendedOffset(), end) - start);
Fix bugs
2019-12-02 18:26:14 +00:00
}
new allocator interface
2020-04-17 20:15:36 +00:00
if (end > extendedOffset()) {
var new_end = end;
if (!extended.isInitialized()) {
// Steal the last page from the memory currently being recycled
// TODO: would it be better if we use the first page instead?
new_end -= 1;
extended.data = @intToPtr([*]u128, new_end * mem.page_size)[0 .. mem.page_size / @sizeOf(u128)];
// Since this is the first page being freed and we consume it, assume *nothing* is free.
mem.set(u128, extended.data, PageStatus.none_free);
}
const clamped_start = std.math.max(extendedOffset(), start);
extended.recycle(clamped_start - extendedOffset(), new_end - clamped_start);
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
}
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
fn resize(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
_: *anyopaque,
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
buf: []u8,
buf_align: u29,
new_len: usize,
len_align: u29,
return_address: usize,
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
) ?usize {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = buf_align;
_ = return_address;
new allocator interface
2020-04-17 20:15:36 +00:00
const aligned_len = mem.alignForward(buf.len, mem.page_size);
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
if (new_len > aligned_len) return null;
new allocator interface
2020-04-17 20:15:36 +00:00
const current_n = nPages(aligned_len);
const new_n = nPages(new_len);
if (new_n != current_n) {
const base = nPages(@ptrToInt(buf.ptr));
freePages(base + new_n, base + current_n);
Brain dump new wasm allocator
2019-11-28 00:46:42 +00:00
}
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
return alignPageAllocLen(new_n * mem.page_size, new_len, len_align);
}
fn free(
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
_: *anyopaque,
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
buf: []u8,
buf_align: u29,
return_address: usize,
) void {
_ = buf_align;
_ = return_address;
const aligned_len = mem.alignForward(buf.len, mem.page_size);
const current_n = nPages(aligned_len);
const base = nPages(@ptrToInt(buf.ptr));
freePages(base, base + current_n);
Merge branch 'wasi-run-tests' of https://github.com/fengb/zig into fengb-wasi-run-tests
2019-11-25 22:53:26 +00:00
}
};
introduce operating system version ranges as part of the target * re-introduce `std.build.Target` which is distinct from `std.Target`. `std.build.Target` wraps `std.Target` so that it can be annotated as "the native target" or an explicitly specified target. * `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a struct which has the tag as well as version range information. * `std.elf` gains some more ELF header constants. * `std.Target.parse` gains the ability to parse operating system version ranges as well as glibc version. * Added `std.Target.isGnuLibC()`. * self-hosted dynamic linker detection and glibc version detection. This also adds the improved logic using `/usr/bin/env` rather than invoking the system C compiler to find the dynamic linker when zig is statically linked. Related: #2084 Note: this `/usr/bin/env` code is work-in-progress. * `-target-glibc` CLI option is removed in favor of the new `-target` syntax. Example: `-target x86_64-linux-gnu.2.27` closes #1907
2020-02-25 06:52:27 +00:00
pub const HeapAllocator = switch (builtin.os.tag) {
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
.windows => struct {
heap_handle: ?HeapHandle,
const HeapHandle = os.windows.HANDLE;
pub fn init() HeapAllocator {
return HeapAllocator{
.heap_handle = null,
};
}
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
pub fn allocator(self: *HeapAllocator) Allocator {
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
return Allocator.init(self, alloc, resize, free);
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
}
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
pub fn deinit(self: *HeapAllocator) void {
if (self.heap_handle) |heap_handle| {
more fixes for windows and wasi
2019-05-27 05:35:58 +00:00
os.windows.HeapDestroy(heap_handle);
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
}
}
new allocator interface
2020-04-17 20:15:36 +00:00
fn getRecordPtr(buf: []u8) *align(1) usize {
return @intToPtr(*align(1) usize, @ptrToInt(buf.ptr) + buf.len);
}
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
fn alloc(
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
self: *HeapAllocator,
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
n: usize,
ptr_align: u29,
len_align: u29,
return_address: usize,
) error{OutOfMemory}![]u8 {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = return_address;
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
new allocator interface
2020-04-17 20:15:36 +00:00
const amt = n + ptr_align - 1 + @sizeOf(usize);
migrate from `std.Target.current` to `@import("builtin").target` closes #9388 closes #9321
2021-10-05 06:47:27 +00:00
const optional_heap_handle = @atomicLoad(?HeapHandle, &self.heap_handle, .SeqCst);
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
const heap_handle = optional_heap_handle orelse blk: {
Use unprotected heap when in single_threaded mode.
2019-05-08 16:14:35 +00:00
const options = if (builtin.single_threaded) os.windows.HEAP_NO_SERIALIZE else 0;
more fixes for windows and wasi
2019-05-27 05:35:58 +00:00
const hh = os.windows.kernel32.HeapCreate(options, amt, 0) orelse return error.OutOfMemory;
migrate from `std.Target.current` to `@import("builtin").target` closes #9388 closes #9321
2021-10-05 06:47:27 +00:00
const other_hh = @cmpxchgStrong(?HeapHandle, &self.heap_handle, null, hh, .SeqCst, .SeqCst) orelse break :blk hh;
more fixes for windows and wasi
2019-05-27 05:35:58 +00:00
os.windows.HeapDestroy(hh);
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
break :blk other_hh.?; // can't be null because of the cmpxchg
};
more fixes for windows and wasi
2019-05-27 05:35:58 +00:00
const ptr = os.windows.kernel32.HeapAlloc(heap_handle, 0, amt) orelse return error.OutOfMemory;
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
const root_addr = @ptrToInt(ptr);
new allocator interface
2020-04-17 20:15:36 +00:00
const aligned_addr = mem.alignForward(root_addr, ptr_align);
const return_len = init: {
if (len_align == 0) break :init n;
const full_len = os.windows.kernel32.HeapSize(heap_handle, 0, ptr);
assert(full_len != std.math.maxInt(usize));
assert(full_len >= amt);
std: Fix metadata corruption in HeapAllocator HeapAllocator stores the pointer returned by HeapAlloc right after the data block and, after the recent allocator refactoring, the space for this pointer was not taken into account in the calculation of the final block size. Fixes #5830
2020-09-19 22:10:53 +00:00
break :init mem.alignBackwardAnyAlign(full_len - (aligned_addr - root_addr) - @sizeOf(usize), len_align);
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
};
new allocator interface
2020-04-17 20:15:36 +00:00
const buf = @intToPtr([*]u8, aligned_addr)[0..return_len];
getRecordPtr(buf).* = root_addr;
return buf;
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
}
improvements & fixes for general purpose allocator integration * std.Mutex API is improved to not have init() deinit(). This API is designed to support static initialization and does not require any resource cleanup. This also happens to work around some kind of stage1 behavior that wasn't letting the new allocator mutex code get compiled. * the general purpose allocator now returns a bool from deinit() which tells if there were any leaks. This value is used by the test runner to fail the tests if there are any. * self-hosted compiler is updated to use the general purpose allocator when not linking against libc.
2020-08-08 06:26:58 +00:00
fn resize(
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
self: *HeapAllocator,
improvements & fixes for general purpose allocator integration * std.Mutex API is improved to not have init() deinit(). This API is designed to support static initialization and does not require any resource cleanup. This also happens to work around some kind of stage1 behavior that wasn't letting the new allocator mutex code get compiled. * the general purpose allocator now returns a bool from deinit() which tells if there were any leaks. This value is used by the test runner to fail the tests if there are any. * self-hosted compiler is updated to use the general purpose allocator when not linking against libc.
2020-08-08 06:26:58 +00:00
buf: []u8,
buf_align: u29,
new_size: usize,
len_align: u29,
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
return_address: usize,
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
) ?usize {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = buf_align;
_ = return_address;
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
new allocator interface
2020-04-17 20:15:36 +00:00
const root_addr = getRecordPtr(buf).*;
const align_offset = @ptrToInt(buf.ptr) - root_addr;
const amt = align_offset + new_size + @sizeOf(usize);
more fixes for windows and wasi
2019-05-27 05:35:58 +00:00
const new_ptr = os.windows.kernel32.HeapReAlloc(
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
self.heap_handle.?,
new allocator interface
2020-04-17 20:15:36 +00:00
os.windows.HEAP_REALLOC_IN_PLACE_ONLY,
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
@intToPtr(*anyopaque, root_addr),
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
amt,
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
) orelse return null;
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
assert(new_ptr == @intToPtr(*anyopaque, root_addr));
new allocator interface
2020-04-17 20:15:36 +00:00
const return_len = init: {
if (len_align == 0) break :init new_size;
const full_len = os.windows.kernel32.HeapSize(self.heap_handle.?, 0, new_ptr);
assert(full_len != std.math.maxInt(usize));
assert(full_len >= amt);
break :init mem.alignBackwardAnyAlign(full_len - align_offset, len_align);
};
getRecordPtr(buf.ptr[0..return_len]).* = root_addr;
return return_len;
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
}
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
fn free(
self: *HeapAllocator,
buf: []u8,
buf_align: u29,
return_address: usize,
) void {
_ = buf_align;
_ = return_address;
stage1, stage2: rename c_void to anyopaque (#10316) zig fmt now replaces c_void with anyopaque to make updating code easy.
2021-12-19 05:24:45 +00:00
os.windows.HeapFree(self.heap_handle.?, 0, @intToPtr(*anyopaque, getRecordPtr(buf).*));
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
}
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
},
else => @compileError("Unsupported OS"),
};
new allocator interface
2020-04-17 20:15:36 +00:00
fn sliceContainsPtr(container: []u8, ptr: [*]u8) bool {
return @ptrToInt(ptr) >= @ptrToInt(container.ptr) and
@ptrToInt(ptr) < (@ptrToInt(container.ptr) + container.len);
}
fn sliceContainsSlice(container: []u8, slice: []u8) bool {
return @ptrToInt(slice.ptr) >= @ptrToInt(container.ptr) and
(@ptrToInt(slice.ptr) + slice.len) <= (@ptrToInt(container.ptr) + container.len);
}
New Zig formal grammar (#1685) Reverted #1628 and changed the grammar+parser of the language to not allow certain expr where types are expected
2018-11-13 13:08:37 +00:00
pub const FixedBufferAllocator = struct {
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
end_index: usize,
buffer: []u8,
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
pub fn init(buffer: []u8) FixedBufferAllocator {
New Zig formal grammar (#1685) Reverted #1628 and changed the grammar+parser of the language to not allow certain expr where types are expected
2018-11-13 13:08:37 +00:00
return FixedBufferAllocator{
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
.buffer = buffer,
.end_index = 0,
};
add test for c_allocator
2017-11-05 17:27:56 +00:00
}
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
/// *WARNING* using this at the same time as the interface returned by `threadSafeAllocator` is not thread safe
pub fn allocator(self: *FixedBufferAllocator) Allocator {
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
return Allocator.init(self, alloc, resize, free);
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
}
/// Provides a lock free thread safe `Allocator` interface to the underlying `FixedBufferAllocator`
/// *WARNING* using this at the same time as the interface returned by `getAllocator` is not thread safe
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
pub fn threadSafeAllocator(self: *FixedBufferAllocator) Allocator {
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
return Allocator.init(
self,
threadSafeAlloc,
Allocator.NoResize(FixedBufferAllocator).noResize,
Allocator.NoOpFree(FixedBufferAllocator).noOpFree,
);
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
}
new allocator interface
2020-04-17 20:15:36 +00:00
pub fn ownsPtr(self: *FixedBufferAllocator, ptr: [*]u8) bool {
return sliceContainsPtr(self.buffer, ptr);
}
pub fn ownsSlice(self: *FixedBufferAllocator, slice: []u8) bool {
return sliceContainsSlice(self.buffer, slice);
}
new allocator interface after Andrew Kelley review
2020-06-27 01:29:06 +00:00
/// NOTE: this will not work in all cases, if the last allocation had an adjusted_index
/// then we won't be able to determine what the last allocation was. This is because
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
/// the alignForward operation done in alloc is not reversible.
new allocator interface
2020-04-17 20:15:36 +00:00
pub fn isLastAllocation(self: *FixedBufferAllocator, buf: []u8) bool {
return buf.ptr + buf.len == self.buffer.ptr + self.end_index;
}
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
fn alloc(self: *FixedBufferAllocator, n: usize, ptr_align: u29, len_align: u29, ra: usize) ![]u8 {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = len_align;
_ = ra;
std: Add helpers to safely align pointers Add two helpers to ensure people won't ignore some edge cases such as pointers overflowing the address space. Also fix #8924 to some degree, the amount of unchecked alignForward is still scary.
2021-06-06 09:12:53 +00:00
const adjust_off = mem.alignPointerOffset(self.buffer.ptr + self.end_index, ptr_align) orelse
return error.OutOfMemory;
const adjusted_index = self.end_index + adjust_off;
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
const new_end_index = adjusted_index + n;
if (new_end_index > self.buffer.len) {
return error.OutOfMemory;
}
behavior tests passing with new pointer deref syntax
2018-05-01 05:53:04 +00:00
const result = self.buffer[adjusted_index..new_end_index];
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
self.end_index = new_end_index;
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
new allocator interface after Andrew Kelley review
2020-06-27 01:29:06 +00:00
return result;
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
}
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
fn resize(
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
self: *FixedBufferAllocator,
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
buf: []u8,
buf_align: u29,
new_size: usize,
len_align: u29,
return_address: usize,
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
) ?usize {
fix code broken from previous commit
2021-06-20 01:10:22 +00:00
_ = buf_align;
_ = return_address;
new allocator interface
2020-04-17 20:15:36 +00:00
assert(self.ownsSlice(buf)); // sanity check
if (!self.isLastAllocation(buf)) {
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
if (new_size > buf.len) return null;
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
return mem.alignAllocLen(buf.len, new_size, len_align);
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
}
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
new allocator interface
2020-04-17 20:15:36 +00:00
if (new_size <= buf.len) {
const sub = buf.len - new_size;
self.end_index -= sub;
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
return mem.alignAllocLen(buf.len - sub, new_size, len_align);
new allocator interface
2020-04-17 20:15:36 +00:00
}
new allocator interface after Andrew Kelley review
2020-06-27 01:29:06 +00:00
const add = new_size - buf.len;
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
if (add + self.end_index > self.buffer.len) return null;
new allocator interface
2020-04-17 20:15:36 +00:00
self.end_index += add;
new allocator interface after Andrew Kelley review
2020-06-27 01:29:06 +00:00
return new_size;
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
}
Add FixedBufferAllocator.reset
2019-09-16 01:35:01 +00:00
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
fn free(
self: *FixedBufferAllocator,
buf: []u8,
buf_align: u29,
return_address: usize,
) void {
_ = buf_align;
_ = return_address;
assert(self.ownsSlice(buf)); // sanity check
if (self.isLastAllocation(buf)) {
self.end_index -= buf.len;
}
}
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
fn threadSafeAlloc(self: *FixedBufferAllocator, n: usize, ptr_align: u29, len_align: u29, ra: usize) ![]u8 {
_ = len_align;
_ = ra;
var end_index = @atomicLoad(usize, &self.end_index, .SeqCst);
while (true) {
const adjust_off = mem.alignPointerOffset(self.buffer.ptr + end_index, ptr_align) orelse
return error.OutOfMemory;
const adjusted_index = end_index + adjust_off;
const new_end_index = adjusted_index + n;
if (new_end_index > self.buffer.len) {
return error.OutOfMemory;
}
end_index = @cmpxchgWeak(usize, &self.end_index, end_index, new_end_index, .SeqCst, .SeqCst) orelse return self.buffer[adjusted_index..new_end_index];
}
}
Remove 'inline', as per @andrewrk
2019-09-21 15:42:26 +00:00
pub fn reset(self: *FixedBufferAllocator) void {
Add FixedBufferAllocator.reset
2019-09-16 01:35:01 +00:00
self.end_index = 0;
}
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
};
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
pub const ThreadSafeFixedBufferAllocator = @compileError("ThreadSafeFixedBufferAllocator has been replaced with `threadSafeAllocator` on FixedBufferAllocator");
add fuzz tests for std.atomic.Stack
2018-04-28 21:53:06 +00:00
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
pub fn stackFallback(comptime size: usize, fallback_allocator: Allocator) StackFallbackAllocator(size) {
New Zig formal grammar (#1685) Reverted #1628 and changed the grammar+parser of the language to not allow certain expr where types are expected
2018-11-13 13:08:37 +00:00
return StackFallbackAllocator(size){
add std.os.cpuCount and have std.event.Loop use it for thread pool size
2018-07-07 05:23:18 +00:00
.buffer = undefined,
.fallback_allocator = fallback_allocator,
.fixed_buffer_allocator = undefined,
};
}
pub fn StackFallbackAllocator(comptime size: usize) type {
New Zig formal grammar (#1685) Reverted #1628 and changed the grammar+parser of the language to not allow certain expr where types are expected
2018-11-13 13:08:37 +00:00
return struct {
remove `this`. add `@This()`. closes #1283
2018-09-13 20:34:33 +00:00
const Self = @This();
add std.os.cpuCount and have std.event.Loop use it for thread pool size
2018-07-07 05:23:18 +00:00
buffer: [size]u8,
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
fallback_allocator: Allocator,
add std.os.cpuCount and have std.event.Loop use it for thread pool size
2018-07-07 05:23:18 +00:00
fixed_buffer_allocator: FixedBufferAllocator,
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
/// WARNING: This functions both fetches a `std.mem.Allocator` interface to this allocator *and* resets the internal buffer allocator
pub fn get(self: *Self) Allocator {
add std.os.cpuCount and have std.event.Loop use it for thread pool size
2018-07-07 05:23:18 +00:00
self.fixed_buffer_allocator = FixedBufferAllocator.init(self.buffer[0..]);
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
return Allocator.init(self, alloc, resize, free);
add std.os.cpuCount and have std.event.Loop use it for thread pool size
2018-07-07 05:23:18 +00:00
}
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
fn alloc(
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
self: *Self,
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
len: usize,
ptr_align: u29,
len_align: u29,
return_address: usize,
Fix StackFallbackAllocator
2020-12-13 17:58:17 +00:00
) error{OutOfMemory}![]u8 {
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
return FixedBufferAllocator.alloc(&self.fixed_buffer_allocator, len, ptr_align, len_align, return_address) catch
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
return self.fallback_allocator.rawAlloc(len, ptr_align, len_align, return_address);
add std.os.cpuCount and have std.event.Loop use it for thread pool size
2018-07-07 05:23:18 +00:00
}
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
fn resize(
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
self: *Self,
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
buf: []u8,
buf_align: u29,
new_len: usize,
len_align: u29,
return_address: usize,
allocgate: change resize to return optional instead of error
2021-11-07 01:40:06 +00:00
) ?usize {
new allocator interface
2020-04-17 20:15:36 +00:00
if (self.fixed_buffer_allocator.ownsPtr(buf.ptr)) {
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
return FixedBufferAllocator.resize(&self.fixed_buffer_allocator, buf, buf_align, new_len, len_align, return_address);
new allocator interface
2020-04-17 20:15:36 +00:00
} else {
allocgate: split free out from resize
2021-11-06 00:54:35 +00:00
return self.fallback_allocator.rawResize(buf, buf_align, new_len, len_align, return_address);
}
}
fn free(
self: *Self,
buf: []u8,
buf_align: u29,
return_address: usize,
) void {
if (self.fixed_buffer_allocator.ownsPtr(buf.ptr)) {
return FixedBufferAllocator.free(&self.fixed_buffer_allocator, buf, buf_align, return_address);
} else {
return self.fallback_allocator.rawFree(buf, buf_align, return_address);
add std.os.cpuCount and have std.event.Loop use it for thread pool size
2018-07-07 05:23:18 +00:00
}
}
};
}
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
test "c_allocator" {
if (builtin.link_libc) {
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
try testAllocator(c_allocator);
try testAllocatorAligned(c_allocator);
try testAllocatorLargeAlignment(c_allocator);
try testAllocatorAlignedShrink(c_allocator);
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
}
breaking change to std.io API * Merge io.InStream and io.OutStream into io.File * Introduce io.OutStream and io.InStream interfaces - io.File implements both of these * Move mem.IncrementingAllocator to heap.IncrementingAllocator Instead of: ``` %return std.io.stderr.printf("hello\n"); ``` now do: ``` std.debug.warn("hello\n"); ``` To print to stdout, see `io.getStdOut()`. * Rename std.ArrayList.resizeDown to std.ArrayList.shrink.
2017-10-31 08:47:55 +00:00
}
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
test "raw_c_allocator" {
if (builtin.link_libc) {
try testAllocator(raw_c_allocator);
}
}
Add WasmPageAllocator tests
2019-12-06 03:54:57 +00:00
test "WasmPageAllocator internals" {
migrate from `std.Target.current` to `@import("builtin").target` closes #9388 closes #9321
2021-10-05 06:47:27 +00:00
if (comptime builtin.target.isWasm()) {
new allocator interface
2020-04-17 20:15:36 +00:00
const conventional_memsize = WasmPageAllocator.conventional.totalPages() * mem.page_size;
const initial = try page_allocator.alloc(u8, mem.page_size);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(@ptrToInt(initial.ptr) < conventional_memsize); // If this isn't conventional, the rest of these tests don't make sense. Also we have a serious memory leak in the test suite.
Rewrite WasmPageAllocator tests to be less flaky on environment
2019-12-09 03:22:07 +00:00
var inplace = try page_allocator.realloc(initial, 1);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expectEqual(initial.ptr, inplace.ptr);
Rewrite WasmPageAllocator tests to be less flaky on environment
2019-12-09 03:22:07 +00:00
inplace = try page_allocator.realloc(inplace, 4);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expectEqual(initial.ptr, inplace.ptr);
Rewrite WasmPageAllocator tests to be less flaky on environment
2019-12-09 03:22:07 +00:00
page_allocator.free(inplace);
const reuse = try page_allocator.alloc(u8, 1);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expectEqual(initial.ptr, reuse.ptr);
Rewrite WasmPageAllocator tests to be less flaky on environment
2019-12-09 03:22:07 +00:00
page_allocator.free(reuse);
// This segment may span conventional and extended which has really complex rules so we're just ignoring it for now.
const padding = try page_allocator.alloc(u8, conventional_memsize);
page_allocator.free(padding);
const extended = try page_allocator.alloc(u8, conventional_memsize);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(@ptrToInt(extended.ptr) >= conventional_memsize);
Rewrite WasmPageAllocator tests to be less flaky on environment
2019-12-09 03:22:07 +00:00
const use_small = try page_allocator.alloc(u8, 1);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expectEqual(initial.ptr, use_small.ptr);
Rewrite WasmPageAllocator tests to be less flaky on environment
2019-12-09 03:22:07 +00:00
page_allocator.free(use_small);
inplace = try page_allocator.realloc(extended, 1);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expectEqual(extended.ptr, inplace.ptr);
Rewrite WasmPageAllocator tests to be less flaky on environment
2019-12-09 03:22:07 +00:00
page_allocator.free(inplace);
const reuse_extended = try page_allocator.alloc(u8, conventional_memsize);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expectEqual(extended.ptr, reuse_extended.ptr);
Rewrite WasmPageAllocator tests to be less flaky on environment
2019-12-09 03:22:07 +00:00
page_allocator.free(reuse_extended);
Add WasmPageAllocator tests
2019-12-06 03:54:57 +00:00
}
}
rename std.heap.direct_allocator to std.heap.page_allocator std.heap.direct_allocator is still available for now but it is marked deprecated.
2019-11-25 22:25:06 +00:00
test "PageAllocator" {
const allocator = page_allocator;
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
try testAllocator(allocator);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
try testAllocatorAligned(allocator);
migrate from `std.Target.current` to `@import("builtin").target` closes #9388 closes #9321
2021-10-05 06:47:27 +00:00
if (!builtin.target.isWasm()) {
Resolve tests to work with or skip WasmPageAllocator
2019-12-06 01:31:49 +00:00
try testAllocatorLargeAlignment(allocator);
try testAllocatorAlignedShrink(allocator);
}
Implement Windows' DirectAllocator on top of VirtualAlloc and VirtualFree.
2019-05-08 16:02:37 +00:00
introduce operating system version ranges as part of the target * re-introduce `std.build.Target` which is distinct from `std.Target`. `std.build.Target` wraps `std.Target` so that it can be annotated as "the native target" or an explicitly specified target. * `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a struct which has the tag as well as version range information. * `std.elf` gains some more ELF header constants. * `std.Target.parse` gains the ability to parse operating system version ranges as well as glibc version. * Added `std.Target.isGnuLibC()`. * self-hosted dynamic linker detection and glibc version detection. This also adds the improved logic using `/usr/bin/env` rather than invoking the system C compiler to find the dynamic linker when zig is statically linked. Related: #2084 Note: this `/usr/bin/env` code is work-in-progress. * `-target-glibc` CLI option is removed in favor of the new `-target` syntax. Example: `-target x86_64-linux-gnu.2.27` closes #1907
2020-02-25 06:52:27 +00:00
if (builtin.os.tag == .windows) {
Implement Windows' DirectAllocator on top of VirtualAlloc and VirtualFree.
2019-05-08 16:02:37 +00:00
// Trying really large alignment. As mentionned in the implementation,
// VirtualAlloc returns 64K aligned addresses. We want to make sure
rename std.heap.direct_allocator to std.heap.page_allocator std.heap.direct_allocator is still available for now but it is marked deprecated.
2019-11-25 22:25:06 +00:00
// PageAllocator works beyond that, as it's not tested by
Implement Windows' DirectAllocator on top of VirtualAlloc and VirtualFree.
2019-05-08 16:02:37 +00:00
// `testAllocatorLargeAlignment`.
const slice = try allocator.alignedAlloc(u8, 1 << 20, 128);
slice[0] = 0x12;
slice[127] = 0x34;
allocator.free(slice);
}
Fix issue 5741, use after free
2020-06-28 20:33:41 +00:00
{
var buf = try allocator.alloc(u8, mem.page_size + 1);
defer allocator.free(buf);
buf = try allocator.realloc(buf, 1); // shrink past the page boundary
}
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
}
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
test "HeapAllocator" {
introduce operating system version ranges as part of the target * re-introduce `std.build.Target` which is distinct from `std.Target`. `std.build.Target` wraps `std.Target` so that it can be annotated as "the native target" or an explicitly specified target. * `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a struct which has the tag as well as version range information. * `std.elf` gains some more ELF header constants. * `std.Target.parse` gains the ability to parse operating system version ranges as well as glibc version. * Added `std.Target.isGnuLibC()`. * self-hosted dynamic linker detection and glibc version detection. This also adds the improved logic using `/usr/bin/env` rather than invoking the system C compiler to find the dynamic linker when zig is statically linked. Related: #2084 Note: this `/usr/bin/env` code is work-in-progress. * `-target-glibc` CLI option is removed in favor of the new `-target` syntax. Example: `-target x86_64-linux-gnu.2.27` closes #1907
2020-02-25 06:52:27 +00:00
if (builtin.os.tag == .windows) {
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
var heap_allocator = HeapAllocator.init();
defer heap_allocator.deinit();
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = heap_allocator.allocator();
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
try testAllocator(allocator);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
try testAllocatorAligned(allocator);
Duplicate windows's DirectAllocator as HeapAllocator, which it is in reality.
2019-05-08 15:53:58 +00:00
try testAllocatorLargeAlignment(allocator);
try testAllocatorAlignedShrink(allocator);
}
}
fix ArenaAllocator
2019-06-22 05:13:10 +00:00
test "ArenaAllocator" {
rename std.heap.direct_allocator to std.heap.page_allocator std.heap.direct_allocator is still available for now but it is marked deprecated.
2019-11-25 22:25:06 +00:00
var arena_allocator = ArenaAllocator.init(page_allocator);
fix ArenaAllocator
2019-06-22 05:13:10 +00:00
defer arena_allocator.deinit();
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = arena_allocator.allocator();
fix ArenaAllocator
2019-06-22 05:13:10 +00:00
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
try testAllocator(allocator);
try testAllocatorAligned(allocator);
try testAllocatorLargeAlignment(allocator);
try testAllocatorAlignedShrink(allocator);
fix ArenaAllocator
2019-06-22 05:13:10 +00:00
}
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
Resolve tests to work with or skip WasmPageAllocator
2019-12-06 01:31:49 +00:00
var test_fixed_buffer_allocator_memory: [800000 * @sizeOf(u64)]u8 = undefined;
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
test "FixedBufferAllocator" {
new allocator interface after Andrew Kelley review
2020-06-27 01:29:06 +00:00
var fixed_buffer_allocator = mem.validationWrap(FixedBufferAllocator.init(test_fixed_buffer_allocator_memory[0..]));
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = fixed_buffer_allocator.allocator();
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
try testAllocator(allocator);
try testAllocatorAligned(allocator);
try testAllocatorLargeAlignment(allocator);
try testAllocatorAlignedShrink(allocator);
remove std.heap.IncrementingAllocator Use std.heap.FixedBufferAllocator combined with std.heap.DirectAllocator instead. std.mem.FixedBufferAllocator is moved to std.heap.FixedBufferAllocator
2018-02-12 07:27:02 +00:00
}
Add FixedBufferAllocator.reset
2019-09-16 01:35:01 +00:00
test "FixedBufferAllocator.reset" {
enable test coverage for armv8-linux-musleabihf * fix libc prototypes of offsets to have correct integer sizes and signedness. This gets all behavior tests to pass for armv8-linux-musleabihf * fix linux mmap syscall to have correct integer size and signedness for offset * disable failing armv8-linux-musleabihf std lib tests. See 3289. I suspect compiler-rt issue. Note these tests fail with this target triple regardless of whether musl is actually linked (-lc).
2019-09-25 19:57:13 +00:00
var buf: [8]u8 align(@alignOf(u64)) = undefined;
Add FixedBufferAllocator.reset
2019-09-16 01:35:01 +00:00
var fba = FixedBufferAllocator.init(buf[0..]);
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = fba.allocator();
Add FixedBufferAllocator.reset
2019-09-16 01:35:01 +00:00
const X = 0xeeeeeeeeeeeeeeee;
const Y = 0xffffffffffffffff;
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
var x = try allocator.create(u64);
Add FixedBufferAllocator.reset
2019-09-16 01:35:01 +00:00
x.* = X;
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
try testing.expectError(error.OutOfMemory, allocator.create(u64));
Add FixedBufferAllocator.reset
2019-09-16 01:35:01 +00:00
fba.reset();
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
var y = try allocator.create(u64);
Add FixedBufferAllocator.reset
2019-09-16 01:35:01 +00:00
y.* = Y;
// we expect Y to have overwritten X.
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(x.* == y.*);
try testing.expect(y.* == Y);
Add FixedBufferAllocator.reset
2019-09-16 01:35:01 +00:00
}
Fix StackFallbackAllocator
2020-12-13 17:58:17 +00:00
test "StackFallbackAllocator" {
const fallback_allocator = page_allocator;
var stack_allocator = stackFallback(4096, fallback_allocator);
try testAllocator(stack_allocator.get());
try testAllocatorAligned(stack_allocator.get());
try testAllocatorLargeAlignment(stack_allocator.get());
try testAllocatorAlignedShrink(stack_allocator.get());
}
Improve realloc on fixed buffer allocator (#1238) * Add test to check re-use of memory * Check if realloc has to reallocate the last allocated memory block. If so extend that block instead of allocating a new one. * Also check if the realloc actually preserves the data.
2018-07-14 20:31:11 +00:00
test "FixedBufferAllocator Reuse memory on realloc" {
var small_fixed_buffer: [10]u8 = undefined;
// check if we re-use the memory
{
var fixed_buffer_allocator = FixedBufferAllocator.init(small_fixed_buffer[0..]);
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = fixed_buffer_allocator.allocator();
Improve realloc on fixed buffer allocator (#1238) * Add test to check re-use of memory * Check if realloc has to reallocate the last allocated memory block. If so extend that block instead of allocating a new one. * Also check if the realloc actually preserves the data.
2018-07-14 20:31:11 +00:00
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
var slice0 = try allocator.alloc(u8, 5);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice0.len == 5);
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
var slice1 = try allocator.realloc(slice0, 10);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice1.ptr == slice0.ptr);
try testing.expect(slice1.len == 10);
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
try testing.expectError(error.OutOfMemory, allocator.realloc(slice1, 11));
Improve realloc on fixed buffer allocator (#1238) * Add test to check re-use of memory * Check if realloc has to reallocate the last allocated memory block. If so extend that block instead of allocating a new one. * Also check if the realloc actually preserves the data.
2018-07-14 20:31:11 +00:00
}
// check that we don't re-use the memory if it's not the most recent block
{
var fixed_buffer_allocator = FixedBufferAllocator.init(small_fixed_buffer[0..]);
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = fixed_buffer_allocator.allocator();
Improve realloc on fixed buffer allocator (#1238) * Add test to check re-use of memory * Check if realloc has to reallocate the last allocated memory block. If so extend that block instead of allocating a new one. * Also check if the realloc actually preserves the data.
2018-07-14 20:31:11 +00:00
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
var slice0 = try allocator.alloc(u8, 2);
Improve realloc on fixed buffer allocator (#1238) * Add test to check re-use of memory * Check if realloc has to reallocate the last allocated memory block. If so extend that block instead of allocating a new one. * Also check if the realloc actually preserves the data.
2018-07-14 20:31:11 +00:00
slice0[0] = 1;
slice0[1] = 2;
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
var slice1 = try allocator.alloc(u8, 2);
var slice2 = try allocator.realloc(slice0, 4);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice0.ptr != slice2.ptr);
try testing.expect(slice1.ptr != slice2.ptr);
try testing.expect(slice2[0] == 1);
try testing.expect(slice2[1] == 2);
Improve realloc on fixed buffer allocator (#1238) * Add test to check re-use of memory * Check if realloc has to reallocate the last allocated memory block. If so extend that block instead of allocating a new one. * Also check if the realloc actually preserves the data.
2018-07-14 20:31:11 +00:00
}
}
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
test "Thread safe FixedBufferAllocator" {
var fixed_buffer_allocator = FixedBufferAllocator.init(test_fixed_buffer_allocator_memory[0..]);
add fuzz tests for std.atomic.Stack
2018-04-28 21:53:06 +00:00
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
try testAllocator(fixed_buffer_allocator.threadSafeAllocator());
try testAllocatorAligned(fixed_buffer_allocator.threadSafeAllocator());
try testAllocatorLargeAlignment(fixed_buffer_allocator.threadSafeAllocator());
try testAllocatorAlignedShrink(fixed_buffer_allocator.threadSafeAllocator());
add fuzz tests for std.atomic.Stack
2018-04-28 21:53:06 +00:00
}
add std.heap.raw_c_allocator This API does what `std.heap.c_allocator` previously did. See the new doc comments for more details.
2020-11-19 05:09:34 +00:00
/// This one should not try alignments that exceed what C malloc can handle.
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
pub fn testAllocator(base_allocator: mem.Allocator) !void {
new allocator interface after Andrew Kelley review
2020-06-27 01:29:06 +00:00
var validationAllocator = mem.validationWrap(base_allocator);
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = validationAllocator.allocator();
new allocator interface
2020-04-17 20:15:36 +00:00
use * for pointer type instead of & See #770 To help automatically translate code, see the zig-fmt-pointer-reform-2 branch. This will convert all & into *. Due to the syntax ambiguity (which is why we are making this change), even address-of & will turn into *, so you'll have to manually fix thes instances. You will be guaranteed to get compile errors for them - expected 'type', found 'foo'
2018-05-31 14:56:59 +00:00
var slice = try allocator.alloc(*i32, 100);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 100);
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
for (slice) |*item, i| {
`std.mem.Allocator.create` replaced with better API `std.mem.Allocator.createOne` is renamed to `std.mem.Allocator.create`. The problem with the previous API is that even after copy elision, the initalization value passed as a parameter would always be a copy. With the new API, once copy elision is done, initialization functions can directly initialize allocated memory in place. Related: * #1872 * #1873
2019-02-03 21:13:28 +00:00
item.* = try allocator.create(i32);
item.*.* = @intCast(i32, i);
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
}
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
slice = try allocator.realloc(slice, 20000);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 20000);
Improve realloc on fixed buffer allocator (#1238) * Add test to check re-use of memory * Check if realloc has to reallocate the last allocated memory block. If so extend that block instead of allocating a new one. * Also check if the realloc actually preserves the data.
2018-07-14 20:31:11 +00:00
for (slice[0..100]) |item, i| {
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(item.* == @intCast(i32, i));
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
allocator.destroy(item);
}
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
slice = allocator.shrink(slice, 50);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 50);
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
slice = allocator.shrink(slice, 25);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 25);
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
slice = allocator.shrink(slice, 0);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 0);
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
slice = try allocator.realloc(slice, 10);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 10);
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
allocator.free(slice);
Test 0 bit allocation
2020-09-03 23:33:47 +00:00
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
// Zero-length allocation
var empty = try allocator.alloc(u8, 0);
allocator.free(empty);
// Allocation with zero-sized types
Test 0 bit allocation
2020-09-03 23:33:47 +00:00
const zero_bit_ptr = try allocator.create(u0);
zero_bit_ptr.* = 0;
allocator.destroy(zero_bit_ptr);
std: ArenaAllocator tries to resize before allocating Closes #5116
2020-09-24 22:21:57 +00:00
const oversize = try allocator.allocAdvanced(u32, null, 5, .at_least);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(oversize.len >= 5);
std: ArenaAllocator tries to resize before allocating Closes #5116
2020-09-24 22:21:57 +00:00
for (oversize) |*item| {
item.* = 0xDEADBEEF;
}
allocator.free(oversize);
introduce std.heap.ArenaAllocator and std.heap.DirectAllocator * DirectAllocator does the underlying syscall for every allocation. * ArenaAllocator takes another allocator as an argument and allocates bytes up front, falling back to DirectAllocator with increasingly large allocation sizes, to avoid calling it too often. Then the entire arena can be freed at once. The self hosted parser is updated to take advantage of ArenaAllocator for the AST that it returns. This significantly reduces the complexity of cleanup code. docgen and build runner are updated to use the combination of ArenaAllocator and DirectAllocator instead of IncrementingAllocator, which is now deprecated in favor of FixedBufferAllocator combined with DirectAllocator. The C allocator calls aligned_alloc instead of malloc, in order to respect the alignment parameter. Added asserts in Allocator to ensure that implementors of the interface return slices of the correct size. Fixed a bug in Allocator when you call realloc to grow the allocation.
2018-02-12 07:14:44 +00:00
}
Added DirectAllocator support for alignments > os.page_size on posix systems
2018-04-22 01:41:49 +00:00
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
pub fn testAllocatorAligned(base_allocator: mem.Allocator) !void {
new allocator interface after Andrew Kelley review
2020-06-27 01:29:06 +00:00
var validationAllocator = mem.validationWrap(base_allocator);
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = validationAllocator.allocator();
new allocator interface
2020-04-17 20:15:36 +00:00
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
// Test a few alignment values, smaller and bigger than the type's one
inline for ([_]u29{ 1, 2, 4, 8, 16, 32, 64 }) |alignment| {
// initial
var slice = try allocator.alignedAlloc(u8, alignment, 10);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 10);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
// grow
slice = try allocator.realloc(slice, 100);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 100);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
// shrink
slice = allocator.shrink(slice, 10);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 10);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
// go to zero
slice = allocator.shrink(slice, 0);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 0);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
// realloc from zero
slice = try allocator.realloc(slice, 100);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 100);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
// shrink with shrink
slice = allocator.shrink(slice, 10);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 10);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
// shrink to zero
slice = allocator.shrink(slice, 0);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice.len == 0);
std: Make C allocator respect the required alignment Use posix_memalign where available and the _aligned_{malloc,free} API on Windows. Closes #3783
2020-09-20 16:54:23 +00:00
}
Create unit test that tests aligned reallocation.
2018-07-14 16:03:06 +00:00
}
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
pub fn testAllocatorLargeAlignment(base_allocator: mem.Allocator) !void {
new allocator interface after Andrew Kelley review
2020-06-27 01:29:06 +00:00
var validationAllocator = mem.validationWrap(base_allocator);
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = validationAllocator.allocator();
new allocator interface
2020-04-17 20:15:36 +00:00
run zig fmt on the codebase See #1003
2018-05-29 00:23:55 +00:00
//Maybe a platform's page_size is actually the same as or
Added DirectAllocator support for alignments > os.page_size on posix systems
2018-04-22 01:41:49 +00:00
// very near usize?
clean up references to os
2019-05-26 17:17:34 +00:00
if (mem.page_size << 2 > maxInt(usize)) return;
behavior tests passing with new pointer deref syntax
2018-05-01 05:53:04 +00:00
Make std.meta.Int accept a signedness parameter
2020-10-17 12:09:59 +00:00
const USizeShift = std.meta.Int(.unsigned, std.math.log2(std.meta.bitCount(usize)));
update more of the std lib to use `@as`
2019-11-07 23:52:09 +00:00
const large_align = @as(u29, mem.page_size << 2);
behavior tests passing with new pointer deref syntax
2018-05-01 05:53:04 +00:00
Added DirectAllocator support for alignments > os.page_size on posix systems
2018-04-22 01:41:49 +00:00
var align_mask: usize = undefined;
update more of the std lib to use `@as`
2019-11-07 23:52:09 +00:00
_ = @shlWithOverflow(usize, ~@as(usize, 0), @as(USizeShift, @ctz(u29, large_align)), &align_mask);
behavior tests passing with new pointer deref syntax
2018-05-01 05:53:04 +00:00
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
var slice = try allocator.alignedAlloc(u8, large_align, 500);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(@ptrToInt(slice.ptr) & align_mask == @ptrToInt(slice.ptr));
behavior tests passing with new pointer deref syntax
2018-05-01 05:53:04 +00:00
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
slice = allocator.shrink(slice, 100);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(@ptrToInt(slice.ptr) & align_mask == @ptrToInt(slice.ptr));
behavior tests passing with new pointer deref syntax
2018-05-01 05:53:04 +00:00
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
slice = try allocator.realloc(slice, 5000);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(@ptrToInt(slice.ptr) & align_mask == @ptrToInt(slice.ptr));
behavior tests passing with new pointer deref syntax
2018-05-01 05:53:04 +00:00
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
slice = allocator.shrink(slice, 10);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(@ptrToInt(slice.ptr) & align_mask == @ptrToInt(slice.ptr));
behavior tests passing with new pointer deref syntax
2018-05-01 05:53:04 +00:00
breaking changes to std.mem.Allocator interface API Before, allocator implementations had to provide `allocFn`, `reallocFn`, and `freeFn`. Now, they must provide only `reallocFn` and `shrinkFn`. Reallocating from a zero length slice is allocation, and shrinking to a zero length slice is freeing. When the new memory size is less than or equal to the previous allocation size, `reallocFn` now has the option to return `error.OutOfMemory` to indicate that the allocator would not be able to take advantage of the new size. For more details see #1306. This commit closes #1306. This commit paves the way to solving #2009. This commit also introduces a memory leak to all coroutines. There is an issue where a coroutine calls the function and it frees its own stack frame, but then the return value of `shrinkFn` is a slice, which is implemented as an sret struct. Writing to the return pointer causes invalid memory write. We could work around it by having a global helper function which has a void return type and calling that instead. But instead this hack will suffice until I rework coroutines to be non-allocating. Basically coroutines are not supported right now until they are reworked as in #1194.
2019-03-15 21:47:47 +00:00
slice = try allocator.realloc(slice, 20000);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(@ptrToInt(slice.ptr) & align_mask == @ptrToInt(slice.ptr));
Added DirectAllocator support for alignments > os.page_size on posix systems
2018-04-22 01:41:49 +00:00
allocator.free(slice);
}
fix heap allocators when shrinking an object but growing its alignment
2019-04-20 00:54:53 +00:00
allocgate: std Allocator interface refactor
2021-10-28 23:37:25 +00:00
pub fn testAllocatorAlignedShrink(base_allocator: mem.Allocator) !void {
new allocator interface after Andrew Kelley review
2020-06-27 01:29:06 +00:00
var validationAllocator = mem.validationWrap(base_allocator);
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const allocator = validationAllocator.allocator();
new allocator interface
2020-04-17 20:15:36 +00:00
fix heap allocators when shrinking an object but growing its alignment
2019-04-20 00:54:53 +00:00
var debug_buffer: [1000]u8 = undefined;
allocgate: renamed getAllocator function to allocator
2021-10-29 01:08:41 +00:00
const debug_allocator = FixedBufferAllocator.init(&debug_buffer).allocator();
fix heap allocators when shrinking an object but growing its alignment
2019-04-20 00:54:53 +00:00
clean up references to os
2019-05-26 17:17:34 +00:00
const alloc_size = mem.page_size * 2 + 50;
fix heap allocators when shrinking an object but growing its alignment
2019-04-20 00:54:53 +00:00
var slice = try allocator.alignedAlloc(u8, 16, alloc_size);
defer allocator.free(slice);
var stuff_to_free = std.ArrayList([]align(16) u8).init(debug_allocator);
Implement Windows' DirectAllocator on top of VirtualAlloc and VirtualFree.
2019-05-08 16:02:37 +00:00
// On Windows, VirtualAlloc returns addresses aligned to a 64K boundary,
// which is 16 pages, hence the 32. This test may require to increase
// the size of the allocations feeding the `allocator` parameter if they
// fail, because of this high over-alignment we want to have.
clean up references to os
2019-05-26 17:17:34 +00:00
while (@ptrToInt(slice.ptr) == mem.alignForward(@ptrToInt(slice.ptr), mem.page_size * 32)) {
fix heap allocators when shrinking an object but growing its alignment
2019-04-20 00:54:53 +00:00
try stuff_to_free.append(slice);
slice = try allocator.alignedAlloc(u8, 16, alloc_size);
}
while (stuff_to_free.popOrNull()) |item| {
allocator.free(item);
}
slice[0] = 0x12;
slice[60] = 0x34;
// realloc to a smaller size but with a larger alignment
std.mem.Allocator: add return_address to the interface The high level Allocator interface API functions will now do a `@returnAddress()` so that stack traces captured by allocator implementations have a return address that does not include the Allocator overhead functions. This makes `4` a more reasonable default for how many stack frames to capture.
2020-08-08 07:34:13 +00:00
slice = try allocator.reallocAdvanced(slice, mem.page_size * 32, alloc_size / 2, .exact);
std: update usage of std.testing
2021-05-04 17:47:26 +00:00
try testing.expect(slice[0] == 0x12);
try testing.expect(slice[60] == 0x34);
fix heap allocators when shrinking an object but growing its alignment
2019-04-20 00:54:53 +00:00
}
std: fix LoggingAllocator, add simple test
2020-01-09 05:37:29 +00:00
test "heap" {
_ = @import("heap/logging_allocator.zig");
Add a logging allocator that uses std.log (#8511)
2021-06-09 18:42:07 +00:00
_ = @import("heap/log_to_writer_allocator.zig");
std: fix LoggingAllocator, add simple test
2020-01-09 05:37:29 +00:00
}
Reference in New Issue Copy Permalink