AndreaOrru/zen 314
Experimental operating system written in Zig
andrewrk/clashos 193
multiplayer arcade game for bare metal Raspberry Pi 3 B+
andrewrk/chem 176
2d canvas-based rapid prototyping game engine
connect middleware for server sent events (EventSource)
static file server middleware for connect. loads files once at startup and saves gzipped versions in memory
linux yo
browserify, minus some of the advanced features and heavy dependencies
andrewrk/evo 14
specification, reference implementation, and examples of Evo, the programming language made for being the DNA of genetic algorithms
connect middleware to insert no cache headers
my personal site
issue commentziglang/zig
Consider running tests in parallel by deafult
I think this could make a lot of sense especially given that the build runner has become a sort of "driver" for the test runner recently:
https://github.com/ziglang/zig/blob/7e0a02ee2593d99236bc9a1ff98a8f957add6ec4/lib/std/Build/Step/Run.zig#L936-L1071
One really hard problem here is what to do with test output (logging or direct use of stderr). If you run a bunch of tests in parallel, and something gets printing to stderr, how do you know which test is responsible? As far as I know, this problem isn't really solvable, and, to get test output, the tests should be run sequentially.
This could be solved in a related way as #1356 which is to execute each test in parallel in a different sub-process. This gives up a bit of efficiency, especially on Windows where spawning child processes is more expensive, but it does enable excellent error reporting, and enable the use case of testing that an expected safety check indeed was triggered.
comment created time in a day
push eventziglang/zig
commit sha 6eee16c513b120c9b22d2ffabefa17e134eb623a
Module: remove incorrect assertion in initNewAnonDecl This assertion is *almost* correct, but there are many parts of Sema which call initNewAnonDecl with a `val` which is initially `undefined` (the idea being that the value is set later on). That was causing this assertion to trigger a crash when bootstrapping the compiler.
commit sha 0088667d4101bbe5c91b1a56cc315d83480faab4
Sema: fix crashes accessing undefined values
commit sha 7c86b48269b6056064c3903e96306b880fff9dc7
wasm: address behavior test regressions
push time in a day
PR closed ziglang/zig
pr closed time in 2 days
PR opened ziglang/zig
pr created time in 2 days
push eventziglang/zig
commit sha bda645d911fd79c5aba1f93c0877c7eb915ed709
std.mem: Split `split` and `splitBackwards` into 3 versions by delimiter type: full, any, and scalar This allows users to choose which version they need for their particular use case, as the previous default (now the 'full' version) was (1) not always the desired type of delimiter and (2) performed worse than the scalar version if the delimiter was a single item.
commit sha 9da3a9733db037d8391eb2b4ccb6c7db774a5dd5
std.mem: Split `tokenize` into 3 versions by delimiter type: full, any, and scalar This allows users to choose which version they need for their particular use case, as the previous default (now the 'any' version) was (1) not always the desired type of delimiter and (2) performed worse than the scalar version if the delimiter was a single item.
commit sha ce9f3ec990cd556f2a9d06a6db2bb53e97a61172
Fix SplitIterator and TokenIterator type instantiation
commit sha 815e53b147a321d0bdb47dc008aa8181f57175ac
Update all std.mem.tokenize calls to their appropriate function Everywhere that can now use `tokenizeScalar` should get a nice little performance boost.
commit sha 2129f28953b72da2f1bb58ff063a044d737c59c4
Update all std.mem.split calls to their appropriate function Everywhere that can now use `splitScalar` should get a nice little performance boost.
commit sha 104f4053a2c3c6a1a2bf801ca5bf88ce4fee7a2a
std.mem: Rename splitFull/tokenizeFull to splitSequence/tokenizeSequence I think this makes the name less ambiguous and more obvious that the suffix applies to the `delimiter`.
commit sha b91d6ff9e8d3b224d8cfa4ce7f206264fe814f6f
add runtime safety for noreturn function returning Closes #15221
commit sha fdf1675c58f029bc08e2aebdad06a5ef5924e81d
Sema: avoid safety slice safety check with comptime-known start and end Oopsie
commit sha ebbc521a8742fe264bf9738082ce1e6b1894e04d
workaround AstGen's love for copying arrays
commit sha f65e8c78621c90c9b9932903a9ed99c973dbcf63
Deduplicate uses of the same package across dependencies
commit sha db7496d6efb64d8210816e49008753b5a81d46f4
Only add build.zig module dependencies once
commit sha eb77e3381fac5f5bba2254cf5a7369aaea930e4e
wasm: implement `@frameAddress`
commit sha 8236a26c605c8c4276f2062b3b6f55497cd45e53
wasm: implement mul, shl and xor for big ints Uses compiler-rt for multiplication and shifting left, while lowers it down using regular instructions for xor.
commit sha 992de8e61718a1a77666e473dc37872afbb80c98
wasm: implement `@addWithOverflow` for 64bit ints
commit sha d353d208e295a01d6f844ccdb7e641a94e6fcb11
wasm: implement `@mulWithOverflow` for big ints Currently we only support exact 128 bit *unsigned* integers
commit sha e20976b7f209a768cb55a37e6a58ed177d76013e
wasm: fix miscompilation for shifting This fix ensures that when we are shifting left or right, both operands have the same WebAssembly type. e.g. it's not possible to shift a 64 bit integer and 32 bit integer together and will fail WebAssembly's validator. By first coercing the values to the same type, we ensure we satisfy the validator.
commit sha 67d27dbe631d1292be363f4121217d66e2d7fd0f
wasm: fix liveness bugs Make sure to increase the reference count for `intcast` when the operand doesn't require any casting of the respective WebAssembly type. Function arguments have a reserved slot, and therefore cannot be re-used arbitrarily
commit sha f2860bb4f40565e43f51757e6cb604bb2df16ae0
wasm: more liveness fixes
commit sha 99422cb5284f3e15c1b5a8598a6b1622c0e7b6ca
wasm: add `dead` tag to `WValue` This new tag is used for freed locals that are not allowed to have any remaining references pointing to it. This new tag allows us to easily identify liveness bugs. Previously we would set the entire region to `undefined` which would incorrectly set the tag to `function_index`, making codegen think it was a valid `WValue` while it wasn't.
commit sha 8be69f41328ebc0331434fd9d4008985463188c9
wasm: simplify merging of branches Rather than adding all values that were generated in the child branch, we simply discard them as outer branches cannot refer to values produced from an inner branch.
push time in 2 days
issue closedziglang/zig
Unreachable code reached when using UNC paths on Windows
Using absolute paths related to network shares (starting with \\...) will result in stdlib reaching unreachable, meaning it's impossible to recover from. Preferred would be if it spat an error such as error.UnsupportedDrive so it can be handled gracefully, or if support for them was added.
Note that getting the working dir handle from std.fs.cwd() whilst on a network share works, and you can use openDir, openFile, and all other relative methods from there. Meaning if you've somehow gotten a handle, you can access and read/write files. Afaik it only breaks when you try to use absolute paths.
Getting working dir from std.fs.cwd() whilst on a network share and then running realpath on the handle also breaks with unreachable.
I've only experienced this because I've done development in WSL2, which Windows has mounted as a network share.
Example:
const std = @import("std");
pub fn main() !void {
const path = "\\\\wsl$\\Arch\\home\\jerwuqu";
std.debug.print("Is absolute: {}\n", .{std.fs.path.isAbsolute(path)});
_ = try std.fs.openDirAbsolute(path, .{});
std.debug.print("OK!\n", .{});
}
Output:
Is absolute: true
thread 19092 panic: reached unreachable code
Panicked during a panic. Aborting.
closed time in 2 days
JerwuQu
push eventziglang/zig
commit sha bda645d911fd79c5aba1f93c0877c7eb915ed709
std.mem: Split `split` and `splitBackwards` into 3 versions by delimiter type: full, any, and scalar This allows users to choose which version they need for their particular use case, as the previous default (now the 'full' version) was (1) not always the desired type of delimiter and (2) performed worse than the scalar version if the delimiter was a single item.
commit sha 9da3a9733db037d8391eb2b4ccb6c7db774a5dd5
std.mem: Split `tokenize` into 3 versions by delimiter type: full, any, and scalar This allows users to choose which version they need for their particular use case, as the previous default (now the 'any' version) was (1) not always the desired type of delimiter and (2) performed worse than the scalar version if the delimiter was a single item.
commit sha ce9f3ec990cd556f2a9d06a6db2bb53e97a61172
Fix SplitIterator and TokenIterator type instantiation
commit sha 815e53b147a321d0bdb47dc008aa8181f57175ac
Update all std.mem.tokenize calls to their appropriate function Everywhere that can now use `tokenizeScalar` should get a nice little performance boost.
commit sha 2129f28953b72da2f1bb58ff063a044d737c59c4
Update all std.mem.split calls to their appropriate function Everywhere that can now use `splitScalar` should get a nice little performance boost.
commit sha 104f4053a2c3c6a1a2bf801ca5bf88ce4fee7a2a
std.mem: Rename splitFull/tokenizeFull to splitSequence/tokenizeSequence I think this makes the name less ambiguous and more obvious that the suffix applies to the `delimiter`.
commit sha 629f0d23b5c0768b5957688591f6fa6216ae4dd3
Merge pull request #15579 from squeek502/mem-delimiters Split `std.mem.split` and `tokenize` into `sequence`, `any`, and `scalar` versions
push time in 2 days
PR merged ziglang/zig
- This allows users to choose which version they need for their particular use case, as the previous default (now the 'sequence' version for split, and the 'any' version for tokenize) was (1) not always the desired type of delimiter and (2) performed worse than the scalar version if the delimiter was a single item.
std.mem.splitis now an alias forstd.mem.splitSequenceandstd.mem.tokenizeis now an alias forstd.mem.tokenizeAnywith a deprecation doc comment on each ofsplitandtokenize- Everywhere that can now use the 'scalar' version should get a nice little performance boost. EDIT: The performance boost might only apply to
split, didn't check the performance difference oftokenizeAnyversustokenizeScalar
Any suggestions for a better name for the Full version are welcome. EDIT: Maybe Sequence? EDIT#2: Went with Sequence for now
pr closed time in 2 days
push eventziglang/zig
commit sha 3add9d8257d9414421acf91823917d9d49b28c6f
std.c: fix return type of recv/recvfrom on windows Windows defines `recv` and `recvfrom` to return a value of type `int`, rather than a pointer-sized signed integer, and so should use `c_int` rather than `isize` for their return types.
push time in 2 days
PR merged ziglang/zig
Windows defines recv and recvfrom to return a value of type int, rather than a pointer-sized signed integer, and so should use c_int rather than isize for their return types.
This would cause a bug where windows may return -1 to indicate an error, but std.os.recv will not recognize it as an error, rather interpretting it as (and returning) 2^32 - 1, indicating that 2^32 - 1 bytes were read into the buffer. I'm unsure if this is UB as it depends on whatever the top 32 bits of the isize end up being, but this behaviour seemed reliable in debug builds.
Here are the docs for recv and recvfrom, the headers containing the declarations are in the headers lib/libc/include/any-windows-any/winsock.h and lib/libc/include/any-windows-any/winsock2.h.
pr closed time in 2 days
issue closedziglang/zig
`ReleaseSmall` builds fail with `FileNotFound` error
Zig Version
0.11.0-dev.5+ddb9eac05
Steps to Reproduce and Observed Behavior
I'm getting a FileNotFound error when building any projects with -Drelease-small on Windows, both with -fstage1 and -fnostage1. The 0.10 release notes mention -fstrip should be the default, but interestingly enough, adding exe.strip = true; to build.zig seems to fix the issue.
❯ zig init-exe
info: Created build.zig
info: Created src\main.zig
info: Next, try `zig build --help` or `zig build run`
❯ zig build run
All your codebase are belong to us.
Run `zig build test` to run the tests.
❯ zig build -Drelease-small
error: FileNotFound
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\os\windows.zig:126:35: 0x7ff71b062809 in OpenFile (build.exe.obj)
.OBJECT_NAME_NOT_FOUND => return error.FileNotFound,
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\fs.zig:1224:23: 0x7ff71b065eb3 in openFileW (build.exe.obj)
.handle = try w.OpenFile(sub_path_w, .{
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\fs.zig:1096:13: 0x7ff71b04324e in openFile (build.exe.obj)
return self.openFileW(path_w.span(), flags);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\fs.zig:2532:24: 0x7ff71b0c6392 in updateFile (build.exe.obj)
var src_file = try source_dir.openFile(source_path, .{});
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:1089:29: 0x7ff71b0db14d in updateFile (build.exe.obj)
const prev_status = try fs.Dir.updateFile(cwd, source_path, cwd, dest_path, .{});
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:3427:13: 0x7ff71b0bd448 in make (build.exe.obj)
try builder.updateFile(self.artifact.getOutputPdbSource().getPath(builder), full_pdb_path);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:3649:9: 0x7ff71b02e449 in make (build.exe.obj)
try self.makeFn(self);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:509:9: 0x7ff71b01c76c in makeOneStep (build.exe.obj)
try s.make();
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:503:17: 0x7ff71b01c6e5 in makeOneStep (build.exe.obj)
return err;
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:464:13: 0x7ff71b01c438 in make (build.exe.obj)
try self.makeOneStep(s);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\build_runner.zig:223:21: 0x7ff71b01f3aa in main (build.exe.obj)
else => return err,
^
error: the following build command failed with exit code 1:
F:\projects\repros\zig-release-small\zig-cache\o\34a1f8d987ce9d091d0e0b5a87f3042a\build.exe C:\Users\mrkishi\scoop\apps\zig-dev\current\zig.exe F:\projects\repros\zig-release-small F:\projects\repros\zig-release-small\zig-cache C:\Users\mrkishi\AppData\Local\zig -Drelease-small
❯ zig build -Drelease-small -fstage1
error: FileNotFound
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\os\windows.zig:134:35: 0x7ff6b140285e in std.os.windows.OpenFile (build.obj)
.OBJECT_NAME_COLLISION => return error.PathAlreadyExists,
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\os.zig:2833:21: 0x7ff6b13feaf9 in std.os.mkdiratW (build.obj)
else => |e| return e,
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\os.zig:2756:9: 0x7ff6b13fea2b in std.os.mkdirat (build.obj)
return mkdiratW(dir_fd, sub_dir_path_w.span(), mode);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\fs.zig:1426:9: 0x7ff6b13fe94a in std.fs.Dir::std.fs.Dir.makeDir (build.obj)
try os.mkdirat(self.fd, sub_path, default_new_dir_mode);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\os\windows.zig:134:35: 0x7ff6b140285e in std.os.windows.OpenFile (build.obj)
.OBJECT_NAME_COLLISION => return error.PathAlreadyExists,
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\os.zig:2833:21: 0x7ff6b13feaf9 in std.os.mkdiratW (build.obj)
else => |e| return e,
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\os.zig:2756:9: 0x7ff6b13fea2b in std.os.mkdirat (build.obj)
return mkdiratW(dir_fd, sub_dir_path_w.span(), mode);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\fs.zig:1426:9: 0x7ff6b13fe94a in std.fs.Dir::std.fs.Dir.makeDir (build.obj)
try os.mkdirat(self.fd, sub_path, default_new_dir_mode);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\os\windows.zig:126:35: 0x7ff6b1402730 in std.os.windows.OpenFile (build.obj)
.OBJECT_NAME_NOT_FOUND => return error.FileNotFound,
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\fs.zig:1224:23: 0x7ff6b13bf8cf in std.fs.Dir::std.fs.Dir.openFileW (build.obj)
.handle = try w.OpenFile(sub_path_w, .{
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\fs.zig:1096:13: 0x7ff6b13dc1aa in std.fs.Dir::std.fs.Dir.openFile (build.obj)
return self.openFileW(path_w.span(), flags);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\fs.zig:2532:24: 0x7ff6b1470a2a in std.fs.Dir::std.fs.Dir.updateFile (build.obj)
var src_file = try source_dir.openFile(source_path, .{});
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:1089:29: 0x7ff6b147e55e in std.build.Builder::std.build.Builder.updateFile (build.obj)
const prev_status = try fs.Dir.updateFile(cwd, source_path, cwd, dest_path, .{});
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:3427:13: 0x7ff6b146524f in std.build.InstallArtifactStep::std.build.InstallArtifactStep.make (build.obj)
try builder.updateFile(self.artifact.getOutputPdbSource().getPath(builder), full_pdb_path);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:3649:9: 0x7ff6b13ff070 in std.build.Step::std.build.Step.make (build.obj)
try self.makeFn(self);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:509:9: 0x7ff6b13fe481 in std.build.Builder::std.build.Builder.makeOneStep (build.obj)
try s.make();
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:503:17: 0x7ff6b13fe42d in std.build.Builder::std.build.Builder.makeOneStep (build.obj)
return err;
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\std\build.zig:464:13: 0x7ff6b13f2dd1 in std.build.Builder::std.build.Builder.make (build.obj)
try self.makeOneStep(s);
^
C:\Users\mrkishi\scoop\apps\zig-dev\current\lib\build_runner.zig:223:21: 0x7ff6b13b80c7 in main (build.obj)
else => return err,
^
error: the following build command failed with exit code 1:
F:\projects\repros\zig-release-small\zig-cache\o\fe30788cd52cfff3817985b5c5b11dbb\build.exe C:\Users\mrkishi\scoop\apps\zig-dev\current\zig.exe F:\projects\repros\zig-release-small F:\projects\repros\zig-release-small\zig-cache C:\Users\mrkishi\AppData\Local\zig -Drelease-small -fstage1 -fstage1
Expected Behavior
A successful build! :P
closed time in 2 days
mrkishi
push eventziglang/zig
commit sha ff57a264ad99562de16d9565d5bd4ac9cd964e5d
Build: fix producesPdbFile logic (#15756) Fixes bug causing ReleaseSmall to fail on Windows. Due to the change in default behavior of ReleaseSmall, debug info are stripped by default. However because `Compile.create` still defaults to null, `producesPdbFile` will report true for `lib/std/Build/Step/InstallArtifact.zig` causing it to fail on copying a file that does not exist. This commit change the default of strip depending on `optimize`.
push time in 2 days
PR merged ziglang/zig
closes: #13405
Due to the change in default behavior of ReleaseSmall (#13067), debug info (pdb here) are stripped by default. However because Compile.create defaults to null, producesPdbFile will report true for
lib/std/Build/Step/InstallArtifact.zig
https://github.com/ziglang/zig/blob/378264d404e71da878f9e6934c045ad64193877f/lib/std/Build/Step/InstallArtifact.zig#L33
Because of this, this only affects zig build and not zig build-exe. After this change, a simple program compiled with ReleaseSmall will successfully compile and won't try to copy/produce the pdb file.
Of course, this still allows the user to choose not to strip the build and will produce a pdb file.
pr closed time in 2 days
push eventziglang/zig
commit sha 105078519a831878066fa995749d4b918d940b2c
fix missing insertion of module to all_modules on first download
push time in 2 days
PR merged ziglang/zig
fixes an issue where multiple dependencies have a common module will crash on download because all_modules do not add the package unless it is cached ->
thread 15204 panic: reached unreachable code
C:\Git\zigcode\zig\lib\std\debug.zig:260:14: 0x7ff622d5e847 in assert (zig.exe.obj)
if (!ok) unreachable; // assertion failure
^
C:\Git\zigcode\zig\src\Package.zig:297:36: 0x7ff622f419bd in fetchAndAddDependencies (zig.exe.obj)
assert(other_sub == sub.mod);
^
C:\Git\zigcode\zig\src\main.zig:4412:67: 0x7ff622d905ae in cmdBuild (zig.exe.obj)
const fetch_result = build_pkg.fetchAndAddDependencies(
pr closed time in 2 days
issue closedziglang/zig
Replace `readUntilDelimiter*` with `streamUntilDelimiter`
readUntilDelimiterArrayList is great if you want to read delimited data into a dynamically allocated buffer. However, sometimes you may want to read delimited data into a BoundedArray (for allocation-free code, or when you know the max size will be small), or send it directly to a writer (eg. for converting a newline-separated stream into a NUL-separated stream or similar).
These usecases are possible today, but they're a bit awkward and don't work as well as they should:
const delimiter = '\n';
// Read into BoundedArray
// Doesn't properly handle EndOfStream like readUntilDelimiterArrayList does
var buf = std.BoundedArray(u8, 4096);
buf.len = (try reader.readUntilDelimiter(buf.buffer, delimiter)).len;
// Read into writer
// Literally just an adaptation of what readUntilDelimiterArrayList does
const writer = std.io.getStdOut().writer();
while (true) {
const byte = try reader.readByte();
if (byte == delimiter) break;
try writer.writeByte(byte);
}
I propose to replace readUntilDelimiter, readUntilDelimiterOrEof, and readUntilDelimiterArrayList with a new function, streamUntilDelimiter (name up for discussion, I don't hugely like this one but can't think of anything better):
// Read into ArrayList
var array = std.ArrayList(u8).init(allocator);
try reader.streamUntilDelimiter(array.writer(), delimiter, max_size);
// Read into BoundedArray
var buf = std.BoundedArray(u8, 4096);
try reader.streamUntilDelimiter(buf.writer(), delimiter, null); // null max_size = unbounded
// Read into writer
const writer = std.io.getStdOut().writer();
try reader.streamUntilDelimiter(writer, delimiter, null);
I would recommend removing readUntilDelimiterAlloc and readUntilDelimiterOrEofAlloc, since they're already pretty much superseded by readUntilDelimiterArrayList which is never slower, often faster, and rarely significantly harder to use.
closed time in 2 days
silversquirl
push eventziglang/zig
commit sha 5c6f111379d81cf017c8c6eaa4c7c76632acf4d6
std.io.reader.Reader: add `streamUntilDelimiter` Signed-off-by: Eric Joldasov <bratishkaerik@getgoogleoff.me>
push time in 2 days
PR merged ziglang/zig
Resolves https://github.com/ziglang/zig/issues/15528
pr closed time in 2 days
issue closedziglang/zig
Taking reference to `errdefer` capture payload crashes compiler
Zig Version
0.11.0-dev.3289+16dbb960f
Steps to Reproduce and Observed Behavior
test {
errdefer |x| {
@import("std").mem.doNotOptimizeAway(&x);
}
return error.Foo;
}
Segfaults.
Expected Behavior
Successful compilation.
This is caused by AstGen not handling ref_table for the errdefer_err_code instruction created by errdefer captures; fix will be fairly simple, just filing this so I don't forget about it.
closed time in 2 days
mlugg
push eventziglang/zig
commit sha 1a4b0d979027069d55ef354e6d04ea10cf455344
AstGen: handle ref_table for errdefer captures Resolves: #15861
push time in 2 days
issue closedziglang/zig
`std.http.Client` hangs indefinitely on response body sizes > 4096 bytes
Zig Version
0.11.0-dev.3107+6547d2331
Steps to Reproduce and Observed Behavior
Make a https request using the std.http.Client and set transfer_encoding to .chunked on the request like so:
var client: http.Client = .{ .allocator = allocator };
var req = try client.request(http.Method.POST, http_endpoint, default_http_headers, .{});
req.transfer_encoding = .chunked;
defer req.deinit();
try req.start();
try req.writer().writeAll(reqBody); // reqBody is just []u8
try req.finish();
try req.wait();
if (req.response.status != http.Status.ok) {
return Error.ResponseNotStatusOk;
}
const respBody = try req.reader().readAllAlloc(self.allocator, 12 * 1024 * 1024); // 12Mb max size set
defer self.allocator.free(body);
// do something with respBody
Once you surpass a certain body size (greater than 4096 i believe), it will remain stuck in an infinite loop. The response body size of the request which causes this issue is much smaller than 12Mb - it's around 3.4Mb. If I make request under 4Kb it seems to work fine.
I tracked the bug to BufferedConnection.readAtLeast and it seems it's stuck in the while (out_index < len) loop.
I logged values of the vars that affect this loop like so:
pub fn readAtLeast(bconn: *BufferedConnection, buffer: []u8, len: usize) ReadError!usize {
var out_index: u16 = 0;
while (out_index < len) {
std.log.debug("readAtLeast.while - begin", .{});
const available = bconn.read_end - bconn.read_start;
const left = buffer.len - out_index;
if (available > 0) {
const can_read = @intCast(u16, @min(available, left));
std.log.debug("readAtLeast.while - available: {}, left: {}, can_read: {}", .{ available, left, can_read });
@memcpy(buffer[out_index..][0..can_read], bconn.read_buf[bconn.read_start..][0..can_read]);
out_index += can_read;
bconn.read_start += can_read;
continue;
}
std.log.debug("readAtLeast.while - passed continue", .{});
if (left > bconn.read_buf.len) {
// skip the buffer if the output is large enough
return bconn.conn.read(buffer[out_index..]);
}
try bconn.fill();
std.log.debug("readAtLeast.while - end", .{});
}
std.log.debug("readAtLeast - returning", .{});
return out_index;
}
and it produces these logs continuously:
...
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
debug: readAtLeast.while - begin
debug: readAtLeast.while - available: 12536, left: 0, can_read: 0
...
Also, sometimes it doesn't get stuck in loop and returns this error instead (~20% of the time):
thread 1290939 panic: @memcpy arguments have non-equal lengths
/Users/bkk/bin/zig/lib/std/crypto/tls/Client.zig:1062:25: 0x104c0671b in readvAdvanced__anon_13130 (main)
@memcpy(frag[0..in], first);
^
/Users/bkk/bin/zig/lib/std/crypto/tls/Client.zig:898:53: 0x104c046cf in readvAtLeast__anon_13129 (main)
var amt = try c.readvAdvanced(stream, iovecs[vec_i..]);
^
/Users/bkk/bin/zig/lib/std/crypto/tls/Client.zig:859:24: 0x104c044b3 in readAtLeast__anon_13127 (main)
return readvAtLeast(c, stream, &iovecs, len);
^
/Users/bkk/bin/zig/lib/std/crypto/tls/Client.zig:864:23: 0x104c043d3 in read__anon_13126 (main)
return readAtLeast(c, stream, buffer, 1);
^
/Users/bkk/bin/zig/lib/std/http/Client.zig:173:46: 0x104bc9e77 in read (main)
.tls => conn.tls_client.read(conn.stream, buffer),
^
/Users/bkk/bin/zig/lib/std/http/Client.zig:267:57: 0x104b7966b in fill (main)
const nread = try bconn.conn.read(bconn.read_buf[0..]);
^
/Users/bkk/bin/zig/lib/std/http/protocol.zig:553:35: 0x104b78f9f in read__anon_7953 (main)
try bconn.fill();
^
/Users/bkk/bin/zig/lib/std/http/Client.zig:686:111: 0x104b7a87b in transferRead (main)
const amt = try req.response.parser.read(&req.connection.data.buffered, buf[index..], req.response.skip);
^
/Users/bkk/bin/zig/lib/std/io/reader.zig:27:31: 0x104bdee07 in read (main)
return readFn(self.context, buffer);
^
/Users/bkk/bin/zig/lib/std/io/reader.zig:243:50: 0x104d1240b in readByte (main)
const amt_read = try self.read(result[0..]);
^
/Users/bkk/bin/zig/lib/std/compress/deflate/decompressor.zig:822:47: 0x104bdb8d7 in moreBits (main)
var c = self.inner_reader.readByte() catch |e| {
^
/Users/bkk/bin/zig/lib/std/compress/deflate/decompressor.zig:412:30: 0x104b8443f in nextBlock (main)
self.moreBits() catch |e| {
^
/Users/bkk/bin/zig/lib/std/compress/deflate/decompressor.zig:471:26: 0x104be05db in read (main)
self.step(self) catch |e| {
^
/Users/bkk/bin/zig/lib/std/compress/gzip.zig:128:45: 0x104b88573 in read (main)
const r = try self.inflater.read(buffer);
^
/Users/bkk/bin/zig/lib/std/http/Client.zig:814:39: 0x104b87c17 in read (main)
.gzip => |*gzip| gzip.read(buffer) catch return error.DecompressionFailure,
^
/Users/bkk/bin/zig/lib/std/io/reader.zig:27:31: 0x104be46b3 in read (main)
return readFn(self.context, buffer);
^
/Users/bkk/bin/zig/lib/std/io/reader.zig:46:49: 0x104b8929f in readAtLeast (main)
const amt = try self.read(buffer[index..]);
^
/Users/bkk/bin/zig/lib/std/io/reader.zig:34:52: 0x104b69727 in readAll (main)
return readAtLeast(self, buffer, buffer.len);
^
/Users/bkk/bin/zig/lib/std/io/reader.zig:80:52: 0x104b6913b in readAllArrayListAligned__anon_6214 (main)
const bytes_read = try self.readAll(dest_slice);
^
/Users/bkk/bin/zig/lib/std/io/reader.zig:65:48: 0x104b68f03 in readAllArrayList (main)
return self.readAllArrayListAligned(null, array_list, max_append_size);
^
/Users/bkk/bin/zig/lib/std/io/reader.zig:105:38: 0x104b698e3 in readAllAlloc (main)
try self.readAllArrayList(&array_list, max_size);
Expected Behavior
It should not hang indefinitely.
closed time in 2 days
ultd
push eventziglang/zig
commit sha 8136123aa7cdd6d53c682572405eb6c1d5e0f0a0
std.http.Client: collapse BufferedConnection into Connection
commit sha 0e5e6cb10c12a5ae9fd83f85a82eafbe5eac1106
std.http: add TlsAlert descriptions so that they can at least be viewed in err return traces
commit sha 23ccff9cce2a5264fc84998bd2c897682ac266ea
std.http.Server: collapse BufferedConnection into Connection
commit sha 77b40d6ecb8d04bd9d8b95b04b8ba6ce3a6ea604
Merge pull request #15927 from truemedian/http-bugs std.http: fix infinite read loop, deduplicate connection code, add TlsAlert errors
push time in 2 days
PR merged ziglang/zig
Collapses BufferedConnection and StoredConnection into the existing Connection struct.
Adds errors for the different tls alerts so that an error return trace can point out which problem tls hit specifically.
Should fix #15710 and #15902
pr closed time in 2 days
push eventziglang/zig
commit sha bffa88a3d2ee876be546e7a9286bb6ee89dcf400
stage2: pass most test cases under InternPool All but 2 test cases now pass (tested on x86_64 Linux, native only). The remaining two signify an issue requiring a larger refactor, which I will do in a separate commit. Notable changes: * Fix uninitialized memory when allocating objects from free lists * Implement TypedValue printing for pointers * Fix some TypedValue printing logic * Work around non-existence of InternPool.remove implementation
commit sha a53f5228af50c05727a5518501f2140d02084e91
Sema: fix int arithmetic overflow checks Previously, these checks worked by performing the arithmetic operation, then checking whether the result fit in the type in question. Since all values are now typed, this approach was no longer valid, and was tripping some assertions due to trying to store too-large values in smaller types. Now, `intAdd`, `intSub`, `intMul` and `intDiv` all check for overflow, and if it happens, re-do the operation with the result being a `comptime_int`, and reporting the error (and vector index) to the caller so that the error can be reported. After this change, all test cases are passing.
push time in 2 days
push eventziglang/zig
commit sha c4c0a353efcf45bccd036e9b6f48947d44d6e987
std.hash: improve small-key hashing in Wyhash Instead of carrying an optimized version of wyhash in the compiler for small keys, put it into the std lib where it belongs. ...except it does not match the official test cases. This will need to be fixed before merging into master. This is an extremely contributor-friendly task. Related issue: #15916
push time in 3 days
push eventziglang/zig
commit sha 1e0ff37fc9a7b7f030a125cd0ab8a68bcd649b8c
InternPool: fix yet more key lifetime issues
commit sha cef7caabbac8547a28a95349b669c54aef357f82
Sema: reword compile error about LLVM extensions and C import
push time in 3 days
push eventziglang/zig
commit sha 08a353c41a05ffccf0d388f5955529738036a358
llvm: fix undefined pointer type
commit sha ac48ee02498a85a60d37ad667ea6793aa569fca5
InternPool: fix more key lifetime issues
commit sha cd8830b6a1188c8dc66e234184f5d88575956eea
InternPool: add optional coercion
commit sha 6b3c304f0cbc546568d3e108aa114aa364ae11ef
Sema: rewrite `monomorphed_funcs` usage In an effort to delete `Value.hashUncoerced`, generic instantiation has been redesigned. Instead of just storing instantiations in `monomorphed_funcs`, partially instantiated generic argument types are also cached. This isn't quite the single `getOrPut` that it used to be, but one `get` per generic argument plus one get for the instantiation, with an equal number of `put`s per unique instantiation isn't bad.
commit sha f09398bf2ef45b0a5ad624421156324a8592f344
Sema: hack around UAF
commit sha cf31932c3d4946c9b288094a1f244014d7eb0a99
Sema: remove leftover references to value_arena Notably, there was a bug where the fields of reified structs and unions were allocated into an arena which was leaked. These are now in the Module.tmp_hack_arena.
commit sha 5255c681a468a72968a872b6713b9815887d0994
InternPool: avoid aggregate null bytes storage This is a workaround for InternPool currently not handling non-null-terminated strings. It avoids using the `bytes` storage for aggregates if there are any null bytes. In the future this should be changed so that the `bytes` storage can be used regardless of whether there are any null bytes. This is important for use cases such as `@embedFile`. However, this fixes a bug for now, and after this commit, stage2 self-hosts again. mlugg: stage5 passes all enabled behavior tests on my system. Commit message edited by Andrew Kelley <andrew@ziglang.org>
commit sha e2f1d3c976c1e4dd6aa389f751aab53e139277a5
InternPool: optimize previous fix Just because we can't dedup, doesn't mean we can't use `string_bytes`.
commit sha 7bb8987503c64253f27a5f7024ef913a3b6aa85c
Sema: handle generic types when coercing functions in memory This used to be handled by `Type.eql`, but that is now a single comparison.
push time in 3 days