Commit graph

1007 commits

Author SHA1 Message Date
Justine Tunney 18bb5888e1
Make more fixes and improvements
- Remove PAGESIZE constant
- Fix realloc() documentation
- Fix ttyname_r() error reporting
- Make forking more reliable on Windows
- Make execvp() a few microseconds faster
- Make system() a few microseconds faster
- Tighten up the socket-related magic numbers
- Loosen restrictions on mmap() offset alignment
- Improve GetProgramExecutableName() with getenv("_")
- Use mkstemp() as basis for mktemp(), tmpfile(), tmpfd()
- Fix flakes in pthread_cancel_test, unix_test, fork_test
- Fix recently introduced futex stack overflow regression
- Let sockets be passed as stdio to subprocesses on Windows
- Improve security of bind() on Windows w/ SO_EXCLUSIVEADDRUSE
2023-07-29 18:44:15 -07:00
Justine Tunney 7926aa8bfa
Remove ELF binaries from tests 2023-07-28 07:20:57 -07:00
Justine Tunney 5018171fa5
Fix a bunch of Windows bugs reported on Discord
This change addresses everything from stack smashing to %SYSTEMROOT%
breaking socket(). Issues relating to compile.com not reporting text
printed to stderr has been resolved for Windows builds.
2023-07-28 06:17:34 -07:00
Justine Tunney 7e0a09feec
Mint APE Loader v1.5
This change ports APE Loader to Linux AARCH64, so that Raspberry Pi
users can run programs like redbean, without the executable needing
to modify itself. Progress has also slipped into this change on the
issue of making progress better conforming to user expectations and
industry standards regarding which symbols we're allowed to declare
2023-07-26 13:54:49 -07:00
Justine Tunney 6843150e0c
Mint APE Loader v1.4
This change also incorporates more bug fixes and improvements to a wide
variety of small things. For example this fixes #860 so Windows console
doesn't get corrupted after exit. An system stack memory map issue with
aarch64 has been fixed. We no longer use O_NONBLOCK on AF_UNIX sockets.
Crash reports on Arm64 will now demangle C++ symbols, even when c++filt
isn't available. Most importantly the Apple M1 version of APE Loader is
brought up to date by this change. A prebuilt unsigned binary for it is
being included in build/bootstrap/. One more thing: retrieving the term
dimensions under --strace was causing the stack to become corrupted and
now that's been solved too. PSS: We're now including an ELF PT_NOTE for
APE in the binaries we build, that has the APE Loader version.
2023-07-25 05:48:08 -07:00
Justine Tunney e0c2b91b3e
Remove _Hide keyword
It never did anything and isn't worthwhile as documentation.
2023-07-24 08:34:58 -07:00
Justine Tunney 94ea34367a
Tune the page sizes 2023-07-24 00:49:06 -07:00
Justine Tunney 3d172c99fe
Mint APE Loader v1.3
This version has better error messages and safety checks. It supports
loading static position-independent executables. It correctly handles
more kinds of weird ELF program header layouts. A force flag has been
added to avoid system execve(). Finally the longstanding misalignment
with our ELF PT_NOTE section has been addressed.
2023-07-23 17:08:14 -07:00
Danny Robinson ac92f25296
Update definitions.lua with latest progress (#852)
* Update definitions.lua with latest progress

* Remove duplicate definition
2023-07-23 11:14:55 -07:00
tkchia 0ffc0dd461
Let lz4toasm accept LZ4 files that lack extracted-size field (#858)
lz4toasm should now more easily accept LZ4 files output by
compressor programs that do not support the extracted-size
field, such as Stephan Brumme's smallz4.

This patch also proposes to add a new lz4len() function to
the libc: it parses an LZ4 compressed block to compute the
unpacked content size, without really unpacking the block.

Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2023-07-23 11:12:22 -07:00
Justine Tunney 18536950b3
Fix the build 2023-07-11 05:48:39 -07:00
Justine Tunney 1ee2e89326
Make improvements
- This commit mints a new release of APE Loader v1.2 which supports
  loading ELF programs with a non-contiguous virtual address layout
  even though we've never been able to take advantage of it, due to
  how `objcopy -SO binary` fills any holes left by PT_LOAD. This'll
  change soon, since we'll have a new way of creating APE binaries.

- The undiamonding trick with our ioctl() implementation is removed
  since POSIX has been killing ioctl() for years and they've done a
  much better job. One problem it resolves, is that ioctl(FIONREAD)
  wasn't working earlier and that caused issues when building Emacs
2023-07-11 04:41:50 -07:00
Justine Tunney a1b1fdd1a4
Reconfigure GitHub Actions 2023-07-10 12:17:18 -07:00
Justine Tunney a2d269dc38
Brush up some more code 2023-07-10 10:17:26 -07:00
Justine Tunney ee6566a152
Further improve usability of cosmocc
- Support -s flag in cosmocc
- Support posix_spawn() setsid() feature
- Disable monorepo debug path prefix stripping
2023-07-10 05:55:00 -07:00
Justine Tunney f7ae50462a
Make improvements
- Fix unused local variable errors
- Remove yoinks from sigaction() header
- Add nox87 and aarch64 to github actions
- Fix cosmocc -fportcosmo in linking mode
- It's now possible to build `make m=llvm o/llvm/libc`
2023-07-10 04:35:14 -07:00
Justine Tunney 3dc86ce154
Fix breakages in Linux-only build modes
- compile.com now polyfills -march=native which gcc/clang removed
- Guarantee zero Windows code is linked into non-Windows binaries
- MODE=tinylinux binaries are now back to being as tiny as ~4kb
- Improve the runtime's stack allocation / alignment hack
- GitHub Actions now tests Linux modes for assurance
2023-07-09 19:51:44 -07:00
Justine Tunney 41396ff48a
Make fixes and improvements
- Fix handling of precision in hex float formatting
- Enhance the cocmd interpreter for system() and popen()
- Manually ran the Lua unit tests, which are now passing
- Let stdio i/o operations happen when file is in error state
- We're now saving and restoring xmm in ftrace out of paranoia
2023-07-09 05:21:11 -07:00
Justine Tunney 9ae230afad
Fix more build configuration errors 2023-07-08 09:08:13 -07:00
Justine Tunney a75175fe94
Make build hermetically sealed again
It turned out that Landlock Make hasn't been applying sandboxing for a
while, due to a mistyped if statement for `$(USE_SYSTEM_TOOLCHAIN)` it
should have had the opposite meaning. Regressions in the build configs
have been fixed. The rmrf() function works better now. The rm.com tool
works according to POSIX with the exception of supporting prompts.
2023-07-08 07:06:25 -07:00
Justine Tunney 0d3c1c8b1a
Do work on curl/mbedtls/zstd
This change fixes stderr to be unbuffered. Added hardware AES on ARM64
to help safeguard against timing attacks. The curl.com command will be
somewhat more pleasant to use.
2023-07-07 10:13:35 -07:00
Justine Tunney a186143f62
Add EncodeHex() and DecodeHex() to Redbean 2023-07-06 15:38:08 -07:00
Justine Tunney 00acd81b2f
Delete more dead code 2023-07-06 09:12:28 -07:00
Justine Tunney 0a24b4fc3c
Clean up more code
The *NSYNC linked list API is good enough that it deserves to be part of
the C libray, so this change writes an improved version of it which uses
that offsetof() trick from the Linux Kernel. We vendor all of the *NSYNC
tests in third_party which helped confirm the needed refactoring is safe

This change also deletes more old code that didn't pan out. My goal here
is to work towards a vision where the Cosmopolitan core libraries become
less experimental and more focused on curation. This better reflects the
current level of quality we've managed to achieve.
2023-07-06 08:03:24 -07:00
Justine Tunney 88612a2cd7
Restore original redbean build linking 2023-07-05 20:04:43 -07:00
Justine Tunney 97b7116953
Hunt down more bugs
After going through the MODE=dbg and MODE=zero build modes, a bunch of
little issues were identified, which have been addressed. Fixing those
issues created even more troubles for the project, because it improved
our ability to detect latent problems which are getting fixed so fast.
2023-07-03 18:43:29 -07:00
Justine Tunney 73c0faa1b5
Remove some dead code 2023-07-03 02:48:29 -07:00
Justine Tunney fe044e22cc
Switch public headers to getopt_long() entirely
Cosmopolitan's getopt() is now redefined as __getopt().
2023-07-02 19:57:43 -07:00
Justine Tunney 0c630d95b5
Rewrite Cosmopolitan Ar
The build/bootstrap/ar.com program is now tinier. This change reduces
its size from 140kb to 53kb. Nothing was traded away. Cosmopolitan Ar
performance is now 2x better than llvm-ar largely thanks to using the
copy_file_range() system call. This change homebrews a new allocation
API that addresses the shortcomings of the C standard library design.
Using these new balloc() and reballoc() functions I managed to reduce
memory consumption so much that Cosmpolitan Ar should now use roughly
100x fewer bytes of peak resident memory compared to llvm-ar. Correct
behavior with better compatibility has been assured. Binary output is
now pretty much bit-identical to llvm-ar, as of this change. This can
and should be the living proof we need to show that a better world is
possible for software.
2023-07-02 10:19:16 -07:00
Justine Tunney 197aa0d465
Implement swapcontext() and makecontext()
This change introduces support for Linux-style uc_context manipulation
that's fast and works well on all supported OSes and architectures. It
also integrates with the Cosmpolitan runtime which can show backtraces
comprised of multiple stacks and fibers. See the test and example code
for further details. This will be used by Mold once it's been vendored
2023-07-02 09:01:44 -07:00
Paul Kulchenko c0348330a5
Add path information in logged error messages (#832) 2023-07-01 19:47:59 -07:00
Justine Tunney 40eb3b9d5d
Fully support OpenBSD 7.3
This change (1) upgrades to OpenBSD's newer kernel ABIs, and (2)
modifies APE to have a read-only data segment. Doing this required
creating APE Loader v1.1, which is backwards and forwards compatible
with the previous version.

If you've run the following commands in the past to install your APE
Loader systemwide, then you need to run them again. Ad-hoc installations
shouldn't be impacted. It's also recommended that APE binaries be remade
after upgrading, since they embed old versions of the APE Loader.

    ape/apeuninstall.sh
    ape/apeinstall.sh

This change does more than just fix OpenBSD. The new loader is smarter
and more reliable. We're now able create much tinier ELF and Mach-O data
structures than we could before. Both APE Loader and execvpe() will now
normalize ambiguous argv[0] resolution the same way as the UNIX shell.
Badness with TLS linkage has been solved.

Fixes #826
2023-07-01 18:14:27 -07:00
Justine Tunney 226375933a
Implement more toolchain fixes 2023-06-18 05:39:31 -07:00
Justine Tunney 0409096658
Get us closer to building busybox
This change undefines __linux__ and adds APIs like clock_settime(). The
gosh darned getopt_long() API has been reintroduced, thanks to OpenBSD.
2023-06-18 04:13:45 -07:00
Justine Tunney d7c79f43ef
Clean up more code
- Found some bugs in LLVM compiler-rt library
- The useless LIBC_STUBS package is now deleted
- Improve the overflow checking story even further
- Get chibicc tests working in MODE=dbg mode again
- The libc/isystem/ headers now have correctly named guards
2023-06-18 01:00:05 -07:00
Justine Tunney 12452976bd
Remove getopt_long() from headers
We removed this API a long time ago and it was breaking the GNU make
open source build.
2023-06-17 16:03:16 -07:00
Justine Tunney b881c0ec9e
Remove printf() linking hack 2023-06-17 10:13:50 -07:00
Justine Tunney 562a1384cd
Make blink support conditionally linkable into APE 2023-06-17 07:55:35 -07:00
Justine Tunney 52d28966f7
Remove old zip base skew hack
Since 8ff48201ca we no longer need the
hack where, when running .com.dbg files, we scanned for the embedded
.com file offset, and then computed zip offsets realtive to that. It
wasn't very reliable in the first place, and was causing issues with
running our new .com.dbg executables, which are true zip files.
2023-06-17 04:20:16 -07:00
Justine Tunney 2a1c588826
Add tokenbucket module to python.com 2023-06-16 15:32:09 -07:00
Justine Tunney e6b7c16a53
Make changes needed for new demo 2023-06-15 23:22:49 -07:00
Justine Tunney c3440d040c
Make improvements
- More timspec_*() and timeval_*() APIs have been introduced.
- The copyfd() function is now simplified thanks to POSIX rules.
- More Cosmo-specific APIs have been moved behind the COSMO define.
- The setitimer() polyfill for Windows NT is now much higher quality.
- Fixed build error for MODE=aarch64 due to -mstringop-strategy=loop.
- This change introduces `make MODE=nox87 toolchain` which makes it
  possible to build programs using your cosmocc toolchain that don't
  have legacy fpu instructions. This is useful, for example, if you
  want to have a ~22kb tinier blink virtual machine.
2023-06-15 14:50:53 -07:00
Justine Tunney 4778cd4d27
Fix bugs in termios library and cleanup code
This change fixes an issue with the tcflow() magic numbers that was
causing bash to freeze up on Linux. While auditing termios polyfills,
several other issues were identified with XNU/BSD compatibility.

Out of an abundance of caution this change undefines as much surface
area from libc/calls/struct/termios.h as possible, so that autoconf
scripts are less likely to detect non-POSIX teletypewriter APIs that
haven't been polyfilled by Cosmopolitan.

This is a *breaking change* for your static archives in /opt/cosmos if
you use the cosmocc toolchain. That's because this change disables the
ioctl() undiamonding trick for code outside the monorepo, specifically
because it'll lead to brittle ABI breakages like this. If you're using
the cosmocc toolchain, you'll need to rebuild libraries like ncurses,
readline, etc. Yes diamonds cause bloat. To work around that, consider
using tcgetwinsize() instead of ioctl(TIOCGWINSZ) since it'll help you
avoid pulling every single ioctl-related polyfill into the linkage.

The cosmocc script was specifying -DNDEBUG for some reason. It's fixed.
2023-06-14 19:30:52 -07:00
Justine Tunney 06b749ae03
Remove blinkenlights
New home: https://github.com/jart/blink
2023-06-14 19:30:52 -07:00
Justine Tunney 5a6a4819d4
Fix -iquote in cosmocc toolchain 2023-06-10 16:35:53 -07:00
Justine Tunney 2676ec55de
Make improvements to cosmocc toolchain 2023-06-10 15:50:01 -07:00
Justine Tunney 8ff48201ca
Rewrite .zip.o file linker
This change takes an entirely new approach to the incremental linking of
pkzip executables. The assets created by zipobj.com are now treated like
debug data. After a .com.dbg is compiled, fixupobj.com should be run, so
it can apply fixups to the offsets and move the zip directory to the end
of the file. Since debug data doesn't get objcopy'd, a new tool has been
introduced called zipcopy.com which should be run after objcopy whenever
a .com file is created. This is all automated by the `cosmocc` toolchain
which is rapidly becoming the new recommended approach.

This change also introduces the new C23 checked arithmetic macros.
2023-06-10 09:29:44 -07:00
Justine Tunney 4a59210008
Introduce #include <cosmo.h> to toolchain users
This change improves the way internal APIs are being hidden behind the
`COSMO` define. The cosmo.h header will take care of defining that, so
that a separate define statement isn't needed. This change also does a
lot more to define which APIs are standard, and which belong to Cosmo.
2023-06-09 18:03:05 -07:00
Justine Tunney 9b55dbe417
Get GCC to mostly build with Cosmo 2023-06-09 06:41:34 -07:00
Justine Tunney 4b2023ffab
Disable linker map generation and improve tinyness 2023-06-09 03:29:26 -07:00
Justine Tunney 23e235b7a5
Fix bugs in cosmocc toolchain
This change integrates e58abc1110b335a3341e8ad5821ad8e3880d9bb2 from
https://github.com/ahgamut/musl-cross-make/ which fixes the issues we
were having with our C language extension for symbolic constants. This
change also performs some code cleanup and bug fixes to getaddrinfo().
It's now possible to compile projects like ncurses, readline and python
without needing to patch anything upstream, except maybe a line or two.
Pretty soon it should be possible to build a Linux distro on Cosmo.
2023-06-08 23:44:03 -07:00
Justine Tunney 22f81a8d50
Improve cosmocc / cosmoc++ toolchain scripts
- Get out of the red zone
- Generate --ftrace nops unless -Os is passed
- Intercept -o path to generate .com / .com.dbg appropriately
2023-06-08 14:29:22 -07:00
Justine Tunney 25678db2a0
Fix remaining regressions in ASAN support
See #829
2023-06-08 08:00:50 -07:00
Justine Tunney 32682f0ce7
Remove some problematic APIs
In order to improve our chances of success building other open source
projects we shouldn't define APIs that'll lead any ./configure script
astray. For example:

- brk() and sbrk() can break mac/windows support
- syscall() is a superb way to break portability
- arch_prctl() is the greatest of all horror shows
2023-06-08 06:12:26 -07:00
Justine Tunney 7512318a2a
Fix MODE=aarch64 build 2023-06-08 05:17:37 -07:00
Justine Tunney daf4454a06
Validate privileged code relationships
- Work towards improving non-optimized build support
- Introduce MODE=zero which is -O0 without ASAN/UBSAN
- Use system GCC when ~/.cosmo.mk has USE_SYSTEM_TOOLCHAIN=1
- Have package.com check .privileged code doesn't call non-privileged
2023-06-08 04:38:06 -07:00
Justine Tunney 9793d3524f
Remove superfluous RET instructions 2023-06-06 19:46:56 -07:00
Justine Tunney 61b9677c05
Make improvements
- Get mprotect_test working on aarch64
- Get completion working on python.com repl again
- Improve quality of printvideo.com and printimage.com
- Fix bug in openpty() so examples/script.c works again
2023-06-06 09:12:30 -07:00
Justine Tunney b8a6a989c0
Create ELF aliases for identical symbols
This change greatly reduces the number of modules that need to be
compiled. The only issue right now is that sometimes when viewing
symbol table entries, the aliased symbol is chosen.
2023-06-06 03:33:49 -07:00
Justine Tunney eb40cb371d
Get --ftrace working on aarch64
This change implements a new approach to function call logging, that's
based on the GCC flag: -fpatchable-function-entry. Read the commentary
in build/config.mk to learn how it works.
2023-06-05 23:35:31 -07:00
Justine Tunney 5b908bc756
Fix some build errors 2023-06-05 15:53:44 -07:00
Justine Tunney f554dd800e
Make it possible to log kprintf() to file
It's now possible to compile Emacs using cosmocc. However we need to
troubleshoot why it's event loop isn't working correctly at runtime.
2023-06-05 04:16:15 -07:00
Justine Tunney 39f20dbb13
Upgrade to Cosmopolitan GCC 11.2.0 for x86_64 2023-06-05 02:06:18 -07:00
Justine Tunney 4aa1d09b9e
Improve aarch64 native support some more
This change introduces partial support for automating remote testing of
aarch64 binaries on Raspberry Pi and Apple Silicon.
2023-06-04 08:58:47 -07:00
Justine Tunney fc34ba2596
Fix Linenoise REPL on AARCH64 2023-06-04 02:57:17 -07:00
Justine Tunney 8f522cb702
Make improvements
This change progresses our AARCH64 support:

- The AARCH64 build and tests are now passing
- Add 128-bit floating-point support to printf()
- Fix clone() so it initializes cosmo's x28 TLS register
- Fix TLS memory layout issue with aarch64 _Alignas vars
- Revamp microbenchmarking tools so they work on aarch64
- Make some subtle improvements to aarch64 crash reporting
- Make kisdangerous() memory checks more accurate on aarch64
- Remove sys_open() since it's not available on Linux AARCH64

This change makes general improvements to Cosmo and Redbean:

- Introduce GetHostIsa() function in Redbean
- You can now feature check using pledge(0, 0)
- You can now feature check using unveil("",0)
- Refactor some more x86-specific asm comments
- Refactor and write docs for some libm functions
- Make the mmap() API behave more similar to Linux
- Fix WIFSIGNALED() which wrongly returned true for zero
- Rename some obscure cosmo keywords from noFOO to dontFOO
2023-06-03 08:12:22 -07:00
Paul Kulchenko 5655c9a4e7
Extend Pledge with anet (same as inet, but with no connect) (#827)
* Add `anet` pledge for `inet` without connect

This is useful for configurations where it's desirable to start redbean
under these restrictions, but not to allow `connect` socket calls.

* Update message on protected/unpledged syscalls for clarity

* Update redbean to add reporting for unpledged sigaction

Previously it would abort without indicating what signal it failed to
install when sigaction is not pledged (although it fails all of them).

* Move GetHostIps before processing command line options

This allows using unix.pledge as part of the options without affecting
retrieving host IP addresses (which requires `connect`). It may still
fail under external `pledge` command as expected; in this case IPs
would need to be passed manually.

* Update tests for pledge anet promise
2023-06-03 07:50:29 -07:00
Connor 992a4638ae
Fix rgbtoxterm tool to output the proper colors (#819) 2023-05-26 18:30:24 -07:00
Paul Kulchenko 5c40800d69
Update redbean ProgramDirectory to update package.path (#823) 2023-05-26 18:29:00 -07:00
Justine Tunney 1422e96b4e
Introduce native support for MacOS ARM64
There's a new program named ape/ape-m1.c which will be used to build an
embeddable binary that can load ape and elf executables. The support is
mostly working so far, but still chasing down ABI issues.
2023-05-20 04:17:03 -07:00
Justine Tunney b852650c08
Fiddle around with Mach-O 2023-05-20 04:13:49 -07:00
Paul Kulchenko 6881a2ecea
Extend redbean Fetch to add option to keeping connection open (#818) 2023-05-17 20:49:26 -07:00
Paul Kulchenko 021c71b651
Fix redbean to avoid compressing the first yielded chunk. (#813)
This may happen if no Status or Content-Type is set and if the written
content is longer than 100B, the compression is going to be applied
after coroutine.yield, even though it shouldn't be.
2023-05-17 20:46:27 -07:00
Justine Tunney e7eb0b3070
Make more ML improvements
- Fix UX issues with llama.com
- Do housekeeping on libm code
- Add more vectorization to GGML
- Get GGJT quantizer programs working well
- Have the quantizer keep the output layer as f16c
- Prefetching improves performance 15% if you use fewer threads
2023-05-16 08:07:23 -07:00
Justine Tunney 80db9de173
Make the intrinsics more readable 2023-05-15 23:12:11 -07:00
Justine Tunney 210187cf77
Perform some code cleanup 2023-05-15 16:32:10 -07:00
Justine Tunney cc1732bc42
Make AARCH64 harder, better, faster, stronger
- Perform some housekeeping on scalar math function code
- Import ARM's Optimized Routines for SIMD string processing
- Upgrade to latest Chromium zlib and enable more SIMD optimizations
2023-05-15 02:15:34 -07:00
Justine Tunney 550b52abf6
Port a lot more code to AARCH64
- Introduce epoll_pwait()
- Rewrite -ftrapv and ffs() libraries in C code
- Use more FreeBSD code in math function library
- Get significantly more tests passing on qemu-aarch64
- Fix many Musl long double functions that were broken on AARCH64
2023-05-14 09:37:26 -07:00
Justine Tunney 5a4cf9560f
Add support for new GGJT v2 quantizers
This change makes quantized models (e.g. q4_0) go 10% faster on Macs
however doesn't offer much improvement for Intel PC hardware.

This change syncs llama.cpp 699b1ad7fe6f7b9e41d3cb41e61a8cc3ea5fc6b5
which recently made a breaking change to nearly all its file formats
without any migration. Since that'll break hundreds upon hundreds of
models on websites like HuggingFace llama.com will support both file
formats because llama.com will never ever break the GGJT file format
2023-05-13 08:08:32 -07:00
Justine Tunney 802e7eb4ef
Mop up more test regressions 2023-05-13 01:09:44 -07:00
Justine Tunney fd34ef732d
Make considerably more progress on AARCH64
- Utilities like pledge.com now build
- kprintf() will no longer balk at 48-bit addresses
- There's a new aarch64-dbg build mode that should work
- gc() and defer() are mostly pacified; avoid using them on aarch64
- THIRD_PART_STB now has Arm Neon intrinsics for fast image handling
2023-05-12 22:42:57 -07:00
Justine Tunney 414667b1c9
Get TEST_LIBC_STR passing on AARCH64
It's now possible to run commands like:

    make -j8 m=aarch64 o/aarch64/test/libc/str

Which will cross-compile and run the test suites in a qemu-aarch64
binary that's vendored in the third_party/qemu/ folder within your
x86_64 build environment.
2023-05-12 18:09:23 -07:00
Justine Tunney 4edbc98811
Get MbedTLS and its unit tests passing AARCH64 2023-05-11 21:53:15 -07:00
Justine Tunney c91de6a356
Avoid running clang-format on ggml.c 2023-05-10 14:59:33 -07:00
Justine Tunney f312f706f4
Bring MODE=tiny binary sizes down to 20kb minimum
aarch64 binaries start at 4kb.
2023-05-10 04:20:48 -07:00
Justine Tunney 5a455eaa0b
Work on magic numbers for aarch64 2023-05-10 04:20:48 -07:00
Justine Tunney 59766efd3e
Do some more aarch64 fixups 2023-05-10 04:20:47 -07:00
Justine Tunney 4c093155a3
Get llama.com building as an aarch64 native binary 2023-05-10 04:20:47 -07:00
Justine Tunney d04430f4ef
Get LIBC_MEM and LIBC_STDIO building with aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney ae0ee59614
Get aarch64 hello world working
$ m=aarch64-tiny
    $ make -j8 m=$m o/$m/tool/hello/hello.com o/third_party/qemu/qemu-aarch64
    $ o/third_party/qemu/qemu-aarch64 o/$m/tool/hello/hello.com
    hello world
    $ ls -hal o/$m/tool/hello/hello.com
    -rwxr-xr-x 1 jart jart 4.0K May  9 05:04 o/aarch64-tiny/tool/hello/hello.com
2023-05-10 04:20:47 -07:00
Justine Tunney 036b9a0002
Make further progress on non-x86 support 2023-05-10 04:20:47 -07:00
Justine Tunney aef9a69a60
Make more progress on aarch64 2023-05-10 04:20:47 -07:00
Justine Tunney 2b73e72d59
Make more code aarch64 friendly 2023-05-10 04:20:46 -07:00
Justine Tunney ca2860947f
Make progress towards aarch64 build 2023-05-10 04:20:46 -07:00
Justine Tunney 8303e23b3a
Do some basic build tuning 2023-05-10 04:20:46 -07:00
Paul Kulchenko a493b64233
Merge pull request #807 from jart/evalarg
Fix the behavior of `redbean -i -e CODE`
2023-05-02 07:48:43 -07:00
Justine Tunney fc82f77a46
Fix bug in cosmocc / cosmoc++ scripts
The necessary flags weren't being passed when compiling programs as a
single step (i.e. not using `-c` to make intermediate objects).
2023-05-01 13:46:15 -07:00
Justine Tunney bf6459e324
Fix bugs with cli flags in gzip.com 2023-04-28 05:15:58 -07:00
Justine Tunney b7bf052a4b
Add Intel intrinsics headers 2023-04-27 05:44:38 -07:00
Justine Tunney 369f9740de
Run clang-format on most sources 2023-04-27 05:44:32 -07:00
Justine Tunney 497d4fa6d1
Fix the behavior of redbean -i -e CODE
When redbean is functioning as a Lua interpreter, the `-e` flag should
behave the same way as other open source language interpreters. Namely
it should exit after evaluating the code rather than showing the REPL.
2023-04-25 01:52:13 -07:00
Justine Tunney 4a6be2154d
Fix hermeticity issue with new test 2023-04-18 06:37:01 -07:00
Paul Kulchenko 58b2f9bd6b
Fix missing detail in the error message in redbean (#768)
This only happens on calls that need to happen during active connection.
2023-03-20 11:10:32 -04:00
Paul Kulchenko e2f429b742
Redbean documentation typo fixes. (#756)
* Redbean documentation typo fixes.

* Add brief description to some unix.* calls missing it
2023-03-06 00:15:09 -08:00
Justine Tunney ba180e754d
Improve cosmocc toolchain
- Decouple zlib from libc
- Add some underscores to mostly internal names
2023-03-05 23:52:49 -08:00
Justine Tunney 713d4424c6
Don't add loopback address to generated SSL cert 2023-03-05 22:57:22 -08:00
Paul Kulchenko 22fcab131c
Fix redbean SQLite to report results on failure to open db (#760) 2023-03-05 14:26:56 -08:00
Paul Kulchenko 0312898979
Make redbean cache directive configurable (#750)
Thank you @johnmuhl for the proposal. Fixes #739
2023-02-23 22:24:40 -08:00
Paul Kulchenko db16f0129a
Redbean sqlite fix multi close (#749)
* Update redbean SQLite config to handle more options

This requires moving sqlite3_initialize call to open, as configuration
should be allowed before initialization is done. This call is effective
only for the first time and then no-op after that.

* Fix redbean SQLite for closing db with already finalized statements

There is a loop in cleanupdb that finalizes all vms that are associated
with that db when it's being closed. Under some circumstances (detailed
below) that loop may contain references pointing to already collected
objects, thus leading to SIGSEGV when those references are used.

This may happen with the following sequence of events ("VM" is the name
used in lsqlite and describes the same thing as "statement"):
1. A finalized statement is created (for example, by preparing an empty
string or a string with no statement that is still grammatically valid).
2. This statement goes out of scope before the DB object it's associated
with does and is garbage collected.
3. When it's garbage collected, dbvm_gc method is called, which checks
for svm->vm being not NULL.
4. Since the VM is already finalized, cleanupvm method is not called,
so the VM reference is not removed from the table of VMs tracked for
that DB.
5. When the DB is finally closed or garbage collected, all the VMs
associated with it are accessed to be finalized, including the ones that
have been garbage collected and have invalid references (thus leading
to a memory access error).

Here is an example of a stacktrace from the resulting SIGSEGV:

70000003de20 5df71a getgeneric+26
70000003fac0 5dfc7f luaH_get+111
70000003faf0 5e06c8 luaH_set+40
70000003fb20 5c5bd7 aux_rawset+55
70000003fb50 5c70cb lua_rawset+27
70000003fb60 4fa8e7 cleanupvm+71
70000003fb80 4fa988 cleanupdb+88
70000003fbc0 4fe899 db_gc+41

One way to fix this is to use userdata references (which anchor their
targets) instead of lightuserdata references (which do not), but this
would prevent the targets (VMs) from being garbage collected until the
DB itself is garbage collected, so this needs to be combined with
weakening the keys in the DB table. The code in cleanupdb to remove the
VM references is no longer needed, as this is handled by having weak keys.

The patch also switches to using close_v2, as it is intended for use
with garbage collected languages where the order in which destructors
are called is arbitrary, as is the case here.

* Remove GC collection from redbean SQLite session

The behavior of sqlite3session_delete is undefined after the DB
connection is closed, so we need to avoid calling it from gc handler.
2023-02-23 20:10:03 -08:00
Michael Ford ca88fc34c3
doc: remove "everything everything" from ParseUrl doc (#704)
Closes: #702.
2023-02-23 09:34:48 -08:00
Justine Tunney 8b69acc1cc
Make mkdeps more helpful when files are missing 2023-02-23 08:11:08 -08:00
James Hulce e4ee1da652
Fix filenames in help.txt (#722)
Many functions incorrectly direct users to see a .c file, when they should be .S
2023-02-22 19:00:49 -08:00
Paul Kulchenko e323527ffa
Remove slowloris handling from Redbean (#737)
It's been superseded by token bucket processing, does not take time
into considerations (only the number of fragments), and affects file
uploads that may require a large number of reads.
2023-02-22 18:56:02 -08:00
Gavin Hayes 94dcf81954
Add glob to cocmd (#740) 2023-02-21 10:31:04 -08:00
Justine Tunney 2b6261a52d
Make some system call fixes
- Fix minor ABI issue with SIOCGIFCONF
- Fix ABI translation issues with statfs() on BSDs
- Fix SQLite angled header line
2023-02-12 22:16:34 -08:00
Justine Tunney cf19e3e46a
Fix SO_LINGER on Apple 2023-02-02 20:05:37 -08:00
Justine Tunney 7486a2cd51
Perform minor cleanups 2023-02-02 02:28:58 -08:00
Paul Kulchenko 6d36584ff2
Redbean lsqlite config addition (#724)
* Fix mislabeled constant in redbean SQLite code
* Simplify callback handling in redbean SQLite
* Add config options to redbean SQLite
* Update redbean SQLite API to return current error callback
2023-01-29 19:37:19 -08:00
James Hulce f422edaf15
FIx link in gensvg demo (#723)
Code link in the gensvg demo is broken, fix that
2023-01-28 09:12:13 -08:00
Justine Tunney 1473eafd1a
Fix the build and introduce some APIs
The tcgetwinsize() and tcsetwinsize() APIs are now available. The
printargs.com example also now displays the baud rate.
2023-01-20 09:25:45 -08:00
Justine Tunney 006c44ff5d
Update tests and CPU detection for Blink 2023-01-18 00:56:09 -08:00
Justine Tunney 2bad436abf
Return error on fopen(NULL)
https://sqlite.org/forum/forumpost/d1c96a9032e564f8
2023-01-08 15:17:44 -08:00
Justine Tunney 7ab4630cc9
Fix MODE=asan build regression 2023-01-08 05:47:18 -08:00
Justine Tunney 5dab97b6d4
Implement program for finding start of zip content 2023-01-06 19:55:00 -08:00
Justine Tunney d73523864a
Fix bugs with Redbean Fetch() uploading
This change makes Fetch() work correctly when large bodies are supplied,
or bodies containing NUL characters are supplied.

Fixes #573
2023-01-05 09:52:36 -08:00
Justine Tunney 651826b660
Update assimilate to work with modern APE format 2023-01-04 06:34:48 -08:00
Justine Tunney a6586cafb2
Retry SQLITE_BUSY when stepping in TurfWar 2023-01-03 06:10:39 -08:00
Justine Tunney e496ee7a80
Be less pushy about redbean ddos protection 2023-01-03 05:14:34 -08:00
Justine Tunney 10d1c6da18
Support argv[0] munging on Windows 2023-01-03 03:22:40 -08:00
Danny Robinson 70ca8c4016
Update redbean lsqlite definitions (#709) 2023-01-01 16:28:34 -08:00
Ronsor d2f811eff3
Add support for the -i (include headers) and -d (POST data) options in curl.com (#708)
This commit also paves the way for initial IPv6 support in `curl.com` by specifying
`AF_UNSPEC` for the address family in the hints passed to `getaddrinfo()`.
2022-12-23 17:06:48 -08:00
Justine Tunney 2e5181666f
Fix the build 2022-12-21 23:06:55 -08:00
Ronsor f6b3019796
Make curl example into a complete tool (#706)
You can now handily use `curl` with popen()/system() by
placing `STATIC_YOINK("_curl")` in your main source file.
2022-12-21 16:12:02 -08:00
Justine Tunney dd04aeba1c
Increase stack size to 128k and guard size to 16k
This improves our compatibility with Apple M1.
2022-12-18 22:58:29 -08:00
Justine Tunney 6d39c6f646
Have redbean avoid gzip when emulated without jit 2022-12-18 02:22:58 -08:00
Justine Tunney 3da887c58f
Introduce CPUID detection for Blink 2022-12-17 00:38:07 -08:00
Justine Tunney ed161b240e
Clean up some code 2022-12-11 14:30:59 -08:00
Paul Kulchenko cd0b12ed1a
Add diff and iterator to SQLite session (#700) 2022-11-28 13:40:04 -08:00
Paul Kulchenko 0dc0758574
Upgrade SQLite to 3.40 (#699) 2022-11-28 12:54:48 -08:00
Paul Kulchenko bcae817215
Update SQLite deserialize to allow resizing restored DB (#701)
Using either option (FREEONCLOSE or RESIZEABLE) requires using
sqlite3_malloc. Ref. #436
2022-11-28 12:53:20 -08:00
Justine Tunney b850b14300
Improve cosmocc / cosmoc++ toolchains
They'll now automatically create empty static archives for system
libraries that are provided by Cosmopolitan Libc. This helps make
configure scripts less confused. Musl does the same thing.
2022-11-14 13:45:57 -08:00
Justine Tunney f2af97711b
Make improvements
- Improve compatibility with Blink virtual machine
- Add non-POSIX APIs for joining threads and signal masks
- Never ever use anything except 32-bit integers for atomics
- Add some `#undef` statements to workaround `ctags` problems
2022-11-10 21:52:47 -08:00
Justine Tunney bf7843833f
Rename hidden keyword to _Hide 2022-11-08 12:55:28 -08:00
Paul Kulchenko 251dcb07eb
Add redbean sqlite session support (#680) 2022-11-08 12:53:37 -08:00
Justine Tunney b407327972
Make fixes and improvements
- clock_nanosleep() is now much faster on OpenBSD and NetBSD
- Thread joining is now much faster on NetBSD
- FreeBSD timestamps are now more accurate
- Thread spawning now goes faster on XNU
- Clean up the clone() code
2022-11-08 10:11:46 -08:00
Justine Tunney c995838e5c
Make improvements
- Clean up sigaction() code
- Add a port scanner example
- Introduce a ParseCidr() API
- Clean up our futex abstraction code
- Fix a harmless integer overflow in ParseIp()
- Use kernel semaphores on NetBSD to make threads much faster
2022-11-07 02:26:06 -08:00
Danny Robinson 543c93f623
bestline keybindings and redbean definitions (#678)
* Add ctrl-left/right keys to bestline
* Update redbean API definitions
2022-11-06 00:28:08 -07:00
Justine Tunney 3f0bcdc3ef
Improve cancellations, randomness, and time
- Exhaustively document cancellation points
- Rename SIGCANCEL to SIGTHR just like BSDs
- Further improve POSIX thread cancellations
- Ensure asynchronous cancellations work correctly
- Elevate the quality of getrandom() and getentropy()
- Make futexes cancel correctly on OpenBSD 6.x and 7.x
- Add reboot.com and shutdown.com to examples directory
- Remove underscore prefix from awesome timespec_*() APIs
- Create assertions that help verify our cancellation points
- Remove bad timespec APIs (cmp generalizes eq/ne/gt/gte/lt/lte)
2022-11-05 23:45:32 -07:00
Justine Tunney 37d40e087f
Ignore SIGSYS on BSD by default 2022-11-03 09:32:12 -07:00
Justine Tunney 89d1e5b8f2
Rearrange code and make a faster sha256sum program 2022-11-02 23:19:16 -07:00
Justine Tunney 5e60e5ad10
Release redbean v2.2 2022-11-02 09:52:54 -07:00
Justine Tunney 6b06a8176d
Fix some glitches in redbean
This change includes a fix to Fetch() where an out of bounds memory read
could happen, when the reverse proxied endpoint omits the content-length
header. This caused a bunch of NUL chars to appear on TurfWar's /statusz
since it wouldn't actually overrun the buffer, and if it did it would've
been caught by MODE=asan builds.
2022-11-02 09:46:34 -07:00
Justine Tunney 14d036b68d
Add WSL to test fleet
All tests pass now under WSL2. They should pass under WSL1 too, but only
WSL2 is integrated into the test fleet right now. This change also fills
in some gaps in the error numbers.

Fixes #665
2022-11-02 06:49:42 -07:00
Justine Tunney fc96af058b
Workaround MAP_GROWSDOWN unavailability on WSL 2022-11-02 01:38:06 -07:00
Justine Tunney e522aa3a07
Make more threading improvements
- ASAN memory morgue is now lockless
- Make C11 atomics header more portable
- Rewrote pthread keys support to be lockless
- Simplify Python's unicode table unpacking code
- Make crash report write(2) closer to being atomic
- Make it possible to strace/ftrace a single thread
- ASAN now checks nul-terminated strings fast and properly
- Windows fork() now restores TLS memory of calling thread
2022-11-01 23:28:26 -07:00
Paul Kulchenko d7b88734cd
Make redbean compress w/o explicit content-type (#671) 2022-11-01 23:15:20 -07:00
Justine Tunney da336b3ea8
Make some more fixes to prod 2022-10-19 13:10:24 -07:00
Justine Tunney 69bee64a59
Make some last minute production changes 2022-10-19 10:00:29 -07:00
Justine Tunney f7ff77d865
Make fixes and improvements
- Invent iso8601us() for faster timestamps
- Improve --strace descriptions of sigset_t
- Rebuild the Landlock Make bootstrap binary
- Introduce MODE=sysv for non-Windows builds
- Permit OFD fcntl() locks under pledge(flock)
- redbean can now protect your kernel from ddos
- Have vfork() fallback to sys_fork() not fork()
- Change kmalloc() to not die when out of memory
- Improve documentation for some termios functions
- Rewrite putenv() and friends to conform to POSIX
- Fix linenoise + strace verbosity issue on Windows
- Fix regressions in our ability to show backtraces
- Change redbean SetHeader() to no-op if value is nil
- Improve fcntl() so SQLite locks work in non-WAL mode
- Remove some unnecessary work during fork() on Windows
- Create redbean-based SSL reverse proxy for IPv4 TurfWar
- Fix ape/apeinstall.sh warning when using non-bash shells
- Add ProgramTrustedIp(), and IsTrustedIp() APIs to redbean
- Support $PWD, $UID, $GID, and $EUID in command interpreter
- Introduce experimental JTqFpD APE prefix for non-Windows builds
- Invent blackhole daemon for firewalling IP addresses via UNIX named socket
- Add ProgramTokenBucket(), AcquireToken(), and CountTokens() APIs to redbean
2022-10-19 07:19:19 -07:00
Justine Tunney 60cb435cb4
Implement pthread_atfork()
If threads are being used, then fork() will now acquire and release and
runtime locks so that fork() may be safely used from threads. This also
makes vfork() thread safe, because pthread mutexes will do nothing when
the process is a child of vfork(). More torture tests have been written
to confirm this all works like a charm. Additionally:

- Invent hexpcpy() api
- Rename nsync_malloc_() to kmalloc()
- Complete posix named semaphore implementation
- Make pthread_create() asynchronous signal safe
- Add rm, rmdir, and touch to command interpreter builtins
- Invent sigisprecious() and modify sigset functions to use it
- Add unit tests for posix_spawn() attributes and fix its bugs

One unresolved problem is the reclaiming of *NSYNC waiter memory in the
forked child processes, within apps which have threads waiting on locks
2022-10-16 12:25:13 -07:00
Justine Tunney 0f89140882
Further improve cocmd interpreter 2022-10-12 10:44:54 -07:00
Justine Tunney b41f91c658
Greatly expand system() shell code features
The cosmopolitan command interpreter now has 13 builtin commands,
variable support, support for ; / && / || syntax, asynchronous support,
and plenty of unit tests with bug fixes.

This change fixes a bug in posix_spawn() with null envp arg. strace
logging now uses atomic writes for scatter functions. Breaking change
renaming GetCpuCount() to _getcpucount(). TurfWar is now updated to use
the new token bucket algorithm. WIN32 affinity masks now inherit across
fork() and execve().
2022-10-11 21:30:31 -07:00
tkchia 7a06760e6f
Rename LINUX to _HOSTLINUX etc. to reduce clashes (#655)
Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-10-10 20:31:25 -07:00
Justine Tunney e557058ac8
Improve cosmo's conformance to libc-test
This change addresses various open source compatibility issues, so that
we pass 313/411 of the tests in https://github.com/jart/libc-test where
earlier today we were passing about 30/411 of them, due to header toil.
Please note that Glibc only passes 341/411 so 313 today is pretty good!

- Make the conformance of libc/isystem/ headers nearly perfect
- Import more of the remaining math library routines from Musl
- Fix inconsistencies with type signatures of calls like umask
- Write tests for getpriority/setpriority which work great now
- conform to `struct sockaddr *` on remaining socket functions
- Import a bunch of uninteresting stdlib functions e.g. rand48
- Introduce readdir_r, scandir, pthread_kill, sigsetjmp, etc..

Follow the instructions in our `tool/scripts/cosmocc` toolchain to run
these tests yourself. You use `make CC=cosmocc` on the test repository
2022-10-10 17:52:41 -07:00
Justine Tunney 7ae556463a
Rename rand64() to _rand64() 2022-10-10 04:12:06 -07:00
Justine Tunney 1c0adb556d
Release redbean v2.1 2022-10-08 09:03:51 -07:00
Justine Tunney 8aca94f951
Add shared memory hit counter example to redbean 2022-10-08 08:39:13 -07:00
Justine Tunney 9849b4c7ba
Add *NSYNC unit test suite
This change also fixes the clock_nanosleep() api and polyfills futexes
on Windows, Mac, and NetBSD using exponential backoff.
2022-10-07 21:34:15 -07:00
Justine Tunney 7d6835c0dd
Release pledge v1.8 2022-10-07 03:41:50 -07:00
Justine Tunney 05197afca2
Do some work on TurfWar 2022-10-07 03:11:07 -07:00
Danny Robinson 7bc24888fe
Update redbean api definitions (#652) 2022-10-06 16:27:23 -07:00
Justine Tunney 59ac141e49
Improve the affinity system calls 2022-10-06 15:08:29 -07:00
Justine Tunney 60b68d7152
Fix naming for redbean shared memory atomics 2022-10-06 09:41:22 -07:00
Justine Tunney 7822917fc2
Add shared memory apis to redbean
You can now do things like implement mutexes using futexes in your
redbean lua code. This provides the fastest possible inter-process
communication for your production systems when SQLite alone as ipc
or things like pipes aren't sufficient.
2022-10-06 04:55:26 -07:00
Daniel Oltmanns afc07b9339
Proof of concept of sqlite serialization (#436)
* Proof of concept of sqlite serialization

This is a minimal proof of concept in order to show that it is easily possible to store the sqlite database within the zip file itself not requiring creating an external file first. Changes include compiling the sqlite library with the serialization flag, adding serialize/deserialize to the lua sqlite library and demonstrating the work via the redbean demo.

* Change demo for sqlite serialization

As explained in https://github.com/jart/cosmopolitan/pull/436#issuecomment-1164706893 the original use case is not possible with sqlite serialization, as an in-memory database cannot be shared across multiple processes. Thereby, this use case simply creates a backup of the in-memory database created in '.init.lua' and loads it to do a query.

* Fix sqlite3_deserialize parameters

The call to the sqlite3 library for the deserilization wasn't fully correct. This should fix the size parameters.
2022-10-05 07:09:53 -07:00
Justine Tunney 950a1b310b
Embed cocmd.com interpreter for system() / open()
This change lets you use system() in an easier and portable way. The
problem with the call in the past has always been that bourne and
cmd.com on Windows have less than nothing in common, so pretty much the
only command system() could be used for across platforms was maybe echo.
cmd.exe is also a security liability due to its escaping rules.

Since cocmd.com implements 85% of what we need from bourne, in a really
tiny way, it makes perfect sense to be embedded in these functionss. We
get a huge performance boost too.

Fixes #644
2022-10-02 15:29:57 -07:00
Danny Robinson 2526a9b8c7
Fix re.NOSUB memory error (#645) 2022-10-02 12:49:23 -07:00
Justine Tunney 3f3cb0650b
Have execve() escape double quotes in cmd.exe's preferred style
This makes it possible for us to use system() and popen() with paths
that redirect to filenames that contain spaces, e.g.

    system("echo.com hello >\"hello there.txt\"")

It's difficult to solve this problem, because WIN32 only allows passing
one single argument when launching programs and each program is allowed
to tokenize that however it wants. Most software follows the convention
of cmd.exe which is poorly documented and positively byzantine.

In the future we're going to solve this by not using cmd.exe at all and
instead embedding the cocmd.com interpreter into the system() function.
In the meantime, our documentation has been updated to help recalibrate
any expectation the user might hold regarding the security of using the
Windows command interpreter.

Fixes #644
2022-10-02 09:15:46 -07:00
Justine Tunney 0b5f84dd20
Introduce double quote support in cocmd.com shell 2022-10-02 09:15:46 -07:00
Justine Tunney 5005f2e446
Rewrite brk() and sbrk()
This change reduces the .bss memory requirement for all executables by
O(64kb). The brk system calls are now fully tested and figured out and
might be useful for tiny programs that only target System Five.
2022-10-01 23:11:56 -07:00
Justine Tunney b46f763ee7
Permit multiple certs with same common name
This is needed to support Cloudflare interop.
2022-09-27 18:17:04 -07:00
Gavin Hayes 87708c5d6e
Change accept type to struct sockaddr * (#630) 2022-09-20 07:49:16 -07:00
Justine Tunney 6cc9e08f1b
Introduce cosmocc and cosmoc++ toolchain commands 2022-09-20 04:34:03 -07:00
Justine Tunney 2cc1d5ac4c
Improve redbean method=get parameter handling 2022-09-19 20:04:39 -07:00
Justine Tunney c7a8cd21e9
Improve system call wrappers
This change improves copy_file_range(), sendfile(), splice(), openpty(),
closefrom(), close_range(), fadvise() and posix_fadvise() in addition to
writing tests that confirm things like errno and seeking behavior across
platforms. We now less aggressively polyfill behavior with some of these
functions when the platform support isn't available. Please see:

https://justine.lol/cosmopolitan/functions.html
2022-09-19 15:06:25 -07:00
Justine Tunney bc3d84341d
Fix redbean unix.nanosleep() documentation 2022-09-18 03:02:26 -07:00
Justine Tunney 3665232d66
Release redbean 2.0.19 2022-09-17 02:18:19 -07:00
Justine Tunney 775944a2d0
Fix important bugs in redbean
This change upgrades to the latest Chromium Zlib, fixes bugs in redbean,
and introduces better support for reverse proxies like Cloudflare. This
change improves the security of redbean and it's recommended that users
upgrade to the release that'll follow. This change also updates the docs
to clarify how to use the security tools redbean provides e.g. pledge(),
unveil(), and the MODE=asan builds which improve memory safety.
2022-09-17 02:18:06 -07:00
Justine Tunney 134ffee519
Change support vector to Windows 8+
Doing this makes binaries tinier, since we don't need to have all the
extra code for supporting a 32-bit address space. It also benefits us
because we're able to use WIN32 futexes, which makes locking simpler.

b69f3d2488 is what officially ended our
Windows 7 support. This change is merely a formalization. You can use
old versions of Cosmo now and forevermore if you need Windows 7 since
our repository is hermetic and vendors all its dependencies.

Won't fix #617
2022-09-15 03:55:05 -07:00
Paul Kulchenko 418bcb39af
Allow optional double quotes in dd parameters on macOS (#621) 2022-09-14 22:53:20 -07:00
Justine Tunney 3f49889841
Make important improvements
- Fix preadv() and pwritev() for old distros
- Introduce _npassert() and _unassert() macros
- Prove that file locks work properly on Windows
- Support fcntl(F_DUPFD_CLOEXEC) on more systems
2022-09-14 22:39:08 -07:00
Justine Tunney aab4ee4072
Add sys_ prefix to unwrapped system calls
This change also implements getlogin() and getlogin_r().
2022-09-13 11:20:35 -07:00
Justine Tunney 9fa6725067
Fix bug with sendfile() on XNU 2022-09-13 04:32:29 -07:00
Justine Tunney 06f9a5b627
Get repository to build with GCC 11
See #594
2022-09-13 04:14:55 -07:00
tkchia df08b541af
Bare metal VGA: implement "status report" escape codes (#613)
* Bare metal VGA: implement "status report" escape codes
* Minor fix to pseudoteletypewriter code

Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-09-13 02:14:10 -07:00
Connor ed5b763a0c
Refine V=0 command logging (#602)
This fixes off-by-one bugs, as well as missing carriage returns
caused by command truncation. When the terminal's width is 0 or
unknown, line feeds are used instead. Otherwise, the command is
padded with spaces to clear the line when the terminal is dumb.
2022-09-13 01:48:28 -07:00
Justine Tunney 6f7d0cb1c3
Pay off more technical debt
This makes breaking changes to add underscores to many non-standard
function names provided by the c library. MODE=tiny is now tinier and
we now use smaller locks that are better for tiny apps in this mode.
Some headers have been renamed to be in the same folder as the build
package, so it'll be easier to know which build dependency is needed.
Certain old misguided interfaces have been removed. Intel intrinsics
headers are now listed in libc/isystem (but not in the amalgamation)
to help further improve open source compatibility. Header complexity
has also been reduced. Lastly, more shell scripts are now available.
2022-09-12 23:36:56 -07:00
Justine Tunney b69f3d2488
Optimize memory layout 2022-09-12 04:26:52 -07:00
Paul Kulchenko 55ddf73484
Update redbean doc to correct return values (#610) 2022-09-11 19:51:18 -07:00
Justine Tunney b5cb71ab84
Use *NSYNC for POSIX threads locking APIs
Condition variables, barriers, and r/w locks now work very well.
2022-09-11 11:04:50 -07:00
Justine Tunney 155b378a39
Tidy up the threading implementation
The organization of the source files is now much more rational.
Old experiments that didn't work out are now deleted. Naming of
things like files is now more intuitive.
2022-09-10 02:56:25 -07:00
Justine Tunney 3265324e00
Don't relocate file descriptor memory
This change fixes #496 where ASAN spotted a race condition that could
happen in multithreaded programs, with more than OPEN_MAX descriptors
when using ZipOS or Windows NT, which require tracking open file info
and this change fixes that table so it never relocates, thus allowing
us to continue to enjoy the benefits of avoiding locks while reading.
2022-09-09 16:54:28 -07:00
Justine Tunney e97f1a99cf
Fix pthread stacks with larger guard size 2022-09-09 06:41:22 -07:00
Paul Kulchenko f68151c010
Add sigpending to redbean unix.* module (#605) 2022-09-09 04:20:13 -07:00
Paul Kulchenko 5140897c27
Update redbean to include params in slash redirect (#604) 2022-09-08 19:14:54 -07:00
Justine Tunney 9f963dc597
Clean up some of the threading code 2022-09-08 12:31:56 -07:00
Justine Tunney b73e35c6fa
Improve open source compatibility
This change tunes the default stack size for the outside world to 8mb
while at the same time, reducing Cosmopolitan's default stack size to
64kb. You can override the stack size using STATIC_STACK_SIZE(). Your
build scripts should point to o//ape/public/ape.lds

This change also fixes the definition of SOMAXCONN and removes AF_RDS
since it's not polyfilled and Python 3.11 complained.
2022-09-08 03:19:35 -07:00
Justine Tunney 0e2b1bfeed
Make garbage collection thread safe
- You can now use _gc(malloc()) in multithreaded programs
- This change fixes a bug where fork() on NT disabled TLS
- Fixed TLS code morphing on XNU/NT, for R8-R15 registers
2022-09-08 02:33:01 -07:00
Justine Tunney 571c2c3c69
Fix race condition in makedirs() 2022-09-07 23:12:44 -07:00
Justine Tunney dbf12c30b0
Make more compatibility improvements 2022-09-06 12:36:34 -07:00
Justine Tunney 55c6297e13
Make more compatibility improvements 2022-09-06 07:04:13 -07:00
Justine Tunney d721ff8938
Remove testonly keyword 2022-09-05 08:41:43 -07:00
Justine Tunney 9be364d40a
Implement POSIX threads API 2022-09-05 08:27:15 -07:00
Paul Kulchenko b0287ef34c
Tune a log message priority in redbean (#589) 2022-09-04 07:43:06 -07:00
Justine Tunney 6a04bc3318
Release redbean 2.0.18 2022-09-04 06:57:59 -07:00
Justine Tunney 8dd4ec68d0
Add more missing C / C++ headers 2022-09-04 04:53:52 -07:00
Justine Tunney b9dc74b672
Do some work on unbuffer command 2022-09-04 02:25:34 -07:00
Justine Tunney 3c00d8c29c Fix TLS linker warning
Fixes #565
Closes #577
Co-authored-by: tkchia <tkchia-cosmo@gmx.com>
2022-09-04 00:05:30 -07:00
Justine Tunney 0c70e8963d Add notpossible keyword
This is the same as `unreachable` except it always traps violations,
even if we're not running in MODE=dbg. This is useful for impossible
conditions relating to system calls. It avoids terrifying bugs where
control falls through to an unrelated function.
2022-09-03 20:35:31 -07:00
Justine Tunney c5c4dfcd21 Improve quality of raise(), abort(), and tkill()
This change fixes a nasty bug where SIG_IGN and SIG_DFL weren't working
as advertised on BSDs. This change also fixes the tkill() definition on
MacOS so it maps to __pthread_kill().
2022-09-03 20:17:54 -07:00
Gavin Hayes 263711965f
Change sigaction_f to match sysv signature (#585) 2022-09-02 05:08:35 -07:00
Paul Kulchenko 33b5b5b312
Remove path escaping from LaunchBrowser (#584)
This is so you can have `?query=args`.
2022-08-31 22:14:06 -07:00
Paul Kulchenko 164e2ab27b
Rename MapContentType to ProgramContentType (#570) 2022-08-24 09:35:17 -07:00
Paul Kulchenko d2cec808dc
Fix redbean to check for file read result when -F option is used (#569) 2022-08-23 20:54:51 -07:00
Paul Kulchenko 91a3252434
Add MapContentType to redbean (#566) 2022-08-22 22:02:58 -07:00
Paul Kulchenko 04cd62c319
Update redbean GetResponseBody to take response compression into account (#561) 2022-08-21 22:26:41 -07:00
Paul Kulchenko e204fc0820
Update redbean If-Modified-Since check to accept same timestamp (#562) 2022-08-21 22:03:30 -07:00
Justine Tunney 14e3025749 Release redbean 2.0.17 2022-08-21 18:35:53 -07:00
Justine Tunney 51fa5535ac Fix regression in Fetch() 2022-08-21 18:32:57 -07:00
Justine Tunney bea0b55576 Fix If-Modified-Since regression 2022-08-21 18:05:05 -07:00
Justine Tunney 57b39d8793 Revert recent fixupobj.com change 2022-08-21 17:33:54 -07:00
Justine Tunney d52170ca34 Avoid zip -d redbean.com foo issue on Windows 2022-08-21 16:32:57 -07:00
Justine Tunney bf757c4646 Fix _Thread_local with fork() on Windows 2022-08-21 15:51:44 -07:00
Justine Tunney 99a92048b4 Get awk to build and make it hackable 2022-08-21 13:39:20 -07:00
Justine Tunney 73845be1f0 Restore zip.com and .symtab files
This change restores the .symtab symbol table files in our flagship
programs (e.g. redbean.com, python.com) needed to show backtraces. This
also rolls back earlier changes to zip.com w.r.t. temp directories since
the right way to do it turned out to be the -b DIR flag.

This change also improves the performance of zip.com. It turned out
mmap() wasn't being used, because zip.com was assuming a 4096-byte
granularity, but cosmo requires 65536. There was also a chance to speed
up stdio scanning using the unlocked functions.
2022-08-21 00:17:20 -07:00
Justine Tunney 35203c0551 Do some string library work 2022-08-20 22:17:14 -07:00
Justine Tunney 83d41e4588 Clean up some code 2022-08-20 12:32:51 -07:00
Paul Kulchenko a51edaa0db
Remove .heartbeat.lua from redbean (#554)
It's been replaced by `OnServerHeartbeat`.
2022-08-20 01:31:58 -07:00
Justine Tunney 8835b82a7c Avoid sandboxing directory prerequisites
Landlock Make will no longer sandbox prerequisites that end with a
trailing slash. This means you can use use directory prerequisites
for detecting deleted files when using using globbing, without the
effect of unveiling the entire directory. When you do want make to
unveil directories, you can omit the trailing slash.
2022-08-19 13:02:25 -07:00
Justine Tunney 6bb9ce553e Remove some includes 2022-08-18 17:46:34 -07:00
Jared Miller db509ee158
LaunchBrowser by default in redbean-demo.com (#547) 2022-08-18 12:01:44 -07:00
Justine Tunney 30e1c5bca9 Add statfs() and fstatfs() to redbean unix module 2022-08-17 23:27:17 -07:00
Justine Tunney 2d479f7b11 Update definitions with latest help.txt changes 2022-08-17 21:57:08 -07:00
Justine Tunney d7372e651d Fix arg redbean lua definition 2022-08-17 21:23:44 -07:00
Justine Tunney 8e176fb026 Reduce build graph by another 14 percent
That's a 37% reduction total from what it was on 2022-05-23.
2022-08-17 20:01:21 -07:00
Justine Tunney f7ee9d7d99 Polyfill statfs() and fstatfs() on BSD distros 2022-08-17 14:54:03 -07:00
Danny Robinson fbdab72e3b
Create definitions.lua (#540) 2022-08-17 14:20:07 -07:00
Justine Tunney e5f705ace3 Fix redbean doc errors reported by stellartux 2022-08-17 00:10:17 -07:00
Justine Tunney a1aaf23dc1 Add more apis to redbean unix module
- Document unix.fcntl()
- Add POSIX Advisory Locks
- Add mask parameter to unix.poll()
- Add lowest parameter to unix.dup()
2022-08-16 23:23:34 -07:00
Justine Tunney ce588dd56b Release pledge.com 1.7 and landlockmake.com 1.3
- pledge("chown") now supported
- pledge("stdio") now allows killing self
- Write tests for pselect() and ppoll()
2022-08-15 19:52:00 -07:00