Merge branch 'nightly' into moar_pointer

.github/CODEOWNERS

@@ -4,8 +4,8 @@
 /stdlib/       @modularml/mojo-standard-library
 
 # Documentation
-/docs/         @scottamain @arthurevans
-/stdlib/docs/  @scottamain @arthurevans
+/docs/         @modularml/mojo-docs
+/stdlib/docs/  @modularml/mojo-docs @modularml/mojo-standard-library
 
 # Examples
 /examples/     @jackos

CODE_OF_CONDUCT.md

@@ -1,4 +1,4 @@
-# Code of Conduct
+# Code of conduct
 
 In the interest of fostering an open and welcoming environment, we as
 contributors and maintainers pledge to make participation in our project and
@@ -7,7 +7,7 @@ body size, disability, ethnicity, gender identity and expression, level of
 experience, nationality, personal appearance, race, religion, or sexual
 identity and orientation.
 
-## Our Standards
+## Our standards
 
 All community forums and spaces are meant for professional interactions that
 are friendly, inclusive, helpful, and collaborative. Examples of behavior that
@@ -32,7 +32,7 @@ participants include:
 - Conduct which could reasonably be considered inappropriate for the forum in
   which it occurs.
 
-## Our Responsibilities
+## Our responsibilities
 
 Project maintainers are responsible for clarifying the standards of acceptable
 behavior and are expected to take appropriate and fair corrective action in

CONTRIBUTING.md

@@ -21,7 +21,7 @@ Also, before opening a new issue, take a moment to search through the already
 submitted issues to avoid creating duplicate issues for the maintainers to
 address.
 
-### Writing high-quality bugs
+### Writing high-quality bug descriptions
 
 We encourage you to provide as much information about the issue as practical.
 The more details you provide, the faster we can resolve the issue. The following
@@ -157,7 +157,8 @@ experience with the process.
 
 You can use a pull request to propose a change or bug fix to the Mojo Standard
 Library, Mojo examples, or Mojo documentation. This page gives an overview of
-the process, especially for first-time contributors.
+the process. For a more detailed walkthrough, see
+[How to contribute to the Mojo standard library: a step-by-step guide](https://www.modular.com/blog/how-to-contribute-to-mojo-standard-library-a-step-by-step-guide).
 
 **Note:** Pull requests should be submitted against the `nightly` branch,
 which represents the most recent nightly build.

Kernels/scripts/stdlib/check-licenses.mojo

@@ -0,0 +1,46 @@
+# ===----------------------------------------------------------------------=== #
+# Copyright (c) 2024, Modular Inc. All rights reserved.
+#
+# Licensed under the Apache License v2.0 with LLVM Exceptions:
+# https://llvm.org/LICENSE.txt
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ===----------------------------------------------------------------------=== #
+
+import sys
+from pathlib import Path
+
+# We can't check much more than this at the moment, because the license year
+# changes and the language is not mature enough to do regex yet.
+alias LICENSE = String(
+    """
+# ===----------------------------------------------------------------------=== #
+# Copyright (c)
+"""
+).strip()
+
+
+def main():
+    target_paths = sys.argv()
+    if len(target_paths) < 2:
+        raise Error("A file path must be given as a command line argument.")
+
+    files_without_license = List[Path]()
+    for i in range(len(target_paths)):
+        if i == 0:
+            # this is the current file
+            continue
+        file_path = Path(target_paths[i])
+        if not file_path.read_text().startswith(LICENSE):
+            files_without_license.append(file_path)
+
+    if len(files_without_license) > 0:
+        print("The following files have missing licences 💥 💔 💥")
+        for file in files_without_license:
+            print(file[])
+        print("Please add the license to each file before commiting.")
+        sys.exit(1)

docs/changelog.md

@@ -16,10 +16,10 @@ what we publish.
 
 ### 🔥 Legendary
 
-- Tuple now works with memory-only element types like String.  Also, `Tuple.get`
-  now supports a form that just takes an element index but does not
-  require you to specify the result type.  Instead of `tup.get[1, Int]()` you
-  can now just use `tup.get[1]()`.
+- Tuple now works with memory-only element types like String and allows you to
+  directly index into it with a parameter exprssion.  This means you can now
+  simply use `x = tup[1]` like Python instead of `x = tup.get[1, Int]()`.  You
+  can also assign into tuple elements now as well with `tup[1] = x`.
 
 ### ⭐️ New
 
@@ -67,43 +67,147 @@ what we publish.
       print(d)  # prints `{'foo': [1, 2, 'something else'], 7: 'bar'}`
   ```
 
-- `List` now has a `pop(index)` API for removing an element
-  at a particular index.  By default, `List.pop()` removes the last element
-  in the list.
+- `List` now has several new methods:
+  - `pop(index)` for removing an element at a particular index.\
+    By default, `List.pop()` removes the last element in the list.
+  - `resize(new_size)` for resizing the list without the need to
+    specify an additional value.
+  - `insert(index, value)` for inserting a value at a specified index
+    into the `List`.
+  - constructor from `(ptr, size, capacity)` to to avoid needing to do a deep
+    copy of an existing contiguous memory allocation when constructing a new `List`.
 
 - `Dict` now has a `update()` method to update keys/values from another `Dict`.
 
+- `Set` now has named methods for set operations:
+  - `Set.difference()` mapping to `-`
+  - `Set.difference_update()` mapping to `-=`
+  - `Set.intersection_update()` mapping to `&=`
+  - `Set.update()` mapping to `|=`
+
+- `String` now has `removeprefix()` and `removesuffix()` methods.
+
+- `Optional` now implements `__is__` and `__isnot__` methods so that you can compare
+  an `Optional` with `None`, e.g. `Optional(1) is not None` for example.
+
+- The `ord` and `chr` functions have been improved to accept any Unicode character.
+
+- `Atomic` is now movable.
+
+- `Dict` and `List` are both `Boolable` now.
+
+- `atol` now handles whitespaces so `int(String( " 10 "))` gives back `10`
+  instead of raising an error.
+
+- `SIMD` now implements `__rmod__`.
+
+- `bool(None)` is now implemented.
+
+- The `DTypePointer` type now implements `gather` for gathering a `SIMD`
+  vector from offsets of a current pointer.  Similarly, support for `scatter`
+  was added to scatter a `SIMD` vector into offsets of the current pointer.
+
+- The `len` function for unary `range` with negative end values has been fixed
+  so that things like `len(range(-1))` work correctly now.
+
 - A low-level `__get_mvalue_as_litref(x)` builtin was added to give access to
   the underlying memory representation as a `!lit.ref` value without checking
   initialization status of the underlying value.  This is useful in very
   low-level logic but isn't designed for general usability and will likely
   change in the future.
 
+- The `testing.assert_equal[SIMD]()` now raises if any of the elements
+  mismatch in the two `SIMD` arguments being compared.
+
 - The `testing.assert_almost_equal` and `math.isclose` functions now have an
   `equal_nan` flag. When set to True, then NaNs are considered equal.
 
+- Mojo now supports declaring functions that have both optional and variadic
+  arguments, both positional and keyword-only. E.g. this now works:
+
+  ```mojo
+  fn variadic_arg_after_default(
+    a: Int, b: Int = 3, *args: Int, c: Int, d: Int = 1, **kwargs: Int
+  ): ...
+  ```
+
+  Positional variadic parameters also work in the presence of optional
+  parameters, i.e.:
+
+  ```mojo
+  fn variadic_param_after_default[e: Int, f: Int = 2, *params: Int]():
+    pass
+  ```
+
+  Note that variadic keyword parameters are not supported yet.
+
+- The `__getitem__`/`__getattr__` and related methods can now take indices as
+  parameter values instead of argument values.  This is enabled when defining
+  these as taking no arguments other than 'self' and the set value in a setter.
+  This enables types that can only be subscript into with parameters, as well
+  as things like:
+
+  ```mojo
+   struct RGB:
+     fn __getattr__[name: StringLiteral](self) -> Int:
+       @parameter
+       if name == "r":   return ...
+       elif name == "g": return ...
+       else:
+         constrained[name == "b", "can only access with r, g, or b members"]()
+         return ...
+    ```
+
+- Added `reversed()` for creating reversed iterators. Several range types,
+  `List`, and `Dict` now support iterating in reverse.
+  ([PR #2215](https://github.com/modularml/mojo/pull/2215),
+   [PR #2327](https://github.com/modularml/mojo/pull/2327))
+
+- Added left and right shift operations for `object`
+  ([PR #2247](https://github.com/modularml/mojo/pull/2247))
+
+- Added checked arithmetic operations.
+  ([PR #2138](https://github.com/modularml/mojo/pull/2138))
+
+  SIMD integral types (including the sized integral scalars like `Int64`) can
+  now perform checked additions, substractions, and multiplications using the
+  following new methods:
+
+  - `SIMD.add_with_overflow`
+  - `SIMD.sub_with_overflow`
+  - `SIMD.mul_with_overflow`
+
+  Checked arithimetic allows the caller to determine if an operation exceeded
+  the numeric limits of the type.
+
+- Added `os.remove()` and `os.unlink()` for deleting files.
+  ([PR #2310](https://github.com/modularml/mojo/pull/2310))
+
 ### 🦋 Changed
 
 - The behavior of `mojo build` when invoked without an output `-o` argument has
   changed slightly: `mojo build ./test-dir/program.mojo` now outputs an
   executable to the path `./program`, whereas before it would output to the path
   `./test-dir/program`.
+
 - The REPL no longer allows type level variable declarations to be
   uninitialized, e.g. it will reject `var s: String`.  This is because it does
   not do proper lifetime tracking (yet!) across cells, and so such code would
   lead to a crash.  You can work around this by initializing to a dummy value
   and overwriting later.  This limitation only applies to top level variables,
   variables in functions work as they always have.
+
 - `AnyPointer` got renamed to `UnsafePointer` and is now Mojo's preferred unsafe
   pointer type.  It has several enhancements, including:
   1) The element type can now be `AnyType`: it doesn't require `Movable`.
   2) Because of this, the `take_value`, `emplace_value`, and `move_into` methods
      have been changed to be top-level functions, and were renamed to
-     `move_from_pointee`, `initialize_pointee` and `move_pointee` respectively.
+     `move_from_pointee`, `initialize_pointee_*` and `move_pointee` respectively.
   3) A new `destroy_pointee` function runs the destructor on the pointee.
   4) `UnsafePointer` can be initialized directly from a `Reference` with
      `UnsafePointer(someRef)` and can convert to an immortal reference with
      `yourPointer[]`.  Both infer element type and address space.
+
 - All of the pointers got a pass of cleanup to make them more consistent, for
   example the `unsafe.bitcast` global function is now a consistent `bitcast`
   method on the pointers, which can convert element type and address space.
@@ -115,11 +219,66 @@ what we publish.
      `Reference` is a safe type - use `UnsafePointer` to do unsafe operations.
 
 - The `mojo package` command no longer supports the `-D` flag. All compilation
-  environment flags should be provided at the point of package use
-  (e.g. `mojo run` or `mojo build`).
+  environment flags should be provided at the point of package use (e.g.
+  `mojo run` or `mojo build`).
+
+- `parallel_memcpy` function has moved from the `buffer` package to the
+  `algorithm` package. Please update your imports accordingly.
+
+- `FileHandle.seek()` now has a whence argument that defaults to `os.SEEK_SET`
+  to seek from the beginning of the file. You can now set to `os.SEEK_CUR` to
+  offset by the current `FileHandle` seek position:
+
+  ```mojo
+  var f = open("/tmp/example.txt")
+  # Skip 32 bytes
+  f.seek(os.SEEK_CUR, 32)
+  ```
+
+  Or `os.SEEK_END` to offset from the end of file:
+
+  ```mojo
+  # Start from 32 bytes before the end of the file
+  f.seek(os.SEEK_END, -32)
+  ```
+
+  - `FileHandle.read()` can now read straight into a `DTypePointer`:
+
+    ```mojo
+    var file = open("/tmp/example.txt", "r")
+
+    # Allocate and load 8 elements
+    var ptr = DTypePointer[DType.float32].alloc(8)
+    var bytes = file.read(ptr, 8)
+    print("bytes read", bytes)
+    print(ptr.load[width=8]())
+    ```
+
+- `Optional.value()` will now return a reference instead of a copy of the
+  contained value.
+  ([PR #2226](https://github.com/modularml/mojo/pull/2226))
+
+  To perform a copy manually, dereference the result:
+
+  ```mojo
+  var result = Optional(123)
+
+  var value = result.value()[]
+  ```
+
+- Per the accepted community proposal
+  [`proposals/byte-as-uint8.md`](https://github.com/modularml/mojo/blob/main/proposals/byte-as-uint8.md),
+  began transition to using `UInt8` by changing the data pointer of `Error`
+  to `DTypePointer[DType.uint8]`.
+  ([PR #2318](https://github.com/modularml/mojo/pull/2318))
 
-- `parallel_memcpy` function has moved from the `buffer` package to the `algorithm`
-  package.  Please update your imports accordingly.
+- Continued transition to `UnsafePointer` away from the legacy `Pointer` type
+  in various standard library APIs and internals.
+  ([PR #2365](https://github.com/modularml/mojo/pull/2365),
+   [PR #2367](https://github.com/modularml/mojo/pull/2367),
+   [PR #2368](https://github.com/modularml/mojo/pull/2368),
+   [PR #2370](https://github.com/modularml/mojo/pull/2370),
+   [PR #2371](https://github.com/modularml/mojo/pull/2371))
 
 ### ❌ Removed
 
@@ -163,9 +322,6 @@ what we publish.
   longer cuts off hover previews for functions with functional arguments,
   parameters, or results.
 
-- [#1245](https://github.com/modularml/mojo/issues/1245) [Feature Request]
-  Parameter Inference from Other Parameters.
-
 - [#1901](https://github.com/modularml/mojo/issues/1901) Fixed Mojo LSP and
   documentation generation handling of inout arguments.
 
@@ -196,3 +352,12 @@ what we publish.
 
 - [#1675](https://github.com/modularml/mojo/issues/1675) Ensure `@value`
   decorator fails gracefully after duplicate field error.
+
+- [#2068](https://github.com/modularml/mojo/issues/2068)
+  Fix simd.reduce for size_out == 2
+  ([PR #2102](https://github.com/modularml/mojo/pull/2102))
+
+- [#2224](https://github.com/modularml/mojo/issues/2224)
+  `object` now implements `__truediv__`, `__floordiv__` and related divison
+  and modulo operators.
+  ([PR #2230](https://github.com/modularml/mojo/pull/2230))

docs/faq.md

@@ -285,8 +285,30 @@ You can [get the Mojo SDK here](https://developer.modular.com/download)!
 
 ### Is the Mojo Playground still available?
 
-Yes, you can [get access today](https://www.modular.com/get-started)
-to the Mojo Playground, a hosted set of Mojo-supported Jupyter notebooks.
+Yes, but it's different. When we first announced Mojo, it was available
+only through login, in a JupyterLab environment. Now that Mojo is available
+for local development, we've shut down that service (you can instead [run
+Mojo notebooks
+locally](https://github.com/modularml/mojo/tree/main/examples/notebooks#readme)).
+
+The new [Mojo Playground](/mojo/playground) is built into the docs website
+and does not require login.
+
+- It provides access to Mojo and the Mojo standard library. It does not have
+  network access, so you can't install additional Mojo or Python packages.
+
+- It doesn't include any Python packages by default. In the future,
+  we intend to make some common Python packages available to import in the
+  Playground.
+
+- You can download your code or share it as a gist, but there's no mechanism
+  for saving code in the Playground itself. Any changes will be lost when you
+  switch code examples (as well as in the event of a server refresh or update).
+  If you come up with something you want to save, download it or share it
+  using buttons in the Playground toolbar.
+
+- There might be some bugs. Please [report issues and feedback on
+  GitHub](https://github.com/modularml/mojo/issues/new/choose).
 
 ### What are the license terms for the SDK?