New mpy 6 file format with improved bytecode, addition of renesas-ra port
In this release of MicroPython the .mpy file format has been significantly
reworked and bumped to version 6. The new format now contains data and
bytecode organised to be mostly static and ready to run in-place. To
achieve this, qstr references from bytecode (and native machine code) now
go through a lookup table, to convert from local qstr number in the module
to global qstr number in the runtime that the module executes within. This
means bytecode (and most native code) does not need rewriting when it is
imported. Along with this change, the jump opcodes have been optimised to
emit 1-byte jump offsets when possible, which gives a good reduction in
bytecode size, saving RAM and reducing the size of .mpy files.
The change to the bytecode means that code constants (like long strings,
bytes objects and floats) are now shared and de-duplicated across the whole
module, rather than each code object having a separate constant table. The
parser can also now turn a tuple of constants into a constant object, so
the tuple doesn't need to be built at runtime each time it is used.
Furthermore, any constant object can now be used in "X = const(obj)"
statements.
With these bytecode changes, .mpy files see around a 10% reduction in size
and frozen .mpy code sees about a 5% size reduction. Heap memory use after
import is also down by around 5% (more savings are seen if the code uses
lots of constants). Due to the qstr indirection performance of the VM is
slightly reduced on some benchmarks by a few percent, but the benefits of
smaller code and reduced RAM usage are deemed to outweigh this. See commit
f2040bfc7ee033e48acef9f289790f3b4e6b74e5 for more details.
In the runtime, support has been added for multiple *args and **args in a
function call, following PEP 448. Built-in modules can now have (optional)
mutable attributes, and this is used to implement sys.ps1/sys.ps2 which
allow the user to customise the REPL prompt. This can be useful, for
example, to configure a different REPL for different boards so they can be
easily distinguished. The sys module has also had the MicroPython git
version and build date appended to sys.version, sys.implementation._machine
has been added, and sys.implementation.mpy renamed to
sys.implementation._mpy. The sys module can now be used to fully identify
the MicroPython version and target it is running on (and the platform
module provides additional information when it's available).
The MP_REGISTER_MODULE macro is now used to register all built-in C modules
and the third argument to this macro has been removed. Any user-C-modules
will need to be updated. Also support for MICROPY_PORT_BUILTIN_MODULES is
removed because this is no longer needed, and should be replaced by usage
of MP_REGISTER_MODULE (see commit 0a92469c10542d57ef0ea6b6cf2ed0cae9491dbe
for example).
The MP_REGISTER_MODULE macro is now used to register all built-in C modules
and the third argument to this macro has been removed. Also support for
MICROPY_PORT_BUILTIN_MODULES is removed because this is no longer needed.
mpy-cross now supports the -march=armv6m option, which can be used with the
rp2 port. The -mno-unicode has been removed.
The littlefs2 library has been updated to v2.5.0 which includes some
performance improvements and minor fixes.
At the REPL, a tab will now insert an indent when it follows whitespace,
and auto-indent is automatically disabled if a space or tab follows an
auto-indent. This means that Python code can be pasted into the standard
REPL without messing up the indentation.
mpremote is now at version 0.3.0, with a progress indicator shown when
copying large files, and recursive copy fixed on Windows. It also has the
following commands added: --help, --version, version, resume, soft-reset,
umount.
The esp32 port has a new machine.ADCBlock class and updated ADC class.
Fixes have been made to ADC2 channel mapping, low PWM frequencies,
UART initialisation of unspecified parameters, and RMT channel on S2/S3/C3
variants. The Pin.PULL_HOLD constant has been replaced by a "hold" keyword
argument to the Pic constructor, and it's now possible to configure pin
drive strength. I2C write-then-read transfers have been optimised to be
faster. WLAN.config() now supports setting/getting the txpower. New
boards include LOLIN C3 MINI ESP32-C3, LilyGO LoRa32, and UM ESP32-S3 based
boards.
The mimxrt port has updated the nxp_driver to v2.10 and fixed USB CDC RX
handling to not block. Pin drive constants have been renamed to DRIVE_x to
match other ports, and the machine.I2S class has been added. New boards
include Olimex RT1010 and MIMXRT1015_EVK.
The nrf port now includes uasyncio in the default board manifest, and
additional features have been enabled on nrf52840 and nrf9160 boards to
match the features on stm32, rp2 and esp32. This includes enabling the
json, re, zlib and framebuf modules.
There is a new renesas-ra port which targets Renesas RA4M1, RA4W1, RA6M1
and RA6M2 MCUs, with board definitions for evaluation kits for these MCUs,
along with RA4M1 clicker. This port currently supports Pin, ADC, SoftI2C,
I2C, SoftSPI, SPI, UART, Timer and RTC classes in the machine module, along
with the VFS and an internal flash filesystem.
The rp2 port now has optional USB MSC support, and optional dupterm
support. The machine.UART class has init and deinit methods added, and
the ucryptolib module is now available. The lwIP network stack has been
integrated, along with mbedtls for the ssl module, and these can be used
with the new Wiznet driver to provide Ethernet support. The Wiznet
W5100S-EVB-Pico board has been added which makes use of this driver. The
day-of-week value reported by time.localtime() has been fixed, memory
corruption when a thread is created on core1 has been fixed, and the UART
now uses a mutex when reading/writing to prevent character duplication in
code that uses both cores.
The stm32 port has added support for G0, G4 and WL MCUs. The new boards
for these MCUs are NUCLEO_G0B1RE, NUCLEO_G474RE and NUCLEO_WL55. The
ARDUINO_PORTENTA_H7 board has also been added, with full WiFi and BLE
support. The rfcore on WB55 MCUs has been improved by removing an extra
layer of buffering, and the restriction on the use of address resolution
has been removed, meaning that the rfcore WS firmware should be updated
to v1.12.0 or later; see commit 26b1d31eda292d9cd5cfc68f14ce8055256fade8.
The board-level configuration of mboot has been significantly improved to
allow more control over system clocks, more hooks into the start up
process, and a hook for state change. And the CRC32 of DFU files is now
verified before writing them.
The pyb.CAN class has been improved to support FD frames and buses with
mixed classic/extended-ID nodes, allowing extended IDs to be specified on a
per-message basis. CAN.recv also now returns a value indicating if the
received message had an extended ID or not. And CAN.initfilterbanks has
been removed, replaced with the num_filter_banks keyword argument to
CAN.init. These are all breaking changes to the pyb.CAN class API and code
will need to be updated accordingly; see commit
5cdf9645711cc12b45e602ef5795c33895116fc4 for details.
The unix port now shows compiler information in the REPL banner to make it
consistent with how bare-metal ports work, and the "-X realtime"
command-line option has been added for macOS to get improved timing. All
variants have switched to use the VFS subsystem, and use the VfsPosix class
for mounting the host's filesystem. The os module has been consolidated
with all other ports and the minimal and standard variants gain the unlink,
chdir, getcwd and listdir functions.
The windows port has also switched to the VFS subsystem and VfsPosix. It
adds support for micropython.schedule and uasyncio on the dev variant. And
BCryptGenRandom has been used to implement os.urandom.
The zephyr port upgraded Zephyr to v3.0.0.
The change in code size since the previous release for various ports is
(absolute and percentage change in the text section):
bare-arm: +792 +1.394%
minimal x86: +2091 +1.286%
unix x64: +10231 +1.995%
unix nanbox: +9777 +2.162%
stm32: +184 +0.047% PYBV10
cc3200: +904 +0.493%
esp8266: -1240 -0.177% GENERIC
esp32: +6464 +0.425% GENERIC
nrf: +11288 +6.472% pca10040
rp2: +14432 +2.929% PICO
samd: +1352 +0.970% ADAFRUIT_ITSYBITSY_M4_EXPRESS
The changes that dominate these numbers are:
- bare-arm, minimal, cc3200, samd: reworked bytecode and mpy format,
optimisation of jump opcodes to use 1-byte offsets, de-duplication of
constants at the module level, and PEP 448 support
- unix: switching to use VFS, and new mpy format and related changes
- esp8266: removal of esp.info and machine.WDT.deinit
- esp32: addition of machine.ADCBlock, and new mpy format changes
- nrf: enabling uasyncio, enabling MICROPY_PY_IO
- rp2: enabling ucryptolib with mbedtls
Note that the stm32 port's size remained roughly unchanged. This is
because it has a decent amount of frozen Python code and the reduction in
frozen code size due to the reworked bytecode has offset the increase in
runtime/VM size due to this change. Ports like bare-arm, minimal, cc3200
and samd do not have any frozen code so only see the increase in runtime/VM
size. Production-level projects that have a significant amount of frozen
code have seen around a 4% overall reduction in their firmware size.
Thanks to everyone who contributed to this release: Algy Tynan, Andrew
Leech, Andrew Scheller, Artyom Skrobov, Asensio Lorenzo Sempere, Bradley
Wogsland, Cem Eliguzel, Chris Wilson, Christian Decker, Christian Zietz,
Christophe Priouzeau, Clayton Mills, Damien George, Daniel Jour, Daniël van
de Giessen, Dave Hylands, David Lechner, Emil Kondayan, Herwin Grobben,
iabdalkader, IhorNehrutsa, Jan, Jared Hancock, jason, Jeff Epler, Jeremy
Herbert, Jim Mussared, Jon Bjarni Bjarnason, Jonathan Hogg, Jos Verlinde,
Kattni Rembor, Lars Kellogg-Stedman, Luiz Brandao, marcidy, Martin Fischer,
Maureen Helm, Meriç SARIIŞIK, Michael Himing, Michael O'Cleirigh,
MikeTeachman, Peter D. Gray, Peter Hinch, Peter Züger, Phil Howard, Philipp
Ebensberger, Rob Knegjens, robert-hh, Sean Coates, Seon Rozenblum, Stephane
Smith, stijn, Takeo Takahashi, ubi de feo, Waterlens, wemos, YoungJoon
Chun, Yukai Li.
What follows is a detailed list of changes, generated from the git commit
history, and organised into sections.
Main components
===============
all:
- fix MICROPY_OBJ_REPR_D compilation with msvc
- update Python formatting to latest Black version 22.1.0
- use mp_obj_malloc everywhere it's applicable
- reformat remaining C code that doesn't have a space after a comma
- remove third argument to MP_REGISTER_MODULE
- rename MICROPY_PY_WIZNET5K to MICROPY_PY_NETWORK_WIZNET5K
py core:
- objstr: support '{:08}'.format("Jan") like Python 3.10
- modmath: add math.tau, math.nan and math.inf constants
- qstr: separate hash and len from string data
- qstr: use `const` consistently to avoid a cast
- gc: update debug code to compile with changes to qstr pool types
- rework bytecode and .mpy file format to be mostly static data
- bc.h: fix C++ compilation of public API
- objmodule: support delegating failed attr lookups
- modsys: add optional attribute delegation
- modsys: add optional sys.tracebacklimit attribute
- modsys: add optional mutable attributes sys.ps1/ps2 and use them
- parse: simplify handling of const int parse nodes
- parse: put const bytes objects in parse tree as const object
- parse: handle check for target small-int size in parser
- showbc: remove global variables and make DECODE_PTR work correctly
- smallint: introduce MP_SMALL_INT_BITS macro
- objgenerator: fix unused variables when native gen extracts prelude
- change jump opcodes to emit 1-byte jump offset when possible
- emitbc: add check for bytecode jump offset overflow
- change jump-if-x-or-pop opcodes to have unsigned offset argument
- builtinimport: alias sys to usys if import weak links aren't enabled
- bc.h: fix C++20 compilation with "volatile"
- compile: only show raw code that is bytecode
- asmxtensa: fix use of l32i/s32i when offset won't fit in encoding
- emitnative: don't store prelude at end of machine code if not needed
- vm: prevent array bound warning when using -MP_OBJ_ITER_BUF_NSLOTS
- runtime: allow multiple **args in a function call
- runtime: allow multiple *args in a function call
- runtime: do not overallocate when len is known
- runtime: drop new_alloc < 4 check
- runtime: use size_t/ssize_t instead of uint/int
- emitbc: assert that a small int fits its encoding when emitting one
- fix compiling and decoding of *args at large arg positions
- runtime: remove unnecessary check for kw_value == MP_OBJ_NULL
- makeqstrdefs: cleanup and extend source file classification
- emitinlinethumb: use 16 bit encodings for PUSH LR and POP PC
- scheduler: add support for scheduling static C-based callbacks
- emitnative: revert "don't store prelude at end of machine code if..."
- persistentcode: define enum values for obj types instead of letters
- parse: factor obj extract code to mp_parse_node_extract_const_object
- parse: print const object value in mp_parse_node_print
- parse: add MICROPY_COMP_CONST_TUPLE option to build const tuples
- persistentcode: support loading and saving tuples in .mpy files
- objtype: convert result of user __contains__ method to bool
- modsys: append MicroPython git version and build date to sys.version
- modsys: rename sys.implementation.mpy to sys.implementation._mpy
- modsys: introduce sys.implementation._machine constant
- emitcommon: don't implicitly close class vars that are assigned to
- obj: introduce mp_obj_malloc macro to allocate, and set object type
- objfloat: explain why mp_obj_malloc isn't used
- malloc: introduce m_tracked_calloc, m_tracked_free functions
- objmodule: move stray #include to top of file
- persistentcode: remove obsolete comment about qstr window size
- persistentcode: remove unicode feature flag from .mpy file
- asmx64: support full range of regs in asm_x64_lea_disp_to_r64
- bc: provide separate code-state setup funcs for bytecode and native
- emitnative: put a pointer to the native prelude in child_table array
- compile: allow new qstrs to be allocated at all compiler passes
- compile: de-duplicate constant objects in module's constant table
- parse: allow all constant objects to be used in "X = const(o)"
- makemoduledefs.py: allow multiple ways to register a module
- make builtin modules use MP_REGISTER_MODULE
- remove support for MICROPY_PORT_BUILTIN_MODULES
- asmthumb: fix PC relative load by sign extending the constant
- emitnative: simplify generation of code that loads prelude pointer
- emitnative: provide dedicated local for exception unwind handler ptr
- asmthumb: fix offset variable name in ldr, ldrh and strh functions
- asmthumb: add asm_thumb_ldrh_reg_reg_i12_optimised() helper func
- asmarm: add asm_arm_ldrh_reg_reg_offset() helper func
- asm: add ASM_LOAD16_REG_REG_OFFSET macro for load-u16 with offset
- emitnative: access qstr values using indirection table qstr_table
- asmthumb: make ARMv7-M instruction use dynamically selectable
- persistentcode: select ARMV6M as maximum when __thumb2__ not defined
- dynruntime.mk: add basic support for armv6m architecture
- asmthumb: provide implementations of clz/ctz for msvc
- builtin: clean up and simplify import_stat and builtin_open config
- emitinlinethumb: make ARMv7-M instruction use dynamically selectable
- emitinlinethumb: make float instruction use dynamically selectable
- change makemoduledefs process so it uses output of qstr extraction
- persistentcode: remove remaining native qstr linking support
- bc: remove unused mp_opcode_format function
- compile: give the compiler a hint about num nodes being non-zero
- makemoduledefs.py: remove shebang line and adjust style of comment
- parse: work around xtensa esp-2020r3 compiler bug
- dynruntime: add macros to create a new dict and store to dicts
- dynruntime: add macros to access more types and mp_const_empty_bytes
- makemoduledefs.py: emit useful error for legacy MP_REGISTER_MODULE
extmod:
- network_ninaw10: implement MP_STREAM_POLL in ioctl
- modusocket: make setsockopt return if NIC is not connected
- modusocket: add makefile() method and common socket options
- modusocket: create new sockets in blocking mode
- modusocket: support additional args to getaddrinfo
- machine_i2c: increase default SoftI2C timeout to 50ms
- modubinascii: add newline keyword to b2a_base64 function
- moduplatform: move platform PP definitions into a header
- moduzlib: fix parsing zlib header dict size
- moduos: create general uos module to be used by all ports
- modure: set subject begin_line so ^ doesn't match interior
- uasyncio: allow task state to be a callable
- uasyncio: fix gather cancelling and handling of exceptions
- extmod.cmake: add micropy_lib_mbedtls component
- modusocket: implement optional socket.listen backlog argument
- modusocket: provide config macro for socket.listen backlog deflt
- uasyncio: make Python Task match C version with use of asserts
- uasyncio: fix bug with task ending just after gather is cancel'd
- uasyncio: rename and merge TaskQueue push/pop methods
- move font_petme128_8x8.h from ports/stm32 to extmod
- make extmod modules use MP_REGISTER_MODULE
- make port-included extmod modules use MP_REGISTER_MODULES
- revert accidental usocket->socket rename
- webrepl: fix setting password in foreground mode and factor code
- modusocket: fix polling of a new socket
- modusocket: bind unconnected socket to default NIC in setsockopt
- modusocket: fix errcode returned from socket read/write
- modusocket: add socket type print function
- modusocket: add support for socket events callback
- modusocket: fix socket_make_new argument parsing
- modusocket: add timeout and callback to socket object
- modusocket: add sendall function
- network_ninaw10: add support for socket events callback
- machine_i2c: add optional support for write-then-read transfers
- uasyncio: fix edge case for cancellation of wait_for
- nimble: add support for reading whole HCI UART packets
- network_wiznet5k: add Wiznet Ethernet network interface
- modussl_mbedtls: poll EVENT_POLL_HOOK in ssl handshake loop
- modurandom: fix missing void in empty argument list
- extmod.cmake: require components to be explicitly enabled
- modusocket: add socket state to track new/listening/conn/closed
- modusocket: fix polling of closed sockets
- extmod.cmake: fix hard-coded mbedtls config file path
shared:
- readline: make tab insert an indent when it follows whitespace
- readline: disable auto-indent if space/tab follows added indent
drivers:
- ninaw10: add function to check socket state/data availability
- ninaw10: fix timeout handling to match modusocket
- ninaw10: return standard error numbers
- ninaw10: add NIC-level ioctl function
- sdcard: allow setting the final SPI baudrate
- codec: add driver for the WM8960 codec
- ninaw10: fix isconnected to return False when inactive
- dht/dht.py: change default import module as machine
- ninaw10: update driver to support firmware 1.5.0
- bus: rename MP_SPI_ADDR_IS_32B to MICROPY_HW_SPI_ADDR_IS_32BIT
- lsm6dsox: add support for SPI mode
- sdcard: fix CSD version 1.0 device size calculation
- sdcard: make ioctl(4), ioctl(5) return num blocks, block size
- sdcard: fix address calculation on v2 SDSC cards
mpy-cross:
- add armv6m architecture option
lib:
- re1.5: distinguish between subject start-of-line and start-of-srch
- stm32lib: update library for G0 v1.5.1
- fsp: add renesas fsp git repository as submodule
- littlefs: update littlefs2 to v2.5.0
- littlefs: guard lfs2_file_rawopen with LFS2_NO_MALLOC
- littlefs: remove assignment of variables to themselves
- nxp_driver: update nxp_driver to v2.10
- wiznet5k: add submodule for Wiznet Ethernet drivers
Support components
==================
docs:
- library: specify additional ADC methods and new ADCBlock class
- esp32: document expanded ADC API in quickref
- differences: update differences now that math.tau/inf/nan exist
- differences: update python_36 now that b2a_base64 accepts newline
- remove reference to obsolete neopixel_write function
- develop/porting.rst: fix build and import problems in the example
- reference/isr_rules.rst: fix inconsistent variable name in example
- esp32/quickref: update ADC documentation
- reference/isr_rules.rst: describe uasyncio-IRQ interface
- library/collections.rst: use class for deque and OrderedDict
- use the correct * keyword-only notation
- library/socket.rst: document socket as a class
- library/pyb.Timer.rst: add pyb.Timer class constants
- library/pyb.rst: add pyb.hid_mouse and pyb.hid_keyboard constants
- library/pyb.DAC.rst: add DAC class constants
- library/machine.SPI.rst: add class constant SoftSPI.MSB and .LSB
- library/machine.WDT.rst: use correct case for WDT.feed
- library/esp.rst: document the osdebug function
- library/pyb.SPI.rst: document default for prescaler argument
- library/machine.Pin.rst: document defaults for Pin.init
- library/uasyncio.rst: fix description of ThreadSafeFlag.wait
- library/machine.UART.rst: add details for `invert` parameter
- differences/python_35: mark PEP 486 as not applicable
- library/esp32.rst: mark esp32.Partition as not taking kw args
- esp32/quickref: document GPIO drive strength
- library/machine.Pin: update to use preferred DRIVE_x constants
- esp32/quickref: refine deep-sleep power-saving notes
- library/machine.I2S: clarify what rate refers to
- library/pyb.CAN: update CAN docs to match revised API
- add quickref and docs for mimxrt, including network.LAN docs
- reference/mpremote: add details about mount usage over soft reboot
- differences: update Python 3.5 diff, with optional listen backlog
- reference: update constrained docs now that tuples can be const
- library/network.WLAN: add "txpower" to list of known WLAN configs
- renesas-ra: add renesas-ra docs files
- update files for renesas-ra port
- renesas-ra: remove unused image files
- reference/mpyfiles: remove docs about mpy flags and qstr win size
- mimxrt: add i.MX RT1015 to general.rst, and a few clarifications
- mimxrt: move the pinout tables to a separate document
- reference/manifest: add link to details about opt-level for freeze
- reference/mpyfiles: update .mpy description to match latest format
- update Raspberry Pi URLs from .org to .com
examples:
- embedding: fix build with updated sys and os modules
- natmod/features3: add example to test more natmod features
tests:
- run-multitests.py: ignore lld_pdu_get_tx_flush_nb msgs from IDF
- run-multitests.py: set HOST_IP so tests work between PC and board
- multi_net: add testing key/cert to SSL server/client test
- multi_net: close accepted sockets when tests are done
- multi_net/udp_data.py: allow reusing port before bind
- multi_net/udp_data.py: make UDP test more reliable
- run-perfbench.py: allow a test to SKIP, and to have a .exp file
- perf_bench: add perf tests for qstr interning and importing .mpy
- perf_bench: add perf test for yield-from execution
- run-perfbench.py: use SKIP consistently, and increase print width
- extmod/vfs_fat_finaliser.py: make finalisation more robust
- run-tests.py: skip repl tests when running windows underneath
- perf_bench: skip bm_chaos test if random.randrange is unavailable
- unix: add coverage test for freezing various objects
- basics: add test for creating small-ints in nan-box builds
- run-tests.py: include test files ending in _set as set tests
- micropython: switch from set.pop to raise-0 to test exc strings
- extmod/vfs_posix.py: only test statvfs if it exists
- cmdline/cmd_showbc: fix spelling of sequence
- extmod: add test for machine.I2S data rate
- extmod: update I2S rate test to work on mimxrt
- basics/fun_callstardblstar: add coverage test
- basics/fun_callstardblstar: add test for large arg allocation
- pyb: update CAN tests to match revised CAN API
- inlineasm: add test for PUSH LR and POP PC
- perf_bench: update import tests for changes to .mpy consts
- micropython: add tests that const tuples don't use the heap
- extmod/uasyncio_gather: make double-raise gather test reliable
- run-tests.py: add timeout for running PC-based MicroPython test
- cmdline: add test for REPL auto-indent
- renesas-ra: add tests for renesas-ra port
- run-tests.py: update for renesas-ra port
- renesas-ra: remove unsupported feature test of Pin
- extmod: increase timing on uasyncio tests to make more reliable
- fix tests to use sys.implementation._mpy
- perf_bench: update .mpy file header to remove old unicode flag
- run-perfbench.py: return error code if any test fails on target
- thread: use less resources for stress_aes if settrace enabled
- run-tests.py: exclude settrace tests when using native emitter
- multi_net: skip SSL test if relevant modules aren't available
- micropython: fully unlink nested list in extreme exc test
- extmod: use bytearray instead of bytes for uctypes test
- micropython: add more test cases for native generators
- run-tests.py: allow running tests via mpy-cross on remote targets
- run-perfbench.py: allow running tests via mpy and native emitter
- basics: unlock heap if skipping nanbox small-int test
- run-tests.py: handle case where mpy-cross fails to compile script
- run-tests.py: enable `-X realtime` option for macOS tests
- extmod: change expected errno code from 36 to 30 in VfsLfs2 test
- multi_net: fix TCP accept test when using system error numbers
- run-multitests.py: use LAN for IP address if WLAN doesn't exist
- micropython: make import_mpy_native_gc run on ARMv6-M and above
- micropython: make import_mpy_native test run on all architectures
- run-tests.py: add rp2 test target
- move native while test from pybnative to micropython
- net_inet: remove broken api.telegram.org from tests
- run-multitests.py: read IP address from boot nic if available
- basics: add .exp file for sys.tracebacklimit test
- stress: adjust bytecode_limit test so it can SKIP if no memory
tools:
- mpremote: use machine instead of umachine in commands
- ci: test math constants with MICROPY_OBJ_REPR_D
- uf2conv.py: update to latest version
- autobuild: provide .uf2 images for esp32 builds when available
- mpremote: fix "fs cp -r" on Windows
- mpremote: make ConsolePosix work without .raw attribute
- mpremote: during soft reboot wait long enough for 115200 data
- mpremote: accept both --help and help to show usage
- mpremote: correctly manage mounted flag during soft-reset
- mpremote: fix special handling of ctrl-D when host FS is mounted
- verifygitlog.py: ignore line length in body if it's a URL
- ci.sh: update IDF v4.4 build to use v4.4 tag
- ci.sh: run urandom test scripts as part of native module tests
- mpremote: add "resume" and "soft-reset" commands
- mpremote: add "umount" command
- upip.py: remove unused op_basename() function
- mpy-tool.py: fix frozen comment generation to escape chars
- ci.sh: run performance benchmark as part of all unix test runs
- mpremote: bump version to 0.2.0
- gen-cpydiff: skip Black fmt comments
- mpremote: support any prompt string when detecting soft reset
- mpremote: allow running mpremote with `python -m`
- mpremote: improve reliability of mount after soft reboot
- mpremote: show progress indicator when copying large files
- mpremote: add option to mount cmd to allow "unsafe" symlinks
- mpy-tool.py: support loading tuples from .mpy files
- mpy-tool.py: support freezing tuples and other consts
- mpy-tool.py: optimise freezing of ints that can fit a small int
- mpy-tool.py: optimise freezing of empty str and bytes objects
- mpy-tool.py: make global qstr list a dedicated class
- mpy-tool.py: optimise freezing of str when str data is a qstr
- mpy-tool.py: intern more strings when freezing
- codeformat.py: remove ports/unix/*.py from exclusion list
- codeformat.py: include more ports .c/.h files in formatting
- ci.sh: don't run uasyncio_gather test on macOS
- ci.sh: update for ports/renesas-ra
- uncrustify: make sure a space exists after a comma
- autobuild: update for ports/renesas-ra
- mpy_ld.py: remove obsolete QSTR_WINDOW_SIZE constant
- ci.sh: run full test suite on stackless and settrace builds
- ci.sh: for esp8266 CI install esptool 3.3.1
- mpy-tool.py: remove obsolete unicode flag in .mpy header
- ci.sh: add build of W5100S_EVB_PICO board to rp2 CI
- mpy-tool.py: rework .mpy merging feature
- ci.sh: add test for mpy-tool's merging feature
- mpy-tool.py: improve generated frozen identifiers
- mpremote: add command to print the version
- mpremote: bump version to 0.3.0
- ci.sh: build NUCLEO_H743ZI with -O2 optimisation level
- mpy_ld.py: support R_XTENSA_PDIFF32 relocation
- ci.sh: build native .mpy examples for armv7emsp, xtensawin in CI
- mpy_ld.py: support GOT entries that reference inside mp_fun_table
CI:
- workflows: show context for qemu-arm test failures
- workflows: add new workflow to test .mpy file format and tools
- workflows: add new workflow to test embedding example
- workflows: add workflow for renesas-ra port
The ports
=========
all ports:
- consolidate inclusion of umachine module in built-ins
- recompile bytecode tests now that .mpy format changed
- allow boards to define additional network interfaces
- recompile bytecode tests now that .mpy format changed
- reformat more C and Python source code
- rename thread_t to mp_thread_t
- use default VFS config for import_stat and builtin_open
bare-arm port: no changes specific to this component/port
cc3200 port:
- make port-specific modules use MP_REGISTER_MODULE
esp8266 port:
- moduos: convert module to use extmod version
- mpconfigport.h: remove config values that are the defaults
- modesp: remove esp.info() function
- make port-specific modules use MP_REGISTER_MODULE
- machine_wdt: remove deinit method for watchdog
- etshal.h: remove unneeded function declaration
esp32 port:
- machine_adcblock: add new machine.ADCBlock class and update ADC
- pin MicroPython to core 1 again
- esp32_rmt: call rmt_driver_install directly if running on core 1
- main: allocate at most 1/2 of available IDF heap for MP heap
- README.md: fix URL for esp-idf installation
- modnetwork: fix test for WIFI_AUTH_MAX for IDF v4.3.0
- main: use heap_caps_get_info on IDF <4.1 to compute total heap
- boards: provide custom deploy_c3.md for ESP32-C3 boards
- partitions: increase size of app-part from 0x180000 to 0x1F0000
- machine_pwm: fix PWM not allowing frequencies < 611 Hz
- machine_pwm: clean up macro names and their use
- esp32_rmt: select correct last RMT channel on S2, S3, C3 variants
- main: automatically size SPIRAM heap when allocated using malloc
- adc: fix wrong mapping between ADC2 channel and GPIO number
- remove unneeded modesp.h
- create .uf2 binaries for S2 and S3 chips
- partitions-16MiB.csv: increase 14MiB filesystem to maximum size
- machine_adcblock: fix ADC bit width for ESP32-S3
- machine_adc: fix configuration of default ADC atten value
- boards: add three UM ESP32-S3 based boards
- mpconfigport.h: use the "extra" feature level
- mpconfigport.h: enable remaining features to get to "extra" level
- esp32_partition: add support for specifying block_size
- modules: create ffat partition object with block_size=512
- README.md: update list of supported IDF versions
- boards: add specific deploy instructions for S3 variant
- uart: correctly init low-level UART driver for REPL
- machine_uart: allow limited configuration of REPL UART
- sleep one tick in MICROPY_EVENT_POLL_HOOK
- machine_pwm: always set the duty cycle when setting the frequency
- machine_uart: make UART.init preserve unspecified parameters
- machine_pin: add support for pin drive strength
- moduos: convert module to use extmod version
- machine_pin: add new hold keyword argument and remove PULL_HOLD
- modesp32: add new gpio_deep_sleep_hold function
- boards/UM_TINYPICO: remove use of PULL_HOLD
- boards: add LOLIN C3 MINI ESP32-C3 based board
- machine_hw_spi: use automatic DMA channel selection for SPI on C3
- boards: add support for LilyGO LoRa32 boards
- machine_pin: expose pin 20 for ESP32
- allow overriding cmake frozen manifest from the command line
- boards: add test manifest to test freezing during build
- add MICROPY_HW_ENABLE_UART_REPL and enable on generic S2/S3
- boards: compile GENERIC_SPIRAM and UM_TINYPICO with -Os
- machine_pin: don't expose pins 33-37 when octal SPIRAM is enabled
- modesp32: properly define RTC_VALID_EXT_PINS on S2/S3 variants
- network_wlan: support setting/getting txpower in WLAN.config()
- modnetwork: fix build when MICROPY_PY_NETWORK_WLAN is disabled
- make port-specific modules use MP_REGISTER_MODULE
- boards/UM_FEATHERS3: use correct sdkconfig.board
- boards/LILYGO_TTGO_LORA32: remove ID from TTGO Lora32 board json
- machine_pin: fix ESP32C3 IDFv4.4.1 build
- track allocated iRAM and free it on soft reset
- machine_i2c: implement write-then-read I2C transfers
- machine_pin: only expose IO20 on esp32 for IDF 4.3.2 and above
javascript port:
- make port-specific modules use MP_REGISTER_MODULE
mimxrt port:
- support gaps in the SPI an I2C device numbers
- allow for board-specific flash driver files
- compensate for a bug in the fsl_lpspi.c file
- remove two files from the Seeed Arch Mix directory
- replace Pin-config constants by a function call
- allow to select cs0 or cs1 for SPI
- add USB ID elements
- add support for the Olimex RT1010 board
- simplify Makefile in selection of flash type
- machine_pin: change pin drive constants to DRIVE_x naming
- machine_spi: add omitted GPIO config
- boards: add board files for MIMXRT1015 and MIMXRT1015_EVK
- eth: fix an Ethernet transmit error
- boards/TEENSY41: use the same SPI1 pins for Teensy 4.0 and 4.1
- boards: support using an optional board-specific manifest.py
- eth: avoid a race condition for Ethernet
- hal/pwm_backport: make PWM symmetric, and round division calcs
- moduos: convert module to use extmod version
- mbedtls: add NULL pointer check in m_free_mbedtls
- machine_i2s: add I2S protocol support
- boards: add board.json file for OLIMEX RT1010Py
- modules: append /flash/lib to the default sys.path
- machine_rtc: start RTC at boot and set datetime if not set
- machine_spi: improve handling of busy SPI peripheral
- mphalport: fix USB CDC RX handling to not block when unprocessed
- mbedtls: use core-provided tracked alloc instead of custom funcs
- make port-specific modules use MP_REGISTER_MODULE
- boards/OLIMEX_RT1010: document the initial deploy procedure
- boards: extend the deploy_teensy.md instructions
- mphalport: fix strict aliasing error with unique id
- update port to work with new nxp_driver v2.10
- machine_uart: fix an inconsistency for UART.init() arg handling
minimal port: no changes specific to this component/port
nrf port:
- drivers/usb: fix background events/scheduling while at USB REPL
- drivers/usb: fix MP_STREAM_POLL_RD support on USB CDC
- modules: include uasyncio in default board manifest
- make port-specific modules use MP_REGISTER_MODULE
- mpconfigport: set MICROPY_CONFIG_ROM_LEVEL defines for each MCU
- mpconfigport: enable extra features as defined by ROM levels
- mpconfigport: fix MICROPY_VFS IO build support
- modules/uos/microbitfs: fix MICROPY_MBFS build to use mp_obj_malloc
pic16bit port:
- make port-specific modules use MP_REGISTER_MODULE
- use 1 byte for qstr hash len and include header to fix build
powerpc port: no changes specific to this component/port
qemu-arm port:
- add tests for freezing viper and asm_thumb code
- moduos: convert module to use extmod version
- test-frzmpy: add test for freezing constant tuples
- mpconfigport: define MICROPY_PY_SYS_PLATFORM
renesas-ra port:
- add new port to Renesas RA
- boards: change device name information to show properly
- rename pyb_uart_ to machine_uart_
- rename pyb_rtc_ to machine_rtc_
- make port-specific modules use MP_REGISTER_MODULE
- modmachine: add missing MP_REGISTER_MODULE for umachine
- machine_pin: remove code for unsupported Pin features
rp2 port:
- machine_i2c: provide more specific error codes from I2C transfer
- mpconfigport.h: use internal error numbers
- mphalport: add optional dupterm support
- machine_uart: add machine.UART init/deinit methods
- machine_i2c: use soft I2C only for len=0, and increase timeout
- modutime: fix time.localtime day-of-week value
- machine_uart: fix UART RTS behaviour so RTS is deasserted
- Makefile: add FROZEN_MANIFEST Makefile option, to override default
- mpthreadport: fix memory corruption when thread is created in core1
- add USB MSC support
- allow Overriding cmake frozen manifest from the command line
- enable ucryptolib, using mbedtls for backend functions
- mphalport: fix USB CDC RX handling to not block when unprocessed
- Makefile: add mbedtls to GIT_SUBMODULES list and use it in CI func
- machine_uart: use read/write mutex to prevent char duplication
- mpconfigport: enable execfile, NotImplemented and REPL EMACS keys
- make port-specific modules use MP_REGISTER_MODULE
- mpconfigport.h: enable static scheduler nodes
- mpconfigport.h: allow boards to override internal errno config
- boards/ARDUINO_NANO_RP2040_CONNECT: disable internal errno
- boards/ARDUINO_NANO_RP2040_CONNECT: enable webrepl
- mbedtls: add support for ssl module with MICROPY_SSL_MBEDTLS
- machine_pin: add mp_hal_pin_interrupt C interface
- Makefile: use cmake for "make submodules" task when needed
- integrate lwIP network stack
- add support for using Wiznet hardware as an Ethernet NIC
- boards/W5100S_EVB_PICO: add Wiznet W5100S-EVB-Pico board
- Makefile: build in debug mode with "make DEBUG=1"
- boards/ARDUINO_NANO_RP2040_CONNECT: update USB PID
- mpnetworkport: fix lwip alarm callback timing to use microseconds
- CMakeLists: simplify qstr sources when enabling extmod components
samd port:
- moduos: convert module to use extmod version
- make port-specific modules use MP_REGISTER_MODULE
stm32 port:
- mbedtls: enable MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE
- mboot: remove custom HAL_RCC_GetHCLKFreq and use HAL provided one
- pin: change remaining uses of "af" to "alt"
- dac: deinit all DACs on soft reset
- fdcan: fix FIFO1 usage and handling of error interrupts
- fdcan: fix naming with regards to G4 series
- usbd_conf: set lpm_enable and battery_charging_enable on all MCUs
- powerctrlboot: set HAL uwTickPrio variable when needed
- powerctrl: write bootloader-state as 64-bit word to work on H7
- mboot: use PLL3 for USB clock source on H7 MCUs
- mboot: include hal_rcc_ex.c in source file list
- mboot: support H7 MCUs in fwupdate.py
- mboot: compute and check CRC32 of dfu file in fwupdate.py
- add support for G4 MCUs, and add NUCLEO_G474RE board defn
- mboot: allow HSI to be used as the main clock source
- mboot: add MBOOT_BOARD_ENTRY_INIT for a board to add entry code
- mboot: add support for F469/479 MCUs in fwupdate.py
- rtc: use LL_RTC functions to simplify some MCU-specific code
- Makefile: make stm32lib configurable
- add initial support for STM32WL MCUs
- boards/NUCLEO_WL55: add new board definition
- system_stm32: make SystemClock_Config() a weak symbol
- adc: fix L4 ADC channel numbers
- adc: remove obsolete FIRST/LAST GPIO channel macros
- boards/NUCLEO_L432KC: disable MICROPY_OPT_COMPUTED_GOTO
- boards/NUCLEO_WB55: add Arduino pin alias definitions
- mboot: allow a board to fully configure system clocks
- mboot: allow a board more control over entry initialisation
- moduos: convert module to use extmod version
- boards: convert F4xx and F7xx to new flash FS config
- mbedtls: add NULL pointer check in m_free_mbedtls
- sdcard: use mp_hal_pin_input instead of HAL function
- sdcard: add sdcard_select_sd/mmc functions
- main: support SD cards without a partition table
- dma: add option to disable auto-DMA-turn-off
- mboot: add pragma for GCC to ignore array bounds warning
- mboot: always check the magic number to enter filesystem loading
- mboot: add support for 64-bit mboot address space for reads
- mboot: add support for reading from SD card
- boards/PYBD_SF2: turn on SD card in mboot init if SD enabled
- mboot/fwupdate.py: simplify calculation of CRC32
- mboot: verify signature of fsload packed DFU files before writing
- mboot: verify CRC32 of fsload DFU files before writing
- support building for STM32F745
- machine_i2s: set FullDuplexMode to disabled on F4
- machine_i2s: allow I2S.deinit to be called multiple times
- machine_i2s: fix 16-bit stereo i2s_frame_map
- sdio: use runtime calculation for clock divider of sdio on H7
- modmachine: add deepsleep support to reset_cause() for WB55
- pyb_can: enable CAN FD frame support and BRS
- pyb_can: add support for CAN FD extended frame ID
- pyb_can: add support for bus with mixed Classic/FD nodes
- pyb_can: return the filter ID and type of received messages
- pyb_can: define the maximum bit timing parameters
- pyb_can: replace CAN.initfilterbanks with CAN.init keyword arg
- Makefile: allow overriding COPT in Makefile
- mpconfigport.h: let MICROPY_PY_IO_FILEIO take its default value
- machine_bitstream: include all clobbered regs in Cortex-M0 impl
- mboot: consolidate all UI and add general state change hooks
- mboot: add macros for use in led_state_all()
- sdram: add function to enable SDRAM power-down mode
- Makefile: add H747 to Makefile's MCU flags
- system_stm32: add H7 switched-mode-power-supply support
- system_stm32: set voltage scaling level 0 for H7 Rev-V devices
- system_stm32: allow boards to configure PLL VCI, VCO and FRACN
- system_stm32: improve H7 PLL and OSC configuration
- boards/PYBD_SFx: allow FROZEN_MANIFEST to be overridden
- mpbthciport: switch scheduling callback to use node scheduler
- boards/SPARKFUN_MICROMOD_STM32: fix incorrect UART and I2C config
- mphalport: fix I/O glitches when switching GPIO mode to alt-func
- boards/ADAFRUIT_F405_EXPRESS: use external SPI flash for storage
- boards/LEGO_HUB_NO6: exclude BTstack code from formatting
- qspi: support common flash sizes in MPU configuration
- qspi: wait for a free FIFO location before writing to DR
- qspi: workaround for SR set immediately after setting AR
- ulpi: add USB3320 ULPI low-power functions
- ulpi: include missing py/mperrno.h header
- boards/ARDUINO_PORTENTA_H7: add support for Arduino Portenta H7
- storage: add support for internal storage on Cortex-M0 MCUs
- add support for G0 MCUs
- boards/NUCLEO_G0B1RE: add NUCLEO_G0B1RE board definition
- mbedtls: use core-provided tracked alloc instead of custom funcs
- make port-specific modules use MP_REGISTER_MODULE
- rfcore: revert "intercept addr-resolution HCI cmd to work arou..."
- mboot: move remaining led_state_all call into mboot_state_change
- mboot: add update_app_elements to return mboot fsload elements
- mboot: make 0x70ad0000 values named constants with macros
- mboot: pass initial_r0 to early-init and get-reset-mode funcs
- boards/LEGO_HUB_NO6: make all SPI flash transfers use 32-bit addr
- boards/LEGO_HUB_NO6: add support for mboot to access SPI flash
- boards/LEGO_HUB_NO6: use multi-colour LED for mboot status
- boards/LEGO_HUB_NO6: implement robust filesystem-load updates
- boards/LEGO_HUB_NO6: add helper scripts to update app firmware
- rfcore: remove extra layer of buffering on BLE IPCC
- flash: remove FLASH_OPTR_DBANK condition in L4, H7 get_bank func
- machine_i2c: enable timingr constructor keyword on H7 MCUs
- powerctrl: disable sys tick interrupt in stop mode on some STM32s
- usbd_conf: remove disable of SYSCFG clock
- mboot: add board-configurable SysTick hook
- mboot: remove MP_WEAK on led funcs and make some of them static
- boards/LEGO_HUB_NO6: add comment to readme about powering off
- mboot: only include UI code if at least one board LED is defined
teensy port:
- make port-specific modules use MP_REGISTER_MODULE
- mpconfigport.h: remove obsolete macros
unix port:
- Makefile: remove explicit addition of -std=c++ flag
- show compiler info in REPL banner
- .gitignore: remove *.py from ignore list
- moduos: convert module to use extmod version
- mpconfigport.h: collect together config options from extra level
- variants: use rom feature config for standard, dev, coverage
- variants: enable a few optimisations and features on dev, coverage
- make port-specific modules use MP_REGISTER_MODULE
- implement `-X realtime` command-line option on macOS
- factor out code that generates random bytes to a new func
- variants: enable seeding random module on import in dev, coverage
windows port:
- README.md: fix broken mingw link
- add micropython.schedule support
- mingw: include extmod/shared/lib sources properly
- uselect: enable micropython select in dev variant
- uasyncio: add support for uasyncio to windows dev variant
- appveyor: fix printing of test failures
- Makefile: specify CXXFLAGS in the Makefile
- unify project file headers
- switch to VFS subsystem and use VfsPosix
- consolidate all sleep-related functions into windows_mphal.c
- avoid busy loop when using scheduler
- use BCryptGenRandom to implement os.urandom
- factor out code that generates random bytes to a new func
zephyr port:
- update include path to disk_access.h
- upgrade to Zephyr v3.0.0
- prj.conf: enable CONFIG_BUILD_OUTPUT_HEX to generate .hex output
- moduos: convert module to use extmod version
- make port-specific modules use MP_REGISTER_MODULE
- modusocket: fix function object wrapper for listen method