Michael Brown [Thu, 18 Jun 2026 14:54:19 +0000 (15:54 +0100)]
[crypto] Use private data field for digest algorithms
Following the example of commit 25072c1 ("[crypto] Use private data
field for key exchange algorithms"), extend the definition of a digest
algorithm to include an opaque private data field.
Michael Brown [Thu, 18 Jun 2026 13:21:08 +0000 (14:21 +0100)]
[crypto] Generalise notion of uncompressed elliptic curve points
With algorithm private data pointers now available, the general
mechanism for key exchange using uncompressed elliptic curve points
can be separated from the Weierstrass curve implementation.
Generalise the mechanism for performing elliptic curve key exchange
using uncompressed affine co-ordinates.
Michael Brown [Thu, 18 Jun 2026 12:18:18 +0000 (13:18 +0100)]
[crypto] Use private data field for elliptic curve algorithms
Following the example of commit 25072c1 ("[crypto] Use private data
field for key exchange algorithms"), extend the definition of an
elliptic curve to include an opaque private data field, and use this
to eliminate the wrapper functions generated by WEIERSTRASS_CURVE().
Michael Brown [Wed, 17 Jun 2026 14:13:34 +0000 (15:13 +0100)]
[tls] Accept only explicitly supported FFDHE groups
We do not currently perform any validation on the DHE field prime or
generator. RFC 7919 defines a family of known-safe finite fields, and
TLS version 1.3 completely removes the ability to provide an explicit
field prime and generator.
Verify that the field prime and generator correspond to one of the
explicitly configured groups.
This may break connections to the (now very rare) TLS servers that use
custom FFDHE groups and that choose to use DHE rather than ECDHE (or
that do not support ECDHE). We already advertise ECDHE cipher suites
as preferred over DHE cipher suites, and advertise all ECDHE groups as
preferred over all FFDHE groups. It is therefore very unlikely that
this change will cause any issues in practice.
Michael Brown [Wed, 17 Jun 2026 14:38:01 +0000 (15:38 +0100)]
[tls] Prefer X25519 as a key exchange mechanism
In TLS version 1.3, the expected flow is that the client offers at
least one key share in the initial ClientHello, so that key exchange
can take place as soon as the ServerHello is received (without
requiring a HelloRetryRequest and a second round trip).
We cannot viably offer key shares for all supported groups, since the
FFDHE groups have large public key values. The most likely approach
will be that we offer a single key share for our most preferred group.
Experiments suggest that X25519 is currently the most widely supported
key exchange group. Make this the most preferred group to maximise
the chance that a (future) TLS version 1.3 handshake will avoid the
extra round trip for a HelloRetryRequest.
Michael Brown [Wed, 17 Jun 2026 11:30:52 +0000 (12:30 +0100)]
[crypto] Provide a mechanism to check FFDHE group parameters
Provide is_ffdhe() and ffdhe_has_params() as a way to check if a key
exchange algorithm happens to match against an explicit pair of prime
modulus and generator values.
Michael Brown [Wed, 17 Jun 2026 09:48:18 +0000 (10:48 +0100)]
[crypto] Use private data field for key exchange algorithms
For historical reasons, TLS versions 1.2 and earlier identify FFDHE
groups by specifying the raw group prime and generator (the "dh_p" and
"dh_g" fields in ServerDHParams), rather than using a numeric code to
identify a named group.
This adds complexity to the process of identifying the internal key
exchange algorithm. One option would be to extend the definition of
struct tls_key_exchange_algorithm to include the identifying values
for the field prime and generator, but this is undesirable since the
field prime values may be large, and these values are already
available (indirectly) in ffdhe.c.
Extend our definition of a key exchange algorithm to include an opaque
private data field. This allows us to remove the wrapper functions
currently created by FFDHE_GROUP() and WEIERSTRASS_CURVE(), and opens
up the option of accessing the existing FFDHE field prime and
generator values from within the TLS layer.
The family of finite fields defined in RFC 7919 is almost identical to
that defined in RFC 3526, with the difference being that the older
standard uses the constant "pi" rather than "e".
Extend the definition of an FFDHE group to include a pointer to the
group constant, add the value of "pi", and define the modp2048,
modp3072, and modp4096 FFDHE groups.
Michael Brown [Tue, 16 Jun 2026 15:07:09 +0000 (16:07 +0100)]
[crypto] Use inline assembly for bigint_grow() and bigint_shrink()
The bigint_grow() and bigint_shrink() functions are used on the fast
path for big integer calculations (e.g. within the X25519 Montgomery
ladder step). Use inline assembly implementations of these functions
on all architectures.
Michael Brown [Tue, 16 Jun 2026 13:40:36 +0000 (14:40 +0100)]
[s390x] Use XOR-in-place to zero small fixed-length blocks
The XOR instruction has a storage-and-storage format "xc" that can be
used to zero small blocks of memory without needing to set up the four
registers required for "mvcle".
Michael Brown [Tue, 16 Jun 2026 13:20:41 +0000 (14:20 +0100)]
[tls] Rename "named curve" to "named group"
RFC 7919 renames the NamedCurve enumeration to NamedGroup, reflecting
its extended usage to handle key exchange groups that are not
constructed using elliptic curves.
Michael Brown [Tue, 16 Jun 2026 12:04:01 +0000 (13:04 +0100)]
[virtio] Ignore capabilities that describe inaccessible PCI BARs
In some configurations, newer versions of QEMU will end up placing the
modern interface's BAR4 above 4GB, rendering it inaccessible in a
32-bit build of iPXE. We will currently detect the existence of the
modern interface and attempt to use it, but fail at the point of
attempting to map the PCI BARs.
Fix by ignoring any virtio capabilities that describe an inaccessible
PCI BAR, and thereby allowing iPXE to fall back to using the legacy
interface if the modern interface's BAR cannot be used.
Reported-by: Jan ONDREJ (SAL) <ondrejj@salstar.sk> Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown [Mon, 15 Jun 2026 15:11:12 +0000 (16:11 +0100)]
[settings] Allow system time to be modified via builtin/unixtime
Allow the system time offset to be modified by writing a new time
value to builtin/unixtime, e.g.:
set builtin/unixtime 0x10d1a884
As with the NTP client, this does not attempt to write to the
underlying clock source (e.g. the RTC clock). Only the internal
system time offset is updated.
Any system time offset may be reset by clearing the setting:
clear builtin/unixtime
This will reset the system time offset to zero and so can be used to
undo the effect of a previous "set builtin/unixtime" or "ntp" command.
Requested-by: Christian I. Nilsson <ChristianN@2PintSoftware.com> Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown [Sun, 14 Jun 2026 16:15:55 +0000 (17:15 +0100)]
[s390x] Add optimised TCP/IP checksumming
Add an S/390 assembly language implementation of TCP/IP checksumming,
using the hardware "cksm" instruction to first calculate the 32-bit
one's complement checksum and then folding down to 16 bits.
Use an inline function since the whole checksum calculation (including
folding) requires only six instructions.
Michael Brown [Sat, 13 Jun 2026 15:15:57 +0000 (16:15 +0100)]
[test] Fix RFC 1071 checksum calculation for big-endian targets
Calculation of the TCP/IP checksum is fundamentally endian-agnostic:
the checksum is designed to be symmetric so that both big-endian and
little-endian systems can use native addition in any word size without
any byte swapping. The result is then stored into the checksum field
in the packet header as a native-endian value.
The reference algorithm presented in RFC 1071 (and used in our test
suite) is implicitly little-endian: the trailing byte is on a 16-bit
word boundary and is added to the least significant byte of the 16-bit
checksum value.
Fix by shifting the trailing byte by 8 bits on big-endian targets.
Michael Brown [Sat, 13 Jun 2026 15:07:41 +0000 (16:07 +0100)]
[efi] Fix parsing of EFI signature lists on big-endian targets
Though UEFI is fundamentally little-endian, the EFI signature list
image format is available even on non-EFI platforms (and is covered by
the unit test suite).
Add le32_to_cpu() macros as needed to allow EFI signature lists to be
parsed correctly on big-endian targets.
Michael Brown [Sat, 13 Jun 2026 15:03:33 +0000 (16:03 +0100)]
[peerdist] Fix segment identifier constant on big-endian targets
The "MS_P2P_CACHING" constant (used as part of the HMAC digest
calculation for the segment identifier) is a UTF-16LE string. On a
big-endian target, a wide-character string literal will have the wrong
endianness.
Fix by using a byte array rather than a wide-character string.
Michael Brown [Sat, 13 Jun 2026 15:01:35 +0000 (16:01 +0100)]
[build] Fix building for big-endian targets
Fix build errors that arise when building for a big-endian target such
as s390x. (Runtime endianness errors may remain: this fixes only
those errors that are detected at build time.)
Michael Brown [Sat, 13 Jun 2026 14:28:52 +0000 (15:28 +0100)]
[crypto] Use generic implementations of slow-path big integer functions
In the original big integer implementation, big integers were entirely
opaque to the caller and only the architecture-dependent code knew any
details of the internal structure.
This has long since ceased to be the case: for the sake of arithmetic
efficiency, many portions of the codebase now presume that big
integers are represented as an array of elements, with each element
being a native-endian unsigned value (with the precise type being
chosen by the architecture-specific header file) and with the least
significant element being first in the array.
The functions bigint_init(), bigint_done(), bigint_is_zero(),
bigint_is_geq(), and bigint_max_bit_set() are never used on fast code
paths, and most architectures use a generic C implementation of these
functions.
Provide generic implementations of these slow-path functions to be
used on all architectures.
We currently support fully parameterized finite field Diffie-Hellman
key exchange, where the peer provides not only its public key but also
the (fully arbitrary) selection of the field prime and generator.
RFC 7919 defines a family of finite fields all constructed from the
natural logarithm constant "e", intended to be used as well-known
fields where the peer simply names the field (e.g. "ffdhe2048") rather
than providing the raw prime and generator values.
Add support for this family of finite fields as key exchange
algorithms, to allow for protocols such as TLS version 1.3 where
parameterized fields are not permitted.
We choose to support only up to ffdhe4096, since this is sufficient to
exceed the security strength of our RNG (128 bits).
Support for ffdhe6144 and ffdhe8192 could trivially be added by simply
extending the "euler" constant and adding the relevant FFDHE_GROUP()
declarations. Doing so would approximately double the space
requirements for both read-only data (from 0.5kB to 1kB) and for
uninitialised data (from 3.5kB to 7kB).
Michael Brown [Sat, 6 Jun 2026 15:51:20 +0000 (16:51 +0100)]
[crypto] Remove redundant ECDHE algorithm
Remove the now-unused implementation of ECDHE that requires an
underlying elliptic curve abstraction, since we now use a standalone
key exchange algorithm abstraction instead.
Michael Brown [Sat, 6 Jun 2026 15:43:09 +0000 (16:43 +0100)]
[crypto] Remove elliptic curve abstraction for X25519
X25519 is defined as a key exchange algorithm, not as a generic
elliptic curve. We have never supported arbitrary point addition on
the underlying curve, and we have never supported pure multiplication
(without the clamping defined in RFC7748, which modifies the scalar
multiple).
Now that we have an abstraction for key exchange that exists
independently of the elliptic curve abstraction, there are no further
consumers of the elliptic curve abstraction for X25519. Remove this
redundant abstraction to simplify the codebase.
Michael Brown [Fri, 5 Jun 2026 15:56:41 +0000 (16:56 +0100)]
[crypto] Provide Weierstrass curves as generic key exchange algorithms
Provide the Weierstrass curves P-256 and P-384 as generic key exchange
algorithms (independent of the elliptic curve abstraction). Only the
"uncompressed" point format is supported, and the knowledge of the
format byte is internalised within the key exchange algorithm so that
the caller can just treat all values as opaque byte strings.
Add a random selection of the NIST "ECC CDH Primitive (SP800-56A
Section 5.7.1.2)" key exchange test vectors.
Michael Brown [Fri, 5 Jun 2026 14:07:33 +0000 (15:07 +0100)]
[crypto] Provide X25519 as a generic key exchange algorithm
Provide X25519 as a generic key exchange algorithm (independent of the
elliptic curve abstraction).
The existing RFC7748 test vectors are not structured in a way amenable
to treatment as a generic key exchange algorithm. Retain these test
vectors unaltered for completeness, add the single "Alice/Bob" key
exchange example presented in RFC7748, and add a selection of test
vectors from Project Wycheproof (including some known edge cases).
Michael Brown [Fri, 5 Jun 2026 13:59:29 +0000 (14:59 +0100)]
[crypto] Add a generic concept of a key exchange algorithm
TLS version 1.3 does not use static RSA or parameterized DHE for key
exchange: all key exchange algorithms are identified via a "named
group" enumeration and have predefined group parameters with fixed
input and output sizes.
Add an abstraction of a key exchange algorithm matching this usage
pattern, along with corresponding test support code.
Michael Brown [Mon, 1 Jun 2026 13:12:39 +0000 (14:12 +0100)]
[linux] Disable implicit linking against libatomic
GCC 16 attempts to link against -latomic_asneeded by default, and
expects that this library will be provided by the installed build
toolchain alongside libgcc.
The Fedora cross-gcc packages do not include libatomic, which causes
the build to fail.
We do not require any functions provided by libatomic. Work around
the missing packaged files in Fedora by disabling gcc's implicit
linking via the -fno-link-libatomic build option.
Joseph Wong [Thu, 28 May 2026 05:45:59 +0000 (22:45 -0700)]
[tg3] Use updated DMA APIs
Replace malloc_phys with dma_alloc, free_phys with dma_free, alloc_iob
with alloc_rx_iob, free_iob with free_rx_iob, virt_to_bus with dma or
iob_dma. Replace dma_addr_t with physaddr_t.
Signed-off-by: Joseph Wong <joseph.wong@broadcom.com>
Michael Brown [Sun, 24 May 2026 22:00:08 +0000 (23:00 +0100)]
[loong64] Port the RISC-V optimised TCP/IP checksum implementation
As with most other assembly code in iPXE, LoongArch64 is sufficiently
close to RISC-V that a straightforward transcription of the assembly
language code generally works.
Copy the RISC-V implementation for TCP/IP checksumming, retain the
register names and ABI, and just adjust the syntax to match
LoongArch64 requirements.
to eliminate some unnecessary folding steps, and hold the folded value
in the most significant bits of the register rather than the least
significant bits so that the final one's complement negation can be
accomplished naturally without requiring an explicit 0xffff constant.
Michael Brown [Thu, 21 May 2026 14:41:08 +0000 (15:41 +0100)]
[ci] Include UEFI Secure Boot builds for RISC-V 64 and LoongArch64
The usage pattern for UEFI Secure Boot on RISC-V 64 and LoongArch64 is
not yet well defined: there is no equivalent on those architectures
for the UEFI shim or the Microsoft signing submission infrastructure.
Include signed binaries for these architectures within the release
artifacts. Users may choose to enrol the iPXE Secure Boot CA
certificate on their own systems in order to use these binaries with
UEFI Secure Boot enabled.
OEMs such as Loongson may choose to include the iPXE Secure Boot CA
certificate within their default enrolled certificate list, or to
issue a cross-signed version of the iPXE Secure Boot CA certificate
(which could then be included within the official iPXE binaries in
future releases).
Michael Brown [Thu, 21 May 2026 14:19:10 +0000 (15:19 +0100)]
[loong64] Replace optimised string operations
The current implementation of the optimised string operations appears
to have been ported from the (old) arm64 implementation, and does not
cleanly match the LoongArch64 instruction set.
Replace with code derived from the riscv64 implementation, modified to
use indexed load and store instructions.
Michael Brown [Thu, 21 May 2026 11:43:47 +0000 (12:43 +0100)]
[neighbour] Discard deferred packets before discarding complete entries
Discarding neighbour cache entries for active connections is known to
be extremely disruptive, and is therefore done only as a last resort
when attempting to free up memory for a new allocation attempt.
There is currently no way to discard the deferred packet queue
separately from discarding the complete neighbour cache entry. Under
some conditions (such as a sustained ICMP echo request packet flood
from an IP address that will never complete neighbour resolution),
this can lead to the deferred packet queue growing without limit,
which will eventually lead to complete neighbour cache entries being
discarded.
Split out the logic in neighbour_destroy() for dropping deferred
packets to a separate neighbour_drop() function, and add a separate
cache discarder that will use this to free up memory without requiring
the complete neighbour cache entry to be discarded.
Reported-by: Daniel Kiper <daniel.kiper@oracle.com> Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown [Wed, 13 May 2026 14:32:17 +0000 (15:32 +0100)]
[virtio] Replace the virtio core and network device driver
The existing virtio network driver has been somewhat hacked together
over the past two decades by multiple contributors, and includes a
substantial amount of logic that is almost but not quite duplicated
between the "legacy" and "modern" code paths.
Rip out the existing driver and replace with a completely new driver
written based on the Virtual I/O Device specification document, not
derived from the Linux kernel driver.
Michael Brown [Tue, 12 May 2026 11:09:28 +0000 (12:09 +0100)]
[lacp] Use the same system identifier for all ports
Commit 3d43789 ("[lacp] Detect and ignore erroneously looped back LACP
packets") added protection against LACP packet storms that arise when
our own transmitted packets are somehow looped back to the same port,
but does not protect against a situation in which we have two
different ports that are externally bridged to each other.
This situation is unlikely to arise in practice since a properly
configured link partner should not be both sending and forwarding LACP
packets. Triggering this situation essentially requires our two ports
to be connected to a non-LACP-capable switch, while another port on
the same switch is connected to a separate device that is sending out
LACP packets.
Guard against this situation by using the MAC address of the first
network device as the LACP system identifier, thereby allowing the
loopback detection to reject any packets that were sent from any of
our ports.
Since the system identifier is no longer unique between ports, use the
guaranteed-unique network device scope ID as the group key to indicate
that we do not support aggregation.
Michael Brown [Wed, 6 May 2026 21:06:16 +0000 (22:06 +0100)]
[tls] Add support for RSA-PSS signature scheme
The RSA-PSS signature scheme is crowbarred somewhat awkwardly into TLS
version 1.2. Certificates with the standard rsaEncryption OID in the
public key may be used with either PKCS#1 or RSA-PSS, which breaks the
straightforward mapping between the OID and the signature algorithm.
Extend the definition of a TLS signature hash algorithm to include a
required OID-identified algorithm in the certificate's public key.
This allows us to define signature schemes such as rsa_pss_rsae_sha256
where the signature scheme uses an algorithm that differs from the
algorithm identified in the certificate's public key.
Michael Brown [Wed, 6 May 2026 21:14:17 +0000 (22:14 +0100)]
[crypto] Add support for RSA-PSS signature scheme
Add support for the RSA-PSS signature scheme as defined in RFC 8017
and required for TLS version 1.3.
Signature verification is deliberately implemented by first deriving
the salt value and then reconstructing the entire expected signature.
This is arguably inefficient since it involves two invocations of the
mask generation function when only one is required. However, this
implementation approach keeps the code size minimal (since there is no
need to implement separate verification logic), and makes it provably
impossible to accidentally omit a verification step (such as checking
the leading zero bits or the fixed 0x01 or 0xbc bytes). Since
signature verification is not a fast-path operation, the guaranteed
correctness is more valuable than a marginally faster execution.
Michael Brown [Wed, 6 May 2026 20:49:30 +0000 (21:49 +0100)]
[crypto] Allow for alternative RSA signature schemes
The RSA-PSS signature scheme has the same basic structure as the
existing PKCS#1 signature scheme, with a difference only in how the
digest value is encoded before being enciphered.
Abstract out the digest encoding from the signature and verification
methods, and add an explicit "pkcs1" to the relevant method names.
Joseph Wong [Thu, 30 Apr 2026 21:03:29 +0000 (14:03 -0700)]
[bnxt] Do not abort teardown on command failure
Modify bnxt_hwrm_run() to accept a flag indicating whether to abort
immediately upon a command failure. During initialization path,
driver will continue to abort on first error. During teardown,
sequence will continue executing subsequent cleanup commands even if
one fails. This ensures a best-effort cleanup.
Signed-off-by: Joseph Wong <joseph.wong@broadcom.com>
Joseph Wong [Mon, 27 Apr 2026 22:07:56 +0000 (15:07 -0700)]
[bnxt] Improve code readability and debug output
Enhance code readability in the completion queue servicing logic to
use explicit function calls per case statement, rather than falling
through to the next statement. Add debug print in ring allocation
path. Fix typo in PCI ROM entry.
Signed-off-by: Joseph Wong <joseph.wong@broadcom.com>
Michael Brown [Thu, 30 Apr 2026 13:25:18 +0000 (14:25 +0100)]
[efi] Register EFI IPv6 device path settings as netX.ndp
The EFI device path settings are currently registered as the
"netX.dhcp" settings block, in order that they will be automatically
overridden if a real DHCP configuration takes place. This does not
work as expected in an IPv6-only network, since the IPv6 configurator
will register "netX.ndp" rather than "netX.dhcp".
Fix by registering the EFI device path settings as either "netX.dhcp"
or "netX.ndp" based on the first address family encountered within the
device path.
Michael Brown [Thu, 30 Apr 2026 10:56:41 +0000 (11:56 +0100)]
[tls] Treat signature algorithm identifiers as opaque 16-bit values
RFC 5246 defines the signature_algorithm extension values for TLS
version 1.2 as being tuples of {HashAlgorithm, SignatureAlgorithm}
pairs. RFC 8446 redefines the signature_algorithm extension values
for TLS version 1.3 in a backwards-compatible way as opaque 16-bit
SignatureScheme values, and RFC 8447 updates RFC 5246 to allow these
values to be used with TLS version 1.2.
Redefine our concept of a signature algorithm identifier to remove the
internal structure that no longer exists.
Michael Brown [Wed, 29 Apr 2026 14:05:20 +0000 (15:05 +0100)]
[crypto] Fail all operations for the null public-key algorithm
The null crypto algorithms are intended to do nothing: the null digest
algorithm accepts all input and generates a zero-length digest, and
the null cipher algorithm simply copies the input unmodifed to the
output.
The null public-key algorithm currently does nothing successfully.
Unlike the null digest and cipher algorithms, the null public-key
algorithm's methods are never called.
Change the null public-key algorithm to fail all operations, thereby
allowing its methods to be used as stubs by algorithms such as ECDSA
that do not implement all of the possible public-key operations.
[efi] Fix operator precedence in autoexec network download
The != operator has higher precedence than = in C, so the expressions:
rc = imgacquire ( ..., image ) != 0
are parsed as:
rc = ( imgacquire ( ..., image ) != 0 )
This assigns the boolean result (0 or 1) to rc instead of the actual
return code from imgacquire(). As a result, strerror(rc) reports an
incorrect error message when debugging is enabled.
Add parentheses around each assignment to ensure rc captures the
actual return value, matching the pattern already used in
efi_autoexec_filesystem() within the same file.
Modified-by: Michael Brown <mcb30@ipxe.org> Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown [Thu, 23 Apr 2026 22:03:26 +0000 (23:03 +0100)]
[virtio] Ensure that device is closed before unmapping regions
Commit 988243c ("[virtio] Add virtio-net 1.0 support") erroneously
placed the code to unmap the device regions before the code to
unregister the network device. In the common case that the network
device is still open at the time that we shut down to boot the OS,
this results in the regions being accessed after having been unmapped.
For 32-bit BIOS or for UEFI with no IOMMU enabled, the iounmap()
operation is a no-op and so the driver still happens to work despite
the ordering bug. For 64-bit BIOS or for UEFI with an IOMMU enabled,
the iounmap() operation is not a no-op, and the driver will trigger a
page fault.
Fix by moving the call to unregister_netdev() to before the code that
unmaps the device regions.
Michael Brown [Thu, 23 Apr 2026 13:57:20 +0000 (14:57 +0100)]
[virtio] Fix assertion failures when interface is closed
The unused RX I/O buffers are currently freed without being deleted
from the list, with the list head being reinitialised only after all
buffers have been deleted. This triggers assertion failures due to
the list integrity checks when debugging is enabled.
Fix by deleting each buffer individually, so that the list structure
remains valid at all times.
Michael Brown [Thu, 23 Apr 2026 13:12:00 +0000 (14:12 +0100)]
[virtio] Set MTU for both modern and legacy devices
Commit b9d68b9 ("[ethernet] Use standard 1500 byte MTU unless
explicitly overridden") added code to explicitly set the MTU for
virtio-net devices, but only on the legacy probe path.
Make the behaviour consistent by setting the MTU on both legacy and
modern probe paths.
Michael Brown [Wed, 22 Apr 2026 22:09:42 +0000 (23:09 +0100)]
[cloud] Delete underlying snapshots when deleting Alibaba Cloud images
The underlying snapshots are not automatically deleted along with the
image, and there is no flag that can be set to cause them to be
automatically deleted.
Tag the underlying snapshots for deletion before deleting the image,
delete the image, and then delete any such tagged snapshots (including
any that may remain from a previous failed deletion attempt).
Michael Brown [Tue, 21 Apr 2026 22:44:06 +0000 (23:44 +0100)]
[ci] Add a workflow to import images to Alibaba Cloud
Add a workflow to build and import the official iPXE images for
Alibaba Cloud. As with the AWS and Google Cloud imports, treat this
as a workflow that must be triggered manually.
Michael Brown [Tue, 21 Apr 2026 15:31:52 +0000 (16:31 +0100)]
[cloud] Retry all Alibaba Cloud API calls
Experimentation suggests Alibaba Cloud API calls are extremely
unreliable, with a failure rate around 1%. It is therefore necessary
to allow for retrying basically every API call.
Some API calls (e.g. DescribeImages or ModifyImageAttribute) are
naturally idempotent and so safe to retry. Some non-idempotent API
calls (e.g. CopyImage) support explicit idempotence tokens. The
remaining API calls may simply fail on a retry, if the original
request happened to succeed but failed to return a response.
We could write convoluted retry logic around the non-idempotent calls,
but this would substantially increase the complexity of the already
unnecessarily complex code. For now, we assume that retrying
non-idempotent requests is probably more likely to fix transient
failures than to cause additional problems.
Michael Brown [Tue, 21 Apr 2026 15:06:40 +0000 (16:06 +0100)]
[cloud] Do not rely on CopyImage to import images to Alibaba Cloud
The CopyImage API call does work, but is unacceptably slow due to rate
limiting. Importing a full set of images to all regions can take
several hours (and is likely to fail at some point due to transient
errors in making API calls).
Resort to a mixture of strategies to get images imported to all
regions:
- For regions with working OSS that are not blocked by Chinese state
censorship laws, upload the image files to an OSS bucket and then
import the images.
- For regions with working OSS that are blocked by Chinese state
censorship laws but that have working FC, use a temporary FC
function to copy the image files from the uncensored OSS buckets
and then import the images. Attempt downloads from a variety of
uncensored buckets, since cross-region OSS traffic tends to
experience a failure rate of around 10% of requests.
- For regions that have working OSS but are blocked by Chinese state
censorship laws and do not have working FC, or for regions that
don't even have working OSS, resort to using CopyImage to copy the
previously imported images from another region. Spread the
imports across as many source regions as possible to minimise the
effect of the CopyImage rate limiting.
Michael Brown [Fri, 17 Apr 2026 12:54:49 +0000 (13:54 +0100)]
[cloud] Do not rely on ECS instances to import images to Alibaba Cloud
Spinning up ECS instances is supported in all ECS regions (unlike
Function Compute), but turns out to be unacceptably unreliable since
Alibaba Cloud has a very irritating tendency to fail to launch ECS
instances for a variety of spurious and unpredictable reasons.
Rewrite the censorship bypass mechanism to use the (extremely slow)
CopyImage API call to copy an imported image from an uncensored region
to a censored region.
Michael Brown [Wed, 15 Apr 2026 15:02:23 +0000 (16:02 +0100)]
[cloud] Do not rely on Function Compute to import images to Alibaba Cloud
Function Compute is unsupported in several Alibaba Cloud regions.
Rewrite the censorship bypass mechanism to access OSS buckets using a
temporary ECS instance instead of a temporary Function Compute
function.
Importing images now requires that the account has been prepared using
the "ali-setup" script, which creates the necessary role, VPCs, and
vSwitches to allow ECS instances to be launched in each region.
Michael Brown [Tue, 14 Apr 2026 12:59:38 +0000 (13:59 +0100)]
[cloud] Support creation of a censorship bypass role for Alibaba Cloud
Importing images into Alibaba Cloud currently relies upon using a
temporary Function Compute function to work around Chinese state
censorship laws that prevent direct access to OSS bucket contents in
mainland China regions.
Unfortunately, Alibaba Cloud regions are extremely asymmetric in terms
of feature support. (For example, some regions do not even support
IPv6 networking.) Several mainland China regions do not support
Function Compute, and so this workaround is not available for those
regions.
A possible alternative censorship workaround is to create temporary
ECS virtual machine instances instead of temporary Function Compute
functions. This requires the existence of a role that can be used by
ECS instances to access OSS. We cannot use the AliyunFcDefaultRole
that is currently used by Function Compute, since this role cannot be
assumed by ECS instances.
Creating roles is a privileged operation, and it would be sensible to
assume that the image importer (which may be running as part of a
GitHub Actions workflow) may not have permission to itself create a
suitable temporary role. The censorship bypass role must therefore be
set up once in advance by a suitably privileged user.
Add the ability to create a suitable censorship bypass role to the
Alibaba Cloud setup utility.
Michael Brown [Fri, 10 Apr 2026 14:59:24 +0000 (15:59 +0100)]
[cloud] Add utility to set up VPCs and security groups in Alibaba Cloud
Creating ad hoc instances in Alibaba Cloud is extremely cumbersome and
tedious due to the need to specify an explicit vSwitch and security
group, with no defaults being available.
Add a utility that will create a VPC within each region, a vSwitch
within each zone within each region, and a security group within each
region.
Michael Brown [Thu, 9 Apr 2026 09:47:55 +0000 (10:47 +0100)]
[cloud] Update disk log console tool descriptions
Update the descriptive text for the disk log console tools to remove
references to INT13, since these now work for both BIOS and UEFI disk
log consoles.
Leave the script names as {aws,gce,ali}-int13con, to avoid breaking
any existing tooling that might use these names.
Michael Brown [Wed, 8 Apr 2026 12:55:47 +0000 (13:55 +0100)]
[cloud] Fix architecture detection for partitioned disk images
Allow the UEFI CPU architecture to be detected for the partitioned
disk images generated by genfsimg as of commit 2c84b68 ("[build] Use a
partition table in generated USB disk images").
Michael Brown [Sun, 29 Mar 2026 12:37:55 +0000 (13:37 +0100)]
[disklog] Generalise CONSOLE_INT13 to CONSOLE_DISKLOG
The name "int13" is intrinsically specific to a BIOS environment.
Generalise the build configuration option CONSOLE_INT13 to
CONSOLE_DISKLOG, in preparation for adding EFI disk log console
support.
Existing configurations using CONSOLE_INT13 will continue to work.
Michael Brown [Mon, 6 Apr 2026 23:37:00 +0000 (00:37 +0100)]
[undi] Pad transmit buffer length to work around vendor driver bugs
The workaround used for UEFI in commit 926816c ("[efi] Pad transmit
buffer length to work around vendor driver bugs") is also applicable
to the BIOS UNDI driver.
Apply the same workaround of padding the transmit I/O buffers to the
minimum Ethernet frame length before passing them to the underlying
UNDI driver's transmit function.
Reported-by: Alexander Patrakov <patrakov@gmail.com> Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown [Mon, 6 Apr 2026 21:24:08 +0000 (22:24 +0100)]
[efi] Remove the Dhcp6Dxe driver veto
Commit cb95b5b ("[efi] Veto the Dhcp6Dxe driver on all platforms")
vetoed the Dhcp6Dxe driver to work around the bug described at
https://github.com/tianocore/edk2/issues/10506 that results in
EfiDhcp6Stop() getting stuck in a tight loop waiting for an event that
will never occur.
Since we now call UnloadImage() at TPL_APPLICATION, we no longer
trigger the bug in Dhcp6Dxe, and so the veto may be removed.
Michael Brown [Mon, 6 Apr 2026 21:17:58 +0000 (22:17 +0100)]
[efi] Drop to external TPL when unloading vetoed images
As of commit c3376f8 ("[efi] Drop to external TPL for calls to
ConnectController()"), the veto mechanism will drop to TPL_APPLICATION
for calls to DisconnectController().
Match this behaviour for calls to UnloadImage(), since that is likely
to result in calls to DisconnectController(). For example, any EDK2
driver using NetLibDefaultUnload() as its unload handler will call
DisconnectController() to disconnect itself from all handles.
On Ubuntu/Debian, syslinux-common installs mbr.bin to
/usr/lib/syslinux/mbr/mbr.bin. This path is not currently searched by
find_syslinux_file(), causing USB disk image generation to fail with
"could not find mbr.bin".
Add /usr/lib/syslinux/mbr, /usr/share/syslinux/mbr, and
/usr/local/share/syslinux/mbr to the search paths.
Michael Brown [Thu, 26 Mar 2026 14:46:26 +0000 (14:46 +0000)]
[build] Allow binaries to request a log partition from genfsimg
For UEFI, the USB disk image is constructed from the built EFI binary
(e.g. bin-x86_64-efi/ipxe.efi) by genfsimg, which does not itself have
any way to access the build configuration. We therefore need a way to
annotate the binary such that genfsimg can determine whether or not to
include a log partition within the USB disk image.
The "OEM ID" and "OEM information" fields within the PE header can be
used for this, since they are easily accessed and serve no other
purpose. We define bit 0 of "OEM information" as a flag indicating
that a log partition should be included. If this bit is set, genfsimg
will create a log partition with a layout matching that of the BIOS
build (i.e. using partition 3 and at an offset of 16kB from the start
of the disk).
The PE header is constructed by elf2efi.c, which takes as an input the
linked ELF form of the binary. We use an ELF .note section to allow
any linked-in object to communicate the log partition request through
to elf2efi.c, which then populates the OEM information field
accordingly.
We choose to use the same field locations within the BIOS bzImage
header, since this allows genfsimg to use the same logic for both BIOS
and UEFI binaries. In a BIOS build, there is no external processing
equivalent to elf2efi.c, and so we construct the field value directly
using absolute symbols and explicit relocation records.
(Note that the bzImage header is relevant only when using genfsimg to
construct a combined BIOS/UEFI image. In the common case of building
a BIOS-only image such as bin/ipxe.usb, the partition table is
manually constructed by usbdisk.S and genfsimg is not involved.)
Michael Brown [Thu, 26 Mar 2026 12:51:20 +0000 (12:51 +0000)]
[build] Work around syslinux bugs in FAT cluster counting
The syslinux function check_fat_bootsect() performs some sanity checks
to ensure that the filesystem type string (e.g. "FAT12") is correct
for the total number of clusters in the FAT. There is unfortunately a
bug in its calculation of the number of sectors occupied by the root
directory, which causes it to underestimate the number of sectors by a
factor of 32.
When the total number of clusters is close to the FAT12 limit of 4096,
this bug can cause syslinux to erroneously report that the filesystem
has "more than 4084 clusters but claims FAT12".
Work around this bug by selecting an explicit cluster size in order to
avoid potentially problematic cluster counts. We default to using 4kB
clusters, doubling to 8kB if using 4kB would result in a total cluster
count near 4096 (the FAT12 limit) or near 65536 (the FAT16 limit).
Michael Brown [Thu, 26 Mar 2026 12:40:40 +0000 (12:40 +0000)]
[build] Use sector count values consistently in genfsimg
The calculations around the FAT filesystem layout currently use a
mixture of kilobytes and sector counts. Switch to using sector counts
throughout the calculation, to make the code easier to read.
Michael Brown [Mon, 23 Mar 2026 16:08:09 +0000 (16:08 +0000)]
[build] Use a partition table in generated USB disk images
The USB disk image constructed by util/genfsimg is currently a raw FAT
filesystem, with no containing partition. This makes it incompatible
with the use of CONSOLE_INT13, since there is no way to add a
dedicated log partition without a partition table.
Add a partition table when building a non-ISO image, using the mbr.bin
provided by syslinux (since we are already using syslinux to invoke
the ipxe.lkrn within the FAT filesystem).
The BIOS .usb targets are built using a manually constructed partition
table with C/H/S geometry x/64/32. Match this geometry to minimise
the differences between genfsimg and non-genfsimg USB disk images.
Michael Brown [Tue, 24 Mar 2026 16:41:30 +0000 (16:41 +0000)]
[build] Ensure that generated filesystem images retain no stale content
We use mformat to ensure that the FAT filesystem starts as empty.
However, formatting the filesystem can still leave old data blocks
present (though unreferenced) within the disk image.
Truncate the image to a zero length before extending, to ensure that
no stale content is retained.
Michael Brown [Fri, 20 Mar 2026 13:48:54 +0000 (13:48 +0000)]
[cloud] Add utility to read INT13CON partition in Alibaba Cloud
Following the examples of aws-int13con and gce-int13con, add a utility
that can be used to read the INT13 console log from a used iPXE boot
disk in Alibaba Cloud Elastic Compute Service (ECS).
We cannot reliably access the used iPXE boot disk (or a snapshot
created from it) since OSS buckets in mainland China cannot be
accessed due to Chinese laws. We therefore create a snapshot and
attach this snapshot as a data disk to a temporary Linux instance, as
we do in Google Compute Engine.
Unlike in Google Compute Engine, we cannot reliably capture serial
port output from the temporary Linux instance. Issuing the relevant
GetInstanceConsoleOutput API call will cause the output to be captured
once and (unpredictably) cached. Without knowing in advance precisely
when the output is complete, we cannot use this approach to capture
the relevant part of the output.
We therefore use an Alibaba Cloud Linux image that includes the Cloud
Assistant Agent. This allows us to use the RunCommand API call to run
a command on the instance and capture the output, all done via the
control plane so that we are not dependent on having direct network
access to the temporary instance.
projects / thirdparty / ipxe.git / log
