Acidanthera UEFI Development Kit based on EDK II edk2-stable202311
Go to file
Laszlo Ersek 4cca792399 NetworkPkg/HttpDxe: fix 32-bit truncation in HTTPS download
When downloading over TLS, each TLS message ("APP packet") is returned as
a (decrypted) fragment table by EFI_TLS_PROTOCOL.ProcessPacket().

The TlsProcessMessage() function in "NetworkPkg/HttpDxe/HttpsSupport.c"
linearizes the fragment table into a single contiguous data block. The
resultant flat data block contains both TLS headers and data.

The HttpsReceive() function parses the actual application data -- in this
case: decrypted HTTP data -- out of the flattened TLS data block, peeling
off the TLS headers.

The HttpResponseWorker() function in "NetworkPkg/HttpDxe/HttpImpl.c"
propagates this HTTP data outwards, implementing the
EFI_HTTP_PROTOCOL.Response() function.

Now consider the following documentation for EFI_HTTP_PROTOCOL.Response(),
quoted from "MdePkg/Include/Protocol/Http.h":

> It is the responsibility of the caller to allocate a buffer for Body and
> specify the size in BodyLength. If the remote host provides a response
> that contains a content body, up to BodyLength bytes will be copied from
> the receive buffer into Body and BodyLength will be updated with the
> amount of bytes received and copied to Body. This allows the client to
> download a large file in chunks instead of into one contiguous block of
> memory.

Note that, if the caller-allocated buffer is larger than the
server-provided chunk, then the transfer length is limited by the latter.
This is in fact the dominant case when downloading a huge file (for which
UefiBootManagerLib allocated a huge contiguous RAM Disk buffer) in small
TLS messages.

For adjusting BodyLength as described above -- i.e., to the application
data chunk that has been extracted from the TLS message --, the
HttpResponseWorker() function employs the following assignment:

    HttpMsg->BodyLength = MIN (Fragment.Len, (UINT32) HttpMsg->BodyLength);

The (UINT32) cast is motivated by the MIN() requirement -- in
"MdePkg/Include/Base.h" -- that both arguments be of the same type.

"Fragment.Len" (NET_FRAGMENT.Len) has type UINT32, and
"HttpMsg->BodyLength" (EFI_HTTP_MESSAGE.BodyLength) has type UINTN.
Therefore a cast is indeed necessary.

Unfortunately, the cast is done in the wrong direction. Consider the
following circumstances:

- "Fragment.Len" happens to be consistently 16KiB, dictated by the HTTPS
  Server's TLS stack,

- the size of the file to download is 4GiB + N*16KiB, where N is a
  positive integer.

As the download progresses, each received 16KiB application data chunk
brings the *next* input value of BodyLength closer down to 4GiB. The cast
in MIN() always masks off the high-order bits from the input value of
BodyLength, but this is no problem because the low-order bits are nonzero,
therefore the MIN() always permits progress.

However, once BodyLength reaches 4GiB exactly on input, the MIN()
invocation produces a zero value. HttpResponseWorker() adjusts the output
value of BodyLength to zero, and then passes it to HttpParseMessageBody().

HttpParseMessageBody() (in "NetworkPkg/Library/DxeHttpLib/DxeHttpLib.c")
rejects the zero BodyLength with EFI_INVALID_PARAMETER, which is fully
propagated outwards, and aborts the HTTPS download. HttpBootDxe writes the
message "Error: Unexpected network error" to the UEFI console.

For example, a file with size (4GiB + 197MiB) terminates after downloading
just 197MiB.

Invert the direction of the cast: widen "Fragment.Len" to UINTN.

Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Maciej Rabeda <maciej.rabeda@linux.intel.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Siyuan Fu <siyuan.fu@intel.com>
Reviewed-by: Maciej Rabeda <maciej.rabeda@linux.intel.com>
2020-01-14 10:53:42 +00:00
.azurepipelines .azurepipelines: Add Azure Pipelines YML configuration files 2019-11-11 13:02:47 -08:00
.mergify .mergify: Add Mergify YML pull request rules configuration file 2019-11-11 13:02:51 -08:00
.pytool .pytool: Avoid "is" with a literal Python 3.8 warnings in CI plugins 2019-12-06 03:07:37 +00:00
ArmPkg ArmPkg/MmCommunicationDxe: relay architected PI events to MM context 2019-12-11 11:10:47 +00:00
ArmPlatformPkg ArmPlatformPkg: strip trailing whitespace 2019-10-04 11:18:22 +01:00
ArmVirtPkg ArmVirtPkg: remove EnterS3WithImmediateWake () from ResetSystemLib 2020-01-10 07:00:51 +00:00
BaseTools BaseTools/Scripts/PatchCheck: Address false error conditions 2020-01-13 18:18:03 +00:00
Conf BaseTools:Delete FrameworkDatabase from BaseTools/Conf 2019-05-09 15:03:30 +08:00
CryptoPkg CryptoPkg: Support for SHA384 & SHA512 RSA signing schemes 2020-01-08 06:57:44 +00:00
DynamicTablesPkg DynamicTablesPkg: Arm SRAT Table Generator 2019-10-21 16:16:50 +01:00
EmbeddedPkg EmbeddedPkg: implement EDK2 IoMmu protocol wrapping DmaLib 2019-12-06 14:55:45 +00:00
EmulatorPkg EmulatorPkg DSC and WinHost.inf: Update tool chain name to CLANGPDB 2019-11-15 06:04:21 +00:00
FatPkg FatPkg: Add YAML file for CI builds 2019-11-11 13:02:10 -08:00
FmpDevicePkg FmpDevicePkg: Add YAML file for CI builds 2019-11-11 13:02:14 -08:00
IntelFsp2Pkg IntelFsp2Pkg/SplitFspBin.py: Command crashed when FV almost full. 2019-10-28 10:28:56 +08:00
IntelFsp2WrapperPkg IntelFsp2WrapperPkg: Fix various typos 2019-12-11 01:42:31 +00:00
MdeModulePkg MdeModulePkg/UefiBootManagerLib: log reserved mem allocation failure 2020-01-14 10:53:42 +00:00
MdePkg MdePkg: Add header file for Firmware Interface Table specification. 2020-01-10 06:20:42 +00:00
NetworkPkg NetworkPkg/HttpDxe: fix 32-bit truncation in HTTPS download 2020-01-14 10:53:42 +00:00
OvmfPkg OvmfPkg: remove EnterS3WithImmediateWake () from ResetSystemLib 2020-01-10 07:00:51 +00:00
PcAtChipsetPkg PcAtChipsetPkg: remove EnterS3WithImmediateWake () from ResetSystemLib 2020-01-10 07:00:51 +00:00
SecurityPkg SecurityPkg/Tcg2Pei: Add TCG PFP 105 support. 2020-01-06 06:23:19 +00:00
ShellPkg ShellPkg/ShellProtocol: Return error code while fail parsing cmd-line 2019-12-19 05:11:00 +00:00
SignedCapsulePkg SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time 2019-11-11 13:02:41 -08:00
SourceLevelDebugPkg SourceLevelDebugPkg DebugCommLibUsb3: Address NULL ptr dereference case 2019-08-12 09:46:47 +08:00
StandaloneMmPkg StandaloneMmPkg: Fix build failure - Bug 2253 2019-11-25 13:05:39 +00:00
UefiCpuPkg UefiCpuPkg: Shadow microcode patch according to FIT microcode entry. 2020-01-10 06:20:42 +00:00
UefiPayloadPkg UefiPayloadPkg: remove EnterS3WithImmediateWake () from ResetSystemLib 2020-01-10 07:00:51 +00:00
.gitignore .gitignore: Ignore python compiled files, extdeps, and vscode 2019-11-11 13:01:25 -08:00
.gitmodules ArmPkg: import Berkeley Softfloat library as git submodule 2019-05-31 17:53:44 +02:00
.mailmap .mailmap: Add an entry for Yu-Chen Lin 2020-01-06 18:01:30 +00:00
License-History.txt
License.txt
Maintainers.txt Maintainers.txt: Update email address and role 2019-12-18 06:35:34 +00:00
Readme.md Readme.md: Update EDK II CI build branch name 2019-11-12 03:53:00 +00:00
edksetup.bat edksetup.bat: Simplify the step to use CLANGPDB 2020-01-07 00:44:04 +00:00
edksetup.sh edksetup.sh: remove redundant -?, -h and --help in options parsing 2019-07-23 10:41:55 +02:00
pip-requirements.txt pip-requirements.txt: Add python pip requirements file 2019-11-11 13:01:31 -08:00

Readme.md

EDK II Project

A modern, feature-rich, cross-platform firmware development environment for the UEFI and PI specifications from www.uefi.org.

Build Status

Host Type Toolchain Branch Build Status Test Status Code Coverage
Windows VS2019 master
Ubuntu GCC master

More CI Build information

License Details

The majority of the content in the EDK II open source project uses a BSD-2-Clause Plus Patent License. The EDK II open source project contains the following components that are covered by additional licenses:

The EDK II Project is composed of packages. The maintainers for each package are listed in Maintainers.txt.

Resources

Code Contributions

To make a contribution to a TianoCore project, follow these steps.

  1. Create a change description in the format specified below to use in the source control commit log.

  2. Your commit message must include your Signed-off-by signature

  3. Submit your code to the TianoCore project using the process that the project documents on its web page. If the process is not documented, then submit the code on development email list for the project.

  4. It is preferred that contributions are submitted using the same copyright license as the base project. When that is not possible, then contributions using the following licenses can be accepted:

    For documentation:

    Contributions of code put into the public domain can also be accepted.

    Contributions using other licenses might be accepted, but further review will be required.

Developer Certificate of Origin

Your change description should use the standard format for a commit message, and must include your Signed-off-by signature.

In order to keep track of who did what, all patches contributed must include a statement that to the best of the contributor's knowledge they have the right to contribute it under the specified license.

The test for this is as specified in the Developer's Certificate of Origin (DCO) 1.1. The contributor certifies compliance by adding a line saying

Signed-off-by: Developer Name developer@example.org

where Developer Name is the contributor's real name, and the email address is one the developer is reachable through at the time of contributing.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Sample Change Description / Commit Message

From: Contributor Name <contributor@example.com>
Subject: [Repository/Branch PATCH] Pkg-Module: Brief-single-line-summary

Full-commit-message

Signed-off-by: Contributor Name <contributor@example.com>

Notes for sample patch email

  • The first line of commit message is taken from the email's subject line following [Repository/Branch PATCH]. The remaining portion of the commit message is the email's content.
  • git format-patch is one way to create this format

Definitions for sample patch email

  • Repository is the identifier of the repository the patch applies. This identifier should only be provided for repositories other than edk2. For example edk2-BuildSpecification or staging.
  • Branch is the identifier of the branch the patch applies. This identifier should only be provided for branches other than edk2/master. For example edk2/UDK2015, edk2-BuildSpecification/release/1.27, or staging/edk2-test.
  • Module is a short identifier for the affected code or documentation. For example MdePkg, MdeModulePkg/UsbBusDxe, Introduction, or EDK II INF File Format.
  • Brief-single-line-summary is a short summary of the change.
  • The entire first line should be less than ~70 characters.
  • Full-commit-message a verbose multiple line comment describing the change. Each line should be less than ~70 characters.
  • Signed-off-by is the contributor's signature identifying them by their real/legal name and their email address.

Submodules

Submodule in EDK II is allowed but submodule chain should be avoided as possible as we can. Currently EDK II contains the following submodules

  • CryptoPkg/Library/OpensslLib/openssl
  • ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3

ArmSoftFloatLib is actually required by OpensslLib. It's inevitable in openssl-1.1.1 (since stable201905) for floating point parameter conversion, but should be dropped once there's no such need in future release of openssl.

To get a full, buildable EDK II repository, use following steps of git command

$ git clone https://github.com/tianocore/edk2.git
$ cd edk2
$ git submodule update --init
$ cd ..

If there's update for submodules, use following git commands to get the latest submodules code.

$ cd edk2
$ git pull
$ git submodule update

Note: When cloning submodule repos, '--recursive' option is not recommended. EDK II itself will not use any code/feature from submodules in above submodules. So using '--recursive' adds a dependency on being able to reach servers we do not actually want any code from, as well as needlessly downloading code we will not use.