Separate out components of a TMD signature

Not sure why this hadn't been done already given that the ticket module has always used separate properties for the signature type and data.

.github/workflows/sphinx-docs.yml

@@ -0,0 +1,64 @@
+# Workflow to build libWiiPy documentation with Sphinx and then publish it
+name: Build and publish documentation with Sphinx
+
+on:
+  # Runs on pushes targeting the default branch
+  push:
+    branches: ["main"]
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+
+# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
+# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
+concurrency:
+  group: "pages"
+  cancel-in-progress: false
+
+jobs:
+  # Build job
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Setup Pages
+        uses: actions/configure-pages@v5
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Install Dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+      - name: Build and Install libWiiPy for Sphinx
+        run: |
+          python -m build
+          pip install dist/libwiipy*.tar.gz
+      - name: Build Documentation with Sphinx
+        run: |
+          python -m sphinx -M html docs/source/ docs/build/
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          # Upload only the build/html directory
+          path: 'docs/build/html'
+
+  # Deployment job
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    needs: build
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

.gitignore

@@ -165,6 +165,7 @@ cython_debug/
 *.tmd
 *.wad
 *.arc
+*.ash
 out_prod/
 remakewad.pl
 

README.md

@@ -1,4 +1,4 @@
-![libWiiPy](https://github.com/NinjaCheetah/libWiiPy/assets/58050615/80093c68-b86e-4b96-87b7-db3855382ca8)
+![banner](https://github.com/NinjaCheetah/libWiiPy/assets/58050615/00ea4c41-673c-4a74-addb-fbb40b4313c8)
 # libWiiPy
 libWiiPy is a modern Python 3 library for handling the various files and formats found on the Wii. It aims to be simple to use, well maintained, and offer as many features as reasonably possible in one library, so that a newly-written Python program could reasonably do 100% of its Wii-related work with just one library. It also aims to be fully cross-platform, so that any tools written with it can also be cross-platform.
 
@@ -7,10 +7,12 @@ libWiiPy is inspired by [libWiiSharp](https://github.com/TheShadowEevee/libWiiSh
 
 # Features
 This list will expand as libWiiPy is developed, but these features are currently available:
-- TMD, ticket, and WAD parsing
+- TMD and Ticket parsing (`.tmd`, `.tik`)
-- WAD content extraction, decryption, re-encryption, and packing
+- Title content decryption, re-encryption
+- Packing and unpacking WAD files (`.wad`)
 - Downloading titles from the NUS
-- Packing and unpacking U8 archives (.app, .arc, .carc, .szs)
+- Packing and unpacking U8 archives (`.app`, `.arc`)
+- Decompressing ASH files (`.ash`, both the standard variants and the variants found in My Pokémon Ranch)
 
 # Usage
 A wiki, and in the future a potential documenation site, is being worked on, and can be accessed [here](https://github.com/NinjaCheetah/libWiiPy/wiki). It is currently fairly barebones, but it will be improved in the future.

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/source/conf.py

@@ -0,0 +1,38 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+from datetime import date
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = 'libWiiPy'
+copyright = f'{date.today().year}, NinjaCheetah & Contributors'
+author = 'NinjaCheetah & Contributors'
+release = 'main'
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = ['myst_parser', 'sphinx.ext.napoleon', 'sphinx_copybutton', 'sphinx_tippy', 'sphinx_design']
+
+templates_path = ['_templates']
+exclude_patterns = ["Thumbs.db", ".DS_Store"]
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = 'sphinx_book_theme'
+html_static_path = ['_static']
+html_logo = "banner.png"
+html_title = "libWiiPy API Docs"
+html_theme_options = {
+    "repository_url": "https://github.com/NinjaCheetah/libWiiPy",
+    "use_repository_button": True
+}
+
+# MyST Configuration
+
+myst_enable_extensions = ['colon_fence', 'deflist']

docs/source/index.md

@@ -0,0 +1,46 @@
+---
+sd_hide_title: true
+---
+
+# Overview
+
+# libWiiPy API Docs
+
+Welcome to the API documentation website for libWiiPy! libWiiPy is a modern Python 3 library for handling the various files and formats found on the Wii.
+
+```{toctree}
+:hidden:
+
+self
+```
+
+```{toctree}
+:hidden:
+:caption: The Basics
+
+usage/installation.md
+usage/getting-started.md
+```
+
+```{toctree}
+:hidden:
+:caption: Working with Titles
+
+titles/title-anatomy.md
+titles/extracting-titles.md
+titles/title-module.md
+titles/nus-downloading.md
+```
+
+```{toctree}
+:hidden:
+:caption: Other Useful Pages
+
+modules.md
+```
+
+## Indices and tables
+
+* [Full Index](<project:#genindex>)
+* [Module Index](<project:#modules>)
+* <project:#search>

docs/source/libWiiPy.archive.md

@@ -0,0 +1,28 @@
+# libWiiPy.archive package
+
+## Submodules
+
+### libWiiPy.archive.ash module
+
+```{eval-rst}
+.. automodule:: libWiiPy.archive.ash
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+### libWiiPy.archive.u8 module
+
+```{eval-rst}
+.. automodule:: libWiiPy.archive.u8
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+## Module contents
+
+```{eval-rst}
+.. automodule:: libWiiPy.archive
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```

docs/source/libWiiPy.md

@@ -0,0 +1,34 @@
+# libWiiPy package
+
+## Subpackages
+
+```{toctree}
+:maxdepth: 4
+
+libWiiPy.archive
+libWiiPy.title
+```
+
+## Submodules
+
+### libWiiPy.shared module
+
+libWiiPy's ``shared`` module is private and contains only private functions used by other modules.
+
+```{eval-rst}
+.. automodule:: libWiiPy.shared
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+
+### libWiiPy.types module
+
+libWiiPy's ``types`` module is private and contains only private classes used by other modules.
+
+```{eval-rst}
+.. automodule:: libWiiPy.types
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```

docs/source/libWiiPy.title.md

@@ -0,0 +1,78 @@
+# libWiiPy.title package
+
+## Submodules
+
+### libWiiPy.title.commonkeys module
+
+```{eval-rst}
+.. automodule:: libWiiPy.title.commonkeys
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+
+### libWiiPy.title.content module
+
+```{eval-rst}
+.. automodule:: libWiiPy.title.content
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+
+### libWiiPy.title.crypto module
+
+```{eval-rst}
+.. automodule:: libWiiPy.title.crypto
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+### libWiiPy.title.nus module
+
+```{eval-rst}
+.. automodule:: libWiiPy.title.nus
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+### libWiiPy.title.ticket module
+
+```{eval-rst}
+.. automodule:: libWiiPy.title.ticket
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+### libWiiPy.title.title module
+
+```{eval-rst}
+.. automodule:: libWiiPy.title.title
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+### libWiiPy.title.tmd module
+
+```{eval-rst}
+.. automodule:: libWiiPy.title.tmd
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+### libWiiPy.title.wad module
+
+```{eval-rst}
+.. automodule:: libWiiPy.title.wad
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```
+## Module contents
+
+```{eval-rst}
+.. automodule:: libWiiPy.title
+   :members:
+   :undoc-members:
+   :show-inheritance:
+```

docs/source/modules.md

@@ -0,0 +1,7 @@
+# Modules Overview
+
+```{toctree}
+:maxdepth: 4
+
+libWiiPy
+```

docs/source/titles/extracting-titles.md

@@ -0,0 +1,131 @@
+# Extracting Titles from WAD Files
+
+One of the most common uses for libWiiPy's title subpackage is extracting WAD files so that you can edit their contents. This can open up the doors to modding, like with the [famous DVD image](https://ncxprogramming.com/2023/06/19/wii-dvd-p3.html) in the Wii Menu that actually kicked this project off, or other projects like datamining.
+
+:::{note}
+This guide assumes that you already have a WAD file that you'd like to extract, and that this WAD file doesn't use a personalized ticket, as titles with personalized tickets are not as easy to manipulate. WADs like that aren't very common, as most WADs created from the NUS, dumped from a console, or obtained via other methods will not have this type of ticket, so if in doubt, it will probably work fine.
+
+If you don't currently have a WAD file, you may want to skip ahead to <project:#/titles/nus-downloading> first to obtain one for a free title first.
+:::
+
+:::{hint}
+If you've gotten here, but you're just looking for a tool to do all of this rather than a guide on how to write your own code, you're probably looking for something like [WiiPy](https://github.com/NinjaCheetah/WiiPy). WiiPy is a command line tool that covers all of libWiiPy's features, and is also made by NinjaCheetah.
+:::
+
+With all of that out of the way, let's begin!
+
+## Loading the WAD
+
+The first thing we'll do is import libWiiPy and load up our file:
+```pycon
+>>> import libWiiPy
+>>> wad_data = open("file.wad").read()
+>>>
+```
+
+Then, we can create a new WAD object, and load our data into it:
+```pycon
+>>> wad = libWiiPy.title.WAD()
+>>> wad.load(wad_data)
+>>>
+```
+
+And viola! We have a WAD object that we can use to get each separate part of our title.
+
+## Picking the WAD Apart
+
+Now that we have our WAD loaded, we need to separate it out into its components. On top of the parts we already established, a WAD also contains a certificate, checked by IOS during official title installations to ensure that a title was signed by Nintendo, and potentially two more areas called the footer and the CRL. Footers aren't a necessary part of a WAD, and when they do exist, they typically only contain the build timestamp and the machine it was built on. CRLs are even less common, and have never actually been found inside any WAD, but we know they exist because of things we've seen that Nintendo would really rather we hadn't. Because these three components don't have data we can edit, they're only ever represented as bytes, and do not have their own classes.
+
+### The TMD
+
+To get the TMD, let's create a new TMD object, and then use the method `get_tmd_data()` on our WAD object as the source for our TMD data:
+```pycon
+>>> tmd = libWiiPy.title.TMD()
+>>> tmd.load(wad.get_tmd_data())
+>>>
+```
+
+And now, just like in our <project:#/usage/getting-started> tutorial, we have a TMD object, and can get all the same data from it!
+
+### The Ticket
+
+Next up, we need to get the Ticket. The process for getting the Ticket is very similar to getting the TMD. We'll create a new Ticket object, and then use the method `get_ticket_data()` to get the data:
+```pycon
+>>> ticket = libWiiPy.title.Ticket()
+>>> ticket.load(wad.get_ticket_data())
+>>>
+```
+
+Similarly to the TMD, we can use this Ticket object to get all the properties of a Ticket. This includes getting the decrypted version of the Ticket's encrypted Title Key. In fact, why don't we do that know?
+
+We can use a Ticket's `get_title_key()` method to decrypt the Title Key and return it. This uses the Ticket's `title_key_enc`, `common_key_index`, and `title_id` properties to get the IV and common key required to decrypt the Title Key.
+
+```pycon
+>>> title_key = ticket.get_title_key()
+>>>
+```
+
+:::{danger}
+If the Ticket contained in your WAD is personalized, this Title Key will be invalid! `get_title_key()` won't return any error, as it has no way of validating the output, but the key will not work to decrypt any content.
+:::
+
+### The Contents
+
+Now that we have our TMD and Ticket extracted, we can get to work on extracting and decrypting the content. 
+
+First, we'll need to create a new ContentRegion object, which requires sourcing the raw data of all the WAD's contents (which are stored as one continuous block) using `get_content_data()`, as well as the content records found in our TMD object. We can do this like so:
+
+```pycon
+>>> content_region = libWiiPy.title.ContentRegion()
+>>> content_region.load(wad.get_content_data(), tmd.content_records)
+>>>
+```
+
+The content records from the TMD are used by the `content` module to parse the block of data that the contents are stored in so that they can be separated back out into individual files. Speaking of which, let's try extracting one (still in its encrypted form, for now) just to make sure everything is working. For this example, we'll use `get_enc_content_by_index()`, and get the content at index 0:
+
+```pycon
+>>> encrypted_content = content_region.get_enc_content_by_index(0)
+>>>
+```
+
+As long as that's all good, that means our WAD's content has successfully been parsed, and we can start decrypting it!
+
+Let's try getting the same content again, the one at index 0, but this time in its decrypted form. We can use the method `get_content_by_index()` for this, which takes the index of the content we want, and the Title Key that we saved in the last step.
+```pycon
+>>> decrypted_content = content_region.get_content_by_index(0, title_key)
+>>>
+```
+
+:::{error}
+If you get an error here saying that the hash of your decrypted content doesn't match the expected hash, then something has gone wrong. There are several possibilities, including your Ticket being personalized, causing you to get an invalid Title Key, your WAD having mismatched data, or your content being modified without the hash in the content record having been updated.
+:::
+
+If you don't get any errors, then congratulations! You've just extracted your first decrypted content from a WAD!
+
+Now that we know things are working, why don't we speed things up a little by using the content region's `get_contents()` method, which will return a list of all the decrypted content:
+```pycon
+>>> decrypted_content_list = content_region.get_contents(title_key)
+>>>
+```
+
+And just like that, we have our TMD, Ticket, and decrypted content all extracted! From here, what you do with them is up to you and whatever program you're working on. For example, to make a simple WAD extractor, you may want to write all these files to an output directory.
+
+### The Other Data
+
+As mentioned earlier in this guide, WADs also contain up to three extra regions of data: the certificate, the footer, and the CRL. The procedure for extracting all of these is pretty simple, and follows the same formula as any other data in a WAD:
+```pycon
+>>> certificate = wad.get_cert_data()
+>>> footer = wad.get_meta_data()
+>>> crl = wad.get_crl_data()
+>>>
+```
+
+Beyond getting their raw data, there isn't anything you can directly do with these components with libWiiPy. If one of these components doesn't exist, libWiiPy will simply return an empty bytes object.
+
+:::{note}
+Managed to find a WAD somewhere with CRL data? I'd love to here more, so feel free to email me at [ninjacheetah@ncxprogramming.com](mailto:ninjacheetah@ncxprogramming.com).
+:::
+
+<hr>
+
+Now, that might all seem a bit complicated. What if instead there was a way to manage a title using one object that handles all the individual components for you? Well, you're in luck! On top of the fairly low-level way to extract a WAD provided in this guide, libWiiPy also offers a higher-level method through the <project:#libWiiPy.title.title> module. On the next page, we'll dive into the specifics, and how to use this module.

docs/source/titles/nus-downloading.md

@@ -0,0 +1,5 @@
+# Downloading from the NUS
+
+<project:#libWiiPy.title.nus>
+
+Pardon our dust! This website is still under construction, and we haven't quite gotten to this one yet.

docs/source/titles/title-anatomy.md

@@ -0,0 +1,34 @@
+# Anatomy of a Title
+
+Before we start working with titles, it's important to understand what components make up a title on the Wii, and how each of those components are handled in libWiiPy. If you're here, you likely already understand what a title is in the context of the Wii, but if not, [WiiBrew](https://wiibrew.org/wiki/Main_Page) is a great reference to learn more about the Wii's software.
+
+:::{note}
+"Title" can be used to refer to both digital titles preinstalled on the Wii and distributed via the Wii Shop Channel and system updates, as well as games released on discs. libWiiPy does not currently offer methods to interact with most data found on a game disc, so for all intents and purposes, "title" in this documentation is referring to digital titles only unless otherwise specified.
+:::
+
+There are three major components of a title: the **TMD**, the **Ticket**, and the **contents**. A brief summary of each is provided below.
+
+## TMD
+<project:#libWiiPy.title.tmd>
+
+A **TMD** (**T**itle **M**eta**d**ata) contains basic information about a title, such as its Title ID, version, what IOS and version it's designed to run under, whether it's for the vWii or not, and more related information. The TMD also stores a list of content records that specify the index and ID of each content, as well as the SHA-1 hash of the decrypted content, to ensure that decryption was successful.
+
+In libWiiPy, a TMD is represented by a `TMD()` object, which is part of the `tmd` module in the `title` subpackge, and is imported automatically. A content record is represented by its own `ContentRecord()` object, which is a private class designed to only be used by other modules.
+
+## Ticket
+<project:#libWiiPy.title.ticket>
+
+A **Ticket** primarily contains the encrypted Title Key for a title, as well as the information required to decrypt that key. They come in two forms: common tickets, which are freely available from the Nintendo Update Servers (NUS), and personalized tickets, which are issued to your console specifically by the Wii Shop Channel (or at least they were before it closed, excluding the free titles still available).
+
+In libWiiPy, a Ticket is represented by a `Ticket()` object, which is part of the `ticket` module in the `title` subpackage, and is imported automatically.
+
+## Content
+<project:#libWiiPy.title.content>
+
+**Contents** are the files in a title that contain the actual data, whether that be the main executable or resources required by it. They're usually stored encrypted in a WAD file or on the NUS, until they are decrypted during installation to a console. The Title Key stored in the Ticket is required to decrypt the contents of a title. Each content has a matching record with its index and Content ID, as well as the SHA-1 hash of its decrypted data. These records are stored in the TMD.
+
+In libWiiPy, contents are represented by a `ContentRegion()` object, which is part of the `content` module in the `title` subpackge, and is imported automatically. A content record is represented by its own `ContentRecord()` object, which is a private class designed to only be used by other modules.
+
+To effectively work with a whole title, you'll need to understand the basics of these three components and the libWiiPy classes that are used to represent them.
+
+Now, let's get into how you'd use them to extract a title from a WAD file.

docs/source/titles/title-module.md

@@ -0,0 +1,5 @@
+# The Title Module
+
+<project:#libWiiPy.title.title>
+
+Pardon our dust! This website is still under construction, and we haven't quite gotten to this one yet.

docs/source/usage/getting-started.md

@@ -0,0 +1,42 @@
+# Getting Started
+Once you have libWiiPy installed, it's time to write your first code!
+
+As an example, let's say you have a TMD file with a generic name, `title.tmd`, and because of this you need to find out some information about it, so you know what title it belongs to.
+
+First off, let's import `libWiiPy`, and load up our file:
+```pycon
+>>> import libWiiPy
+>>> tmd_file = open("title.tmd", "rb").read()
+>>>
+```
+
+Then we'll create a new TMD object, and load our file into it:
+```pycon
+>>> tmd = libWiiPy.title.TMD()
+>>> tmd.load(tmd_file)
+>>>
+```
+
+And ta-da! We now have a new TMD object that can be used to find out whatever we need to know about this TMD.
+
+So, to find out what title this TMD is for, let's try looking at the TMD's `title_id` property, like this:
+```pycon
+>>> print(tmd.title_id)
+0000000100000002
+
+>>>
+```
+
+Aha! `0000000100000002`! That means this TMD belongs to the Wii Menu. But what version? Well, we can use the TMD's `title_version` property to check, like so:
+```pycon
+>>> print(tmd.title_version)
+513
+
+>>>
+```
+
+513! So now we know that this TMD is from the Wii Menu, and is version 513, which is the version number used for v4.3U.
+
+So now you know how to identify what title and version a TMD file is from! But, realistically, trying to identify a lone unlabeled TMD file is not something you'll ever really need to do, either in your day-to-day life or in whatever program you're developing. In the next chapter, we'll dive in to working with more components of a title, which is a lot more useful for programs that need to manipulate them.
+
+The full documentation on the TMD class can be found here: <project:#libWiiPy.title.tmd>

docs/source/usage/installation.md

@@ -0,0 +1,20 @@
+# Installation
+The first thing you'll want to do to get set up is to install the `libWiiPy` package. This can be done one of two ways.
+
+**For a more stable experience,** you can install the latest release from PyPI just like any other Python package:
+```shell
+pip install libWiiPy
+```
+
+**If you prefer to live on the edge** (or just want to use features currently in development), you can also build the latest version from git:
+```shell
+pip install git+https://github.com/NinjaCheetah/libWiiPy
+```
+
+If you'd like to check the latest release, our PyPI page can be found [here](https://pypi.org/project/libWiiPy/). Release notes and build files for each release can be found over on our [GitHub releases page](https://github.com/NinjaCheetah/libWiiPy/releases/latest).
+
+:::{caution}
+libWiiPy is under heavy active development! While we try our hardest to not make breaking changes, things move quickly and that sometimes can cause problems.
+:::
+
+For those who are truly brave and want to experiment with the latest features, you can try building from an alternative branch. However, if you're going to do this, please be aware that features on branches other than `main` are likely very incomplete, and potentially completely broken. New features are only merged into `main` once they've been proven to at least work for their intended purpose. This does not guarantee a bug-free experience, but you are significantly less likely to run into show-stopping bugs.

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "libWiiPy"
-version = "0.3.1"
+version = "0.4.0"
 authors = [
     { name="NinjaCheetah", email="ninjacheetah@ncxprogramming.com" },
     { name="Lillian Skinner", email="lillian@randommeaninglesscharacters.com" }

requirements.txt

@@ -1,3 +1,9 @@
 build
 pycryptodome
 requests
+sphinx
+sphinx-book-theme
+myst-parser
+sphinx-copybutton
+sphinx-tippy
+sphinx-design

src/libWiiPy/archive/__init__.py

@@ -1,4 +1,5 @@
 # "archive/__init__.py" from libWiiPy by NinjaCheetah & Contributors
 # https://github.com/NinjaCheetah/libWiiPy
 
+from .ash import *
 from .u8 import *

src/libWiiPy/archive/ash.py

@@ -0,0 +1,233 @@
+# "archive/ash.py" from libWiiPy by NinjaCheetah & Contributors
+# https://github.com/NinjaCheetah/libWiiPy
+#
+# This code in particular is a direct translation of "ash-dec" from ASH0-tools. ASH0-tools is written by Garhoogin and
+# co-authored by NinjaCheetah.
+# https://github.com/NinjaCheetah/ASH0-tools
+#
+# See <link pending> for details about the ASH archive format.
+
+import io
+from dataclasses import dataclass as _dataclass
+
+
+@_dataclass
+class _ASHBitReader:
+    """
+    An _ASHBitReader class used to parse individual words in an ASH file. Private class used by the ASH module.
+
+    Attributes
+    ----------
+    src_data : list[int]
+        The entire data of the ASH file being parsed, as a list of integers for each byte.
+    size : int
+        The size of the ASH file.
+    src_pos : int
+        The position in the src_data list currently being accessed.
+    word : int
+        The word currently being decompressed.
+    bit_capacity : int
+    tree_type : str
+        What tree this bit reader is being used with. Used exclusively for debugging, as this value is only used in
+        error messages.
+    """
+    src_data: list[int]
+    size: int
+    src_pos: int
+    word: int
+    bit_capacity: int
+    tree_type: str
+
+
+def _ash_bit_reader_feed_word(bit_reader: _ASHBitReader):
+    # Ensure that there's enough data to read en entire word, then if there is, read one.
+    if not bit_reader.src_pos + 4 <= bit_reader.size:
+        print(bit_reader.src_pos)
+        raise ValueError("Invalid ASH data! Cannot decompress.")
+    bit_reader.word = int.from_bytes(bit_reader.src_data[bit_reader.src_pos:bit_reader.src_pos + 4], 'big')
+    bit_reader.bit_capacity = 0
+    bit_reader.src_pos += 4
+
+
+def _ash_bit_reader_init(bit_reader: _ASHBitReader, src: list[int], size: int, start_pos: int):
+    # Load data into a bit reader, then have it read its first word.
+    bit_reader.src_data = src
+    bit_reader.size = size
+    bit_reader.src_pos = start_pos
+    _ash_bit_reader_feed_word(bit_reader)
+
+
+def _ash_bit_reader_read_bit(bit_reader: _ASHBitReader):
+    # Reads the starting bit of the current word in the provided bit reader. If the capacity is at 31, then we've
+    # shifted through the entire word, so a new one should be fed. If not, increase the capacity by one and shift the
+    # current word left.
+    bit = bit_reader.word >> 31
+    if bit_reader.bit_capacity == 31:
+        _ash_bit_reader_feed_word(bit_reader)
+    else:
+        bit_reader.bit_capacity += 1
+        bit_reader.word = (bit_reader.word << 1) & 0xFFFFFFFF  # This simulates a 32-bit integer.
+
+    return bit
+
+
+def _ash_bit_reader_read_bits(bit_reader: _ASHBitReader, num_bits: int):
+    # Reads a series of bytes from the current word in the supplied bit reader.
+    bits: int
+    next_bit = bit_reader.bit_capacity + num_bits
+
+    if next_bit <= 32:
+        bits = bit_reader.word >> (32 - num_bits)
+        if next_bit != 32:
+            bit_reader.word = (bit_reader.word << num_bits) & 0xFFFFFFFF  # This simulates a 32-bit integer (again).
+            bit_reader.bit_capacity += num_bits
+        else:
+            _ash_bit_reader_feed_word(bit_reader)
+    else:
+        bits = bit_reader.word >> (32 - num_bits)
+        _ash_bit_reader_feed_word(bit_reader)
+        bits |= (bit_reader.word >> (64 - next_bit))
+        bit_reader.word = (bit_reader.word << (next_bit - 32)) & 0xFFFFFFFF  # Simulate 32-bit int.
+        bit_reader.bit_capacity = next_bit - 32
+
+    return bits
+
+
+def _ash_read_tree(bit_reader: _ASHBitReader, width: int, left_tree: [int], right_tree: [int]):
+    # Read either the symbol or distance tree from the ASH file, and return the root of that tree.
+    work = [0] * (2 * (1 << width))
+    work_pos = 0
+
+    r23 = 1 << width
+    tree_root = 0
+    num_nodes = 0
+
+    while True:
+        if _ash_bit_reader_read_bit(bit_reader) != 0:
+            work[work_pos] = (r23 | 0x80000000)
+            work_pos += 1
+            work[work_pos] = (r23 | 0x40000000)
+            work_pos += 1
+            num_nodes += 2
+            r23 += 1
+        else:
+            tree_root = _ash_bit_reader_read_bits(bit_reader, width)
+            while True:
+                work_pos -= 1
+                node_value = work[work_pos]
+                idx = node_value & 0x3FFFFFFF
+                num_nodes -= 1
+                try:
+                    if node_value & 0x80000000:
+                        right_tree[idx] = tree_root
+                        tree_root = idx
+                    else:
+                        left_tree[idx] = tree_root
+                        break
+                except IndexError:
+                    raise ValueError("Decompression failed while reading " + bit_reader.tree_type + " tree! Incorrect "
+                                     "leaf width may have been used. Try using a different number of bits for the " +
+                                     bit_reader.tree_type + " tree leaves.")
+                # Simulate a do-while loop.
+                if num_nodes == 0:
+                    break
+        # Also a do-while.
+        if num_nodes == 0:
+            break
+
+    return tree_root
+
+
+def _decompress_ash(input_data: list[int], size: int, sym_bits: int, dist_bits: int):
+    # Get the size of the decompressed data by reading the second 4 bytes of the file and masking the first one out.
+    decompressed_size = int.from_bytes(input_data[0x4:0x8]) & 0x00FFFFFF
+    # Array of decompressed data and the position in that array that we're at. Mimics the memory pointer from the
+    # original C source.
+    out_buffer = [0] * decompressed_size
+    out_buffer_pos = 0
+    # Create two empty bit readers, and then initialize them at two different positions for the two trees.
+    bit_reader1 = _ASHBitReader([0], 0, 0, 0, 0, "distance")
+    _ash_bit_reader_init(bit_reader1, input_data, size, int.from_bytes(input_data[0x8:0xC], byteorder='big'))
+    bit_reader2 = _ASHBitReader([0], 0, 0, 0, 0, "symbol")
+    _ash_bit_reader_init(bit_reader2, input_data, size, 0xC)
+    # Calculate the max for the symbol and distance trees based on the bit lengths that were passed. Then, allocate the
+    # arrays for all the trees based on that maximum.
+    sym_max = 1 << sym_bits
+    dist_max = 1 << dist_bits
+    sym_left_tree = [0] * (2 * sym_max - 1)
+    sym_right_tree = [0] * (2 * sym_max - 1)
+    dist_left_tree = [0] * (2 * dist_max - 1)
+    dist_right_tree = [0] * (2 * dist_max - 1)
+    # Read the trees to find the symbol and distance tree roots.
+    sym_root = _ash_read_tree(bit_reader2, sym_bits, sym_left_tree, sym_right_tree)
+    dist_root = _ash_read_tree(bit_reader1, dist_bits, dist_left_tree, dist_right_tree)
+    # Main decompression loop.
+    while True:
+        sym = sym_root
+        while sym >= sym_max:
+            if _ash_bit_reader_read_bit(bit_reader2) != 0:
+                sym = sym_right_tree[sym]
+            else:
+                sym = sym_left_tree[sym]
+        if sym < 0x100:
+            out_buffer[out_buffer_pos] = sym
+            out_buffer_pos += 1
+            decompressed_size -= 1
+        else:
+            dist_sym = dist_root
+            while dist_sym >= dist_max:
+                if _ash_bit_reader_read_bit(bit_reader1) != 0:
+                    dist_sym = dist_right_tree[dist_sym]
+                else:
+                    dist_sym = dist_left_tree[dist_sym]
+            copy_len = (sym - 0x100) + 3
+            srcp_pos = out_buffer_pos - dist_sym - 1
+            # Check to make sure we aren't going to exceed the specified decompressed size.
+            if not copy_len <= decompressed_size:
+                raise ValueError("Invalid ASH data! Cannot decompress.")
+
+            decompressed_size -= copy_len
+            while copy_len > 0:
+                out_buffer[out_buffer_pos] = out_buffer[srcp_pos]
+                out_buffer_pos += 1
+                srcp_pos += 1
+                copy_len -= 1
+        # Simulate a do-while loop.
+        if decompressed_size == 0:
+            break
+
+    return out_buffer
+
+
+def decompress_ash(ash_data: bytes, sym_tree_bits: int = 9, dist_tree_bits: int = 11) -> bytes:
+    """
+    Decompresses the data of an ASH file and returns the decompressed data.
+
+    With the default parameters, this function can decompress ASH files found in the files of the Wii Menu and Animal
+    Crossing: City Folk. Some ASH files, notably the ones found in the WiiWare title My Pokémon Ranch, require setting
+    dist_tree_bits to 15 instead for a successful decompression. If an ASH file is failing to decompress with the
+    default options, trying a dist_tree_bits value of 15 will likely fix it. No other leaf sizes are known to exist,
+    however they might be out there.
+
+    Parameters
+    ----------
+    ash_data : bytes
+        The data for the ASH file to decompress.
+    sym_tree_bits : int, option
+        Number of bits for each leaf in the symbol tree. Defaults to 9.
+    dist_tree_bits : int, option
+        Number of bits for each leaf in the distance tree. Defaults to 11.
+    """
+    # Check the magic number to make sure this is an ASH file.
+    with io.BytesIO(ash_data) as ash_data2:
+        ash_magic = ash_data2.read(4)
+        if ash_magic != b'\x41\x53\x48\x30':
+            raise TypeError("This is not a valid ASH file!")
+    # Begin decompression. Convert the compressed data to an array of ints for processing, then convert the returned
+    # decompressed data back into bytes to return it.
+    ash_size = len(ash_data)
+    ash_data_int = [byte for byte in ash_data]
+    decompressed_data = _decompress_ash(ash_data_int, ash_size, sym_tree_bits, dist_tree_bits)
+    decompressed_data_bin = bytes(decompressed_data)
+
+    return decompressed_data_bin

src/libWiiPy/archive/u8.py

@@ -6,17 +6,17 @@
 import io
 import os
 import pathlib
-from dataclasses import dataclass
+from dataclasses import dataclass as _dataclass
 from typing import List
-from ..shared import align_value
+from ..shared import _align_value
 
 
-@dataclass
+@_dataclass
-class U8Node:
+class _U8Node:
     """
     A U8Node object that contains the data of a single node in a U8 file header. Each node keeps track of whether this
     node is for a file or directory, the offset of the name of the file/directory, the offset of the data for the file/
-    directory, and the size of the data.
+    directory, and the size of the data. Private class used by functions and methods in the U8 module.
 
     Attributes
     ----------
@@ -44,7 +44,7 @@ class U8Archive:
         ----------
         """
         self.u8_magic = b''
-        self.u8_node_list: List[U8Node] = []  # All the nodes in the header of a U8 file.
+        self.u8_node_list: List[_U8Node] = []  # All the nodes in the header of a U8 file.
         self.file_name_list: List[str] = []
         self.file_data_list: List[bytes] = []
         self.u8_file_structure = dict
@@ -86,7 +86,7 @@ class U8Archive:
                 node_name_offset = int.from_bytes(u8_data.read(2))
                 node_data_offset = int.from_bytes(u8_data.read(4))
                 node_size = int.from_bytes(u8_data.read(4))
-                self.u8_node_list.append(U8Node(node_type, node_name_offset, node_data_offset, node_size))
+                self.u8_node_list.append(_U8Node(node_type, node_name_offset, node_data_offset, node_size))
             # Iterate over all loaded nodes and create a list of file names and a list of file data.
             name_base_offset = u8_data.tell()
             for node in self.u8_node_list:
@@ -121,7 +121,7 @@ class U8Archive:
         for file_name in self.file_name_list:
             header_size += len(file_name) + 1
         # The initial data offset is equal to the file header (32 bytes) + node data aligned to 16 bytes.
-        data_offset = align_value(header_size + 32, 16)
+        data_offset = _align_value(header_size + 32, 16)
         # Adjust all nodes to place file data in the same order as the nodes. Why isn't it already like this?
         current_data_offset = data_offset
         for node in range(len(self.u8_node_list)):
@@ -172,9 +172,14 @@ def extract_u8(u8_data, output_folder) -> None:
         The path to a new folder to extract the archive to.
     """
     output_folder = pathlib.Path(output_folder)
-    if pathlib.Path.is_dir(output_folder):
+    # Check if the path already exists, and if it does, ensure that it is both a directory and empty.
-        raise ValueError("Output folder already exists!")
+    if output_folder.exists():
-    os.mkdir(output_folder)
+        if output_folder.is_dir() and next(os.scandir(output_folder), None):
+            raise ValueError("Output folder is not empty!")
+        elif output_folder.is_file():
+            raise ValueError("A file already exists with the provided name!")
+    else:
+        os.mkdir(output_folder)
     # Create a new U8Archive object and load the provided U8 file data into it.
     u8_archive = U8Archive()
     u8_archive.load(u8_data)
@@ -241,7 +246,7 @@ def _pack_u8_dir(u8_archive: U8Archive, current_path, node_count, name_offset):
         node_count += 1
         u8_archive.file_name_list.append(file)
         u8_archive.file_data_list.append(open(current_path.joinpath(file), "rb").read())
-        u8_archive.u8_node_list.append(U8Node(0, name_offset, 0, len(u8_archive.file_data_list[-1])))
+        u8_archive.u8_node_list.append(_U8Node(0, name_offset, 0, len(u8_archive.file_data_list[-1])))
         name_offset = name_offset + len(file) + 1  # Add 1 to accommodate the null byte at the end of the name.
     # For directories, add their name to the file name list, add empty data to the file data list (since they obviously
     # wouldn't have any), find the total number of files and directories inside the directory to calculate the final
@@ -251,7 +256,7 @@ def _pack_u8_dir(u8_archive: U8Archive, current_path, node_count, name_offset):
         u8_archive.file_name_list.append(directory)
         u8_archive.file_data_list.append(b'')
         max_node = node_count + sum(1 for _ in current_path.joinpath(directory).rglob('*'))
-        u8_archive.u8_node_list.append(U8Node(256, name_offset, 0, max_node))
+        u8_archive.u8_node_list.append(_U8Node(256, name_offset, 0, max_node))
         name_offset = name_offset + len(directory) + 1  # Add 1 to accommodate the null byte at the end of the name.
         u8_archive, node_count, name_offset = _pack_u8_dir(u8_archive, current_path.joinpath(directory), node_count,
                                                            name_offset)
@@ -280,7 +285,7 @@ def pack_u8(input_path) -> bytes:
         u8_archive = U8Archive()
         u8_archive.file_name_list.append("")
         u8_archive.file_data_list.append(b'')
-        u8_archive.u8_node_list.append(U8Node(256, 0, 0, sum(1 for _ in input_path.rglob('*')) + 1))
+        u8_archive.u8_node_list.append(_U8Node(256, 0, 0, sum(1 for _ in input_path.rglob('*')) + 1))
         # Call the private function _pack_u8_dir() on the root note, which will recursively call itself to pack every
         # subdirectory and file. Discard node_count and name_offset since we don't care about them here, as they're
         # really only necessary for the directory recursion.
@@ -300,8 +305,8 @@ def pack_u8(input_path) -> bytes:
             u8_archive.file_data_list.append(b'')
             u8_archive.file_data_list.append(file_data)
             # Append generic U8Node for the root, followed by the actual file's node.
-            u8_archive.u8_node_list.append(U8Node(256, 0, 0, 2))
+            u8_archive.u8_node_list.append(_U8Node(256, 0, 0, 2))
-            u8_archive.u8_node_list.append(U8Node(0, 1, 0, len(file_data)))
+            u8_archive.u8_node_list.append(_U8Node(0, 1, 0, len(file_data)))
             return u8_archive.dump()
     else:
         raise FileNotFoundError("Input file/directory: \"" + str(input_path) + "\" does not exist!")

src/libWiiPy/shared.py

@@ -4,12 +4,10 @@
 # This file defines general functions that may be useful in other modules of libWiiPy. Putting them here cuts down on
 # clutter in other files.
 
-import binascii
 
-
+def _align_value(value, alignment=64) -> int:
-def align_value(value, alignment=64) -> int:
     """
-    Aligns the provided value to the set alignment (defaults to 64).
+    Aligns the provided value to the set alignment (defaults to 64). Private function used by other libWiiPy modules.
 
     Parameters
     ----------
@@ -29,9 +27,10 @@ def align_value(value, alignment=64) -> int:
     return value
 
 
-def pad_bytes(data, alignment=64) -> bytes:
+def _pad_bytes(data, alignment=64) -> bytes:
     """
-    Pads the provided bytes object to the provided alignment (defaults to 64).
+    Pads the provided bytes object to the provided alignment (defaults to 64). Private function used by other libWiiPy
+    modules.
 
     Parameters
     ----------
@@ -48,24 +47,3 @@ def pad_bytes(data, alignment=64) -> bytes:
     while (len(data) % alignment) != 0:
         data += b'\x00'
     return data
-
-
-def convert_tid_to_iv(title_id: str) -> bytes:
-    title_key_iv = b''
-    if type(title_id) is bytes:
-        # This catches the format b'0000000100000002'
-        if len(title_id) == 16:
-            title_key_iv = binascii.unhexlify(title_id)
-        # This catches the format b'\x00\x00\x00\x01\x00\x00\x00\x02'
-        elif len(title_id) == 8:
-            pass
-        # If it isn't one of those lengths, it cannot possibly be valid, so reject it.
-        else:
-            raise ValueError("Title ID is not valid!")
-    # Allow for a string like "0000000100000002"
-    elif type(title_id) is str:
-        title_key_iv = binascii.unhexlify(title_id)
-    # If the Title ID isn't bytes or a string, it isn't valid and is rejected.
-    else:
-        raise TypeError("Title ID type is not valid! It must be either type str or bytes.")
-    return title_key_iv

src/libWiiPy/title/content.py

@@ -4,10 +4,10 @@
 # See https://wiibrew.org/wiki/Title for details about how titles are formatted
 
 import io
-import sys
 import hashlib
 from typing import List
-from ..types import ContentRecord
+from ..types import _ContentRecord
+from ..shared import _pad_bytes, _align_value
 from .crypto import decrypt_content, encrypt_content
 
 
@@ -18,20 +18,20 @@ class ContentRegion:
 
     Attributes
     ----------
-    content_records : List[ContentRecord]
+    content_records : List[_ContentRecord]
         The content records for the content stored in the region.
     num_contents : int
         The total number of contents stored in the region.
     """
 
     def __init__(self):
-        self.content_records: List[ContentRecord] = []
+        self.content_records: List[_ContentRecord] = []
         self.content_region_size: int = 0  # Size of the content region.
         self.num_contents: int = 0  # Number of contents in the content region.
         self.content_start_offsets: List[int] = [0]  # The start offsets of each content in the content region.
         self.content_list: List[bytes] = []
 
-    def load(self, content_region: bytes, content_records: List[ContentRecord]) -> None:
+    def load(self, content_region: bytes, content_records: List[_ContentRecord]) -> None:
         """
         Loads the raw content region and builds a list of all the contents.
 
@@ -39,13 +39,13 @@ class ContentRegion:
         ----------
         content_region : bytes
             The raw data for the content region being loaded.
-        content_records : list[ContentRecord]
+        content_records : list[_ContentRecord]
             A list of ContentRecord objects detailing all contents contained in the region.
         """
         self.content_records = content_records
+        # Get the total size of the content region.
+        self.content_region_size = len(content_region)
         with io.BytesIO(content_region) as content_region_data:
-            # Get the total size of the content region.
-            self.content_region_size = sys.getsizeof(content_region_data)
             self.num_contents = len(self.content_records)
             # Calculate the offsets of each content in the content region.
             # Content is aligned to 16 bytes, however a new content won't start until the next multiple of 64 bytes.
@@ -56,7 +56,7 @@ class ContentRegion:
                     start_offset += 64 - (content.content_size % 64)
                 self.content_start_offsets.append(start_offset)
             # Build a list of all the encrypted content data.
-            for content in range(len(self.content_start_offsets)):
+            for content in range(self.num_contents):
                 # Seek to the start of the content based on the list of offsets.
                 content_region_data.seek(self.content_start_offsets[content])
                 # Calculate the number of bytes we need to read by adding bytes up the nearest multiple of 16 if needed.
@@ -68,7 +68,7 @@ class ContentRegion:
                 content_enc = content_region_data.read(bytes_to_read)
                 self.content_list.append(content_enc)
 
-    def dump(self) -> bytes:
+    def dump(self) -> tuple[bytes, int]:
         """
         Takes the list of contents and assembles them back into one content region. Returns this content region as a
         bytes object and sets the raw content region variable to this result, then calls load() again to make sure the
@@ -77,19 +77,31 @@ class ContentRegion:
         Returns
         -------
         bytes
-            The full WAD file as bytes.
+            The full ContentRegion as bytes, including padding between content.
+        int
+            The size of the ContentRegion, including padding.
         """
         content_region_data = b''
         for content in self.content_list:
+            # If this isn't the first content, pad the whole region to 64 bytes before the next one.
+            if content_region_data is not b'':
+                content_region_data = _pad_bytes(content_region_data, 64)
             # Calculate padding after this content before the next one.
             padding_bytes = 0
-            if (len(content) % 64) != 0:
+            if (len(content) % 16) != 0:
-                padding_bytes = 64 - (len(content) % 64)
+                padding_bytes = 16 - (len(content) % 16)
             # Write content data, then the padding afterward if necessary.
             content_region_data += content
             if padding_bytes > 0:
                 content_region_data += b'\x00' * padding_bytes
-        return content_region_data
+        # Calculate the size of the whole content region.
+        content_region_size = 0
+        for record in range(len(self.content_records)):
+            if record is len(self.content_records) - 1:
+                content_region_size += self.content_records[record].content_size
+            else:
+                content_region_size += _align_value(self.content_records[record].content_size, 64)
+        return content_region_data, content_region_size
 
     def get_enc_content_by_index(self, index: int) -> bytes:
         """
@@ -173,7 +185,7 @@ class ContentRegion:
         # Compare the hash and throw a ValueError if the hash doesn't match.
         if content_dec_hash != content_record_hash:
             raise ValueError("Content hash did not match the expected hash in its record! The incorrect Title Key may "
-                             "have been used!.\n"
+                             "have been used!\n"
                              "Expected hash is: {}\n".format(content_record_hash) +
                              "Actual hash is: {}".format(content_dec_hash))
         return content_dec
@@ -256,7 +268,7 @@ class ContentRegion:
             if (index + 1) > num_contents + 1:
                 raise ValueError("You are trying to set the content at position " + str(index) + ", but no content "
                                  "exists at position " + str(index - 1) + "!")
-            self.content_records.append(ContentRecord(cid, index, content_type, content_size, content_hash))
+            self.content_records.append(_ContentRecord(cid, index, content_type, content_size, content_hash))
         # If it does, reassign the values in it.
         else:
             self.content_records[index].content_id = cid

src/libWiiPy/title/crypto.py

@@ -2,10 +2,32 @@
 # https://github.com/NinjaCheetah/libWiiPy
 
 import struct
+import binascii
 from .commonkeys import get_common_key
-from ..shared import convert_tid_to_iv
+from Crypto.Cipher import AES as _AES
 
-from Crypto.Cipher import AES
+
+def _convert_tid_to_iv(title_id: str) -> bytes:
+    # Converts a Title ID in various formats into the format required to act as an IV. Private function used by other
+    # crypto functions.
+    title_key_iv = b''
+    if type(title_id) is bytes:
+        # This catches the format b'0000000100000002'
+        if len(title_id) == 16:
+            title_key_iv = binascii.unhexlify(title_id)
+        # This catches the format b'\x00\x00\x00\x01\x00\x00\x00\x02'
+        elif len(title_id) == 8:
+            pass
+        # If it isn't one of those lengths, it cannot possibly be valid, so reject it.
+        else:
+            raise ValueError("Title ID is not valid!")
+    # Allow for a string like "0000000100000002"
+    elif type(title_id) is str:
+        title_key_iv = binascii.unhexlify(title_id)
+    # If the Title ID isn't bytes or a string, it isn't valid and is rejected.
+    else:
+        raise TypeError("Title ID type is not valid! It must be either type str or bytes.")
+    return title_key_iv
 
 
 def decrypt_title_key(title_key_enc: bytes, common_key_index: int, title_id: bytes | str) -> bytes:
@@ -31,11 +53,11 @@ def decrypt_title_key(title_key_enc: bytes, common_key_index: int, title_id: byt
     # Load the correct common key for the title.
     common_key = get_common_key(common_key_index)
     # Convert the IV into the correct format based on the type provided.
-    title_key_iv = convert_tid_to_iv(title_id)
+    title_key_iv = _convert_tid_to_iv(title_id)
     # The IV will always be in the same format by this point, so add the last 8 bytes.
     title_key_iv = title_key_iv + (b'\x00' * 8)
     # Create a new AES object with the values provided.
-    aes = AES.new(common_key, AES.MODE_CBC, title_key_iv)
+    aes = _AES.new(common_key, _AES.MODE_CBC, title_key_iv)
     # Decrypt the Title Key using the AES object.
     title_key = aes.decrypt(title_key_enc)
     return title_key
@@ -64,11 +86,11 @@ def encrypt_title_key(title_key_dec: bytes, common_key_index: int, title_id: byt
     # Load the correct common key for the title.
     common_key = get_common_key(common_key_index)
     # Convert the IV into the correct format based on the type provided.
-    title_key_iv = convert_tid_to_iv(title_id)
+    title_key_iv = _convert_tid_to_iv(title_id)
     # The IV will always be in the same format by this point, so add the last 8 bytes.
     title_key_iv = title_key_iv + (b'\x00' * 8)
     # Create a new AES object with the values provided.
-    aes = AES.new(common_key, AES.MODE_CBC, title_key_iv)
+    aes = _AES.new(common_key, _AES.MODE_CBC, title_key_iv)
     # Encrypt Title Key using the AES object.
     title_key = aes.encrypt(title_key_dec)
     return title_key
@@ -105,7 +127,7 @@ def decrypt_content(content_enc, title_key, content_index, content_length) -> by
     if (len(content_enc) % 16) != 0:
         content_enc = content_enc + (b'\x00' * (16 - (len(content_enc) % 16)))
     # Create a new AES object with the values provided, with the content's unique ID as the IV.
-    aes = AES.new(title_key, AES.MODE_CBC, content_index_bin)
+    aes = _AES.new(title_key, _AES.MODE_CBC, content_index_bin)
     # Decrypt the content using the AES object.
     content_dec = aes.decrypt(content_enc)
     # Trim additional bytes that may have been added so the content is the correct size.
@@ -144,7 +166,7 @@ def encrypt_content(content_dec, title_key, content_index) -> bytes:
     if (len(content_dec) % 16) != 0:
         content_dec = content_dec + (b'\x00' * (16 - (len(content_dec) % 16)))
     # Create a new AES object with the values provided, with the content's unique ID as the IV.
-    aes = AES.new(title_key, AES.MODE_CBC, content_index_bin)
+    aes = _AES.new(title_key, _AES.MODE_CBC, content_index_bin)
     # Encrypt the content using the AES object.
     content_enc = aes.encrypt(content_dec)
     # Trim down the encrypted content.

src/libWiiPy/title/nus.py

@@ -10,7 +10,7 @@ from .title import Title
 from .tmd import TMD
 from .ticket import Ticket
 
-nus_endpoint = ["http://nus.cdn.shop.wii.com/ccs/download/", "http://ccs.cdn.wup.shop.nintendo.net/ccs/download/"]
+_nus_endpoint = ["http://nus.cdn.shop.wii.com/ccs/download/", "http://ccs.cdn.wup.shop.nintendo.net/ccs/download/"]
 
 
 def download_title(title_id: str, title_version: int = None, wiiu_endpoint: bool = False) -> Title:
@@ -68,9 +68,9 @@ def download_tmd(title_id: str, title_version: int = None, wiiu_endpoint: bool =
     # Build the download URL. The structure is download/<TID>/tmd for latest and download/<TID>/tmd.<version> for
     # when a specific version is requested.
     if wiiu_endpoint is False:
-        tmd_url = nus_endpoint[0] + title_id + "/tmd"
+        tmd_url = _nus_endpoint[0] + title_id + "/tmd"
     else:
-        tmd_url = nus_endpoint[1] + title_id + "/tmd"
+        tmd_url = _nus_endpoint[1] + title_id + "/tmd"
     # Add the version to the URL if one was specified.
     if title_version is not None:
         tmd_url += "." + str(title_version)
@@ -109,9 +109,9 @@ def download_ticket(title_id: str, wiiu_endpoint: bool = False) -> bytes:
     # Build the download URL. The structure is download/<TID>/cetk, and cetk will only exist if this is a free
     # title.
     if wiiu_endpoint is False:
-        ticket_url = nus_endpoint[0] + title_id + "/cetk"
+        ticket_url = _nus_endpoint[0] + title_id + "/cetk"
     else:
-        ticket_url = nus_endpoint[1] + title_id + "/cetk"
+        ticket_url = _nus_endpoint[1] + title_id + "/cetk"
     # Make the request.
     ticket_request = requests.get(url=ticket_url, headers={'User-Agent': 'wii libnup/1.0'}, stream=True)
     if ticket_request.status_code != 200:
@@ -142,11 +142,11 @@ def download_cert(wiiu_endpoint: bool = False) -> bytes:
     """
     # Download the TMD and cetk for the System Menu 4.3U.
     if wiiu_endpoint is False:
-        tmd_url = nus_endpoint[0] + "0000000100000002/tmd.513"
+        tmd_url = _nus_endpoint[0] + "0000000100000002/tmd.513"
-        cetk_url = nus_endpoint[0] + "0000000100000002/cetk"
+        cetk_url = _nus_endpoint[0] + "0000000100000002/cetk"
     else:
-        tmd_url = nus_endpoint[1] + "0000000100000002/tmd.513"
+        tmd_url = _nus_endpoint[1] + "0000000100000002/tmd.513"
-        cetk_url = nus_endpoint[1] + "0000000100000002/cetk"
+        cetk_url = _nus_endpoint[1] + "0000000100000002/cetk"
     tmd = requests.get(url=tmd_url, headers={'User-Agent': 'wii libnup/1.0'}, stream=True).content
     cetk = requests.get(url=cetk_url, headers={'User-Agent': 'wii libnup/1.0'}, stream=True).content
     # Assemble the certificate.
@@ -186,9 +186,9 @@ def download_content(title_id: str, content_id: int, wiiu_endpoint: bool = False
     if len(content_id_hex) < 2:
         content_id_hex = "0" + content_id_hex
     if wiiu_endpoint is False:
-        content_url = nus_endpoint[0] + title_id + "/000000" + content_id_hex
+        content_url = _nus_endpoint[0] + title_id + "/000000" + content_id_hex
     else:
-        content_url = nus_endpoint[1] + title_id + "/000000" + content_id_hex
+        content_url = _nus_endpoint[1] + title_id + "/000000" + content_id_hex
     # Make the request.
     content_request = requests.get(url=content_url, headers={'User-Agent': 'wii libnup/1.0'}, stream=True)
     if content_request.status_code != 200:

src/libWiiPy/title/ticket.py

@@ -5,11 +5,33 @@
 
 import io
 import binascii
+from dataclasses import dataclass as _dataclass
 from .crypto import decrypt_title_key
-from ..types import TitleLimit
 from typing import List
 
 
+@_dataclass
+class _TitleLimit:
+    """
+    A TitleLimit object that contains the type of restriction and the limit. The limit type can be one of the following:
+    0 = None, 1 = Time Limit, 3 = None, or 4 = Launch Count. The maximum usage is then either the time in minutes the
+    title can be played or the maximum number of launches allowed for that title, based on the type of limit applied.
+    Private class used only by the Ticket class.
+
+    Attributes
+    ----------
+    limit_type : int
+        The type of play limit applied.
+    maximum_usage : int
+        The maximum value for the type of play limit applied.
+    """
+    # The type of play limit applied.
+    # 0 = None, 1 = Time Limit, 3 = None, 4 = Launch Count
+    limit_type: int
+    # The maximum value of the limit applied.
+    maximum_usage: int
+
+
 class Ticket:
     """
     A Ticket object that allows for either loading and editing an existing Ticket or creating one manually if desired.
@@ -47,12 +69,14 @@ class Ticket:
         self.unknown1: bytes = b''  # Some unknown data, not always the same so reading it just in case.
         self.title_version: int = 0  # Version of the ticket's associated title.
         self.permitted_titles: bytes = b''  # Permitted titles mask
-        self.permit_mask: bytes = b''  # "Permit mask. The current disc title is ANDed with the inverse of this mask to see if the result matches the Permitted Titles Mask."
+        # "Permit mask. The current disc title is ANDed with the inverse of this mask to see if the result matches the
+        # Permitted Titles Mask."
+        self.permit_mask: bytes = b''
         self.title_export_allowed: int = 0  # Whether title export is allowed with a PRNG key or not.
         self.common_key_index: int = 0  # Which common key should be used. 0 = Common Key, 1 = Korean Key, 2 = vWii Key
         self.unknown2: bytes = b''  # More unknown data. Varies for VC/non-VC titles so reading it to ensure it matches.
         self.content_access_permissions: bytes = b''  # "Content access permissions (one bit for each content)"
-        self.title_limits_list: List[TitleLimit] = []  # List of play limits applied to the title.
+        self.title_limits_list: List[_TitleLimit] = []  # List of play limits applied to the title.
         # v1 ticket data
         # TODO: Write in v1 ticket attributes here. This code can currently only handle v0 tickets, and will reject v1.
 
@@ -134,7 +158,7 @@ class Ticket:
             for limit in range(0, 8):
                 limit_type = int.from_bytes(ticket_data.read(4))
                 limit_value = int.from_bytes(ticket_data.read(4))
-                self.title_limits_list.append(TitleLimit(limit_type, limit_value))
+                self.title_limits_list.append(_TitleLimit(limit_type, limit_value))
 
     def dump(self) -> bytes:
         """

src/libWiiPy/title/title.py

@@ -78,7 +78,8 @@ class Title:
         # Dump the Ticket and set it in the WAD.
         self.wad.set_ticket_data(self.ticket.dump())
         # Dump the ContentRegion and set it in the WAD.
-        self.wad.set_content_data(self.content.dump())
+        content_data, content_size = self.content.dump()
+        self.wad.set_content_data(content_data, content_size)
         return self.wad.dump()
 
     def load_tmd(self, tmd: bytes) -> None:

src/libWiiPy/title/tmd.py

@@ -7,7 +7,7 @@ import io
 import binascii
 import struct
 from typing import List
-from ..types import ContentRecord
+from ..types import _ContentRecord
 
 
 class TMD:
@@ -31,8 +31,8 @@ class TMD:
     """
     def __init__(self):
         self.blob_header: bytes = b''
-        self.sig_type: int = 0
+        self.signature_type: int = 0
-        self.sig: bytes = b''
+        self.signature: bytes = b''
         self.issuer: bytes = b''  # Follows the format "Root-CA%08x-CP%08x"
         self.tmd_version: int = 0  # This seems to always be 0 no matter what?
         self.ca_crl_version: int = 0  # Certificate Authority Certificate Revocation List version
@@ -52,7 +52,7 @@ class TMD:
         self.title_version: int = 0  # The version of the associated title.
         self.num_contents: int = 0  # The number of contents contained in the associated title.
         self.boot_index: int = 0  # The content index that contains the bootable executable.
-        self.content_records: List[ContentRecord] = []
+        self.content_records: List[_ContentRecord] = []
 
     def load(self, tmd: bytes) -> None:
         """
@@ -68,8 +68,12 @@ class TMD:
             # ====================================================================================
             # Parses each of the keys contained in the TMD.
             # ====================================================================================
+            # Signature type.
             tmd_data.seek(0x0)
-            self.blob_header = tmd_data.read(320)
+            self.signature_type = tmd_data.read(4)
+            # Signature data.
+            tmd_data.seek(0x04)
+            self.signature = tmd_data.read(256)
             # Signing certificate issuer.
             tmd_data.seek(0x140)
             self.issuer = tmd_data.read(64)
@@ -142,9 +146,9 @@ class TMD:
                 tmd_data.seek(0x1E4 + (36 * content))
                 content_record_hdr = struct.unpack(">LHH4x4s20s", tmd_data.read(36))
                 self.content_records.append(
-                    ContentRecord(int(content_record_hdr[0]), int(content_record_hdr[1]),
+                    _ContentRecord(int(content_record_hdr[0]), int(content_record_hdr[1]),
-                                  int(content_record_hdr[2]), int.from_bytes(content_record_hdr[3]),
+                                   int(content_record_hdr[2]), int.from_bytes(content_record_hdr[3]),
-                                  binascii.hexlify(content_record_hdr[4])))
+                                   binascii.hexlify(content_record_hdr[4])))
 
     def dump(self) -> bytes:
         """
@@ -157,8 +161,12 @@ class TMD:
             The full TMD file as bytes.
         """
         tmd_data = b''
-        # Signed blob header.
+        # Signature type.
-        tmd_data += self.blob_header
+        tmd_data += self.signature_type
+        # Signature data.
+        tmd_data += self.signature
+        # Padding to 64 bytes.
+        tmd_data += b'\x00' * 60
         # Signing certificate issuer.
         tmd_data += self.issuer
         # TMD version.
@@ -310,7 +318,7 @@ class TMD:
             case _:
                 return "Unknown"
 
-    def get_content_record(self, record) -> ContentRecord:
+    def get_content_record(self, record) -> _ContentRecord:
         """
         Gets the content record at the specified index.
 
@@ -321,7 +329,7 @@ class TMD:
 
         Returns
         -------
-        ContentRecord
+        _ContentRecord
             A ContentRecord object containing the data in the content record.
         """
         if record < self.num_contents:

src/libWiiPy/title/wad.py

@@ -5,7 +5,7 @@
 
 import io
 import binascii
-from ..shared import align_value, pad_bytes
+from ..shared import _align_value, _pad_bytes
 
 
 class WAD:
@@ -91,9 +91,11 @@ class WAD:
             # WAD TMD size.
             wad_data.seek(0x14)
             self.wad_tmd_size = int(binascii.hexlify(wad_data.read(4)), 16)
-            # WAD content size.
+            # WAD content size. This needs to be rounded now, because with some titles (primarily IOS?), there can be
+            # extra bytes past the listed end of the content that is needed for decryption.
             wad_data.seek(0x18)
             self.wad_content_size = int(binascii.hexlify(wad_data.read(4)), 16)
+            self.wad_content_size = _align_value(self.wad_content_size, 16)
             # Time/build stamp for the title contained in the WAD.
             wad_data.seek(0x1c)
             self.wad_meta_size = int(binascii.hexlify(wad_data.read(4)), 16)
@@ -102,12 +104,12 @@ class WAD:
             # ====================================================================================
             wad_cert_offset = self.wad_hdr_size
             # crl isn't ever used, however an entry for its size exists in the header, so its calculated just in case.
-            wad_crl_offset = align_value(wad_cert_offset + self.wad_cert_size)
+            wad_crl_offset = _align_value(wad_cert_offset + self.wad_cert_size)
-            wad_tik_offset = align_value(wad_crl_offset + self.wad_crl_size)
+            wad_tik_offset = _align_value(wad_crl_offset + self.wad_crl_size)
-            wad_tmd_offset = align_value(wad_tik_offset + self.wad_tik_size)
+            wad_tmd_offset = _align_value(wad_tik_offset + self.wad_tik_size)
             # meta isn't guaranteed to be used, but some older SDK titles use it, and not reading it breaks things.
-            wad_meta_offset = align_value(wad_tmd_offset + self.wad_tmd_size)
+            wad_meta_offset = _align_value(wad_tmd_offset + self.wad_tmd_size)
-            wad_content_offset = align_value(wad_meta_offset + self.wad_meta_size)
+            wad_content_offset = _align_value(wad_meta_offset + self.wad_meta_size)
             # ====================================================================================
             # Load data for each WAD section based on the previously calculated offsets.
             # ====================================================================================
@@ -159,25 +161,25 @@ class WAD:
         wad_data += int.to_bytes(self.wad_content_size, 4)
         # WAD meta size.
         wad_data += int.to_bytes(self.wad_meta_size, 4)
-        wad_data = pad_bytes(wad_data)
+        wad_data = _pad_bytes(wad_data)
         # Retrieve the cert data and write it out.
         wad_data += self.get_cert_data()
-        wad_data = pad_bytes(wad_data)
+        wad_data = _pad_bytes(wad_data)
         # Retrieve the crl data and write it out.
         wad_data += self.get_crl_data()
-        wad_data = pad_bytes(wad_data)
+        wad_data = _pad_bytes(wad_data)
         # Retrieve the ticket data and write it out.
         wad_data += self.get_ticket_data()
-        wad_data = pad_bytes(wad_data)
+        wad_data = _pad_bytes(wad_data)
         # Retrieve the TMD data and write it out.
         wad_data += self.get_tmd_data()
-        wad_data = pad_bytes(wad_data)
+        wad_data = _pad_bytes(wad_data)
         # Retrieve the meta/footer data and write it out.
         wad_data += self.get_meta_data()
-        wad_data = pad_bytes(wad_data)
+        wad_data = _pad_bytes(wad_data)
         # Retrieve the content data and write it out.
         wad_data += self.get_content_data()
-        wad_data = pad_bytes(wad_data)
+        wad_data = _pad_bytes(wad_data)
         return wad_data
 
     def get_wad_type(self) -> str:
@@ -309,7 +311,7 @@ class WAD:
         # Calculate the size of the new Ticket data.
         self.wad_tik_size = len(tik_data)
 
-    def set_content_data(self, content_data) -> None:
+    def set_content_data(self, content_data, size: int = None) -> None:
         """
         Sets the content data of the WAD. Also calculates the new size.
 
@@ -317,10 +319,15 @@ class WAD:
         ----------
         content_data : bytes
             The new content data.
+        size : int, option
+            The size of the new content data.
         """
         self.wad_content_data = content_data
-        # Calculate the size of the new content data.
+        # Calculate the size of the new content data, if one wasn't supplied.
-        self.wad_content_size = len(content_data)
+        if size is None:
+            self.wad_content_size = len(content_data)
+        else:
+            self.wad_content_size = size
 
     def set_meta_data(self, meta_data) -> None:
         """

src/libWiiPy/types.py

@@ -4,7 +4,7 @@ from dataclasses import dataclass
 
 
 @dataclass
-class ContentRecord:
+class _ContentRecord:
     """
     A content record object that contains the details of a content contained in a title. This information must match
     the content stored at the index in the record, or else the content will not decrypt properly, as the hash of the
@@ -28,24 +28,3 @@ class ContentRecord:
     content_type: int  # Type of content, possible values of: 0x0001: Normal, 0x4001: DLC, 0x8001: Shared.
     content_size: int
     content_hash: bytes
-
-
-@dataclass
-class TitleLimit:
-    """
-    A TitleLimit object that contains the type of restriction and the limit. The limit type can be one of the following:
-    0 = None, 1 = Time Limit, 3 = None, or 4 = Launch Count. The maximum usage is then either the time in minutes the
-    title can be played or the maximum number of launches allowed for that title, based on the type of limit applied.
-
-    Attributes
-    ----------
-    limit_type : int
-        The type of play limit applied.
-    maximum_usage : int
-        The maximum value for the type of play limit applied.
-    """
-    # The type of play limit applied.
-    # 0 = None, 1 = Time Limit, 3 = None, 4 = Launch Count
-    limit_type: int
-    # The maximum value of the limit applied.
-    maximum_usage: int

tests/test_commonkeys.py

@@ -3,18 +3,18 @@
 
 import unittest
 
-from libWiiPy import commonkeys
+from libWiiPy import title
 
 
 class TestCommonKeys(unittest.TestCase):
     def test_common(self):
-        self.assertEqual(commonkeys.get_common_key(0), b'\xeb\xe4*"^\x85\x93\xe4H\xd9\xc5Es\x81\xaa\xf7')
+        self.assertEqual(title.get_common_key(0), b'\xeb\xe4*"^\x85\x93\xe4H\xd9\xc5Es\x81\xaa\xf7')
 
     def test_korean(self):
-        self.assertEqual(commonkeys.get_common_key(1), b'c\xb8+\xb4\xf4aN.\x13\xf2\xfe\xfb\xbaL\x9b~')
+        self.assertEqual(title.get_common_key(1), b'c\xb8+\xb4\xf4aN.\x13\xf2\xfe\xfb\xbaL\x9b~')
 
     def test_vwii(self):
-        self.assertEqual(commonkeys.get_common_key(2), b'0\xbf\xc7n|\x19\xaf\xbb#\x1630\xce\xd7\xc2\x8d')
+        self.assertEqual(title.get_common_key(2), b'0\xbf\xc7n|\x19\xaf\xbb#\x1630\xce\xd7\xc2\x8d')
 
 
 if __name__ == '__main__':