Skip to content

Commit

Permalink
update README
Browse files Browse the repository at this point in the history
  • Loading branch information
Deficuet authored and Deficuet committed Jan 3, 2024
1 parent a375ab5 commit e9284d7
Show file tree
Hide file tree
Showing 6 changed files with 114 additions and 61 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
main.*
124 changes: 78 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
# UnityKt
**Warning: Out of date. Refactored on Jan. 2nd 2024**

[![](https://jitpack.io/v/Deficuet/UnityKt.svg)](https://jitpack.io/#Deficuet/UnityKt)

A read-only Unity assets extractor for Kotlin based on [AssetStudio](https://github.com/Perfare/AssetStudio) and refer to [UnityPy](https://github.com/K0lb3/UnityPy).
A read-only Unity assets extractor for Kotlin based on [AssetStudio](https://github.com/Perfare/AssetStudio) and refers to [UnityPy](https://github.com/K0lb3/UnityPy).

All the C++ code used to decode texture compression comes from [AssetStudio/Texture2DDecoderNative](https://github.com/Perfare/AssetStudio/tree/master/Texture2DDecoderNative).
All the C++ code used to decode texture compression come from [AssetStudio/Texture2DDecoderNative](https://github.com/Perfare/AssetStudio/tree/master/Texture2DDecoderNative).

[7zip-LZMA SDK](https://www.7-zip.org/sdk.html) is used to decompress bundle files.
[7zip-LZMA SDK](https://www.7-zip.org/sdk.html) is included to decompress bundle files.

**If you encounter an error and want to start an new issue, remember to submit the assets files as well.**

Expand All @@ -20,30 +18,37 @@ All the C++ code used to decode texture compression comes from [AssetStudio/Text
- All reachable files under the folder, recursively.
- ByteArray, bytes of a asset bundle file.
- A `String` is required as the identifier "file name" of this `ByteArray`.
- UnityAssetManager
- The entry point of loading object. It contains all objects read from all files that are loaded through it, except `AssetBundle` objects.
- Use `UnityAssetManager.new(...)` with two parameters `assetRootFolder` and `readerConfig` to get an instance.
- `assetRootFolder` is optional and it can be a `java.io.File`, `java.nio.file.Path` or just a `String`. When present, it will be used by `PPtr` to look for object according to the `mDependencies` property in the `AssetBundle` object of the file.
- `readerConfig` is the loading configuration and is also optional.
- Loading Configurations
- When loading file(s)/folder/ByteArray, there is a lambda parameter that can be used to configure loading behaviors.
- The lambda will be applied **before** loading.
- `offsetMode` - `MANUAL` or `AUTO`
- If the `offsetMode` is `AUTO`, all the `\x00` bytes at the beginning of the file will be ignored. The reading of the file will start at the first non-zero byte.
- When loading file(s)/folder/ByteArray, an optional `ReaderConfig` can also be passed to override the configuration given to the manager, or the configuration given to the manager will be used by default.
- The configuration will be applied **before** loading.
- `offsetMode` - `MANUAL` or `AUTO`; Default: `MANUAL`
- If the `offsetMode` is `AUTO`, all the `\x00` bytes at the beginning of the file will be ignored. The reading of the file will start at the first non-zero byte. The config `manualOffset` is ignored under this mode.
- If the `offsetMode` is `MANUAL`, the number of bytes that is ignored depends on `manualOffset`.
- `manualOffset`
- `manualOffset`: `Int`; Default: `0`
- Determine the number of bytes that needs to be ignored at the beginning of the file, no matter the byte is `\x00` or not.
- This propery works under `OffsetMode.MANUAL` mode only, will be ignored under `AUTO` mode.
- e.g. If an asset bundle file has 1024 bytes of useless non-zero data at the beginning, The `offsetMode` can be set to `MANUAL` and `manualOffset` can be set to 1024 in order to skip first 1024 bytes.
- Object Loading
- The data of Object will not be read until you access any of its properties.
- However, an access to the properties related to the Object's info will not cause the initialization of the Object.
- e.g. Its `assetFile` which is a `SerializedFile`, `mPathID`, `unityVersion`, etc. Those are the properties without a `getter`. See [Object class](https://github.com/Deficuet/UnityKt/blob/main/src/main/kotlin/io/github/deficuet/unitykt/data/Object.kt).
- UnityAssetManager & ImportContext
- For each file loaded, an `ImportContext` with file name and directory will be given. An `ImportContext` contains all objects read from the file.
- When load files/folder, a list of `ImportContext` will be returned.
- `UnityAssetManager` contains all objects read from all files that are loaded through it, except `AssetBundle` objects.
- Don't forget to run the method `close()` to close and release the resources used by the manager, or run the static method `closeAll()` in `UnityAssetManager` to release resources used by **all** manager instances. It implements `Closeable` interface, so in Kotlin using the function `use()` would be a good idea.
- e.g. If an asset bundle file has 1024 bytes of useless non-zero data at the beginning, these bytes can be skipped by setting `offsetMode` to `MANUAL` and setting `manualOffset` to 1024.
- ImportContext
- For each file loaded, an `ImportContext` with file name and directory will be returned. An `ImportContext` contains all objects read from the file.
- When load files/folder, an array of `ImportContext` will be returned.
- Don't forget to call the method `close()` to close and release the resources used by the manager, or call the static method `closeAll()` in `UnityAssetManager` to release resources used by **all** manager instances. It implements `Closeable` interface, so the function `use` for `Closeable` can be used.
- Object Reading
- The data of a UnityObject will not be read until you access any of its properties.
- However, an access to the fields related to the UnityObject's metadata will not cause the initialization of the UnityObject.
- e.g. Its `assetFile` which is a `SerializedFile`, `mPathID`, `unityVersion`, etc. Those are the fields enclosed in the metadata region. See [UnityObject class](https://github.com/Deficuet/UnityKt/blob/main/src/main/kotlin/io/github/deficuet/unitykt/classes/UnityObject.kt).


- Shortcuts
- See [utils.kt](https://github.com/Deficuet/UnityKt/blob/main/src/main/kotlin/io/github/deficuet/unitykt/utils.kt) and [PPtrUtils.kt](https://github.com/Deficuet/UnityKt/blob/main/src/main/kotlin/io/github/deficuet/unitykt/PPtrUtils.kt)
- Should always use `PPtr<O>.getObj()` or `PPtr<O>.getObjectAs<>()` to get the object (and cast it).
## Installation
Used openJDK 11.0.10 and Kotlin Language 1.7.20.
- See [utils.kt](https://github.com/Deficuet/UnityKt/blob/main/src/main/kotlin/io/github/deficuet/unitykt/utils.kt) and [PPtr.kt](https://github.com/Deficuet/UnityKt/blob/main/src/main/kotlin/io/github/deficuet/unitykt/classes/PPtr.kt)
- Should always use `PPtr<O>.safeGetObj()` or `PPtr<O>.getObj()` to get the object, or use `PPtr<*>.safeGetObjAs<T: UnityObject>()` to get and cast.

## Build/Installation
Used openJDK 11.0.10 and Kotlin Language 1.8.21.

<ins>**Note:**</ins> The decoding for texture compression of `Texture2D` that needs to call native library is available on <ins>Windows 64-bit JVM</ins> **only**.
- ### Gradle
Expand Down Expand Up @@ -89,58 +94,85 @@ libraryDependencies += "com.github.Deficuet" % "UnityKt" % "{version}"
## Export
So far the objects that can export data includes:
- Mesh
- `exportString` - A string with the same content as exporting mesh to .obj file, see [AssetStudio](https://github.com/Perfare/AssetStudio).
- `exportVertices` - The data of lines starts with "v" in .obj file, grouped by Vector3.
- `exportUV` - The data of lines starts with "vt" in .obj file, grouped by Vector2.
- `exportNormals` - The data of lines starts with "vn" in .obj file, grouped by Vector3.
- `exportFaces` - The data of lines starts with "f" in .obj file, grouped by Vector3.
- `exportString` - A string with the same content as exporting mesh to .obj file using [AssetStudio](https://github.com/Perfare/AssetStudio).
- `exportVertices` - The data of lines starts with "v" in the .obj file, grouped by Vector3.
- `exportUV` - The data of lines starts with "vt" in the .obj file, grouped by Vector2. Only the data `mUV0` is exported.
- `exportNormals` - The data of lines starts with "vn" in the .obj file, grouped by Vector3.
- `exportFaces` - The data of lines starts with "f" in the .obj file, grouped by Vector3.
- Texture2D
- `decompressedImageData` - Image data after decoding. Can be used to create image directly.
- `image` - A BufferedImage created from `decompressedImageData`. **It is usually up-side-down**.
- `getDecompressedData` - Image data as `ByteArray` after decoding. Can be used to create image directly. The color channels are `BGRA`. The size of the array is `mWidth * mHeight * 4`.
- `getImage` - A BufferedImage created from the decompressed data. **It is usually up-side-down**.
- If the format of the texture is unsupported, both functions will return `null`.
- Sprite
- `getImage` - An **upright** BufferedImage cropped from a `Texture2D` image. The function will return `null` if the `Texture2D` object is not found or the format is unsupported.
- TextAsset
- `text()` - The function used to export content in this object as `String`. A `Charset` can be passed as a parameter, by default it is `Charsets.UTF_8`.
- `text(charset)` - This function is used to export content in this object as `String`. A `Charset` can be passed as a parameter, by default it is `Charsets.UTF_8`.
- Shader
- `exportString` - Export as String. **Include** the Spir-V Shader data.
- `exportString` - Export as String. **Include** the Spir-V Shader data (experimental).
- `AudioClip` and `VideoClip`
- `getRawData` - To get the raw data `ByteArray`. The export functions are not implemented yet.
- All objects
- `typeTreeJson` - A [JSONObject](https://stleary.github.io/JSON-java/org/json/JSONObject.html) contains all the properties they have, including those properties that is not implemented yet. The json could be `null`.
- May throw exception for some types of object like `Font` because of some unsolved problems.
## Example
Example for reading and saving an image from a Texture2D object.
```kotlin
import io.github.deficuet.unitykt.*
import io.github.deficuet.unitykt.data.*
import io.github.deficuet.unitykt.classes.*
import java.io.File
import javax.imageio.ImageIO

fun main() {
UnityAssetManager().use {
UnityAssetManager.new().use {
val context: ImportContext = it.loadFile("C:/path/to/AssetBundle.aab")

//If there is no Texture2D object, an IndexOutOfBoundsException will be thrown.
//You can consider the function firstOfOrNull<>()
//Use the function firstOfOrNull<>() if the existence of the object is not guaranteed.
//The data of this Texture2D object has not been read yet.
val tex: Texture2D = context.objects.firstObjectOf<Texture2D>()

//The object data will initialize as long as you access its properties.
ImageIO.write(tex.image, "png", File("C:/whatever/you/want/tex.png"))
tex.getImage()?.let {
ImageIO.write(it, "png", File("C:/whatever/you/want/name.png"))
}

//The manager will automatically close.
//The manager will automatically close under the scope of 'use' function.
}

// Instantiate another manager
UnityAssetManager().use {
UnityAssetManager.new(
"C:/path/to/asset/system/root/folder",
ReaderConfig(OffsetMode.AUTO)
).use {
//Loading configurations
//It will not influence the configurations of previous manager.
//The files and objects loaded by this manager will not show in previous manager as well.
it.loadFolder("C:/foo/bar") {
offsetMode = OffsetMode.MANUAL
manualIgnoredOffset = 217
}
//The manager holds all the objects loaded through it, except those AssetBundle objects, their PathID is usually 1.
//The new config with mode `MANUAL` and offset `217` will be used instead of the config with mode `AUTO` given to the manager.
//The files and objects loaded by this manager will not show in previous manager.
val context = it.loadFolder("C:/foo/bar", ReaderConfig(OffsetMode.MANUAL, 217))
//The manager holds all the objects loaded through it, except those AssetBundle objects, their PathID is usually (always?) 1.
println(it.objects.firstObjectOf<Shader>().exportString)

context.objectMap.getAs<Material>(4054671897103428721) //Map<Long, UnityObject>.getAs<T: UnityObject>(pathId: Long): T
// or .safeGetAs<T: UnityObject>(pathId: Long): T?
.mSavedProperties.mTexEnvs.values.first()[0].mTexture //PPtr<Texture>
.safeGetObj() // if PPtr can not find the object under the same assetFile (SerializedFile) nor others loaded assetFiles
// Then it will look for the file under the directory "C:/path/to/asset/system/root/folder" which was
// given to the manager. The file name comes from the mDependecies property of the AssetBundle object
// in the same file as this Material object being in. The target file will be loaded using the same
// reader config passed to the manager. Finally, PPtr will try to find the object in the new loaded file.
?.safeCast<Texture2D>()?.getImage()?.let { image ->
ImageIO.write(image, "png", File("C:/whatever/you/want/name.png"))
}
}
}
```
## Changelog
- ### 2024.01.02
- Refactor
- Renamed `Object` to `UnityObject`.
- Fixed some errors and incorrect behaviors in `Matrix4x4` and `PPtr`.
- Removed implementation of `Font` object because of some unsolved errors.
- Extended some fields for `GameObject` and `VideoClip`.
- ...many other things.
- ### 2022.12.25
- Create Canvas object, tested on version 2018.4.34f1, not guaranteed to be stable on other versions.
- ### 2022.12.23
Expand Down
25 changes: 15 additions & 10 deletions src/main/kotlin/io/github/deficuet/unitykt/classes/PPtr.kt
Original file line number Diff line number Diff line change
@@ -1,26 +1,31 @@
package io.github.deficuet.unitykt.classes

import io.github.deficuet.unitykt.cast
import io.github.deficuet.unitykt.firstObjectOf
import io.github.deficuet.unitykt.firstOfOrNull
import io.github.deficuet.unitykt.*
import io.github.deficuet.unitykt.internal.impl.PPtrImpl
import io.github.deficuet.unitykt.internal.impl.getObj
import io.github.deficuet.unitykt.internal.impl.safeGetObj
import io.github.deficuet.unitykt.safeCast
import io.github.deficuet.unitykt.internal.impl.getObjInternal
import io.github.deficuet.unitykt.internal.impl.safeGetObjInternal

interface PPtr<out T: UnityObject> {
val mFileID: Int
val mPathID: Long
val isNull: Boolean
}

inline fun <reified T: UnityObject> PPtr<T>.safeGetObj() = (this as PPtrImpl<T>).safeGetObj()
inline fun <reified T: UnityObject> PPtr<T>.safeGetObj(): T? {
return (this as PPtrImpl<T>).safeGetObjInternal()
}

inline fun <reified T: UnityObject> PPtr<T>.getObj(): T = (this as PPtrImpl<T>).getObj()
inline fun <reified T: UnityObject> PPtr<T>.getObj(): T {
return (this as PPtrImpl<T>).getObjInternal()
}

inline fun <reified T: UnityObject> PPtr<*>.safeGetAs(): T? = (this as PPtrImpl<UnityObject>).safeGetObj() as? T
inline fun <reified T: UnityObject> PPtr<*>.safeGetAs(): T? {
return (this as PPtrImpl<UnityObject>).safeGetObjInternal() as? T
}

inline fun <reified T: UnityObject> PPtr<*>.getAs(): T = (this as PPtrImpl<UnityObject>).getObj() as T
inline fun <reified T: UnityObject> PPtr<*>.getAs(): T {
return (this as PPtrImpl<UnityObject>).getObjInternal() as T
}

inline fun <reified O: UnityObject> Iterable<PPtr<*>>.firstObjectOf() =
mapNotNull { it.safeGetObj() }.firstObjectOf<O>()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@ interface UnityObject {
val unityVersion: IntArray
val platform: BuildTarget
val serializedType: SerializedType?

/**
* Mostly be used to create [PPtr] from the type tree of a [MonoBehaviour]
*/
fun <T: UnityObject> createPPtr(fileId: Int, pathId: Long): PPtr<T>
//endregion

val bytes: ByteArray
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ internal class PPtrImpl<out T: UnityObject>: PPtr<T> {
get() = mPathID == 0L || mFileID < 0
private val assetFile: SerializedFile

constructor(reader: ObjectReader) {
internal constructor(reader: ObjectReader) {
mFileID = reader.readInt32()
mPathID = with(reader) {
if (formatVersion < FormatVersion.Unknown_14) reader.readInt32().toLong()
Expand All @@ -25,7 +25,7 @@ internal class PPtrImpl<out T: UnityObject>: PPtr<T> {
assetFile = reader.assetFile
}

constructor(fileId: Int, pathId: Long, assetFile: SerializedFile) {
internal constructor(fileId: Int, pathId: Long, assetFile: SerializedFile) {
mFileID = fileId
mPathID = pathId
this.assetFile = assetFile
Expand Down Expand Up @@ -53,7 +53,10 @@ internal class PPtrImpl<out T: UnityObject>: PPtr<T> {
if (fileIndex >= bundle.mDependencies.size) return null
val dependencyName = bundle.mDependencies[fileIndex]
try {
manager.loadFile(manager.assetRootFolder.resolve(dependencyName), assetFile.root.readerConfig)
manager.loadFile(
manager.assetRootFolder.resolve(dependencyName),
assetFile.root.manager.defaultReaderConfig
)
} catch (e: Exception) {
println("An error occurred during loading dependency file ${dependencyName}: ${e.message}")
return null
Expand Down Expand Up @@ -87,7 +90,7 @@ internal class PPtrImpl<out T: UnityObject>: PPtr<T> {
}

@PublishedApi
internal inline fun <reified T: UnityObject> PPtrImpl<T>.safeGetObj(): T? {
internal inline fun <reified T: UnityObject> PPtrImpl<T>.safeGetObjInternal(): T? {
val cache = getCache()
if (cache != null) return cache
return getAssetFile()?.let {
Expand All @@ -102,4 +105,6 @@ internal inline fun <reified T: UnityObject> PPtrImpl<T>.safeGetObj(): T? {
}

@PublishedApi
internal inline fun <reified T: UnityObject> PPtrImpl<T>.getObj(): T = safeGetObj()!!
internal inline fun <reified T: UnityObject> PPtrImpl<T>.getObjInternal(): T {
return safeGetObjInternal()!!
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package io.github.deficuet.unitykt.internal.impl

import io.github.deficuet.unitykt.classes.PPtr
import io.github.deficuet.unitykt.classes.UnityObject
import io.github.deficuet.unitykt.enums.BuildTarget
import io.github.deficuet.unitykt.internal.file.ObjectInfo
Expand Down Expand Up @@ -33,6 +34,10 @@ internal open class UnityObjectImpl(
return toTypeTreeJson()?.toString(indent) ?: "null"
}

final override fun <T: UnityObject> createPPtr(fileId: Int, pathId: Long): PPtr<T> {
return PPtrImpl(fileId, pathId, assetFile)
}

protected open fun read() {
println("Object($type) path id $mPathID initialized")
reader.position = 0
Expand Down

0 comments on commit e9284d7

Please sign in to comment.