diff --git a/README.md b/README.md index 325125b..c1f98ee 100644 --- a/README.md +++ b/README.md @@ -2,77 +2,17 @@ This repository regenerates debug information (like PDB) from LeviLamina's public data. -## Background +## Usage -- After 1.21.3.01, Mojang removed debug information from BDS. -- Mojang no longer provides any debugging information (both server and client) to the community. -- Mojang has an agreement with LiteLDev to provide them with debug data. -- LiteLDev generates header files and obfuscated symbol to RVA lookup tables from debug data and provides them to the community. +It's all [here](https://bedrockdebug.info/). (for PDB generation, see the **Tutorial**) -### Problems caused by Mojang's collaboration with LiteLDev - -- Mojang's collaboration with LiteLDev is opaque and we have no idea what they do. -- LiteLDev has completed its monopoly, and there will no longer be a second mod loader in the community. -- Due to the obfuscated format, the community can no longer reverse engineer BDS. - -### Header files, obfuscation format and security - -- LeviLamina's design necessitates that they publish header files. -- The header file contains all the declaration information so that symbols can be generated. -- The RVA of the corresponding symbol can be extracted from the obfuscated format at runtime. -- The obfuscated format is actually a carrier of the complete "symbol table", which used to be PDB/DWARF. - -## Tool for restoring original DebugInfo from obfuscated format - -> [!NOTE] -> LiteLDev has not yet released bedrock_runtime_data/magicblob for the Linux server. - -They are dethunk, dumpsym, askrva and makepdb. Each tool is in a directory with the same name as it, and also has a README to help you use it. -In short, the PDB is generated by the following steps: - -- Preprocess the header files published by LiteLDev by dethunk. - -``` -python main.py {HEADER_PROJECT_DIR}/src -``` - -- Compile the header file and load the dumpsym plugin in the compilation parameters. - -``` -xmake f -c -p windows -a x64 -m release --sdk=/opt/msvc --cxflags="-fplugin=/path/to/libdumpsym.so -fplugin-arg-dumpsym-record-decl-name" --toolchain=clang -xmake -v -``` - -- Find the generated symbols file. - -``` -{HEADER_PROJECT_DIR}/build/.objs/bdsheader/windows/x64/release/test/__cpp_main.cpp.cpp.symbols -``` - -- Generate symbol table using askrva. - -``` -./askrva __cpp_main.cpp.cpp.symbols --output succeed.json --output-failed failed.txt --output-format=makepdb -``` - -- Generate PDB using makepdb. - -``` -./makepdb --program bedrock_server.exe --symbol-data succeed.json --output bedrock_server.pdb -``` - -## TODO - -- [ ] Tap into more available symbols. -- [ ] Fully open source HeaderGen. -- [ ] Bootstrap. - -## Be with us +## Community Our vision is to build an open and inclusive Minecraft: Bedrock Edition ecosystem. -- [https://t.me/bdsplugins](https://t.me/s/bdsplugins) +- [Discord](https://discord.gg/fPKVZScPT7) +- [Telegram (Channel)](https://t.me/s/bdsplugins) -## LICENSE +## License -All tools are open source under the MIT license. +This project is open source under the MIT license. diff --git a/docs/CODE_OF_CONDUCT.md b/docs/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..a8ffcd7 --- /dev/null +++ b/docs/CODE_OF_CONDUCT.md @@ -0,0 +1,132 @@ +--- +icon: octicons/code-of-conduct-16 +--- + +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our +community a harassment-free experience for everyone, regardless of age, body +size, visible or invisible disability, ethnicity, sex characteristics, gender +identity and expression, level of experience, education, socio-economic status, +nationality, personal appearance, race, religion, or sexual identity +and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, +diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our +community include: + +- Demonstrating empathy and kindness toward other people +- Being respectful of differing opinions, viewpoints, and experiences +- Giving and gracefully accepting constructive feedback +- Accepting responsibility and apologizing to those affected by our mistakes, + and learning from the experience +- Focusing on what is best not just for us as individuals, but for the + overall community + +Examples of unacceptable behavior include: + +- The use of sexualized language or imagery, and sexual attention or + advances of any kind +- Trolling, insulting or derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information, such as a physical or email + address, without their explicit permission +- Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of +acceptable behavior and will take appropriate and fair corrective action in +response to any behavior that they deem inappropriate, threatening, offensive, +or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject +comments, commits, code, wiki edits, issues, and other contributions that are +not aligned to this Code of Conduct, and will communicate reasons for moderation +decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when +an individual is officially representing the community in public spaces. +Examples of representing our community include using an official e-mail address, +posting via an official social media account, or acting as an appointed +representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported to the community leaders responsible for enforcement at +redbeana44945@gmail.com. +All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the +reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining +the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact**: Use of inappropriate language or other behavior deemed +unprofessional or unwelcome in the community. + +**Consequence**: A private, written warning from community leaders, providing +clarity around the nature of the violation and an explanation of why the +behavior was inappropriate. A public apology may be requested. + +### 2. Warning + +**Community Impact**: A violation through a single incident or series +of actions. + +**Consequence**: A warning with consequences for continued behavior. No +interaction with the people involved, including unsolicited interaction with +those enforcing the Code of Conduct, for a specified period of time. This +includes avoiding interactions in community spaces as well as external channels +like social media. Violating these terms may lead to a temporary or +permanent ban. + +### 3. Temporary Ban + +**Community Impact**: A serious violation of community standards, including +sustained inappropriate behavior. + +**Consequence**: A temporary ban from any sort of interaction or public +communication with the community for a specified period of time. No public or +private interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, is allowed during this period. +Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact**: Demonstrating a pattern of violation of community +standards, including sustained inappropriate behavior, harassment of an +individual, or aggression toward or disparagement of classes of individuals. + +**Consequence**: A permanent ban from any sort of public interaction within +the community. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], +version 2.0, available at +https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. + +Community Impact Guidelines were inspired by [Mozilla's code of conduct +enforcement ladder](https://github.com/mozilla/diversity). + +[homepage]: https://www.contributor-covenant.org + +For answers to common questions about this code of conduct, see the FAQ at +https://www.contributor-covenant.org/faq. Translations are available at +https://www.contributor-covenant.org/translations. diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md new file mode 100644 index 0000000..6aa06ad --- /dev/null +++ b/docs/CONTRIBUTING.md @@ -0,0 +1,24 @@ +--- +icon: octicons/repo-push-16 +--- + +# Contributing Guide + +We welcome contributions of any kind, including but not limited to + +- Open a new issue and tell us what you think. +- Contribute directly to the _DebugInfo_ toolchain. +- Correct documentation errors, including translation errors. +- Write a new tutorial. +- Any others may not be listed. + +## Coding style + +- All code must be formatted before submission (follow `ruff.toml` and `.clang-format`) +- Except for the class name which uses CamelCase, all others use snake_style. +- Private member names begin with `m_` (C++ code only). + +## Interface Stability + +- Great, you don't need to make ABI stability guarantees! +- Tool usage should not be significantly modified (as appropriate) diff --git a/docs/ask-rva.md b/docs/ask-rva.md deleted file mode 100644 index f23be08..0000000 --- a/docs/ask-rva.md +++ /dev/null @@ -1,41 +0,0 @@ -# AskRVA - -This module uses LiteLDev's official PreLoader to load bedrock_runtime_data, calls `pl::symbol_provider::pl_resolve_symbol` to obtain the RVA corresponding to the symbol, and then builds the symbol table. - -> [!WARNING] -> AskRVA must load PreLoader, and the security of PreLoader is unknown (see below for the reason). It is recommended to execute it in a restricted environment. The author does not bear any consequences. - -### PreLoader - -[PreLoader](https://github.com/LiteLDev/PreLoader) is a key component of LeviLamina, which is responsible for parsing bedrock_runtime_data, hooking, calling, loading LeviLamina ontology, etc. -**PreLoader is currently proprietary software.** After v1.10.0, due to the adoption of bedrock_runtime_data (also -known as magicblob), PreLoader no longer handles PDB. The source code in the current repository is old, as of v1.9.2. - -### Build - -- Since there is no Linux/MacOS build for PreLoader, you have to build AskRVA under Windows. -- Just execute `xmake` to complete the build. - -### Usage - -- --output-format can be `auto` / `txt` / `makepdb` - -``` -Usage: askrva [--help] [--version] --output VAR [--output-failed VAR] [--output-format VAR] path - -Positional arguments: - path Path to the symbol list file. [required] - -Optional arguments: - -h, --help shows help message and exits - -v, --version prints version information and exits - -o, --output Path to output. [required] - -of, --output-failed Path to output failed entries. - --output-format Specify output format. [nargs=0..1] [default: "auto"] -``` - -- Example: - -``` -./askrva.exe __cpp_main.cpp.cpp.symbols --output output.json --output-failed failed.txt -``` diff --git a/docs/assets/landscape.png b/docs/assets/landscape.png new file mode 100644 index 0000000..50c8688 Binary files /dev/null and b/docs/assets/landscape.png differ diff --git a/docs/blob-extractor.md b/docs/blob-extractor.md deleted file mode 100644 index 6daceb5..0000000 --- a/docs/blob-extractor.md +++ /dev/null @@ -1 +0,0 @@ -# BlobExtractor diff --git a/docs/bootstrap.md b/docs/bootstrap.md deleted file mode 100644 index b231c53..0000000 --- a/docs/bootstrap.md +++ /dev/null @@ -1,2 +0,0 @@ -# Bootstrap -WIP. diff --git a/docs/community/index.md b/docs/community/index.md new file mode 100644 index 0000000..1d57d78 --- /dev/null +++ b/docs/community/index.md @@ -0,0 +1,21 @@ +# Community + +
+ +- :fontawesome-brands-discord:{ .lg .middle } __Discord__ + + --- + + The best place to communicate together. + + [:octicons-arrow-right-24: Join the group](https://discord.gg/fPKVZScPT7) + +- :fontawesome-brands-telegram:{ .lg .middle } __Telegram__ + + --- + + If you just want to receive notifications. + + [:octicons-arrow-right-24: Subscribe to channel](https://t.me/bdsplugins) + +
diff --git a/docs/de-thunk.md b/docs/de-thunk.md deleted file mode 100644 index c880cbc..0000000 --- a/docs/de-thunk.md +++ /dev/null @@ -1,37 +0,0 @@ -# DeThunk -This is the header preprocessor. - -### About FakeSymbol -
- Click me - - > FakeSymbol is an invention of LiteLDev HeaderGen, which was originally used to access member functions/variables with private/protected permissions. Since the access specifier is part of the function signature, it is not possible to simply change "private" to "public" in the header file. -Now, it is also used to help developers hook special member functions in the class. -As we all know, in C++, it is not possible to directly get the address of special member functions such as constructors/destructors/virtual functions. If you want to hook them, you must use string symbols, but string symbols are dirty. Another solution is to generate thunks and intercept symbol resolution, which is exactly what FakeSymbol does. - -
- -*TL;DR* FakeSymbol has many benefits, but it actually causes incorrect symbols to be generated, so it is necessary to restore them. - -### Where to get the headers? -``` -git clone https://github.com/LiteLDev/LeviLamina.git -git checkout -b header -``` - -### Thunks - - `virtual function thunk` - *Removed.* - - `constructor thunk` - *Removed.* - - `vftables` - *Removed.* - - **TODO** dumpsym is currently unable to emit a symbol of vtable pointer. - - `destructor thunk` - *Removed.* - - **TODO** dumpsym is currently unable to emit a symbol of destructor. - - `static variables thunk` - *Restored.* - - **TODO** Symbols are now generated correctly, but some symbols cannot be queried from bedrock_runtime_data. - -### Usage -> [!NOTE] -> The modification will be done directly on the original file. -``` -python src/main.py path/to/header/ -``` \ No newline at end of file diff --git a/docs/dump-sym.md b/docs/dump-sym.md deleted file mode 100644 index 5134279..0000000 --- a/docs/dump-sym.md +++ /dev/null @@ -1,38 +0,0 @@ -# DumpSYM - -Sometimes, we need to extract symbols from declarations. So this compiler plugin was born. - -### Build - -> As far as I know, there are some problems with the clang plugin executing under Windows, so I recommend that all operations be performed under Linux. - -- Building llvm will consume a lot of time and resources, it is recommended to pre-install llvm from your system package manager, xmake can detect system packages. -- Run `xmake` to build. - -### Usage - -Simply pass `-fplugin=...` to clang and the plugin will run automatically. - -#### Optional Arguments - -- `record-decl-name` - Add the name of the Decl in the output, reference: [FunctionDecl](https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html), [VarDecl](https://clang.llvm.org/doxygen/classclang_1_1VarDecl.html) - -> [!NOTE] -> Because LLVM is used, both ItaniumABI and MicrosoftABI are supported. - -- Example: - -``` -$ clang++ -fplugin=/path/to/plugin/libdumpsym.so -fplugin-arg-dumpsym-record-decl-name test.cpp -``` - -- The result will be generated in the `.symbols` file - -``` -$ cat test.cpp.symbols -Function, main -CXXDestructor, ??_DThreadPool@OS@@QEAAXXZ -CXXConstructor, ??0SpinLockImpl@@QEAA@AEBV0@@Z -CXXMethod, ??4SpinLockImpl@@QEAAAEAV0@AEBV0@@Z -Var, ?Low@OSThreadPriority@Threading@Bedrock@@2V123@B -``` diff --git a/docs/extract-sym.md b/docs/extract-sym.md deleted file mode 100644 index c97d4a5..0000000 --- a/docs/extract-sym.md +++ /dev/null @@ -1 +0,0 @@ -# ExtractSYM diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 0000000..048768c --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,9 @@ +--- +icon: material/chat-question +--- + +# Frequently Asked Questions (FAQ) + +!!! warning "" + + The content of this page is awaiting update. diff --git a/docs/index.md b/docs/index.md index 740abac..cc8e19a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1 +1,7 @@ -# DebugInfo +# Home + +> **Analyzing bedrock is not a privilege for the few.** + +This project is specifically designed to help you reverse engineer Minecraft Bedrock Edition (especially dedicated servers) since version 1.21.3.01. + +![](assets/landscape.png) diff --git a/docs/license.md b/docs/license.md new file mode 100644 index 0000000..24db2b5 --- /dev/null +++ b/docs/license.md @@ -0,0 +1,25 @@ +--- +icon: material/license +--- + +# MIT License + +Copyright (c) 2025 Open LiteLDev + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/docs/makepdb.md b/docs/makepdb.md deleted file mode 100644 index 9e9f939..0000000 --- a/docs/makepdb.md +++ /dev/null @@ -1,23 +0,0 @@ -# MakePDB -Generates a matching PDB file from the symbol data and the original COFF file. - -### Usage - -> [!NOTE] -> Please make sure that the symbol data (generated by askrva) matches the original COFF (bedrock_server.exe). - -``` -Usage: makepdb [--help] [--version] --program VAR --symbol-data VAR --output VAR - -Optional arguments: - -h, --help shows help message and exits - -v, --version prints version information and exits - --program Path to bedrock_server.exe [required] - --symbol-data Path to symbol data. [required] - -o, --output Path to output PDB. [required] -``` - - - Example: -``` -./makepdb --program test.exe --symbol-data dump.json --output test.pdb -``` diff --git a/docs/releases.md b/docs/releases.md new file mode 100644 index 0000000..7b8cf98 --- /dev/null +++ b/docs/releases.md @@ -0,0 +1,9 @@ +--- +icon: material/package-variant-closed-check +--- + +# Releases + +!!! warning "" + + The content of this page is awaiting update. diff --git a/docs/research/FakeSymbol.md b/docs/research/FakeSymbol.md new file mode 100644 index 0000000..4740b1f --- /dev/null +++ b/docs/research/FakeSymbol.md @@ -0,0 +1,5 @@ +# FakeSymbol + +!!! warning "" + + The content of this page is awaiting update. diff --git a/docs/research/HeaderGen.md b/docs/research/HeaderGen.md new file mode 100644 index 0000000..5a0ee9a --- /dev/null +++ b/docs/research/HeaderGen.md @@ -0,0 +1,5 @@ +# HeaderGen + +!!! warning "" + + The content of this page is awaiting update. diff --git a/docs/research/PreLoader.md b/docs/research/PreLoader.md new file mode 100644 index 0000000..cd9b173 --- /dev/null +++ b/docs/research/PreLoader.md @@ -0,0 +1,5 @@ +# PreLoader + +!!! warning "" + + The content of this page is awaiting update. diff --git a/docs/research/index.md b/docs/research/index.md new file mode 100644 index 0000000..73c9483 --- /dev/null +++ b/docs/research/index.md @@ -0,0 +1,7 @@ +# Research + +!!! warning "" + + The content of this page is awaiting update. + +LiteLDev has made great contributions to Bedrock Edition, but their technology is not transparent. Here, I will reveal it for you. diff --git a/docs/toolchain/AskRVA.md b/docs/toolchain/AskRVA.md new file mode 100644 index 0000000..2c801b5 --- /dev/null +++ b/docs/toolchain/AskRVA.md @@ -0,0 +1,42 @@ +# AskRVA + +This tool is used to parse bedrock_runtime_data (MagicBlob), accepts the results of DumpSYM/ExtractSYM as input and outputs a symbol table in JSON format. + +## MagicBlob parsing backend + +The libdi provides two parser backends: `builtin` and `native`. By default, the built-in is used. + +- `builtin` is an open source MagicBlob format implementation that supports cross-platform. +- `native` uses LiteLDev's [PreLoader](https://github.com/LiteLDev/Preloader), which is closed source and only supports Windows platform. + +The choice of which backend to use is controlled by build options, see the Build section. + +!!! warning + + If you do need to use the native backend, make sure you do it in a restricted environment. + +## Usage + +``` +Usage: askrva [--help] [--version] [--magic-blob VAR] --output VAR [--output-failed VAR] path... + +Positional arguments: + path Path to the symbol list file. [nargs: 1 or more] [required] + +Optional arguments: + -h, --help shows help message and exits + -v, --version prints version information and exits + --magic-blob Path to magic blob (for builtin-symbol-resolver only). [nargs=0..1] [default: "bedrock_runtime_data"] + -o, --output Path to output. [required] + -of, --output-failed Path to output failed entries. +``` + +!!! tip + + If the native symbol resolver is used, MagicBlob must be the "bedrock_runtime_data" file in the current directory. Custom read paths are not supported. + +???+ example + + ``` + ./askrva symlist.txt --output succeed.json --output-failed failed.txt + ``` diff --git a/docs/toolchain/BlobExtractor.md b/docs/toolchain/BlobExtractor.md new file mode 100644 index 0000000..e31bfe4 --- /dev/null +++ b/docs/toolchain/BlobExtractor.md @@ -0,0 +1,23 @@ +# BlobExtractor + +This tool is used to extract all data from MagicBlob and is generally used for debugging purposes. + +## Usage + +``` +Usage: blob-extractor [--help] [--version] --output VAR magicblob + +Positional arguments: + magicblob Path to magic blob. [required] + +Optional arguments: + -h, --help shows help message and exits + -v, --version prints version information and exits + -o, --output Path to output symlist. [required] +``` + +???+ example + + ``` + ./blob-extractor bedrock_runtime_data --output dump.json + ``` diff --git a/docs/toolchain/DeThunk.md b/docs/toolchain/DeThunk.md new file mode 100644 index 0000000..c0345b6 --- /dev/null +++ b/docs/toolchain/DeThunk.md @@ -0,0 +1,60 @@ +# DeThunk + +This is the header file preprocessor. The header files are generated by HeaderGen (LiteLDev's non-public software). Some of the generated things may cause incorrect symbols or compilation failures. This tool is used to erase them. + +## Exclusion List + +!!! failure "" + + It is generally considered that excluding something is a desperate measure, so there are plans to [completely rewrite DeThunk](https://github.com/liteldev-free/DebugInfo/issues/1). + +!!! tip + + An exclusion list is maintained based on the branch name of the header files and can be obtained from [here](https://github.com/liteldev-free/DebugInfo/tree/main/scripts/dethunk/exclusion_list). + +The exclusion list is mainly for "types", and the reasons for exclusion are varied. In short, the exclusion list serves "passing compilation". + +## Presets + +We provide preset rules for general use, and generally prefer to use presets instead of detailed options. + +| Preset | Equal to... | +| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--preset-extract-names` | `--remove-constructor-thunk`
`--remove-destructor-thunk`
`--remove-virtual-function-thunk`
`--remove-virtual-table-pointer-thunk`
`--restore-static-variable` | +| `--preset-extract-types` | `--preset-extract-names`
`--restore-member-variable` | + +## Usage + +``` +usage: dethunk [-h] --exclusion-list EXCLUSION_LIST [--remove-constructor-thunk] [--remove-destructor-thunk] + [--remove-virtual-table-pointer-thunk] [--remove-virtual-function-thunk] [--restore-static-variable] + [--restore-member-variable] [--all] [--preset-extract-names] [--preset-extract-types] + path + +positional arguments: + path Path to header project. + +options: + -h, --help show this help message and exit + --exclusion-list EXCLUSION_LIST + Configure ignore rules. + --remove-constructor-thunk + --remove-destructor-thunk + --remove-virtual-table-pointer-thunk + --remove-virtual-function-thunk + --restore-static-variable + --restore-member-variable + --all Apply all remove/restore options. + --preset-extract-names + --preset-extract-types +``` + +!!! warning + + The modification will be done directly on the original file. + +???+ example + + ``` + python main.py --all --exclusion-list exclusion_list/r21_u6.toml path/to/header + ``` diff --git a/docs/toolchain/DumpSYM.md b/docs/toolchain/DumpSYM.md new file mode 100644 index 0000000..8413875 --- /dev/null +++ b/docs/toolchain/DumpSYM.md @@ -0,0 +1,42 @@ +# DumpSYM + +This is a clang compiler plugin that outputs all declared symbols(and their types). + +## Type of declaration + +This depends on the two main base classes in LLVM: [FunctionDecl](https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html), [VarDecl](https://clang.llvm.org/doxygen/classclang_1_1VarDecl.html). The general type name is the class name minus the "Decl" suffix. + +See also `DeclType` for internal implementation details. (1) +{ .annotate } + +1. https://github.com/liteldev-free/DebugInfo/blob/91bf6546da500b74186d26671d020c1b0a30c563/src/data_format/type/decl_type.h + +## Compiler compatibility + +- Only clang is supported. (GCC/MSVC are not supported) +- Only tested on Linux, not sure if it works on Windows. + +## Usage + +!!! note + + Both ItaniumABI and MSABI are supported, depending on the target passed to the compiler. + +Simply pass `-fplugin=...` to clang and the plugin will run automatically. + +???+ example + + ``` + clang++ -fplugin=/path/to/plugin/libdumpsym.so test.cpp + ``` + + The result will be generated in the `.symbols` file + + ``` + $ cat test.cpp.symbols + Function, main + CXXDestructor, ??_DThreadPool@OS@@QEAAXXZ + CXXConstructor, ??0SpinLockImpl@@QEAA@AEBV0@@Z + CXXMethod, ??4SpinLockImpl@@QEAAAEAV0@AEBV0@@Z + Var, ?Low@OSThreadPriority@Threading@Bedrock@@2V123@B + ``` diff --git a/docs/toolchain/ExtractSYM.md b/docs/toolchain/ExtractSYM.md new file mode 100644 index 0000000..f10c144 --- /dev/null +++ b/docs/toolchain/ExtractSYM.md @@ -0,0 +1,23 @@ +# ExtractSYM + +This tool is used to extract symbols from PDB. + +## Usage + +``` +Usage: extractsym [--help] [--version] --output VAR pdb + +Positional arguments: + pdb Path to program database. [required] + +Optional arguments: + -h, --help shows help message and exits + -v, --version prints version information and exits + -o, --output Path to output symlist. [required] +``` + +???+ example + + ``` + ./extractsym bedrock_server.pdb --output symbols.txt + ``` diff --git a/docs/toolchain/MakePDB.md b/docs/toolchain/MakePDB.md new file mode 100644 index 0000000..aee3170 --- /dev/null +++ b/docs/toolchain/MakePDB.md @@ -0,0 +1,31 @@ +# MakePDB + +Integrate data and generate a common format. + +## Where to get TypeInfo + +As we all know, the header files generated by LiteLDev contain type information. But it is not easy to use them directly (we are trying to do so). The current approach is to let the compiler generate PDBs containing type information, and then `makepdb` merges them from them. + +## Usage + +!!! warning + + Please make sure that the symbol data (generated by askrva) matches the original COFF (bedrock_server.exe). + +``` +Usage: makepdb [--help] [--version] --program VAR --symbol VAR [--typeinfo VAR] --output VAR + +Optional arguments: + -h, --help shows help message and exits + -v, --version prints version information and exits + --program Path to bedrock_server.exe [required] + --symbol Path to symbol data. [required] + --typeinfo Path to compiler PDB which contains TPI & IPI (will merged into result PDB). + -o, --output Path to output PDB. [required] +``` + +???+ example + + ``` + ./makepdb --program test.exe --symbol dump.json --output test.pdb + ``` diff --git a/docs/toolchain/index.md b/docs/toolchain/index.md new file mode 100644 index 0000000..b14f604 --- /dev/null +++ b/docs/toolchain/index.md @@ -0,0 +1,47 @@ +# Getting started + +All the tools you need are here! + +## Introduction + +- `DeThunk` - Header file preprocessing. +- `DumpSYM` - Extract symbols from header files. +- `ExtractSYM` - Extracting symbols from elsewhere. +- `AskRVA` - Batch query the RVA of symbols. +- `MakePDB` - Generate debugging information in a common format. +- `BlobExtractor` - Explore MagicBlob. + +## Build + +!!! tip + + You can get prebuilt versions from [Releases](https://github.com/liteldev-free/DebugInfo/releases) or [GitHub Actions](https://github.com/liteldev-free/DebugInfo/actions). + +Install build dependencies + +``` +pacman -S llvm clang git xmake +``` + +Clone the repository + +``` +git clone https://github.com/liteldev-free/DebugInfo.git +``` + +Configure and build + +``` +xmake +``` + +??? danger "Advanced: Use the native symbol resolver" + + Switching to the native symbol resolver is not recommended unless you have a enough reason to do so.
+ **This will rely on PreLoader, which is closed source software of unknown security and is only available on Windows.** + + ``` + xmake f --symbol-resolver=native + ``` + +Will automatically install build dependencies and build all targets. diff --git a/docs/tutorial/generate-a-pdb-from-scratch.md b/docs/tutorial/generate-a-pdb-from-scratch.md new file mode 100644 index 0000000..59ae493 --- /dev/null +++ b/docs/tutorial/generate-a-pdb-from-scratch.md @@ -0,0 +1,184 @@ +# Generate your first PDB + +!!! tip + + All operations in this article are performed under Linux.
+ +:octicons-git-commit-24: DebugInfo: [ccac962](https://github.com/liteldev-free/DebugInfo/commit/ccac9623d90cf22862259bbde4af7c2df05da19f) +:octicons-git-commit-24: Header: [642a16f](https://github.com/liteldev-free/Header/commit/642a16f8a6db364b1ff3d0a8e917c77de42d588b) +:material-package-variant-closed-check: Bedrock Dedicated Server: [1.21.60.10](https://www.minecraft.net/bedrockdedicatedserver/bin-linux/bedrock-server-1.21.60.10.zip) + +Today we will generate a complete PDB, including symbol tables and type information. +Before we start, I assume that you have followed the build instructions, or downloaded a pre-built, all in all you should have a complete _DebugInfo_ toolchain. + +## Install build dependencies + +### Setup msvc-wine + +Since everything is done under Linux, and the header depends on MSSTL, you must install [msvc-wine](https://github.com/mstorsjo/msvc-wine) (not just clang or clang-cl). If you are working under Windows, you should only need to install Visual Studio. + +``` +yay -S msvc-wine +``` + +### Setup clang & xmake + +Because the header file project uses the xmake build system. + +``` +pacman -S clang xmake +``` + +## Get the header files + +We use `liteldev-free/Header` instead of the header branch of `liteldev/LeviLamina` because we made some patches on the former. +`liteldev-free/Header` is synced from upstream for each version from time to time. + +``` +git clone https://github.com/liteldev-free/Header.git +``` + +Switch to the desired branch. The `_p` suffix means "patched". + +``` +git checkout -b mc/r21_u6_p +``` + +## Prepare other things + +
+ +- Current version of `bedrock_server.exe` (1) +- Matching version of `bedrock_runtime_data` (2) +- The last version with debug info `bedrock_server.pdb` (3) + +
+ +1. [Download from the official page](https://www.minecraft.net/en-us/download/server/bedrock) +2. [Published by LiteLDev here](https://github.com/LiteLDev/bedrock-runtime-data/releases) +3. [Download 1.21.3.01 (Windows)](https://www.minecraft.net/bedrockdedicatedserver/bin-linux/bedrock-server-1.21.3.01.zip) + +## Extracting symbols from header + +Enter the header file directory and configure + +``` +xmake f -m debug -p windows -a x64 --toolchain=clang --cxflags=-fplugin=/path/to/libdumpsym.so --sdk=/opt/msvc +``` + +Then, use DeThunk for preprocessing + +!!! tips + + The path passed in should end with `mc` instead of `src` because it is not expected that the header files of `ll` should be traversed. + +``` +cd /path/to/scripts/dethunk +python main.py "/path/to/header/mc" --preset-extract-names --exclusion-list exclusion_list/r21_u6.toml +``` + +If there are no errors, start the build. + +``` +xmake build +``` + +If everything is OK, extract the symbol list from the directory below and copy it out. + +``` +build/.objs/bdsheader/windows/x64/debug/test/__cpp_main.cpp.cpp.symbols +``` + +The contents of the file look like this + +``` +CXXMethod, ?Submit@ThreadPool@OS@@QEAAXXZ +CXXConstructor, ??0WebviewDownloadInfo@@QEAA@XZ +CXXMethod, ??4WebviewDownloadInfo@@QEAAAEAU0@AEBU0@@Z +CXXMethod, ?getBatteryLevel@FakeBatteryMonitorInterface@@UEBA +... +``` + +## Extracting symbols from former PDB + +As we all know, many functions in C++ programs are generated at compile time, so there are too few symbols that can be directly extracted from header files. +There is currently a lack of tools to directly generate "template symbols". (1) +So we need to extract symbols from the previous PDB, use the following command +{ .annotate } + +1. You may be interested in [this issue](https://github.com/liteldev-free/DebugInfo/issues/2). + +``` +./extractsym "/path/to/former/bedrock_server.pdb" -o 1.21.3.01.symbols +``` + +Similarly, copy this file out, its content is similar to... + +``` +Function, ??1I18nImpl@@UEAA@XZ +Function, ??_9PeerConnectionInterface@webrtc@@$BPI@AA +Function, ??1CommandArea@@QEAA@XZ +Var, ??_C@_0M@DPBEHEC@Destination@ +... +``` + +## Extracting types from header + +???+ warning "Optional step" + + This step is a bit complicated and unstable, and has some "workaround" meaning. If you can't figure it out, you can skip it. + +First, you still need to use the header file, so restore it first. + +``` +cd /path/to/header/proj +git restore . +``` + +Then run DeThunk. + +``` +python main.py "/path/to/header/mc" --preset-extract-types --exclusion-list exclusion_list/r21_u6.toml +``` + +Then rebuild. + +``` +xmake build +``` + +After completion, you should be able to find `bdsheader.pdb` in the following path. In general it should be larger than 40M, copy it out. + +``` +build/windows/x64/release +``` + +## Generate symbol table + +Since we just extracted the symbol lists from two different places, we pass them both to `askrva`. + +!!! tip + + By the way, this will read bedrock_runtime_data from the current directory by default, make sure it exists. + +``` +./askrva __cpp_main.cpp.cpp.symbols 1.21.3.01.symbols -o succeed.json -of failed.txt +``` + +## Generate PDB + +Now everything is ready, execute the following command to generate PDB. + +!!! question + + You might be wondering why you are passing in bedrock_server.exe, this is so that the generated PDB matches the COFF file (so that the debugger can use it). + +``` +./makepdb --program bedrock_server.exe --symbol succeed.json --typeinfo bdsheader.pdb -o bedrock_server.pdb +``` + +## Finish + +Everything is done! According to the information mentioned at the beginning of the article, this will generate at least 70M of PDB, which will be very useful to you, so enjoy using it! + +If you have any questions, please join the community for discussion. diff --git a/mkdocs.yml b/mkdocs.yml index ce5c1c3..1dbd388 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -3,13 +3,41 @@ site_url: https://bedrockdebug.info/ repo_url: https://github.com/liteldev-free/DebugInfo edit_uri: edit/main/docs/ repo_name: liteldev-free/DebugInfo +nav: + - Home: + - index.md + - releases.md + - FAQ: faq.md + - License: license.md + - Toolchain: + - toolchain/index.md + - toolchain/DeThunk.md + - toolchain/DumpSYM.md + - toolchain/ExtractSYM.md + - toolchain/AskRVA.md + - toolchain/MakePDB.md + - toolchain/BlobExtractor.md + - Tutorial: + - Generate a PDB from scratch: tutorial/generate-a-pdb-from-scratch.md + - Research: + - research/index.md + - research/HeaderGen.md + - research/PreLoader.md + - research/FakeSymbol.md + - Community: + - community/index.md + - Code of Conduct: CODE_OF_CONDUCT.md + - Contribution Guidelines: CONTRIBUTING.md theme: name: material logo: assets/logo.png favicon: assets/favicon.ico language: en + icon: + annotation: material/arrow-right-circle features: - navigation.instant + - navigation.indexes - navigation.instant.progress - navigation.tracking - navigation.tabs @@ -17,7 +45,6 @@ theme: - navigation.path - navigation.top - navigation.footer - - toc.integrate - search.suggest - search.highlight - content.action.edit @@ -43,6 +70,7 @@ theme: icon: material/lightbulb-outline name: Switch to system preference plugins: + - search - git-revision-date-localized: enabled: true enable_creation_date: true @@ -53,20 +81,28 @@ plugins: extra: alternate: - name: English - link: /en/ + link: / lang: en - - name: 简体中文 - link: /zh/ - lang: zh version: provider: mike social: - icon: fontawesome/brands/github link: https://github.com/liteldev-free - icon: fontawesome/brands/discord - link: https://discord.com/ + link: https://discord.gg/fPKVZScPT7 - icon: fontawesome/brands/telegram - link: https://telegram.org + link: https://t.me/s/bdsplugins markdown_extensions: - admonition -copyright: Copyright © 2025 - present Open LiteLDev \ No newline at end of file + - attr_list + - md_in_html + - pymdownx.details + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format + - pymdownx.emoji: + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg +copyright: Copyright © 2025 - present Open LiteLDev diff --git a/requirements.txt b/requirements.txt index 26aa290..f5766ed 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,9 +1,14 @@ babel==2.17.0 backrefs==5.8 +cairocffi==1.7.1 +CairoSVG==2.7.1 certifi==2025.1.31 +cffi==1.17.1 charset-normalizer==3.4.1 click==8.1.8 colorama==0.4.6 +cssselect2==0.8.0 +defusedxml==0.7.1 ghp-import==2.1.0 gitdb==4.0.12 GitPython==3.1.44 @@ -21,7 +26,9 @@ mkdocs-material-extensions==1.3.1 packaging==24.2 paginate==0.5.7 pathspec==0.12.1 +pillow==10.4.0 platformdirs==4.3.6 +pycparser==2.22 Pygments==2.19.1 pymdown-extensions==10.14.3 python-dateutil==2.9.0.post0 @@ -31,5 +38,7 @@ pyyaml_env_tag==0.1 requests==2.32.3 six==1.17.0 smmap==5.0.2 +tinycss2==1.4.0 urllib3==2.3.0 watchdog==6.0.0 +webencodings==0.5.1 diff --git a/src/tools/askrva/main.cpp b/src/tools/askrva/main.cpp index df31587..c3d3ca8 100644 --- a/src/tools/askrva/main.cpp +++ b/src/tools/askrva/main.cpp @@ -24,8 +24,6 @@ using namespace di::data_format; std::optional m_output_failed_path; } args; - std::string output_format; - // clang-format off program.add_argument("path") @@ -47,12 +45,6 @@ using namespace di::data_format; program.add_argument("--output-failed", "-of") .help("Path to output failed entries."); - program.add_argument("--output-format") - .help("Specify output format.") - .choices("auto", "text", "makepdb") - .default_value("auto") - .store_into(output_format); - // clang-format on program.parse_args(argc, argv); diff --git a/src/tools/extractsym/main.cpp b/src/tools/extractsym/main.cpp index c405fa7..92a2b8e 100644 --- a/src/tools/extractsym/main.cpp +++ b/src/tools/extractsym/main.cpp @@ -10,7 +10,7 @@ using namespace di::object_file; using namespace di::data_format; auto load_args(int argc, char* argv[]) { - argparse::ArgumentParser program("extractpdb"); + argparse::ArgumentParser program("extractsym"); struct { std::string m_program_database_path;