* docs: Add missing testing toc entries and other minor formatting
* fix: Adjust nix dev shell packages to properly support jemalloc and qemu tests
* fix(jemalloc): Add more robust error handling to jemalloc commands and fix test
* fix: point JEMALLOC_PATH to correct jemalloc package path
* fix: Use correct aglib-compatible symbol resolution function
* fix: re-enable test_jemalloc_heap test and make slightly more forgiving
* splitting tests passed and skipped variable into two, solving issue #2485
* Fixing bugs in PR #2495
* jemalloc.py: remove unused Arena class (#2492)
* Fixes canary command when no canaries are there (#2496)
Fixes the following bug uncovered by our test_commands[canary] test suite:
```
(.venv) root@pwndbg:~/pwndbg# RUN_FLAKY=1 ./tests.sh 'test_commands' --pdb
Will run tests in serial and with Python debugger
ZIGPATH set to /root/pwndbg/.zig
make: Nothing to be done for 'all'.
pwndbg: loaded 173 pwndbg commands and 47 shell commands. Type pwndbg [--shell | --all] [filter] for a list.
pwndbg: created $rebase, $base, $bn_sym, $bn_var, $bn_eval, $ida GDB functions (can be used with print/break)
Launching pytest with args: ['/root/pwndbg/tests/gdb-tests/tests/test_commands.py::test_commands[canary]', '-vvv', '-s', '--showlocals', '--color=yes', '--pdb']
======================================================================== test session starts ========================================================================
platform linux -- Python 3.11.6, pytest-8.0.2, pluggy-1.5.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /root/pwndbg/tests
plugins: cov-4.1.0
collected 1 item
gdb-tests/tests/test_commands.py::test_commands[canary] Running command canary
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.ubuntu.com>
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
Program stopped.
0x00007ffff7fe4b40 in _start () from /lib64/ld-linux-x86-64.so.2
AT_RANDOM = 0x7fffffffe3d9 # points to (not masked) global canary value
Canary = 0x37492280ae2b3000 (may be incorrect on != glibc)
No canaries found.
Traceback (most recent call last):
File "/root/pwndbg/pwndbg/commands/__init__.py", line 187, in __call__
return self.function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/pwndbg/pwndbg/commands/__init__.py", line 378, in _OnlyWhenRunning
return function(*a, **kw)
^^^^^^^^^^^^^^^^^^
File "/root/pwndbg/pwndbg/commands/canary.py", line 91, in canary
if some_canaries_not_shown is True:
^^^^^^^^^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'some_canaries_not_shown' where it is not associated with a value
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/pwndbg/pwndbg/dbg/gdb.py", line 871, in invoke
self.handler(self.debugger, args, from_tty)
File "/root/pwndbg/pwndbg/commands/__init__.py", line 105, in _handler
self.invoke(arguments, is_interactive)
File "/root/pwndbg/pwndbg/commands/__init__.py", line 153, in invoke
self(*args, **kwargs)
File "/root/pwndbg/pwndbg/commands/__init__.py", line 192, in __call__
pwndbg.exception.handle(self.function.__name__)
File "/root/pwndbg/pwndbg/exception.py", line 106, in handle
raise e
File "/root/pwndbg/pwndbg/commands/__init__.py", line 187, in __call__
return self.function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/pwndbg/pwndbg/commands/__init__.py", line 378, in _OnlyWhenRunning
return function(*a, **kw)
^^^^^^^^^^^^^^^^^^
File "/root/pwndbg/pwndbg/commands/canary.py", line 91, in canary
if some_canaries_not_shown is True:
^^^^^^^^^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'some_canaries_not_shown' where it is not associated with a value
```
* Fix gdt command: require address argument (#2497)
Before this commit, the `gdt` command incorrectly did not require the `address` argument, while it was required.
Because of that, the command crashed like this:
```
pwndbg> gdt
'gdt': Decode X86-64 GDT entries at address
See also:
* https://wiki.osdev.org/Global_Descriptor_Table
* https://wiki.osdev.org/GDT_Tutorial
Note:
In 64-bit mode, the Base and Limit values are ignored, each descriptor covers the entire linear address space regardless of what they are set to.
Exception occurred: gdt: int() argument must be a string, a bytes-like object or a real number, not 'NoneType' (<class 'TypeError'>)
For more info invoke `set exception-verbose on` and rerun the command
or debug it by yourself with `set exception-debugger on`
```
* Add better handling for exceptions originating in execution controllers under LLDB (#2488)
* Fix try_free command: make addr argument required (#2499)
Fixes the following problem:
```
pwndbg> try_free
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
> /root/pwndbg/pwndbg/commands/heap.py(1195)try_free()
1194 def try_free(addr: str | int) -> None:
-> 1195 addr = int(addr)
1196
```
* Fix ctxp command (#2498)
* Fix ctxp and ctxn commands
Before this commit the `ctxp` could fail with:
```
Launching pytest with args: ['/root/pwndbg/tests/gdb-tests/tests/test_commands.py::test_commands[ctxp]', '-vvv', '-s', '--showlocals', '--color=yes', '--pdb']
======================================================================== test session starts ========================================================================
platform linux -- Python 3.11.6, pytest-8.0.2, pluggy-1.5.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /root/pwndbg/tests
plugins: cov-4.1.0
collected 1 item
gdb-tests/tests/test_commands.py::test_commands[ctxp] Running command ctxp
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.ubuntu.com>
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
Program stopped.
0x00007ffff7fe4b40 in _start () from /lib64/ld-linux-x86-64.so.2
Traceback (most recent call last):
File "/root/pwndbg/pwndbg/commands/__init__.py", line 187, in __call__
return self.function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/pwndbg/pwndbg/commands/context.py", line 363, in contextprev
longest_history = max(len(h) for h in context_history.values())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: max() arg is an empty sequence
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/pwndbg/pwndbg/dbg/gdb.py", line 871, in invoke
self.handler(self.debugger, args, from_tty)
File "/root/pwndbg/pwndbg/commands/__init__.py", line 105, in _handler
self.invoke(arguments, is_interactive)
File "/root/pwndbg/pwndbg/commands/__init__.py", line 153, in invoke
self(*args, **kwargs)
File "/root/pwndbg/pwndbg/commands/__init__.py", line 192, in __call__
pwndbg.exception.handle(self.function.__name__)
File "/root/pwndbg/pwndbg/exception.py", line 106, in handle
raise e
File "/root/pwndbg/pwndbg/commands/__init__.py", line 187, in __call__
return self.function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/pwndbg/pwndbg/commands/context.py", line 363, in contextprev
longest_history = max(len(h) for h in context_history.values())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: max() arg is an empty sequence
XFAIL (flaky test)
```
* fix
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
Co-authored-by: Matt. <dark.ryu.550@gmail.com>
* Fix terminal width to 80 columns in tests
Set a `PWNDBG_IN_TEST`environment variable when running gdb in tests. Use the dimensions in `LINES` and `COLUMNS` when looking up the window size when that envvar is set.
This makes context output always be 80 columns wide which allows to compare to hardcoded output.
* Use `width` and `height` parameters for cmd window size
They are updated to the correct values when switching between tui and cli mode. So it's unnecessary to parse `info win`.
* Remove workarounds for terminal size in DEVELOPING
* Don't bother updating the env of the debuggee
* Add history of context output
Every context section is cached individually to allow to display older output again. You can scroll through the old context output using `contextprev` and `contextnext`.
This allows to "scroll" in TUI mode.
* Add button TUI window to control context history
* Simplify history status output generation
* Fix attachp tests when ptrace_scope is missing
Assume we can attach to any process when ptrace_scope is not available. This is the case in WSL2.
* Update tests/gdb-tests/tests/test_attachp.py
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Make last-signal context section selectable
There exists a context section to display the reason for the last stop, but it wasn't registered anywhere.
Add it to the list of available sections to choose from.
* Fix default context sections "heap-tracker"
The section name is derived from the function name.
The function is called `context_heap_tracker` so `heap-tracker` wouldn't work:
```
pwndbg> set context-sections heap-tracker
Invalid section: heap-tracker, valid values: args, regs, disasm, stack, backtrace, code, expressions, ghidra, heap_tracker, threads
```
The validator doesn't appear to run for the default value :D
* Update help text to include all available context sections
* Replace bash with sleep in attachp commands to fix crash due to reading from terminal
* Update tests/gdb-tests/tests/test_attachp.py
* Update test_attachp.py
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Fixed bug where plist()'s argument sentinel had incorrect type
* Fixed bug where if sentinel was different from zero, plist tried to dereference a null address
* Fixed bug where some error message in plist displayed 0x0x{address} due to incorrect formatting
* Added additional tests for plist's bugfixes
* Added types to hexdump()'s parameters
* hexdump(): upgraded format strings to f-strings, while fixing the offset when data is empty
* hexdump(): offset is now correctly computed and updated accross repeated calls to hexdump.
This fixes two bugs:
1. Offset was increased by the number of lines displayed by the hexdump() command,
which was incorrect, as some lines may be compressed;
2. When dumping a number of bytes that is not divisible by the number of bytes per line,
offset would be incorrectly updated.
* Linter
* hexdump: fixed type of retrieved config entry hexdump_group_use_big_endian
* Fixed test_hexdump_saved_address_and_offset test to account for random stack address
* Upgraded the linked-lists.c test program to 6 items in order to test the "deference-limit" setting
* Chain: changes to the deference-limit parameter were not reflected in the program due to an early cast to int()
* Linter
* Added support for `--offset` and `--count` for plist
* Added typing to commands.plist's prototype
* Updated documentation for plist tests
* Add informational message when plist --count is zero or under
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Handle case, in plist, where the default value for count is zero
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Added types to hexdump()'s parameters
* hexdump(): upgraded format strings to f-strings, while fixing the offset when data is empty
* hexdump(): offset is now correctly computed and updated accross repeated calls to hexdump.
This fixes two bugs:
1. Offset was increased by the number of lines displayed by the hexdump() command,
which was incorrect, as some lines may be compressed;
2. When dumping a number of bytes that is not divisible by the number of bytes per line,
offset would be incorrectly updated.
* Linter
* hexdump: fixed type of retrieved config entry hexdump_group_use_big_endian
* Fixed test_hexdump_saved_address_and_offset test to account for random stack address
* Upgraded the linked-lists.c test program to 6 items in order to test the "deference-limit" setting
* Chain: changes to the deference-limit parameter were not reflected in the program due to an early cast to int()
* Linter
* Port Pwndbg to LLDB
* Fix splitting mistakes
* I love typos
* We already check for Python 3.11 manually
* Update pwndbg/dbg/lldb/__init__.py
Co-authored-by: patryk4815 <bux.patryk@gmail.com>
* Fix use of wrong variable name that was preventing the vmmap from being used in LLDB memory reads
* Use cached vmmap for `read_memory` in LLDB
* Fix Pwndbg CLI jank
* Try using SBAddress to resolve objfile names in LLDB vmmap
* Fix missing changes in `pwndbg.gdblib`
* Update pwndbg/aglib/arch.py
* Update pwndbg/aglib/arch.py
* Remove outdated comment about `pwndbg.aglib.arch`
* Update pwndbg/dbg/lldb/repl/io.py
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Fix lints
* Fix new test so that it points to `pwndbg.aglib.disasm`
* Fix lints
---------
Co-authored-by: patryk4815 <bux.patryk@gmail.com>
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Merge Arm tests
* Integrate MIPS, AArch64, RISC-V tests, and the rest of the ARM edges cases
* add fsbase/gsbase annotation test
* add function call / fin test
* lint
* Fix all bugs found - MIPS JAL, delay slots splits, arm/aarch unconditional jumps, and forcing targets when we know they go to the next instruction in memory, and add a bunch of comments. Arm now detects instructions that write to PC as branches too.
* Fix Arm exclusive stores
* comment update
* Fix arm/aarch64 post-indexed stores - add test
* AArch64 post-index stores
* Fix arm shifts
* comment typo
* Initial version of qemu-user tests
* Refactor testing files to reduce file duplication, introduce qemu-user-tests
* lint and edit github actions workflow file. Move old qemu-user tests to seperate directory
* Add iproute2 so ss command is available
* test ubuntu 24
* funkiness with current working directory...
* Further remote old test_qemu.sh and integrate into a Pytest fixture
* lint
* Disable ASLR, add test for aarch64 jumps
* Use Popen.kill() function to make sure it closes.
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* qemu.kill() on the other fixture as well
* comment
* comment
* lint
* system test path stuff
* remove old try-catch block
* revert
* revert path change
* Use os._exit to pass return code, and move qemu-user tests above system tests because they run significantly faster
* lint
* Flush stdout before os._exit
* Comment out flaky check for the address of main in old qemu tests
* rename qemu-user to cross-arch
* rename qemu-user to cross-arch and hotfix to not run pytest when
cross-arch is used
* remove todo comment
* another comment
* Test pwndbg.gdblib.symbol.address is not None and revert setarch -R
* Revert os.exit change
* Revert os.exit change
* Revert os.exit change
* readd os.exit in new exit places
* lint
* rebase
* delete file introduced in rebase
* break up tests into 3 files to invoke separately. Update GitHub workflow, remove code duplication in existing test
* code coverage
* fix code coverage
* lint
* test difference between Ubuntu 22 and 24 in Kernel tests
* lint
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* basic go value dumping
* better error handling and misc improvements
* minor documentation changes
* satisfy mypy
* struct parsing and bug fix
* satisfy mypy
* deal with evacuated buckets
* better error message for invalid expressions
* convert bytearray to bytes before repr
* support for recursive types and better type dumping
* better QOL for go-dump command
* formatting options and debug/pretty printing
* add go dumping unit tests
* deal with differences in old go version
* lint
* old go versions missing any alias
* proper name dumping for go versions prior to 1.17
* lint
* go is being weird on CI
* warn instead of erroring
* function and interface dumping
* Add event system to the Debugger-agnostic API
* Move uses of `pwndbg.gdblib.events` to the Debugger-agnostic API
* Fix rebase mistakes
* Update pwndbg/commands/context.py
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Update context.py
* Update __init__.py
* Fix nits
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* basic binary ninja integration
* start centralizing integrations into uniform interface
* finish most integrations
* make binja integration contain all ida features
* allow switching servers
* remove unused functions
* format
* fix unit tests
* satisfy mypy + warn instead of error on invalid provider
* fix decompilation not working in function prologue
* better config options and il config
* lint
* fix bn_sym test
* add decomp command
* add bn_eval gdb function
* format binja_script.py
* satisfy mypy again
* doc update
* move integrations to separate subdir + make test_loads more versatile
* format
* forgot to push a file
* format
* highlight next instruction instead of previous one in decomp
* fix disassembly il level
* prefer function tags over data tags
* add bn_var function
* add variable labeling
* more configurability, documentation, bug fixes
* code cleanup
* format
* remove code duplication between chain.py and memory.py
* add short-lived cache for some integration functions
* use every register available to help with frame inference
* lint
* don't waste time checking irrelevant registers
* move performance hit in cache only to except case
* fix issue with conflict resolution
* lint
* fix rebase mistake
* Added types to hexdump()'s parameters
* hexdump(): upgraded format strings to f-strings, while fixing the offset when data is empty
* hexdump(): offset is now correctly computed and updated accross repeated calls to hexdump.
This fixes two bugs:
1. Offset was increased by the number of lines displayed by the hexdump() command,
which was incorrect, as some lines may be compressed;
2. When dumping a number of bytes that is not divisible by the number of bytes per line,
offset would be incorrectly updated.
* Linter
* hexdump: fixed type of retrieved config entry hexdump_group_use_big_endian
* Fixed test_hexdump_saved_address_and_offset test to account for random stack address
* Improve bin corruption checks
* Update pwndbg/heap/ptmalloc.py
* factor out and clean up bin corruption check
* check chunks even if bin is longer than limit
* add empty bin check
* lint.sh, remove testing if
* dont modify chain in check, allow corruption=0, cleanup
* typing, more reliable empty bin check
* cast params to int, otherwise not detected properly
* add regression test for corruption check
* lint.sh
---------
Co-authored-by: Gulshan Singh <gsingh2011@gmail.com>
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Implement bitwise math rotation operations on numbers of discrete width. Will be used in manually evaluating arm instruction offsets and shifts
* fixes
* Fix caching for pwndbg.gdblib.elf functions
* Add test for cache clear priority
* Fix formatting in files
* switch priority to an enum instead of boolean
* fix logging for start event
* decouple gdblib.events import from lib.cache
* Move syscall number evaluation into instruction.py. This allows us to determine and display future syscalls
* Move string manipulation to color.disasm.py
* lint
* fix padding
* Fix x86 syscall
* comment
* disable debug mode
* Fix a test - we now remember previous syscalls as well
* Move x86 specific syscall logic to the x86 subclass
* lint
* @override decorator added to methods
* comments
* lint
* add test with emulation disabled for syscall annotation
* Fix x86/x86_64 edge cases with syscall register reading, and add test for emulation off for syscalls
* Update an outdated comment
* Tests depend on width of context banner
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Add gdb_version to mock gdblib
* Re-enable unit tests
* Only collect unit test coverage if --cov is passed
* Source venv before running tests in github action
* Add venv path PATH in to Dockerfile
* Only check for "/ls" in `which` test
* Fix i386-32 syscall name printing
pwndbg-git from AUR shows hexadecimal constants in masm syntax
(e.g. 80h) for some reason (as if the option CS_OPT_SYNTAX_MASM was set).
This commit makes syscall name printing work regardless of hex syntax.
* riscv: Fix AssertionError on "jalr ra, ra, 0x252"
When the PC was on this instruction, the pwndbg context would not be
printed due to this AssertionError.
* riscv: Fix AssertionError on "c.jalr a5"
According to the specification, "C.JALR expands to jalr x1, 0(rs1)".
* Modify python test scripts to work from nix
* Update utils.py
* address review feedback
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Only look for readable address in retaddr command
* Rename stack.py to retaddr.py
* Add pwndbg.gdblib.stack.callstack and use it in retaddr
* Add callstack gdb test
* Add QEMU callstack test
* V1 - annotations for values of registers and memory to display result of instructions.
* Emulator telescope(), more x86 instructions
* Emulation change - keep track of before & after instruction execution. Telescope format correctly, read size taken into account
* Add config options to configure emulation and annotations, vmovaps alignment warning, string length in disasm telescope, cache previously annotated instructions
* Create PwndbgInstruction type for better typing and easier future development
* More consistent spacing, options to disable annotations, ADD instruction shows operands
* Rebase from dev
* Correctly go to .next address in disasm view (was incorrectly going to call targets)
* Precompute register str to reduce code duplication
* Correct telescope memory read width, bring target printing back to previous behavior when symbol can be resolved
* More consistent looking annotation spacing/padding, fixed edge case bugs with cached instructions
* Even cleaner padding
* Additional comments and debugging, ironed out last bugs
* debugging tight loops
* Cache fixed - nearpc only annotates what can be statically resolved
* lint and show instructions that cannot dereference
* Reapply btrace crash fix after rebase
* Less code duplication, implement XCHG and AND instructions, moved more methods from x86 subclass to superclass
* aarch64 set flags register in Unicorn correctly
* fix
* Don't recreate emulator regname->const map every time the emulator is instantiated
* Use emulation to set .next within enhancement
* Improve ret instruction target address setting
* Green checkmarks for jumps on all architectures
* Fixed .next and .target setting
* All architectures now have correct logic for determining .next and .target. Green checkmarks for taking conditional branches now appear for all architectures, added things to determine that type of branch being taken, and simplified printing by replacing symbol_addr with new field 'target_string'
* Instruction debug print fix
* Correct jump instruction checking
* Fixed target_string resolution
* Fix conditional jump check, also make default target resolution better
* target_const determined more accurately
* reverse iteration order of last change
* Pwndbg.condition is more retyped and more correct, make manual determinations of condition override the emulators (it can be incorrect in cases). Uncover why MIPS sometimes takes incorrect jumps in the emulator (delay slot)
* MIPS annotations work really good now. Jumps are correctly predicted (with green checkmarks). Implemented manual condition() function for MIPS. Only highlight the correct instance of instruction at PC when there are multiple in view (tight loops). Allow manual .condition to override emulator in determiningg .next.
* Additional debug info on instructions
* Print arch in instruction
* aarch64 branch fix
* aarch64 branch fix (real)
* lint
* Final changes - fixing .size error
* lint
* Add dev_dump_instruction command, add default memory read in resolve_used_value, update comments and remove .size from enhancedoperand as it only exists on x86
* More uniform spacing on annotations
* Various comments converted to docstrings, aarch64 enhancer created, post-rebase
* import aarch64
* Aarch64 mov, ldr, add, sub
* adrp
* ADR
* lint
* Fun with git rebase
* lint
* lint again after re-installing dependencies
* New caching strategy implemented to ensure no state caching when jumping large distances. Handled edge cases of user manually setting a register or memory, 'set regname=2'
* lint
* Fixed two regressions (nearpc shouldn't take jumps, even ones we know statically, and replace all constants in the assembly with symbols). Tweak tests to reflect new annotations
* lint
* one last test
* Fix chain format dereferencing for non-singleton lists, now correctly deferences and displays chains for future instructions when not emulating (dereference until pointer goes to writable memory)
* Add jumps-only setting to emulation (on, off, jumps-only), fixes to chain deferencing and enhancing
* Properly dereference memory before and after execution of instructions, adding a new before_value_resolved field (same for after). This also reduces code duplication.
* Debogusify the format()/telescoping dereferencing logic
* lint
* post-rebase fixes
* Fix case the breaks a test - don't attempt to read larger than ptrsize such as in SIMD instruction memory reads
* Typo in emulate setting
* Developer docs for annotations
* Fix case where emulator attempts to read and unpack very large, 16 byte+ wide values while telescoping
* Add a helper command to find valid one_gadget for current context
* Refactor the function for getting section address
* Rename the command to onegadget for more convenient typing
* Make the output format cleaner
* Add a simple cache mechanism for the one_gadget output
* Update the warning message
* Use MD5 instead of BLAKE2 for computing the file hash
I thought that BLAKE2 was faster than MD5, but it doesn't seem correct here somehow (probably because of the implementation of Python!?)
Here's the script I used for benchmarking:
```python
import hashlib
import timeit
def compute_file_hash_1() -> str:
h = hashlib.blake2b()
with open("/lib/x86_64-linux-gnu/libc.so.6", "rb") as f:
h.update(f.read())
return h.hexdigest()
def compute_file_hash_2() -> str:
h = hashlib.md5()
with open("/lib/x86_64-linux-gnu/libc.so.6", "rb") as f:
h.update(f.read())
return h.hexdigest()
print(timeit.timeit(compute_file_hash_1, number=1000))
print(timeit.timeit(compute_file_hash_2, number=1000))
```
I executed the above script on various machines, and the results seem to show that MD5 outperforms BLAKE2 in this scenario. (On my x86 VM running through QEMU on my M1 MacBook, BLAKE2 even takes almost twice as long as MD5.)
* Add the tests for `onegadget` command
* Fix lint issue
* Try to cover more code
* Fix lint issue
* Fix illogical tests
* Rename one_gadget to onegadget
* Use `pwndbg.lib.tempfile.cachedir` for `onegadget`
* Call `pwndbg.lib.tempfile.cachedir` only once
* Add support for breaking on UAF
* Small fixes and documentation
* Add a command to enable and disable tracking, better diagnostics
* Add initial support for calloc and realloc
* Better safeguard against matching ld.so malloc
* Small fixes
* Better interface for managing the heap tracker. More terse and information dense diagnostics
* Add warning and fix lints
* Update poetry lock
Hopefully fixes#1947 by fetching stacks only when they are used instead
of doing it on each stop event. It will also first try to compute stacks
dictionary based on vmmap and if it fallbacks to exploring stacks if
vmmap is not present.
* [WIP] Port gdb-tests from bash to python
* Use threads instead of processes
* Port gdb tests to python
* Linting
* Fix coverage "again"
* Remove bash tests
---------
Co-authored-by: intrigus <abc123zeus@live.de>
Previously test scripts would just indiscriminately kill all qemu
processes on the system. This would kill other debug sessions I had
running. These changes make the test scripts record the qemu pids they
run and only kill those.
The old scripts would also not allow you to specify a gdb port, so
if you were already running a debug session with port 1234, the tests
would fail. This update allows you to pass --gdb-port=NNNN to use a
non-default port. You can pass -Q to preserve failing qemu instances.
The scripts now also will show qemu errors to console, and will warn
the user if there is a qemu port conflict.
Also update gdb-pt-dump submodule as it has been updated recently to not
throw an exception when multiple qemu processes are running. The
exception thrown in the event of a failure also changed, so
this has also been updated on the pwndbg side.
* get_one_instruction: clear "cont" cache on mem/reg changed
Fixes#1818.
Note that this makes a substantial change: it changes all caches that
are refreshed on `gdb.ContinueEvent` to also be cleared on memory/regs
changed.
This change is needed so that the `get_one_instruction` function which
uses this cache will get its cache cleared when user invokes a command
that changes memory or registers.
While this may sound as too big change: we are changing the whole "cont"
cache to be cleared on two additional events, this should not be an
issue. This is because:
1. We should notice it if we start clearing an important cache too often
2. The "cont" cache is currently only used by the `get_one_instruction`
at this moment.
The 2) also creates a question: when should one use "cont" vs "start"
caches? It is not so clear to me right now.
* Add test for issue #1818
* Clear caches on MemoryChanged events from gdblib.write
Regarding the last part:
Interestingly implementing tests here uncovered another bug: the gdblib.memory.write(..) or rather the gdb.selected_inferior().write_memory(...) API used there does not trigger a gdb.MemoryChanged event. As a result, we never cleared certain caches that should have been cleared when the user used that API.
I have added two tests here, one changes the instructions at $RIP to nops via gdblib.memory.write(..) and another via executing the patch $rip nop;nop;nop;nop;nop command. As a result, we test both scenarios: 1) when we depend on memory changed event being fired via GDB to clear caches; and 2) when we depend on gdblib.memory.write(..) to clear the caches.
This PR also makes a fix to the gdblib.memory.write(..) to actually clear caches that depend on (or rather: are hooked to in order to be cleared) memory changed events.
* Fix glibc-fastbin-bug option of find_fake_fast
Using the find_fake_fast option --glibc-fastbin-bug always resulted in an error, at least on 64-bit platforms.
This was because the option caused only 4 bytes to be read for the size, but then that gets passed to unpack() which expects 8 bytes.
Closes#1773
* Address review comment
* Update arch.py
* Update pwndbg/commands/heap.py
* Fix lint
* Update arch.py
* Update arch.py
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
This commit adds a command that traverses the linked list beginning at a given
element, dumping its contents and the contents of all the elements that come
after it in the list. Traversal is configurable and can handle multiple types
of chains.
This commit adds the `break-if-taken` and `break-if-not-taken` commands,
which attach breakpoints to branch instructions that will stop the
inferior if said branch is taken or is not taken, respectively. It adds
an extra class, `pwndbg.gdblib.bpoint.Breakpoint`, which clears caches
before calling `stop()`, allowing for the use of register values inside
that function in breakpoint classes that derive from it. Additionally,
checking of whether the conditions for a branch to be taken have been
fulfilled is done through `DisassemblyAssistant.condition()`.
* Add `stepuntilasm` command
This commit adds a `stepuntilasm` command that, given a mnemonic and,
optionally, a set of operands, will step until a instruction that
matches both is found. Matching is string-based, as the user will likely
want to spell out the asm directive they want as text, and interpreting
assembly language conventions for all of the platforms pwndbg supports
is probably outside the scope of this change.
* next.py: small code cleanup
* next.py: fix bug introduced in previous commit
op.str -> op_str
* Update next.py
* Update next.py
* Update next.py
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Only run arch for testing
* Remove outdated arch repo
* Actually build the docker image
* Do not include site packages in sys.path
* Ignore `.relr.dyn` section; skip lines w/o spaces
Newer binaries can contain a `.relr.dyn` section to compress `R_X86_64_RELATIVE` relocation entries.
These binaries can be found for example on archlinux but also on Debian 12 for example.
`readelf` prints the content of the section similarly to this:
```
Relocation section '.relr.dyn' at offset 0x25220 contains 35 entries:
1198 offsets
00000000001ce8d0
00000000001ce8e0
```
Compared to `00000000001d2000 0000000000000025 R_X86_64_IRELATIVE 9f330` for
`.rela.plt`.
Pwndbg now chokes on the new format because it expects a space seperator where there is none.
It might be, that this is actually an upstream problem with binutils, because llvm-readelf prints this:
```
Relocation section '.relr.dyn' at offset 0x25220 contains 1198 entries:
Offset Info Type Symbol's Value Symbol's Name
00000000001ce8d0 0000000000000008 R_X86_64_RELATIVE
00000000001ce8e0 0000000000000008 R_X86_64_RELATIVE
```
Nevertheless, we aren't actually interested in `R_X86_64_RELATIVE` relocations so I guess it's fine to
just skip all lines that contain no spaces at all.
`.relr.dyn` can only containt `R_X86_64_RELATIVE` relocations as far as I understand
https://maskray.me/blog/2021-10-30-relative-relocations-and-relr
* Accept Full RELRO in test
Archlinux has libc and ld with Full RELRO.
We now just accept Partial and Full RELRO.
* Do not copy binaries from host to docker
The `Dockerfile` copies the whole pwndbg folder to the image.
If we have built binaries on the host before, these binaries will contain references to
the host system and *copied* to the image.
If we now run `context code` (inside docker) to have a look at the source code this will
fail, because we will try to refer to a path on the host system.
* Do not use loop index after loop
Do not use loop index after the loop. The tests assumed that the loop in line 186
would run at least once, thereby *resetting* `i` to zero. If we never enter the
loop, `i` will *continue* to have the value it had at the end of line 172.
This will cause the test to fail in mysterious ways because `i` is now not reset
to zero but still has the value `31` for example.
The solution is to never use `i` outside of a loop.
* Re-enable archlinux and temporarily disabled ones
* Fix coverage combine toml issue
This commit should fix this issue:
```
Run coverage combine
coverage combine
coverage xml
shell: /usr/bin/bash -e {0}
Can't read 'pyproject.toml' without TOML support. Install with [toml] extra
Error: Process completed with exit code 1.
```
* setup.sh: cleanup the --user flag since we use venv now
Cleans up the --user flag from setup.sh since it is unused after we changed setup.sh to install Python dependencies in a virtual environment
* Remove --user flag from CI workflows
* Fix codecov problem
We need to run the python `coverage` library to collect coverage.
However, gdb was failing to find it.
Recently, pwndbg moved to using venvs. When pwndbg is initialized
it setups the venv "manually", that is, no "source .venv/bin/activate"
is needed. When we run gdb tests, we pass the `gdbinit.py` of pwndbg as a
command to gdb to be executed like this:
`gdb --silent --nx --nh -ex 'py import coverage;coverage.process_startup()' --command PATH_TO_gdbinit.py`
The problem is that *order* matters. This means that *first* coverage
is imported (by `-ex py ...`) and only *then* the init script is executed.
When `coverage` is first imported, it's library search path only looks
in system libraries of python, and not the venv that gdbinit.py would load.
So we would try to import an old version of coverage and fail.
One solution would be to move around the commands, but this would be an
ugly hack IMHO. **Instead**, we should just tell gdb that this is an **init**
command that has to be executed before other commands.
Previously, the order did not matter. All of pwndbg's dependencies were
installed directly as system libraries to python. So the library search path
was the same before and after loading `gdbinit.py`.
---------
Co-authored-by: disconnect3d <dominik.b.czarnota@gmail.com>
Co-authored-by: intrigus <abc123zeus@live.de>
* Refactor the `got` command to support more use cases
- Create some function to parse the information of loaded shared object libraries from `info sharedlibrary`
- Make got command can show the entries of other libraries loaded in memory
- Make got command can show more various relocations to support not only the `JUMP_SLOT` type relocation but also supports `IRELATIVE` and `GLOB_DAT` type relocation.
* Update tests for the `got` command
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update the comment
https://github.com/pwndbg/pwndbg/pull/1771#discussion_r1251054080
* Update the tests
* Add some hints for the qemu users
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Improve RISCV support
This is a resurrection of #829
Co-authored-by: Tobias Faller <faller@endiio.com>
* Silence bogus vermin warning
* Fix relative backwards jump calculations
The target address wouldn't be truncated to the pointer size.
* Add basic qemu-user test
* Run qemu-user tests in CI
* Make shfmt happy
* Fix pwntools < 4.11.0 support
* Support RISCV32 for pwntools < 4.11.0 as well
---------
Co-authored-by: Tobias Faller <faller@endiio.com>
* fix: remove minor bugs and complete address translation
* feat: add 5lvl paging
* feat: add address translation tests
* fix: remove unnecessary comments
* fix: add references for magic values
* fix: add X86_FEATURE_LA57 reference
* fix: move x86 specific functions to x86_64Ops
* fix: extend tests and remove faulty code
* fix: only test address translation for lowmem
* fix: adjust arch_ops test to pytest
* fix: add reference for memory models in linux
* fix: do not memoize staticmethods
* Fix and test ctx disasm when disassembly-flavor changes
* New lib/cache.py: make caching great again
This commit fixes bugs with old caching (memoize.py) and makes it more
readable.
See also https://github.com/pwndbg/pwndbg/issues/1453
* Update pwndbg/lib/cache.py
Co-authored-by: Gulshan Singh <gsingh2011@gmail.com>
* lib.cache: address PR comments and add debug mode
* Fix lint
* Remove leftover memoize usages
* Add cache benchmark
* fix lint
---------
Co-authored-by: Gulshan Singh <gsingh2011@gmail.com>
* feature: Add `killthreads` command (closes#1580)
This command allows the user to quickly kill multiple threads by
specyfying their ids as arguments to this command. It also supports
the `--all` flag, which will kill overy thread except the currently
selected one. This is useful for use with the `checkpoint` command.
The killing is done by calling `pthread_exit(0)`.
* fix: try fixing building test binaries by enabling -lpthread
* fix: remove error message check when calling pthread_exit
Removed the message check, because the error messages difffer between
versions of GDB.
* fix: Improve UX of the killthreads command
Add an extended description of the command, some validation on the thread IDs
and suppress GDB output.
* fix: lint
* fix: put the multiline help text in the correct place
* tests: fix test failing due to a race condition when running in parrallel to other tests
Replaced asserts with loops which wait for a cundition to be met, so that the tests doesn't fail due to scheduling issues.
* tests: add more fixes for race conditions in test_killthreads
* fix: lint
* Update pwndbg/commands/killthreads.py
* tests: Wait for exactly three threads
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Changing the arguments to vis_heap_chunks to be clearer
1. --native to --beyond_top
2. --display_all to --no_truncate
* Add print all chunks to vis_heap_chunks
* Preventing the use of the all_chunks argument together with the count argument in vis_heap_chunks
* Use linting for heap.py
* Fix test_vis_heap_chunks.py
According to cdd71a1d82 --display_all/-d moved to --no_truncate/-n
---------
Co-authored-by: Nerya Zadkani <nerya@tokagroup.com>
This commit adds a fix and tests for #1600 and #752.
* https://github.com/pwndbg/pwndbg/issues/1600
* https://github.com/pwndbg/pwndbg/issues/752
Generally, for an example like this:
```cpp
struct A {
void foo(int, int) { };
};
int main() {
A a;
a.foo(1, 1);
}
```
The output for `info symbol <address of A::foo>` returns:
```
'A::foo(int, int) [clone.isra.0] + 3 in section .text of /root/pwndbg/tests/gdb-tests/tests/binaries/a.out\n'
```
We then used this code to parse this:
```py
# Expected format looks like this:
# main in section .text of /bin/bash
# main + 3 in section .text of /bin/bash
# system + 1 in section .text of /lib/x86_64-linux-gnu/libc.so.6
# No symbol matches system-1.
a, b, c, _ = result.split(maxsplit=3)
if b == "+":
return "%s+%s" % (a, c)
if b == "in":
return a
return ""
```
The `result.split(maxsplit=3)` here splitted the string to:
```py
['A::foo(int,',
'int)',
'[clone.isra.0] + 3 in section .text of /root/pwndbg/tests/gdb-tests/tests/binaries/a.out\n']
```
And since `b` was not `"+"` or `"in"` we eventually returned an empty
string instead of the `A::foo(int, int)` which would be expected here.
* Fix the bug when using LD_PRELOAD to load libc
The heap heuristics will try to find `libc.so.6` in the output of `info sharedlibrary`, but if we load libc with `LD_PRELOAD`, the filename of the libc might not be `libc.so.6`.
* Add test for `glibc.get_libc_filename_from_info_sharedlibrary`
* Refactor `pwndbg.glibc`
- Add type hints
- Use `info sharedlibrary` to find libc
- Update the regex of libc filename
- Rename `get_data_address()` to `get_data_section_address()`
* Add a function to dump libc ELF file's .data section
* Use the new methods to find `main_arena` and `mp_`
With ELF of libc, we can use the default value of `main_arena` and `mp_` to find their address
* Drop some unreliable methods for the heap heuristics
* Update the tests for the heap heuristics
* Show `main_arena` address in the `arenas` command output
* Make the heap hueristics support statically linked targets
* Drop some deprecated TLS functions and refactor the command
- Drop some deprecated TLS functions for the deprecated heap heuristics
- Don't call `pthread_self()` in the `tls` command without `-p` option
- Show the page of TLS in the `tls` command output
* Update the hint for the heap heuristics for multi-threaded
* Fix the wrong usage of the exception
* Fix the outdated description
* Return the default global_max_fast when we cannot find the address
* Enhance the output of `arena` and `mp`
- Show the address of the arena we print in the output of `arena` command if we didn't specify the address by ourselves.
- Avoid the bug that `arena` command might get an error if thread_arena doesn't allocate yet.
- Show the address of `mp_` in the output of the `mp` command
* Remove wrong hint
* Support using brute-force to find the address of main_arena
If the user allows, brute-force the left and right sides of the TLS address to find the closest possible value to the TLS address.
* Refactor the code about thread_arena and add the new brute-force strategy
In the .got section, brute-force search for possible TLS-reference values to find possible thread_arena locations
* Add tests for thread_arena and global_max_fast
- Check if we can get default global_max_fast
- Check if we can use brute-force to find thread_arena
* Update the output of `arenas`
* Add the test for the `tls` command
Add two tests for the `tls` command:
```
test_tls_address_and_command[x86-64] PASSED
test_tls_address_and_command[i386] PASSED
```
* Update and refactor the heuristics for `thread_arena` and `tcache`
- We provide an option for users to brute force `tcache` like what we did for `thread_arena`
- Cache `thread_arena` even when we are single-threaded
- Refactor the code for `thread_arena`, to make it work for `tcache` as well
- Update the tests for `tcache`
- Remove some redundant hint
* Fix the wrong cache mechanism
Cache the address of the arena instead of the instance of `Arena`, because `Arena` will cache the value of the field, resulting in getting the old value the next time the same property is used
* Update the description of some configs about heap heuristics
* Handling the case when tcache is NULL
* Handling the case when thread_arena is NULL
* Fix a bug that occurred when the TLS address could not be found
* Fix#1550
* Show tid only if no address is specified
* Update pwndbg/commands/__init__.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Fix lint
* Move some code into `pwndbg.gdblib.elf`
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Fix plt and gotplt commands
* Add plt gotplt commands tests
* Fix got and plt commands and test them
* Revert accidental change
* Extend system path
* Hopefully fix PATH problems once and for all?
* fix import
* remove redundant part
* Enhance the checks before accessing the memory
- Use `pwndbg.gdblib.memory.peek()` instead of `pwndbg.gdblib.vmmap.find()` to check if the address is valid
- Directly access the memory when searching the `main_arena` in memory and catch the exception
* Make finding `main_arena` in memory more efficient and reliable
We only try the address that is aligned to `pwndbg.gdblib.arch.ptrsize`
* Avoid unnecessary memory accessing if possible
- Before we used `pwndbg.gdblib.memory.peek()` to check if an address is readable for GDB, we used `pwndbg.gdblib.vmmap.find()` to make sure that this address is in one of the pages, since accessing memory for embedded targets might be slow and expensive
- Create a new function: `is_readable_address` for `pwndbg.gdblib.memory`
* Fix wrong test for `main_arena`
The heap object should be reset before testing the multi-threaded condition
* Add the test to make sure the heap heuristics won't be affected by the vmmap result
Previously, we used `pwndbg.gdblib.vmmap.find()` to check whether the address is valid or not, but this might be a false positive for the address in the `[vsyscall]` page or in the page with a range from 0~0xffffffffffffffff (e.g. qemu-user).
This commit aims to include this scenario during the tests, to make sure the heap heuristics won't be affected by this.
* Use `gdb.MemoryError` instead of `Exception`
* Refactor TLS module
- Replace unreliable `__errno_location()` trick with `pthread_self()` to acquire TLS address
- Consolidate heap heuristics checks about TLS within the `pwndbg.gdblib.tls` module for better organization
* Bug fix for the `errno` command
Calling `__errno_location()` without locking the scheduler can cause another thread to inadvertently continue execution
* Refactor code about heap heuristics of thread-local variables
- Replace some checks with some functions in `pwndbg.gdblib.tls`
- Try to find tcache with `mp_.sbrk_base + 0x10` if the target is single-threaded
* Add tests for heap heuristics with multi-threaded
* Refacotr scheduler-locking related functions
- Move these functions into `pwndbg.gdblib.scheduler`
- Fetch the parameter value once (https://github.com/pwndbg/pwndbg/pull/1536#discussion_r1082549746)
* Avoid bug caused by GLIBC_TUNABLES
See https://github.com/pwndbg/pwndbg/pull/1536#discussion_r1083202815
* Add note about `set scheduler-locking on`
* Add comment for `lock_scheduler`
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Update DEVELOPING.md
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
patryk4815
Dejan
patryk4815
Aaron Adams
Aadish Jain
jetchirag
Disconnect3d
Matt.
charif
OBarronCS
peace-maker
Charles Fol
Will
Jason An
Costinteo
k4lizen
Gulshan Singh
Lennard Hofmann
Piotr Zarycki
CptGibbon
jetchirag
Aryaman Sharma
Alan Li
Albert Gierlach
Hubert Jasudowicz
keymoon
intrigus-lgtm
Aaron Adams
duckl1ng
smiley
ntsleep
duckl1ng
SlidyBat
peace-maker
theguy147
dan "smiley" murray
Unknown Sentinel
Albert Koczy
neryaz