* Modify python test scripts to work from nix
* Update utils.py
* address review feedback
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Only look for readable address in retaddr command
* Rename stack.py to retaddr.py
* Add pwndbg.gdblib.stack.callstack and use it in retaddr
* Add callstack gdb test
* Add QEMU callstack test
* V1 - annotations for values of registers and memory to display result of instructions.
* Emulator telescope(), more x86 instructions
* Emulation change - keep track of before & after instruction execution. Telescope format correctly, read size taken into account
* Add config options to configure emulation and annotations, vmovaps alignment warning, string length in disasm telescope, cache previously annotated instructions
* Create PwndbgInstruction type for better typing and easier future development
* More consistent spacing, options to disable annotations, ADD instruction shows operands
* Rebase from dev
* Correctly go to .next address in disasm view (was incorrectly going to call targets)
* Precompute register str to reduce code duplication
* Correct telescope memory read width, bring target printing back to previous behavior when symbol can be resolved
* More consistent looking annotation spacing/padding, fixed edge case bugs with cached instructions
* Even cleaner padding
* Additional comments and debugging, ironed out last bugs
* debugging tight loops
* Cache fixed - nearpc only annotates what can be statically resolved
* lint and show instructions that cannot dereference
* Reapply btrace crash fix after rebase
* Less code duplication, implement XCHG and AND instructions, moved more methods from x86 subclass to superclass
* aarch64 set flags register in Unicorn correctly
* fix
* Don't recreate emulator regname->const map every time the emulator is instantiated
* Use emulation to set .next within enhancement
* Improve ret instruction target address setting
* Green checkmarks for jumps on all architectures
* Fixed .next and .target setting
* All architectures now have correct logic for determining .next and .target. Green checkmarks for taking conditional branches now appear for all architectures, added things to determine that type of branch being taken, and simplified printing by replacing symbol_addr with new field 'target_string'
* Instruction debug print fix
* Correct jump instruction checking
* Fixed target_string resolution
* Fix conditional jump check, also make default target resolution better
* target_const determined more accurately
* reverse iteration order of last change
* Pwndbg.condition is more retyped and more correct, make manual determinations of condition override the emulators (it can be incorrect in cases). Uncover why MIPS sometimes takes incorrect jumps in the emulator (delay slot)
* MIPS annotations work really good now. Jumps are correctly predicted (with green checkmarks). Implemented manual condition() function for MIPS. Only highlight the correct instance of instruction at PC when there are multiple in view (tight loops). Allow manual .condition to override emulator in determiningg .next.
* Additional debug info on instructions
* Print arch in instruction
* aarch64 branch fix
* aarch64 branch fix (real)
* lint
* Final changes - fixing .size error
* lint
* Add dev_dump_instruction command, add default memory read in resolve_used_value, update comments and remove .size from enhancedoperand as it only exists on x86
* More uniform spacing on annotations
* Various comments converted to docstrings, aarch64 enhancer created, post-rebase
* import aarch64
* Aarch64 mov, ldr, add, sub
* adrp
* ADR
* lint
* Fun with git rebase
* lint
* lint again after re-installing dependencies
* New caching strategy implemented to ensure no state caching when jumping large distances. Handled edge cases of user manually setting a register or memory, 'set regname=2'
* lint
* Fixed two regressions (nearpc shouldn't take jumps, even ones we know statically, and replace all constants in the assembly with symbols). Tweak tests to reflect new annotations
* lint
* one last test
* Fix chain format dereferencing for non-singleton lists, now correctly deferences and displays chains for future instructions when not emulating (dereference until pointer goes to writable memory)
* Add jumps-only setting to emulation (on, off, jumps-only), fixes to chain deferencing and enhancing
* Properly dereference memory before and after execution of instructions, adding a new before_value_resolved field (same for after). This also reduces code duplication.
* Debogusify the format()/telescoping dereferencing logic
* lint
* post-rebase fixes
* Fix case the breaks a test - don't attempt to read larger than ptrsize such as in SIMD instruction memory reads
* Typo in emulate setting
* Developer docs for annotations
* Fix case where emulator attempts to read and unpack very large, 16 byte+ wide values while telescoping
* Add a helper command to find valid one_gadget for current context
* Refactor the function for getting section address
* Rename the command to onegadget for more convenient typing
* Make the output format cleaner
* Add a simple cache mechanism for the one_gadget output
* Update the warning message
* Use MD5 instead of BLAKE2 for computing the file hash
I thought that BLAKE2 was faster than MD5, but it doesn't seem correct here somehow (probably because of the implementation of Python!?)
Here's the script I used for benchmarking:
```python
import hashlib
import timeit
def compute_file_hash_1() -> str:
h = hashlib.blake2b()
with open("/lib/x86_64-linux-gnu/libc.so.6", "rb") as f:
h.update(f.read())
return h.hexdigest()
def compute_file_hash_2() -> str:
h = hashlib.md5()
with open("/lib/x86_64-linux-gnu/libc.so.6", "rb") as f:
h.update(f.read())
return h.hexdigest()
print(timeit.timeit(compute_file_hash_1, number=1000))
print(timeit.timeit(compute_file_hash_2, number=1000))
```
I executed the above script on various machines, and the results seem to show that MD5 outperforms BLAKE2 in this scenario. (On my x86 VM running through QEMU on my M1 MacBook, BLAKE2 even takes almost twice as long as MD5.)
* Add the tests for `onegadget` command
* Fix lint issue
* Try to cover more code
* Fix lint issue
* Fix illogical tests
* Rename one_gadget to onegadget
* Use `pwndbg.lib.tempfile.cachedir` for `onegadget`
* Call `pwndbg.lib.tempfile.cachedir` only once
* Add support for breaking on UAF
* Small fixes and documentation
* Add a command to enable and disable tracking, better diagnostics
* Add initial support for calloc and realloc
* Better safeguard against matching ld.so malloc
* Small fixes
* Better interface for managing the heap tracker. More terse and information dense diagnostics
* Add warning and fix lints
* Update poetry lock
Hopefully fixes#1947 by fetching stacks only when they are used instead
of doing it on each stop event. It will also first try to compute stacks
dictionary based on vmmap and if it fallbacks to exploring stacks if
vmmap is not present.
* [WIP] Port gdb-tests from bash to python
* Use threads instead of processes
* Port gdb tests to python
* Linting
* Fix coverage "again"
* Remove bash tests
---------
Co-authored-by: intrigus <abc123zeus@live.de>
* get_one_instruction: clear "cont" cache on mem/reg changed
Fixes#1818.
Note that this makes a substantial change: it changes all caches that
are refreshed on `gdb.ContinueEvent` to also be cleared on memory/regs
changed.
This change is needed so that the `get_one_instruction` function which
uses this cache will get its cache cleared when user invokes a command
that changes memory or registers.
While this may sound as too big change: we are changing the whole "cont"
cache to be cleared on two additional events, this should not be an
issue. This is because:
1. We should notice it if we start clearing an important cache too often
2. The "cont" cache is currently only used by the `get_one_instruction`
at this moment.
The 2) also creates a question: when should one use "cont" vs "start"
caches? It is not so clear to me right now.
* Add test for issue #1818
* Clear caches on MemoryChanged events from gdblib.write
Regarding the last part:
Interestingly implementing tests here uncovered another bug: the gdblib.memory.write(..) or rather the gdb.selected_inferior().write_memory(...) API used there does not trigger a gdb.MemoryChanged event. As a result, we never cleared certain caches that should have been cleared when the user used that API.
I have added two tests here, one changes the instructions at $RIP to nops via gdblib.memory.write(..) and another via executing the patch $rip nop;nop;nop;nop;nop command. As a result, we test both scenarios: 1) when we depend on memory changed event being fired via GDB to clear caches; and 2) when we depend on gdblib.memory.write(..) to clear the caches.
This PR also makes a fix to the gdblib.memory.write(..) to actually clear caches that depend on (or rather: are hooked to in order to be cleared) memory changed events.
* Fix glibc-fastbin-bug option of find_fake_fast
Using the find_fake_fast option --glibc-fastbin-bug always resulted in an error, at least on 64-bit platforms.
This was because the option caused only 4 bytes to be read for the size, but then that gets passed to unpack() which expects 8 bytes.
Closes#1773
* Address review comment
* Update arch.py
* Update pwndbg/commands/heap.py
* Fix lint
* Update arch.py
* Update arch.py
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
This commit adds a command that traverses the linked list beginning at a given
element, dumping its contents and the contents of all the elements that come
after it in the list. Traversal is configurable and can handle multiple types
of chains.
This commit adds the `break-if-taken` and `break-if-not-taken` commands,
which attach breakpoints to branch instructions that will stop the
inferior if said branch is taken or is not taken, respectively. It adds
an extra class, `pwndbg.gdblib.bpoint.Breakpoint`, which clears caches
before calling `stop()`, allowing for the use of register values inside
that function in breakpoint classes that derive from it. Additionally,
checking of whether the conditions for a branch to be taken have been
fulfilled is done through `DisassemblyAssistant.condition()`.
* Add `stepuntilasm` command
This commit adds a `stepuntilasm` command that, given a mnemonic and,
optionally, a set of operands, will step until a instruction that
matches both is found. Matching is string-based, as the user will likely
want to spell out the asm directive they want as text, and interpreting
assembly language conventions for all of the platforms pwndbg supports
is probably outside the scope of this change.
* next.py: small code cleanup
* next.py: fix bug introduced in previous commit
op.str -> op_str
* Update next.py
* Update next.py
* Update next.py
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Only run arch for testing
* Remove outdated arch repo
* Actually build the docker image
* Do not include site packages in sys.path
* Ignore `.relr.dyn` section; skip lines w/o spaces
Newer binaries can contain a `.relr.dyn` section to compress `R_X86_64_RELATIVE` relocation entries.
These binaries can be found for example on archlinux but also on Debian 12 for example.
`readelf` prints the content of the section similarly to this:
```
Relocation section '.relr.dyn' at offset 0x25220 contains 35 entries:
1198 offsets
00000000001ce8d0
00000000001ce8e0
```
Compared to `00000000001d2000 0000000000000025 R_X86_64_IRELATIVE 9f330` for
`.rela.plt`.
Pwndbg now chokes on the new format because it expects a space seperator where there is none.
It might be, that this is actually an upstream problem with binutils, because llvm-readelf prints this:
```
Relocation section '.relr.dyn' at offset 0x25220 contains 1198 entries:
Offset Info Type Symbol's Value Symbol's Name
00000000001ce8d0 0000000000000008 R_X86_64_RELATIVE
00000000001ce8e0 0000000000000008 R_X86_64_RELATIVE
```
Nevertheless, we aren't actually interested in `R_X86_64_RELATIVE` relocations so I guess it's fine to
just skip all lines that contain no spaces at all.
`.relr.dyn` can only containt `R_X86_64_RELATIVE` relocations as far as I understand
https://maskray.me/blog/2021-10-30-relative-relocations-and-relr
* Accept Full RELRO in test
Archlinux has libc and ld with Full RELRO.
We now just accept Partial and Full RELRO.
* Do not copy binaries from host to docker
The `Dockerfile` copies the whole pwndbg folder to the image.
If we have built binaries on the host before, these binaries will contain references to
the host system and *copied* to the image.
If we now run `context code` (inside docker) to have a look at the source code this will
fail, because we will try to refer to a path on the host system.
* Do not use loop index after loop
Do not use loop index after the loop. The tests assumed that the loop in line 186
would run at least once, thereby *resetting* `i` to zero. If we never enter the
loop, `i` will *continue* to have the value it had at the end of line 172.
This will cause the test to fail in mysterious ways because `i` is now not reset
to zero but still has the value `31` for example.
The solution is to never use `i` outside of a loop.
* Re-enable archlinux and temporarily disabled ones
* Fix coverage combine toml issue
This commit should fix this issue:
```
Run coverage combine
coverage combine
coverage xml
shell: /usr/bin/bash -e {0}
Can't read 'pyproject.toml' without TOML support. Install with [toml] extra
Error: Process completed with exit code 1.
```
* setup.sh: cleanup the --user flag since we use venv now
Cleans up the --user flag from setup.sh since it is unused after we changed setup.sh to install Python dependencies in a virtual environment
* Remove --user flag from CI workflows
* Fix codecov problem
We need to run the python `coverage` library to collect coverage.
However, gdb was failing to find it.
Recently, pwndbg moved to using venvs. When pwndbg is initialized
it setups the venv "manually", that is, no "source .venv/bin/activate"
is needed. When we run gdb tests, we pass the `gdbinit.py` of pwndbg as a
command to gdb to be executed like this:
`gdb --silent --nx --nh -ex 'py import coverage;coverage.process_startup()' --command PATH_TO_gdbinit.py`
The problem is that *order* matters. This means that *first* coverage
is imported (by `-ex py ...`) and only *then* the init script is executed.
When `coverage` is first imported, it's library search path only looks
in system libraries of python, and not the venv that gdbinit.py would load.
So we would try to import an old version of coverage and fail.
One solution would be to move around the commands, but this would be an
ugly hack IMHO. **Instead**, we should just tell gdb that this is an **init**
command that has to be executed before other commands.
Previously, the order did not matter. All of pwndbg's dependencies were
installed directly as system libraries to python. So the library search path
was the same before and after loading `gdbinit.py`.
---------
Co-authored-by: disconnect3d <dominik.b.czarnota@gmail.com>
Co-authored-by: intrigus <abc123zeus@live.de>
* Refactor the `got` command to support more use cases
- Create some function to parse the information of loaded shared object libraries from `info sharedlibrary`
- Make got command can show the entries of other libraries loaded in memory
- Make got command can show more various relocations to support not only the `JUMP_SLOT` type relocation but also supports `IRELATIVE` and `GLOB_DAT` type relocation.
* Update tests for the `got` command
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update pwndbg/commands/got.py
* Update the comment
https://github.com/pwndbg/pwndbg/pull/1771#discussion_r1251054080
* Update the tests
* Add some hints for the qemu users
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
Aaron Adams
Piotr Zarycki
Gulshan Singh
CptGibbon
jetchirag
OBarronCS
Aryaman Sharma
Alan Li
Matt
Albert Gierlach
Hubert Jasudowicz
Disconnect3d
keymoon
intrigus-lgtm
duckl1ng
Aaron Adams
smiley
ntsleep
duckl1ng
SlidyBat