* fix: remove minor bugs and complete address translation
* feat: add 5lvl paging
* feat: add address translation tests
* fix: remove unnecessary comments
* fix: add references for magic values
* fix: add X86_FEATURE_LA57 reference
* fix: move x86 specific functions to x86_64Ops
* fix: extend tests and remove faulty code
* fix: only test address translation for lowmem
* fix: adjust arch_ops test to pytest
* fix: add reference for memory models in linux
* fix: do not memoize staticmethods
* Fix and test ctx disasm when disassembly-flavor changes
* New lib/cache.py: make caching great again
This commit fixes bugs with old caching (memoize.py) and makes it more
readable.
See also https://github.com/pwndbg/pwndbg/issues/1453
* Update pwndbg/lib/cache.py
Co-authored-by: Gulshan Singh <gsingh2011@gmail.com>
* lib.cache: address PR comments and add debug mode
* Fix lint
* Remove leftover memoize usages
* Add cache benchmark
* fix lint
---------
Co-authored-by: Gulshan Singh <gsingh2011@gmail.com>
* feature: Add `killthreads` command (closes#1580)
This command allows the user to quickly kill multiple threads by
specyfying their ids as arguments to this command. It also supports
the `--all` flag, which will kill overy thread except the currently
selected one. This is useful for use with the `checkpoint` command.
The killing is done by calling `pthread_exit(0)`.
* fix: try fixing building test binaries by enabling -lpthread
* fix: remove error message check when calling pthread_exit
Removed the message check, because the error messages difffer between
versions of GDB.
* fix: Improve UX of the killthreads command
Add an extended description of the command, some validation on the thread IDs
and suppress GDB output.
* fix: lint
* fix: put the multiline help text in the correct place
* tests: fix test failing due to a race condition when running in parrallel to other tests
Replaced asserts with loops which wait for a cundition to be met, so that the tests doesn't fail due to scheduling issues.
* tests: add more fixes for race conditions in test_killthreads
* fix: lint
* Update pwndbg/commands/killthreads.py
* tests: Wait for exactly three threads
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Changing the arguments to vis_heap_chunks to be clearer
1. --native to --beyond_top
2. --display_all to --no_truncate
* Add print all chunks to vis_heap_chunks
* Preventing the use of the all_chunks argument together with the count argument in vis_heap_chunks
* Use linting for heap.py
* Fix test_vis_heap_chunks.py
According to cdd71a1d82 --display_all/-d moved to --no_truncate/-n
---------
Co-authored-by: Nerya Zadkani <nerya@tokagroup.com>
This commit adds a fix and tests for #1600 and #752.
* https://github.com/pwndbg/pwndbg/issues/1600
* https://github.com/pwndbg/pwndbg/issues/752
Generally, for an example like this:
```cpp
struct A {
void foo(int, int) { };
};
int main() {
A a;
a.foo(1, 1);
}
```
The output for `info symbol <address of A::foo>` returns:
```
'A::foo(int, int) [clone.isra.0] + 3 in section .text of /root/pwndbg/tests/gdb-tests/tests/binaries/a.out\n'
```
We then used this code to parse this:
```py
# Expected format looks like this:
# main in section .text of /bin/bash
# main + 3 in section .text of /bin/bash
# system + 1 in section .text of /lib/x86_64-linux-gnu/libc.so.6
# No symbol matches system-1.
a, b, c, _ = result.split(maxsplit=3)
if b == "+":
return "%s+%s" % (a, c)
if b == "in":
return a
return ""
```
The `result.split(maxsplit=3)` here splitted the string to:
```py
['A::foo(int,',
'int)',
'[clone.isra.0] + 3 in section .text of /root/pwndbg/tests/gdb-tests/tests/binaries/a.out\n']
```
And since `b` was not `"+"` or `"in"` we eventually returned an empty
string instead of the `A::foo(int, int)` which would be expected here.
* Fix the bug when using LD_PRELOAD to load libc
The heap heuristics will try to find `libc.so.6` in the output of `info sharedlibrary`, but if we load libc with `LD_PRELOAD`, the filename of the libc might not be `libc.so.6`.
* Add test for `glibc.get_libc_filename_from_info_sharedlibrary`
* Refactor `pwndbg.glibc`
- Add type hints
- Use `info sharedlibrary` to find libc
- Update the regex of libc filename
- Rename `get_data_address()` to `get_data_section_address()`
* Add a function to dump libc ELF file's .data section
* Use the new methods to find `main_arena` and `mp_`
With ELF of libc, we can use the default value of `main_arena` and `mp_` to find their address
* Drop some unreliable methods for the heap heuristics
* Update the tests for the heap heuristics
* Show `main_arena` address in the `arenas` command output
* Make the heap hueristics support statically linked targets
* Drop some deprecated TLS functions and refactor the command
- Drop some deprecated TLS functions for the deprecated heap heuristics
- Don't call `pthread_self()` in the `tls` command without `-p` option
- Show the page of TLS in the `tls` command output
* Update the hint for the heap heuristics for multi-threaded
* Fix the wrong usage of the exception
* Fix the outdated description
* Return the default global_max_fast when we cannot find the address
* Enhance the output of `arena` and `mp`
- Show the address of the arena we print in the output of `arena` command if we didn't specify the address by ourselves.
- Avoid the bug that `arena` command might get an error if thread_arena doesn't allocate yet.
- Show the address of `mp_` in the output of the `mp` command
* Remove wrong hint
* Support using brute-force to find the address of main_arena
If the user allows, brute-force the left and right sides of the TLS address to find the closest possible value to the TLS address.
* Refactor the code about thread_arena and add the new brute-force strategy
In the .got section, brute-force search for possible TLS-reference values to find possible thread_arena locations
* Add tests for thread_arena and global_max_fast
- Check if we can get default global_max_fast
- Check if we can use brute-force to find thread_arena
* Update the output of `arenas`
* Add the test for the `tls` command
Add two tests for the `tls` command:
```
test_tls_address_and_command[x86-64] PASSED
test_tls_address_and_command[i386] PASSED
```
* Update and refactor the heuristics for `thread_arena` and `tcache`
- We provide an option for users to brute force `tcache` like what we did for `thread_arena`
- Cache `thread_arena` even when we are single-threaded
- Refactor the code for `thread_arena`, to make it work for `tcache` as well
- Update the tests for `tcache`
- Remove some redundant hint
* Fix the wrong cache mechanism
Cache the address of the arena instead of the instance of `Arena`, because `Arena` will cache the value of the field, resulting in getting the old value the next time the same property is used
* Update the description of some configs about heap heuristics
* Handling the case when tcache is NULL
* Handling the case when thread_arena is NULL
* Fix a bug that occurred when the TLS address could not be found
* Fix#1550
* Show tid only if no address is specified
* Update pwndbg/commands/__init__.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
* Update pwndbg/commands/heap.py
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Fix lint
* Move some code into `pwndbg.gdblib.elf`
---------
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Fix plt and gotplt commands
* Add plt gotplt commands tests
* Fix got and plt commands and test them
* Revert accidental change
* Extend system path
* Hopefully fix PATH problems once and for all?
* fix import
* remove redundant part
* Enhance the checks before accessing the memory
- Use `pwndbg.gdblib.memory.peek()` instead of `pwndbg.gdblib.vmmap.find()` to check if the address is valid
- Directly access the memory when searching the `main_arena` in memory and catch the exception
* Make finding `main_arena` in memory more efficient and reliable
We only try the address that is aligned to `pwndbg.gdblib.arch.ptrsize`
* Avoid unnecessary memory accessing if possible
- Before we used `pwndbg.gdblib.memory.peek()` to check if an address is readable for GDB, we used `pwndbg.gdblib.vmmap.find()` to make sure that this address is in one of the pages, since accessing memory for embedded targets might be slow and expensive
- Create a new function: `is_readable_address` for `pwndbg.gdblib.memory`
* Fix wrong test for `main_arena`
The heap object should be reset before testing the multi-threaded condition
* Add the test to make sure the heap heuristics won't be affected by the vmmap result
Previously, we used `pwndbg.gdblib.vmmap.find()` to check whether the address is valid or not, but this might be a false positive for the address in the `[vsyscall]` page or in the page with a range from 0~0xffffffffffffffff (e.g. qemu-user).
This commit aims to include this scenario during the tests, to make sure the heap heuristics won't be affected by this.
* Use `gdb.MemoryError` instead of `Exception`
* Refactor TLS module
- Replace unreliable `__errno_location()` trick with `pthread_self()` to acquire TLS address
- Consolidate heap heuristics checks about TLS within the `pwndbg.gdblib.tls` module for better organization
* Bug fix for the `errno` command
Calling `__errno_location()` without locking the scheduler can cause another thread to inadvertently continue execution
* Refactor code about heap heuristics of thread-local variables
- Replace some checks with some functions in `pwndbg.gdblib.tls`
- Try to find tcache with `mp_.sbrk_base + 0x10` if the target is single-threaded
* Add tests for heap heuristics with multi-threaded
* Refacotr scheduler-locking related functions
- Move these functions into `pwndbg.gdblib.scheduler`
- Fetch the parameter value once (https://github.com/pwndbg/pwndbg/pull/1536#discussion_r1082549746)
* Avoid bug caused by GLIBC_TUNABLES
See https://github.com/pwndbg/pwndbg/pull/1536#discussion_r1083202815
* Add note about `set scheduler-locking on`
* Add comment for `lock_scheduler`
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Update DEVELOPING.md
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
- Update the docs of the config: `kernel-vmmap`, `hexdump-group-use-big-endian`, `kernel_vmmap_via_pt`, and `resolve-heap-via-heuristic`
- Update the output of `get_show_string()` to display: ``See `help set <config>` for more information`` in the end of the output of `show <config>`
- Modify `get_set_string()` to match GDB's builtin behaviour
- Make `gcc-compiler-path`'s and `cymbol-editor`'s `set_show_doc` first strings to lowercase
- Change `gcc-compiler-path` and `cymbol-editor` to `gdb.PARAM_OPTIONAL_FILENAME`
- Add resolve_heap_via_heuristic as a gdb.PARAM_ENUM config with options:
- auto: pwndbg will try to use heuristics if debug symbols are missing
- force: pwndbg will always try to use heuristics, even if debug symbols are available
- never: pwndbg will never use heuristics to resolve the heap
- Move some hints to `resolve_heap_via_heuristic`'s `help_docstring`
* Fix Arch CI: install missing netcat (#1400)
The arch linux test_command_procinfo was failing bcoz the netcat was not
installed on arch build. This commit fixes it by:
1) installing gnu-netcat for arch linux setup-dev.sh
2) asserting that nc is available in the test itself, to prevent similar
regressions from happening on future/newer images
* Fix Arch CI: the load binary tests (#1400)
Before this commit we asserted whether the loaded binary in tests report
to find or not find debug symbols but this is irrelevant for the thing
we want to test there which is: pwndbg loading. What eventually cares is
whether Pwndbg got loaded and didn't raise an exception.
This commit fixes those tests so they should now work also on ArchLinux
CI and on all CI builds.
Additionally, it removes the `compile_binary` test utility function
which was redundant as we compile all test binaries via a makefile.
* fix lint
* cleanup tests/binaries/div_zero_binary
The cymbol command did not work on old GDB versions like 8.2 because
they require the ADDR argument to be passed into the `add-symbol-file`
command unlike newer GDB versions in which the argument is optional.
This can be seen below.
```
pwndbg> help add-symbol-file
Load symbols from FILE, assuming FILE has been dynamically loaded.
Usage: add-symbol-file FILE ADDR [-readnow | -readnever | -s SECT-NAME SECT-ADDR]...
ADDR is the starting address of the file's text.
Each '-s' argument provides a section name and address, and
should be specified if the data and bss segments are not contiguous
with the text. SECT-NAME is a section name to be loaded at SECT-ADDR.
The '-readnow' option will cause GDB to read the entire symbol file
immediately. This makes the command slower, but may make future operations
faster.
The '-readnever' option will prevent GDB from reading the symbol file's
symbolic debug information.
pwndbg> version
Gdb: 8.1.1
Python: 3.6.9 (default, Jun 29 2022, 11:45:57) [GCC 8.4.0]
Pwndbg: 1.1.1 build: c5d8800
Capstone: 4.0.1024
Unicorn: 2.0.7
```
vs
```
pwndbg> help add-symbol-file
Load symbols from FILE, assuming FILE has been dynamically loaded.
Usage: add-symbol-file FILE [-readnow | -readnever] [-o OFF] [ADDR] [-s SECT-NAME SECT-ADDR]...
ADDR is the starting address of the file's text.
Each '-s' argument provides a section name and address, and
should be specified if the data and bss segments are not contiguous
with the text. SECT-NAME is a section name to be loaded at SECT-ADDR.
OFF is an optional offset which is added to the default load addresses
of all sections for which no other address was specified.
The '-readnow' option will cause GDB to read the entire symbol file
immediately. This makes the command slower, but may make future operations
faster.
The '-readnever' option will prevent GDB from reading the symbol file's
symbolic debug information.
pwndbg> version
Gdb: 12.1
Python: 3.10.6 (main, Nov 2 2022, 18:53:38) [GCC 11.3.0]
Pwndbg: 1.1.1 build: c5d8800
Capstone: 4.0.1024
Unicorn: 2.0.0
pwndbg>
```
When we optimized tests runs with gnu parallel execution, we broke the
--pdb flag. This commit fixes it and sets the SERIAL flag so that tests
are run one by one when --pdb is passed.
Disconnect3d
theguy147
Gulshan Singh
Albert Koczy
CptGibbon
neryaz
Alan Li