* Refactor TLS module
- Replace unreliable `__errno_location()` trick with `pthread_self()` to acquire TLS address
- Consolidate heap heuristics checks about TLS within the `pwndbg.gdblib.tls` module for better organization
* Bug fix for the `errno` command
Calling `__errno_location()` without locking the scheduler can cause another thread to inadvertently continue execution
* Refactor code about heap heuristics of thread-local variables
- Replace some checks with some functions in `pwndbg.gdblib.tls`
- Try to find tcache with `mp_.sbrk_base + 0x10` if the target is single-threaded
* Add tests for heap heuristics with multi-threaded
* Refacotr scheduler-locking related functions
- Move these functions into `pwndbg.gdblib.scheduler`
- Fetch the parameter value once (https://github.com/pwndbg/pwndbg/pull/1536#discussion_r1082549746)
* Avoid bug caused by GLIBC_TUNABLES
See https://github.com/pwndbg/pwndbg/pull/1536#discussion_r1083202815
* Add note about `set scheduler-locking on`
* Add comment for `lock_scheduler`
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
* Update DEVELOPING.md
Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>
- Update the docs of the config: `kernel-vmmap`, `hexdump-group-use-big-endian`, `kernel_vmmap_via_pt`, and `resolve-heap-via-heuristic`
- Update the output of `get_show_string()` to display: ``See `help set <config>` for more information`` in the end of the output of `show <config>`
- Modify `get_set_string()` to match GDB's builtin behaviour
- Make `gcc-compiler-path`'s and `cymbol-editor`'s `set_show_doc` first strings to lowercase
- Change `gcc-compiler-path` and `cymbol-editor` to `gdb.PARAM_OPTIONAL_FILENAME`
- Add resolve_heap_via_heuristic as a gdb.PARAM_ENUM config with options:
- auto: pwndbg will try to use heuristics if debug symbols are missing
- force: pwndbg will always try to use heuristics, even if debug symbols are available
- never: pwndbg will never use heuristics to resolve the heap
- Move some hints to `resolve_heap_via_heuristic`'s `help_docstring`
* Fix Arch CI: install missing netcat (#1400)
The arch linux test_command_procinfo was failing bcoz the netcat was not
installed on arch build. This commit fixes it by:
1) installing gnu-netcat for arch linux setup-dev.sh
2) asserting that nc is available in the test itself, to prevent similar
regressions from happening on future/newer images
* Fix Arch CI: the load binary tests (#1400)
Before this commit we asserted whether the loaded binary in tests report
to find or not find debug symbols but this is irrelevant for the thing
we want to test there which is: pwndbg loading. What eventually cares is
whether Pwndbg got loaded and didn't raise an exception.
This commit fixes those tests so they should now work also on ArchLinux
CI and on all CI builds.
Additionally, it removes the `compile_binary` test utility function
which was redundant as we compile all test binaries via a makefile.
* fix lint
* cleanup tests/binaries/div_zero_binary
The cymbol command did not work on old GDB versions like 8.2 because
they require the ADDR argument to be passed into the `add-symbol-file`
command unlike newer GDB versions in which the argument is optional.
This can be seen below.
```
pwndbg> help add-symbol-file
Load symbols from FILE, assuming FILE has been dynamically loaded.
Usage: add-symbol-file FILE ADDR [-readnow | -readnever | -s SECT-NAME SECT-ADDR]...
ADDR is the starting address of the file's text.
Each '-s' argument provides a section name and address, and
should be specified if the data and bss segments are not contiguous
with the text. SECT-NAME is a section name to be loaded at SECT-ADDR.
The '-readnow' option will cause GDB to read the entire symbol file
immediately. This makes the command slower, but may make future operations
faster.
The '-readnever' option will prevent GDB from reading the symbol file's
symbolic debug information.
pwndbg> version
Gdb: 8.1.1
Python: 3.6.9 (default, Jun 29 2022, 11:45:57) [GCC 8.4.0]
Pwndbg: 1.1.1 build: c5d8800
Capstone: 4.0.1024
Unicorn: 2.0.7
```
vs
```
pwndbg> help add-symbol-file
Load symbols from FILE, assuming FILE has been dynamically loaded.
Usage: add-symbol-file FILE [-readnow | -readnever] [-o OFF] [ADDR] [-s SECT-NAME SECT-ADDR]...
ADDR is the starting address of the file's text.
Each '-s' argument provides a section name and address, and
should be specified if the data and bss segments are not contiguous
with the text. SECT-NAME is a section name to be loaded at SECT-ADDR.
OFF is an optional offset which is added to the default load addresses
of all sections for which no other address was specified.
The '-readnow' option will cause GDB to read the entire symbol file
immediately. This makes the command slower, but may make future operations
faster.
The '-readnever' option will prevent GDB from reading the symbol file's
symbolic debug information.
pwndbg> version
Gdb: 12.1
Python: 3.10.6 (main, Nov 2 2022, 18:53:38) [GCC 11.3.0]
Pwndbg: 1.1.1 build: c5d8800
Capstone: 4.0.1024
Unicorn: 2.0.0
pwndbg>
```
When we optimized tests runs with gnu parallel execution, we broke the
--pdb flag. This commit fixes it and sets the SERIAL flag so that tests
are run one by one when --pdb is passed.
* fix shlint
* Fix crash when unable to get ehdr and fix vmmap coredump test
This commit fixes two issues and test them.
1. It changes the reads in `get_ehdr` to partial reads so that inability
to read the `vmmap.start` address there will not crash Pwndbg with
`gdb.error` but instead we will simply return `None` as expected from
this function. This crash could happen on Debian 10 (GDB 8.2.1) and
Ubuntu 18.04 (not sure which GDB) when you did:
- gdb ./binary-that-crashes
- `run`
- `generate-core-file /tmp/core`
- `file` - to unload the binary
- `core-file /tmp/core` - to load the generated core
At this point I think we may have preserved the old vmmap info and use
it in `get_ehdr` maybe, which then crashed? I am not sure, but this fix
here works.
To test this behavior properly I also added the `unload_file`
parametrization to the
`test_command_vmmap_on_coredump_on_crash_simple_binary` test.
2. We fix the vmmap coredump test case when the `info proc mappings` returns nothing on core
dumps on old GDBs. In such case we are missing the vmmap info about
the binary mapping, so now we properly remove it in the test.
This fixes the weird error that appeared on debian10 CI:
```
root@98cc3841eab9:/pwndbg/tests/gdb-tests/tests/binaries# ld -Ttext 0x400000 -o memory.out memory.o
ld: section .note.gnu.property LMA [00000000004000e8,0000000000400107] overlaps section .text LMA [0000000000400000,00000000004001a4]
```
It turned out that the .note.gnu.property address was choosen to be the
same as our hardcoded .text address and so we got into this issue.
This PR hardcodes the gnu section address.
* Fix tests reporting in parallel execution
Fix issue where parallel test execution was unable to track failed tests and inform about their number.
* Fix logic in tests.sh
* Add get_sbrk_heap_region() method
* Use SIZE_BITS in Chunk.real_size()
* Add non_contiguous property to Arena class
* Improve Heap class
* More accurate arena detection
* Integrate Heap class into Chunk class
* Don't parse bins when no arena in find_fake_fast
* Add active_heap property to Arena class
* Add more functionality to heap classes
* next_chunk method for Chunk class
* prev property & __str__ method for Heap class
* heaps property for Arena class
* arenas command updated to reflect changes to Arena class
* Use deepcopy() in get_region() to avoid changing vmmap command output
* Import fiddling to deal with unrelated bug
* Attempt at integration with heap commands
With debug syms looks good, still issues to iron out with heuristics
* Remove redundant heap functions
* Remove redundant functions from tests
* Add system_mem property to Arena class
* thread_arena returns main_arena if single thread
* Fix some issues for GDB < 9.x
* GDB < 9.x doesn't have `gdb.lookup_static_symbol`
* GDB < 9.x doesn't have `gdb.PARAM_ZUINTEGER_UNLIMITED`
* Better error handling for the heap commands
* Inform users to `set exception-* on` when they encounter some error during using some heap commands
* Bug fix for heap region finding of `HeuristicHeap`
* Before this commit, `get_heap_boundaries()` of `HeuristicHeap` will always return the page whose name is `[heap]`, this won't work for multithreaded cases and won't work if the heap region of the main thread is not `[heap]` (e.g., when using QEMU, sometimes the name of heap region is something like: `[anon_deadbeaf]`)
* Fallback to `gdb.lookup_symbol` if we do not have `gdb.lookup_static_symbol`
* Add more features for `pwndbg.gdblib.config`
* Support all parameter-class
* Use `get_show_string` to render better output when using `show <param>`
* Show more information when using `help set <param>` and `help show <param>` if we create a config with `help_docstring` parameter.
Some examples of the updates included in this commit:
1. `gdb.PARAM_AUTO_BOOLEAN` with `help_docstring`
In Python script:
```
pwndbg.gdblib.config.add_param(
"test",
None,
"test",
"on == AAAA\noff == BBBB\nauto == CCCC",
gdb.PARAM_AUTO_BOOLEAN,
scope="test",
)
```
In GDB:
```
pwndbg> show test
The current value of 'test' is 'auto'
pwndbg> set test on
Set test to 'on'
pwndbg> set test off
Set test to 'off'
pwndbg> set test auto_with_typo
"on", "off" or "auto" expected.
pwndbg> show test
The current value of 'test' is 'off'
pwndbg> set test auto
Set test to 'auto'
pwndbg> show test
The current value of 'test' is 'auto'
pwndbg> help show test
Show test
on == AAAA
off == BBBB
auto == CCCC
pwndbg> help set test
Set test
on == AAAA
off == BBBB
auto == CCCC
```
2. `gdb.PARAM_AUTO_BOOLEAN` with `help_docstring`
In Python script:
```
pwndbg.gdblib.config.add_param(
"test",
"A",
"test",
"A == AAAA\nB == BBBB\nC == CCCC",
gdb.PARAM_ENUM,
["A", "B", "C"],
scope="test",
)
```
In GDB:
```
pwndbg> show test
The current value of 'test' is 'A'
pwndbg> set test B
Set test to 'B'
pwndbg> set test C
Set test to 'C'
pwndbg> set test D
Undefined item: "D".
pwndbg> show test
The current value of 'test' is 'C'
pwndbg> help show test
Show test
A == AAAA
B == BBBB
C == CCCC
pwndbg> help set test
Set test
A == AAAA
B == BBBB
C == CCCC
```
* Update the tests for gdblib parameter
* Use auto boolean for `safe-linking`
* Fix some comments
* Pass `help_docstring` directly
* Force callers of `add_param` to use keyword arguments
* Create `add_heap_param()` to avoid setting the scope of param everytime
* Add a header to the vmmap table
A simple header has been added to the output of vmmap which helps new users identify the columns.
* fix: lint
* fix: failing test
Adjust the length of expected vmmaps
* fix: tests again
Alan Li
Disconnect3d
Gulshan Singh
CptGibbon
Lonny Wong
Szymon Borecki
George Dhmosxakhs
Tingfeng Yu
alufers