The problem was that after some of the recent changes to chain/get to prevent dereferencing too much addresses and having better display when dereferencing limit is 0 (used for bare metal debugging) the bins command displayed wrong results for everything except fastbins.
This was due to the fact we are adding the dereference start address to the list.
This fixes the `bins` command by adding `include_start=True` keyword argument to the `chain.get` function. The `bins` simply uses `include_start=False`.
Bins command fails on a libc that doesn't use tcache at all, e.g.:
```
GNU C Library (Ubuntu GLIBC 2.23-0ubuntu10) stable release version 2.23, by Roland McGrath et al.
```
Here is the output:
```
pwndbg> bins
Traceback (most recent call last):
File "/root/pwndbg/pwndbg/commands/__init__.py", line 109, in __call__
return self.function(*args, **kwargs)
File "/root/pwndbg/pwndbg/commands/__init__.py", line 200, in _OnlyWhenRunning
return function(*a, **kw)
File "/root/pwndbg/pwndbg/commands/heap.py", line 255, in bins
if pwndbg.heap.current.has_tcache():
File "/root/pwndbg/pwndbg/heap/ptmalloc.py", line 47, in has_tcache
return (self.mp and self.mp['tcache_bins'])
gdb.error: There is no member named tcache_bins.
```
This commit fixes this issue by checking whether `tcache_bins` field is present in the `malloc_par` structure.
* Add stepsyscall (and stepsc) command
So that one can break at a syscall which is e.g. inside a call.
* Rename next_syscall into nextsyscall
* Display context on next/stepsyscall only if process is alive
When using `eX` commands and setting data to hex value prefixed with `0x`, we get an exception:
```
pwndbg> ed 0xffb21ae4 0x55616740
Traceback (most recent call last):
File "/usr/lib/python3.6/encodings/hex_codec.py", line 19, in hex_decode
return (binascii.a2b_hex(input), len(input))
binascii.Error: Non-hexadecimal digit found
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/dc/installed/pwndbg/pwndbg/commands/__init__.py", line 109, in __call__
return self.function(*args, **kwargs)
File "/home/dc/installed/pwndbg/pwndbg/commands/__init__.py", line 200, in _OnlyWhenRunning
return function(*a, **kw)
File "/home/dc/installed/pwndbg/pwndbg/commands/windbg.py", line 141, in ed
return eX(4, address, data)
File "/home/dc/installed/pwndbg/pwndbg/commands/windbg.py", line 180, in eX
data = codecs.decode(bytestr, 'hex')
binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found)
```
This commit fixes this problem so that if the data input has prefix, it is stripped.
* Implement support for ptmalloc's tcache in heap/ (#387)
Glibc 2.26 added per-thread cache of free chunks. This implements
new "tcache" and "tcachebins" commands for displaying information about
this cache.
Note this works well only if pthread is linked in the debugged program.
Otherwise gdb cannot access thread-local variables, so it cannot find
address of tcache main struct. One can though find the address, ex. by stepping
through malloc code, and pass it to the new commands.
* Another round of review fixes.
* handle gracefully older libc, without tcache
* use aligned size for consistency with other bins
* Improve behavior without IDA Pro
* Fix import order
* Improved IDA Pro behaviour more
* Added only_after_first_prompt decorator
* Removed newline after import
* Added documentation
* Improved docstring
* Add prototype of unit tests for pwndbg
* Add test for pwndbg [filter]
* Fix isort, e2e tests, add pytest requirement
* Add comment about not handling exceptions for unittests
* Fixes after rebase
* Fix test_loads_without_crashing
* e2e tests: no colors & loading pwndbg tests
* Fix isort
* Add example of no file loaded test
* Move tests to unit_tests, add binary, add memory tests
* Isort fixes
* Move from e2e/unit tests to tests
* Add info about tests to DEVELOPING.md
* Fix tests
* review fixes
* commands filtering test: check for contents, not for equality
* Add tests launcher bash script
* Change tests launcher name from unittests to pytests
* Cleanup; better test file paths
* Add theme param to disable colors
* Better test_loads
* Skip some tests locally that can run on travis
* Fix test_loads according to travis
* Fix travis tests
* Fix regression made in #432
*This situation pushes me more and more to work on tests engine*
* Fix eX memory write on Python 2
As string literal is unicode, in Py2 the code below would fail if `bytestr` is just a `str`, due to `'0'` being unocide literal:
```
bytestr.rjust(size*2, '0')
```
Here is `readelf --program-headers <binary>` output for different readelf versions
(The `//` are commented lines; the output is truncated so it contains only useful data):
```
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
// GNU readelf (GNU Binutils for Debian) 2.25 (2014):
LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000
0x000000000001bad4 0x000000000001bad4 R E 200000
// GNU readelf (GNU Binutils) 2.29.1 (2017):
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x000000000001e050 0x000000000001e050 R E 0x200000
```
Our parser parsed the line after the one containing `LOAD` and expected that `Align` column value will be always prefixed with `0x`. As we can see this is not always the case...
It seems that pwndbg.memory.write fix for Py2 introduced in 433bf231
wasn't tested properly on Py3.
In Py2 by default the `bytes` is just `str` and so doesn't accept the encoding argument.
Because of that a `from builtins import bytes` has been added.
Some more info on `builtins` module can be found here: http://python-future.org/imports.html#imports-of-builtins
Detailed info is within the issue, but TLDR:
```
(gdb) show osabi
El actual SO ABI es «auto» (actualmente «GNU/Linux»).
El SO ABI predeterminado es «GNU/Linux».
```
* Avoid to use 'type' as varialbe name
* Fix utf8 issue of Parameter.value in python2
* Fix Parameter member funcs
* Operator overwrite of Parameter
* Remove all workaround of Parameter
* Use regex to unwrap the string
* Remove impossible cases in commands/context.py after Parameter class update
* Revert "ensure length padding works with py2 by enforcing unicode awareness (#416)"
This reverts commit 8ecaa67043.
* Add scripts for benchmarking and profiling pwndbg commands
* Fix performance issue in emulator.py
Register to unicorn enum lookup was really ineffective. Replaced with
parsing (consts) on initialization time, and only dict lookup on hot path.
* Fix performance issue in syntax_highlight.
Current code initialized pygments on each syntax_highlight(), which
apparently took some time.
* Minor performance improvements in syntax_highlight
* Memoize IDA availability.
Not sure it this is a valid solution, I have never used pwndbg with IDA.
However, we should not try to connect to ida on each context(), as this
takes 25% of current exec time.
* Explicitly source gdbinit in benchmark scripts.
This works around the issue of python2 not being unicode aware and
the config classes not properly returning instance of decoded raw
strings. This leads to length operations being performed on bytes
rather then logical characters.
We check for python2 and enfore decoding if not a text_type.
Fixes#412
See timothycrosley/isort#652 for more information. tl;dr is `pip install isort==4.3.0` fails, which is what `pip install -Ur requirements.txt` will attempt to do.
Ban this specific version as it causes issues.
* hexdump: adjust shown offset from src while repeating
* nearpc: make command repeatable to show further instructions
The pc gets adjusted to the last instructions address making it
visually easy to follow where to continue reading the assembly.
This also forwards repeating of emulate() and pdisass()
* telescope: make command repeatable with adjusted offset from src
This also forwards stack() to be repeatable.
* Make chain.get() to check vmmap first in bare metal mode
Make chain.get() limit to de-reference within the known page in
bare metal mode.
Since the address are all valid when mmu is not enable and all
the value are valid physical address. It will be de-referenced
even these addresses are not used and actually, it is data in
the most of case. Ex. 0x1 often means the value 1, not the
address 0x1.
Also, for issue #371, some addresses may be the MMIO registers.
The read operation on these address will break the state.
It is better to limit the de-reference address range. This patch
will also fix it, hopefully.
* Add custom vmmap add/del API in vmmap.py
In some cases, ex. bare metal, the pages information can not be
detected automatically. Also, the most of pwndbg feature rely on
page information such as highlighting.
User may want to create page information manually and maintain it
by himself.
This commit add python APIs to manually add/del page information
and they are isolated.
* Fix stack page detection in bare metal mode
We can not detect the stack page size in bare metal mode by
1. finding the ELF location after the stack page
2. page fault
A simple workaround is returning the current $sp page
and assume it is the stack page.
* Add vmmap control command to add/del customized vmmap
In some cases, ex. bare metal, the pages information can not be
detected automatically. Also, the most of pwndbg feature rely on
page information such as highlighting.
User may want to create page information manually and maintain it
by himself.
I add few commands to make user can add/del pages and load page
information from ELF sections.
* Fix the command amount for auto test to pass CI
* Add warning message
* Fix descriptions
* Fix cache issue and use bisect in insert API
* Keep LinuxOnly in find_elf_magic
* remove XXX
This allows to use args section via the context-sections config
setting (default off).
Additionally introduce a nearpc-show-args config value making it
possible to disable showing it trice while using the args section.
Fixes the problem that can be observed below:
```
pwndbg> py import re; flags = 1 | re.MULTILINE
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3.6/enum.py", line 798, in __or__
result = self.__class__(self._value_ | self.__class__(other)._value_)
File "/usr/lib/python3.6/enum.py", line 291, in __call__
return cls.__new__(cls, value)
File "/usr/lib/python3.6/enum.py", line 533, in __new__
return cls._missing_(value)
File "/usr/lib/python3.6/enum.py", line 762, in _missing_
new_member = cls._create_pseudo_member_(value)
File "/usr/lib/python3.6/enum.py", line 788, in _create_pseudo_member_
pseudo_member._name_ = None
AttributeError: 'int' object has no attribute '_name_'
```
* theme: make everything themable by avoiding explicite colors
This makes it posssible to theme everything logically grouped by
message types. This will also make it easier for future features
to keep a consistent way of coloring plus make every non-specific
coloring themeable automatically.
Direct explicit usage of colors should be avoided in future commits.
* theme: make banner fully customizable including positions
Since the python2 use `from __future__ import unicode_literals`, so the string literals will be `unicode` type in python2.
Use `six.string_types` in `isinstance()` instead of using `str` type.
* Fixes `u` command `module object is not callable` (#310)
pwndbg> u 0x404030
'u': Starting at the specified address, disassemble
N instructions (default 5).
Traceback (most recent call last):
File "/home/dc/installed/pwndbg/pwndbg/commands/__init__.py", line 99, in __call__
return self.function(*args, **kwargs)
File "/home/dc/installed/pwndbg/pwndbg/commands/__init__.py", line 191, in _OnlyWhenRunning
return function(*a, **kw)
File "/home/dc/installed/pwndbg/pwndbg/commands/windbg.py", line 292, in u
pwndbg.commands.nearpc(where, n)
TypeError: 'module' object is not callable
* Fix malloc chunk names (#318)
* heap: respect rename of malloc_chunk fields
newer glibc uses different names for the fields of malloc_chunk
* move value_from_type to typeinfo and rename to read_gdbvalue
* add comment about renaming of `[prev_]size`
* Workaround for gdb remote target search bug described in #321 (#322)
* Fixes issue when we try to display context while selected thread is running #299 (#331)
* Fix tag_release (#348)
* Fix "dt" offsets which are sometimes floating-point (#355)
* Fixes#362 - broken entry command (#363)
* Fix upper_stack_boundary not working introduced in 31f468e
The `upper_stack_boundary` we returned wasn't matching the one from `vmmap`.
Previously we determined upper address by having a memory read failure.
Recent changes made it so we got a `None` instead of the address in such situation.
This adds a parameter to `find_elf_magic` which lets us get a result when gdb.MemoryError occurs.
* Small refactor: add missing newlines
Adds `$rebase(offset)` gdbfunction that can be used to set up a breakpoint
over an offset from program image base.
Also changed a bit the pwndbg banner displayed at startup.
* Add dereference-limit and heap-dereference-limit parameters
This allows setting the number of pointers dereferenced during 'telescope'
and in the register context. Separately, the number of heap bins which
are dereferenced can be set.
* Cast LIMIT to an integer, and address off-by-one
Disconnect3d
Maciej Dębski
Benedikt Werner
Maciej Kotowicz
Jörg Thalheim
Meng-Huan Yu
Levente Polyak
Bruce Chen
Zach Riggle
Zach Riggle
Alexandro Sanchez
Bjoern Kerler