Bringt back Archlinux CI. Fixes #1772, closes #1783 (#1800)

* Only run arch for testing

* Remove outdated arch repo

* Actually build the docker image

* Do not include site packages in sys.path

* Ignore `.relr.dyn` section; skip lines w/o spaces

Newer binaries can contain a `.relr.dyn` section to compress `R_X86_64_RELATIVE` relocation entries.
These binaries can be found for example on archlinux but also on Debian 12 for example.
`readelf` prints the content of the section similarly to this:
```
Relocation section '.relr.dyn' at offset 0x25220 contains 35 entries:
  1198 offsets
00000000001ce8d0
00000000001ce8e0
```
Compared to `00000000001d2000  0000000000000025 R_X86_64_IRELATIVE                        9f330` for
`.rela.plt`.

Pwndbg now chokes on the new format because it expects a space seperator where there is none.
It might be, that this is actually an upstream problem with binutils, because llvm-readelf prints this:
```
Relocation section '.relr.dyn' at offset 0x25220 contains 1198 entries:
    Offset             Info             Type               Symbol's Value  Symbol's Name
00000000001ce8d0  0000000000000008 R_X86_64_RELATIVE
00000000001ce8e0  0000000000000008 R_X86_64_RELATIVE
```

Nevertheless, we aren't actually interested in `R_X86_64_RELATIVE` relocations so I guess it's fine to
just skip all lines that contain no spaces at all.

`.relr.dyn` can only containt `R_X86_64_RELATIVE` relocations as far as I understand
https://maskray.me/blog/2021-10-30-relative-relocations-and-relr

* Accept Full RELRO in test

Archlinux has libc and ld with Full RELRO.
We now just accept Partial and Full RELRO.

* Do not copy binaries from host to docker

The `Dockerfile` copies the whole pwndbg folder to the image.
If we have built binaries on the host before, these binaries will contain references to
the host system and *copied*  to the image.
If we now run `context code` (inside docker) to have a look at the source code this will
fail, because we will try to refer to a path on the host system.

* Do not use loop index after loop

Do not use loop index after the loop. The tests assumed that the loop in line 186
would run at least once, thereby *resetting* `i` to zero. If we never enter the
loop, `i` will *continue* to have the value it had at the end of line 172.
This will cause the test to fail in mysterious ways because `i` is now not reset
to zero but still has the value `31` for example.

The solution is to never use `i` outside of a loop.

* Re-enable archlinux and temporarily disabled ones
pull/1804/head
intrigus-lgtm 2 years ago committed by GitHub
parent 2a6a05f2b8
commit f7cb2f9cb9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -1,2 +1,16 @@
# venv
.venv/
# ignore binaries that are potentially tainted by the host system
# E.g. the binary could reference source code on the host system
# this is a problem because inside docker we won't have access to
# the host system
tests/**/binaries/*.o
tests/**/binaries/*.out
tests/**/binaries/gosample.x*
tests/**/binaries/div_zero_binary/core
tests/**/binaries/div_zero_binary/binary
!tests/**/binaries/*.go
# ignore QEMU test images (could also be tainted)
tests/qemu-tests/images

@ -6,8 +6,7 @@ jobs:
strategy:
fail-fast: false
matrix:
# archlinux is removed for now until we use virtualenv for deps install
images: [ubuntu18.04, ubuntu20.04, ubuntu22.04, debian10, debian11]
images: [ubuntu18.04, ubuntu20.04, ubuntu22.04, debian10, debian11, archlinux]
runs-on: ubuntu-latest
timeout-minutes: 30
@ -23,7 +22,7 @@ jobs:
key: ${{ matrix.images }}-cache-images-{hash}
- name: Docker Build ${{ matrix.images }}
run: docker-compose pull ${{ matrix.images }}
run: docker-compose build ${{ matrix.images }}
- name: Test on ${{ matrix.images }}
run: docker-compose run ${{ matrix.images }} ./tests.sh

@ -2,6 +2,7 @@ import cProfile
import glob
import locale
import os
import site
import sys
import time
from glob import glob
@ -30,6 +31,14 @@ if not os.path.exists(venv_path):
site_pkgs_path = glob(os.path.join(venv_path, "lib/*/site-packages"))[0]
# add virtualenv's site-packages to sys.path and run .pth files
site.addsitedir(site_pkgs_path)
# remove existing, system-level site-packages from sys.path
for site_packages in site.getsitepackages():
if site_packages in sys.path:
sys.path.remove(site_packages)
# Set virtualenv's bin path (needed for utility tools like ropper, pwntools etc)
bin_path = os.path.join(venv_path, "bin")
os.environ["PATH"] = bin_path + os.pathsep + os.environ.get("PATH")
@ -37,7 +46,6 @@ os.environ["PATH"] = bin_path + os.pathsep + os.environ.get("PATH")
# Add gdb-pt-dump directory to sys.path so it can be imported
gdbpt = path.join(directory, "gdb-pt-dump")
sys.path.append(directory)
sys.path.append(site_pkgs_path)
sys.path.append(gdbpt)
# warn if the user has different encoding than utf-8

@ -23,7 +23,7 @@ def get_got_entry(local_path: str) -> Dict[RelocationType, List[str]]:
entries: Dict[RelocationType, List[str]] = {category: [] for category in RelocationType}
for line in readelf_out.splitlines():
if not line or not line[0].isdigit():
if not line or not line[0].isdigit() or " " not in line:
continue
category = line.split()[2]
# TODO/FIXME: There's a bug here, somehow the IRELATIVE relocation might point to somewhere in .data.rel.ro, which is not in .got or .got.plt

@ -120,9 +120,6 @@ Include = /etc/pacman.d/mirrorlist
[extra-debug]
Include = /etc/pacman.d/mirrorlist
[community-debug]
Include = /etc/pacman.d/mirrorlist
[multilib-debug]
Include = /etc/pacman.d/mirrorlist
EOF

@ -131,7 +131,8 @@ def test_command_got_for_target_binary_and_loaded_library():
assert out[2] == ""
assert re.match(r"State of the GOT of .*/libc.so.6:", out[3])
m = re.match(
r"GOT protection: Partial RELRO \| Found (\d+) GOT entries passing the filter", out[4]
r"GOT protection: (?:Partial|Full) RELRO \| Found (\d+) GOT entries passing the filter",
out[4],
)
got_entries_count = int(m.group(1))
assert got_entries_count > 0
@ -145,7 +146,8 @@ def test_command_got_for_target_binary_and_loaded_library():
assert out[1] == ""
assert re.match(r"State of the GOT of .*/libc.so.6:", out[2])
m = re.match(
r"GOT protection: Partial RELRO \| Found (\d+) GOT entries passing the filter", out[3]
r"GOT protection: (?:Partial|Full) RELRO \| Found (\d+) GOT entries passing the filter",
out[3],
)
assert int(m.group(1)) > got_entries_count # We should have more entries now
got_entries_count = int(m.group(1))
@ -161,7 +163,8 @@ def test_command_got_for_target_binary_and_loaded_library():
assert out[3] == ""
assert re.match(r"State of the GOT of .*/libc.so.6:", out[4])
m = re.match(
r"GOT protection: Partial RELRO \| Found (\d+) GOT entries passing the filter", out[5]
r"GOT protection: (?:Partial|Full) RELRO \| Found (\d+) GOT entries passing the filter",
out[5],
)
got_entries_count = int(m.group(1))
assert len(out) == (6 + got_entries_count)
@ -176,18 +179,20 @@ def test_command_got_for_target_binary_and_loaded_library():
assert out[2] == ""
assert re.match(r"State of the GOT of .*/ld-linux-x86-64.so.2:", out[3])
m = re.match(
r"GOT protection: Partial RELRO \| Found (\d+) GOT entries passing the filter", out[4]
r"GOT protection: (?:Partial|Full) RELRO \| Found (\d+) GOT entries passing the filter",
out[4],
)
got_entries_count = int(m.group(1))
for i in range(got_entries_count):
assert re.match(r"\[0x[0-9a-f]+\] .* -> .*", out[5 + i])
assert out[5 + i + 1] == ""
assert out[5 + got_entries_count] == ""
# Second should be libc.so.6
out = out[5 + i + 2 :]
out = out[5 + got_entries_count + 1 :]
assert re.match(r"State of the GOT of .*/libc.so.6:", out[0])
m = re.match(
r"GOT protection: Partial RELRO \| Found (\d+) GOT entries passing the filter", out[1]
r"GOT protection: (?:Partial|Full) RELRO \| Found (\d+) GOT entries passing the filter",
out[1],
)
got_entries_count = int(m.group(1))
assert len(out) == (2 + got_entries_count)
@ -207,18 +212,20 @@ def test_command_got_for_target_binary_and_loaded_library():
# Second should be ld-linux-x86-64.so.2
assert re.match(r"State of the GOT of .*/ld-linux-x86-64.so.2:", out[0])
m = re.match(
r"GOT protection: Partial RELRO \| Found (\d+) GOT entries passing the filter", out[1]
r"GOT protection: (?:Partial|Full) RELRO \| Found (\d+) GOT entries passing the filter",
out[1],
)
got_entries_count = int(m.group(1))
for i in range(got_entries_count):
assert re.match(r"\[0x[0-9a-f]+\] .* -> .*", out[2 + i])
assert out[2 + i + 1] == ""
out = out[2 + i + 2 :]
assert out[2 + got_entries_count] == ""
out = out[2 + got_entries_count + 1 :]
# Third should be libc.so.6
assert re.match(r"State of the GOT of .*/libc.so.6:", out[0])
m = re.match(
r"GOT protection: Partial RELRO \| Found (\d+) GOT entries passing the filter", out[1]
r"GOT protection: (?:Partial|Full) RELRO \| Found (\d+) GOT entries passing the filter",
out[1],
)
got_entries_count = int(m.group(1))
assert len(out) == (2 + got_entries_count)
@ -234,8 +241,12 @@ def test_command_got_for_target_binary_and_loaded_library():
assert re.match(r"\[0x[0-9a-f]+\] puts@GLIBC_[0-9.]+ -> .*", out[4])
assert out[5] == ""
assert re.match(r"State of the GOT of .*/ld-linux-x86-64.so.2:", out[6])
assert out[7] == "GOT protection: Partial RELRO | Found 0 GOT entries passing the filter"
assert re.match(
r"GOT protection: (?:Partial|Full) RELRO \| Found 0 GOT entries passing the filter", out[7]
)
assert out[8] == ""
assert re.match(r"State of the GOT of .*/libc.so.6:", out[9])
assert out[10] == "GOT protection: Partial RELRO | Found 0 GOT entries passing the filter"
assert re.match(
r"GOT protection: (?:Partial|Full) RELRO \| Found 0 GOT entries passing the filter", out[10]
)
assert len(out) == 11

Loading…
Cancel
Save