JTool II: Testers wanted

[morpheus]

Hello all,

JTool2 is a radical rewrite of jtool to make it more featureful and extensible, and less buggy.

From the ChangeLog (WhatsNew)

- Matched JTool features:

- -F (find string)
- -a
- -l
- -S, -v
- --pages (automatically in quick)
- -h (for shared cache)

- Everything is faster. Especially disassembly, and in some cases by an order of magnitude:
- When using -q jtool2 is produces almost same output (but faster than) otool/objdump
- Without -q jtool2 is still on par with otool, often faster, AND provides strings + basic decompilation
- When testing on SpringBoard (THE pathological case), -q -d finishes in under 2s! jtool v1 would get take minutes (even with -q)
- Caching in particular is way faster.
- Jtool v1 had an unnecessary cache lookup when adding, which slowed it considerably. No more.
- Sequential address lookups in cache now use cursor

- Decompilation is smarter:

- JTool detects the number of arguments for a given function automatically. Two corner cases are:
- First function call in a given function (won't know how many arguments)
- Functions with no arguments (void) may still show up with their first argument
- Arguments are also autodetected by type! No need to specify the ([ic@b...) etc (although this
will be supported again in a future release, for those cases where the arguments need to be refined)


- New Architectures: arm64e (A12 chipset and later), ARM64_32 (0x2000000c/0x1, for Apple Watch 4)

- ARMv8.1 instructions (CASP, PAN so far)

- ARMv8.3 instructions (so you don't have to wait for IDA plugins from people who won't ever deliver)
B[L]RA[A/B][Z], RETA[A/B], LDRA[A/B] - still need ERETA[A/B] and XPACD, XPACI, XPACLRI (C6-1002)
AUTIA/PACIA/PACGAA Appear to be unused as yet by AAPL

- Can now work directly on *COMPRESSED* kernelcaches! Plan is to integrate joker fully into jtool2 - and soon

- Fix for weird functions like zinit, which seems to ignore (i.e. not get) some arguments (case in point, X2 is not
set, while X3 is! Weird)

- The '-a' and '-o' switch are now much more precise when in shared caches


- Dyld-625 support: BIND_SUBOPCODE_THREADED_SET_BIND_ORDINAL_TABLE_SIZE_ULEB and BIND_SUBOPCODE_THREADED_APPLY
Shows opcodes in -opcodes
Successfully reconstruct the bindings for dyldinfo -bind (even though it's no longer the real table..)

Now, I'm still not caught up on all the features of jtool v1, notably code signing and such, but that's in the works, and when it's done, I'll retire jtool v1 for good.


PLEASE HELP TESTING by looking at WhatsNew.txt and trying out features! If you see something (crash) say something (to me) over this forum. You will help improve the tool, and I'll mention your found bugs in the ChangeLog Thank you!

IMPORTANT NOTE: JTOOL v1 crashes hideously on arm64e (A12+) binaries. This is resolved in jtool2. Going forward, I will not fix jtoolv1 - just use jtoolv2

Also, feature requests are more than requested. I'm currently working on fixing the shared cache extraction so it creates perfect dylibs, as well as in-kernelcache processing, so as to meld joker into jtool2.

Latest build always available at http://NewOSXBook.com/tools/jtool2.tgz, for ARM64, x86_64, and Linux.

[zchee]

Hi, morpheus.
I'm zchee. I was sent twitter mentions and posted http://newosxbook.com/forum/viewtopic.php?f=3&t=16800. I hope you still remember me.

Thank for everything develop great tools :D

I was short testing jtool2 2.0 (alpha 1, Paris), and I found some nits bugs.

1. Run "jtool2 --help", will display the jtool [options] _filename_.

But it's jtool2.

2. In the jtool2 --help, dyldinfo Compatible Options section, display the


Code: Select all

dyldinfo Compatible Options:
   --bind        print addresses dyld will set based on symbolic lookups
   --lazy_bind   print addresses dyld will lazily set on first use
   --opcodes     print opcodes used to generate the rebase and binding information
   --function_starts   print table of function start addresses




But it will show the "--[flag] - Unknown option".
Actually, the dyldinfo Compatible Options flags are available the single hyphen(-)

3. In the jtool2 --help, not shown the "--pages" flag. Maybe adding Newer (JTool 2) Options section...?

Other than that, it works perfectly in my environment so far. Thank you for a nice tool !!


BTW, as before, I wrote a zsh completion script for jtool2. I am happy if morpheus and everyone who uses jtool2 can help.
The script code is below. Or I pushed to my zchee/zsh-completions(https://github.com/zchee/zsh-completions) github repository.
https://github.com/zchee/zsh-completions/blob/master/src/zsh/_jtool2

Code: Select all

#compdef jtool2

# -----------------------------------------------------------------------------
# Copyright (c) 2018, Jonathan Levin (@Morpheus______ / http://newosxbook.com)
# All rights reserved.
# -----------------------------------------------------------------------------
#
# jtool2
#   http://newosxbook.com/tools/jtool2.tgz
#
# version: 2.0 (alpha 1, Paris) compiled on Sep 22 2018 17:55:31
#
# -----------------------------------------------------------------------------
#
# Usage: jtool2 [options] _filename_
#
# OTool Compatible Options:
#    -h            Dump Mach-O (or DYLD Shared Cache) header
#    -l            List sections/commands in binary
#    -L            print shared libraries used
#
# JTool options:
#    -S            List Symbols (like NM)
#    -v            Toggle verbosity
#
# New Options:
#    -q            Quick operation - do not process any symbols in the Mach-O
#    -F            find all occurrences of _string_ in binary
#    -a            Find offset/segment corresponding to virtual address _addr_
#    -o            Find address corresponding to offset _offset_
#    -d            Dump (smart dump, will disassemble text and dump data by autodetecting)
#
# Newer (JTool 2) Options:
#    --analyze     Analyze file and create a companion file
#
# dyldinfo Compatible Options:
#    -bind        print addresses dyld will set based on symbolic lookups
#    -lazy_bind   print addresses dyld will lazily set on first use
#    -opcodes     print opcodes used to generate the rebase and binding information
#    -function_starts   print table of function start addresses
#
# Environment Variables:
#    ARCH                   Select architecture slice. Set to arm64, arm64e, arm64_32, armv7, armv7k, x86_64 or (not for long) i386
#    JDEBUG                 Enhanced debug output. May be very verbose
#    JCOLOR                 ANSI Colors. Note you'll need 'less -R' if piping output
#    JTOOLDIR               path to search for companion jtool files (default: $PWD).
#                           Use this to force create a file, if one does not exist
#    NOPSUP                 Suppress NOPs in disassembly
#
# -----------------------------------------------------------------------------

function _jtool2() {
  local context curcontext=$curcontext state line ret=1
  declare -A opt_args

  _arguments -C \
    '-h[dump Mach-O (or DYLD Shared Cache) header]' \
    '-l[list sections/commands in binary]' \
    '-L[print shared libraries used]' \
    '-S[list Symbols (like NM)]' \
    '-v[toggle verbosity]' \
    '-q[quick operation - do not process any symbols in the Mach-O]' \
    '-F[find all occurrences of _string_ in binary]:__string__' \
    '-a[find offset/segment corresponding to virtual address _addr_]:virtual address (0x...)' \
    '-o[find address corresponding to offset _offset_]:offset (0x...)' \
    '-d[dump \(smart dump, will disassemble text and dump data by autodetecting\)]' \
    '--analyze[analyze file and create a companion file]' \
    '--pages[show pages]' \
    '-bind[print addresses dyld will set based on symbolic lookups]' \
    '-lazy_bind[print addresses dyld will lazily set on first use]' \
    '-opcodes[print opcodes used to generate the rebase and binding information]' \
    '-function_starts[print table of function start addresses]' \
    '*:filename:_files' \
    && ret=0

  return ret
}

_jtool2 "$*"

# vim:ft=zsh:et:sts=2:sw=2


For morpheus, I wrote

Copyright (c) 2018, Jonathan Levin (@Morpheus______ / http://newosxbook.com) All rights reserved.

Please let me know if there is a problem with the license.

Also, I pushed filemon, joker, jtool, jtool2, kdv, and procexp zsh completion script to github. Likewise, if it is a problem I will delete it so please let me know.
https://github.com/zchee/zsh-completions/blob/master/src/zsh/

J Says: Not only is it no problem, it is great to have completion scripts!!! As for --arguments , you are right - I will fix, thank you! As for "jtool2", I still call the tool jtool internally because very soon jtool (1) is going to go away, and I want people to still use the same name in scripts and so on

[jonios]

The -L argument doesn't seem to work. Sorry if thats known, its just the first thing I tried.

Code: Select all

./jtool2 -L myapp
Note: 13701 symbols detected in this file! This will take a little - generate a companion file or use '-q' for quick mode..


Where myapp is a swift arm64 macho. jtool1 spits out the dynamically linked libraries

Code: Select all

$ jtool -L myapp
...
/System/Library/Frameworks/UIKit.framework/UIKit
@rpath/libswiftCore.dylib
@rpath/libswiftCoreFoundation.dylib
@rpath/libswiftCoreGraphics.dylib
...


BTW, just noticed jtool2 is missing a word in the output: "this will take a little" (while is missing)

J says: Fixed, and typo fixed too. I had forgotten to implement -L.... http://NewOSXBook.com/tools/jtool2

[morpheus]

Alpha 3: joker integration , especially for stripped 1469 caches - this will bring back almost 1,000 symbols

http://NewOSXBook.com/tools/jtool2.tgz

[morpheus]

01/11/2018 (Maui!)
------------------

- jtool -l now does 32-bit again..
- Supports MH_PRELOAD. Yes, I know XNU doesn't. But iPhone11,*'s Petra does! So you can now disassemble those images (can your IDA do that?)

- Code signature improvements: --sig does 0x20500 (Thanks to Jeremy Agostino!)
Finally, requirement parsing works (Again, thanks to Jeremy Agostino asking for it)
Colorized output
- (VERY) Initial support for notarization tickets


I'll be bringing back --sign (to self sign) very soon, and probably some rudimentary ARM32/Thumb disassembly again. But not in this version

http://NeWOSXBook.com/tools/jtool2.tgz

[morpheus]

- Bug fixes for -S
- New feature "--tbd" - works directly on dyld_shared_cache_arm64:framework_name and spits a TBDv3 file to stdout. Tested this on a couple of frameworks and it's valid, so you can link with it.

Code: Select all

mkdir -p /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/System/Library/PrivateFrameworks/NetworkStatistics.framework

./jtool2 --tbd /Volumes/PeaceB16B92.N102OS/System/Library/Caches/com.apple.dyld/dyld_shared_cache_arm64:NetworkStatistics  > /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/System/Library/PrivateFrameworks/NetworkStatistics.framework/NetworkStatistics.tbd

Latest build always available at http://NewOSXBook.com/tools/jtool2.tgz, for ARM64, x86_64, and Linux.

[pwm]

I think the ARM64 slice of the FAT binary is the wrong version. If I run the Linux or macOS versions, it shows

Code: Select all

$ jtool2
Welcome to JTool 2.0 (alpha 5, still Maui) compiled on Nov 21 2018 14:12:38. Try "--help" for help


whilst if I run on iOS it shows

Code: Select all

$ jtool2
Welcome to JTool 2.0 (alpha 3, Belize City) compiled on Oct 29 2018 15:46:54. Try "--help" for help


Doing "strings" on the FAT binary shows both Maui and Belize City strings rather than two Maui strings as I would expect.

In addition the iOS version seems to be missing the --ent and --sig commands.

[morpheus]

Oops. Fixed. Also updated to nightly build (Columbia) http://NewOSXBook.com/tools/jtool2.tgz

[jonios]

Code: Select all

$ jtool --version
This is 2.0 (alpha 6, Jo'Burg) compiled on Dec 18 2018 02:45:38


The computation of code signed page sha's is failing in jtool2. Using the binaries in the 12.1 (16B91) ios developer disk image, jtool2 computes hashes that do not match what are in the binary.

Code: Select all

/Volumes/DeveloperDiskImage/usr/libexec $ ARCH=arm64 jtool --sig sysmond
An embedded signature
Code Directory (746 bytes)
      Version:     20400
      Flags:       adhoc  (0x2)
      CodeLimit:   0xee80
      Identifier:  com.apple.sysmond (@0x58)
      Executable Segment: Base 0x00000000 Limit: 0x00000000 Flags: 0x00000000
      CDHash:        6facc4335e8a22a7e2854b35078d8cd38467b0f494a4a2ef2f0f9dd160ce4245 (computed)
      # of Hashes: 15 code + 5 special
      Hashes @266 size: 32 Type: SHA-256
         Slot   0 (File page @0x0000):   524840a4fa0f168418bc2f20f84ca96e5ab9fca3be0cc1b0236512f1a55af76a != 113325e7154799465efd2f23100c2b0aad0a2655babf0a175fb27dc31653f4d4(actual)
         Slot   2 (File page @0x2000):   de102574da91ba351b5e50f9e25d68a95731d170a096735435f6fddb17e14f3a != ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7(actual)
         Slot   3 (File page @0x3000):   49c7adeb9476fa088b9b293c3a109724d1a767b303240197696ae12f3fd49184 != ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7(actual)
         Slot   4 (File page @0x4000):   fffd0349014c0b8ecbdc98c2fe0d9db55c8bc1af0faf1690fa4d1ac14ca38f8e != 524840a4fa0f168418bc2f20f84ca96e5ab9fca3be0cc1b0236512f1a55af76a(actual)
         Slot   5 (File page @0x5000):   4e63a15257b938af8d8f6ee9da71bd82751c16b1e8188ab624cc268065544785 != ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7(actual)
         Slot   6 (File page @0x6000):   00e4c9bfa59a7fa695566387f394cdc2eac56044692c1b722d4df4e651d335fa != de102574da91ba351b5e50f9e25d68a95731d170a096735435f6fddb17e14f3a(actual)
         Slot   7 (File page @0x7000):   ea860e1eae1181f6e1c2e69ddd86a93b801cce65e5e3bfdb0d28328bf1fe6672 != 49c7adeb9476fa088b9b293c3a109724d1a767b303240197696ae12f3fd49184(actual)
         Slot   8 (File page @0x8000):   f281df271883c952ec0a48611307dc4a2c25a1416d43e2ee2fabb48cc09a8dac != fffd0349014c0b8ecbdc98c2fe0d9db55c8bc1af0faf1690fa4d1ac14ca38f8e(actual)
         Slot   9 (File page @0x9000):   480ff7b29e87f68c743951e285d2d8a41559b5cd5074d85381cefab780182b45 != 4e63a15257b938af8d8f6ee9da71bd82751c16b1e8188ab624cc268065544785(actual)
         Slot  10 (File page @0xa000):   d93542dc6c18f593fd93ce178498aa5ef5a724df6a438b5d4f96fccd258f8118 != 00e4c9bfa59a7fa695566387f394cdc2eac56044692c1b722d4df4e651d335fa(actual)
         Slot  11 (File page @0xb000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 != ea860e1eae1181f6e1c2e69ddd86a93b801cce65e5e3bfdb0d28328bf1fe6672(actual)
         Slot  12 (File page @0xc000):   610a99584b4497461c19d368a9ea5a188e3dd5054b1a17a10d3e99b9b57f656a != f281df271883c952ec0a48611307dc4a2c25a1416d43e2ee2fabb48cc09a8dac(actual)
         Slot  13 (File page @0xd000):   e8c010e33ac3e5bb95f00814bd9b3ad70adaf198c044a3795ead72d37e382533 != 480ff7b29e87f68c743951e285d2d8a41559b5cd5074d85381cefab780182b45(actual)
         Slot  14 (File page @0xe000):   aed53efe0c83a40d5e20857f8f7ec3982e6cc63d50e331ef63aa461098e8d4ff != af314160907d46651006a4ce646dd0aaf6434cc537ef9a49b61c89996b1cced1(actual)
 Empty requirement set (12 bytes)
Entitlements (285 bytes) (use --ent to view)
Blob Wrapper (8 bytes) (0x10000 is CMS (RFC3852) signature)


jtool1 doesn't have any trouble (as long as ARCH=arm64)

Code: Select all

/Volumes/DeveloperDiskImage/usr/libexec $ ARCH=arm64 jtool1 --sig sysmond
Blob at offset: 61056 (1104 bytes) is an embedded signature
Code Directory (746 bytes)
      Version:     20400
      Flags:       adhoc (0x2)
      CodeLimit:   0xee80
      Identifier:  com.apple.sysmond (0x58)
      Executable Segment: Base 0x00000000 Limit: 0x00000000 Flags: 0x00000000
      CDHash:        6facc4335e8a22a7e2854b35078d8cd38467b0f494a4a2ef2f0f9dd160ce4245 (computed)
      # of Hashes: 15 code + 5 special
      Hashes @266 size: 32 Type: SHA-256
 Empty requirement set (12 bytes)
Entitlements (285 bytes) (use --ent to view)
Blob Wrapper (8 bytes) (0x10000 is CMS (RFC3852) signature)


In the output of jtool2 there are some strange things
1. Where is slot 1?
2. The slots seem to be offset by 4, meaning slot 4 actual == slot 0 expected, slide 5 actual == slide 1 expected, etc.
3. jtool2 always seems to show only arm64 output.

Code: Select all

$ ARCH=arm64 jtool --sig sysmond
An embedded signature
Code Directory (746 bytes)
      Version:     20400
      Flags:       adhoc  (0x2)
      CodeLimit:   0xee80

$ ARCH=arm64e jtool --sig sysmond
An embedded signature
Code Directory (746 bytes)
      Version:     20400
      Flags:       adhoc  (0x2)
      CodeLimit:   0xee80


Both have a size of 0xee80, but when I extract the individual mach-o's using lipo I can see that jtool does properly see a difference in arm64e.

Code: Select all

$ lipo sysmond -extract arm64e -output sysmond.arm64e
$ jtool --sig sysmond.arm64e
An embedded signature
Code Directory (746 bytes)
      Version:     20400
      Flags:       adhoc  (0x2)
      CodeLimit:   0xea60


Now the size is 0xea60.

The other two binaries in /usr/libexec, testmanagerd and gputoolsd have similar issues.

[morpheus]

Good catch - Minor bug of mine since I didn't account for FAT binaries (rewriting, rather than cut/pasting) Thanks for letting me know. Next update fixes this, but bandwidth here is sooooooooo slow. Give me another day or two to get stable Internet.